cgroups(Control Groups) yes linux A mechanism provided by the kernel , This mechanism can integrate a series of system tasks and their subtasks according to the requirements ( Or separate ) To different groups graded by resource , So as to provide a unified framework for system resource management . In short ,cgroups Can restrict 、 Record the physical resources used by the task group . In essence ,cgroups It's a series of hooks that the kernel attaches to the program (hook), In order to achieve the purpose of resource tracking and restriction, the corresponding hook is triggered by the scheduling of resources during the program running .

This article takes Ubuntu 16.04 The system is introduced as an example cgroups, be-all demo All of them are demonstrated in the system .

Why understand cgroups

In the era of virtualization technology represented by container technology, understanding cgroups Technology is very necessary ! For example, we can easily limit what a container can use CPU、 Memory and other resources , How on earth is this achieved ? By understanding cgroups technology , We can spy on linux The context of the whole resource limitation system in the system . So as to help us better understand and use linux System .

cgroups Main role of

Realization cgroups The main purpose of this paper is to provide a unified interface for resource management at different user levels . From resource control of a single task to virtualization at the operating system level ,cgroups It provides four functions :

  • Resource constraints :cgroups You can limit the total amount of resources that the task is to use . For example, set the maximum amount of memory used by the task when running , Once it's over, it's going to be OOM.
  • Priority allocation : By way of distribution CPU Number of timeslices and disks IO bandwidth , In fact, it controls the priority of the task .
  • Resource Statistics :cgoups You can count the resource usage of the system , such as CPU Use time 、 Memory usage, etc . This function is very suitable for the current cloud based product billing method .
  • Task control :cgroups You can suspend task execution 、 Recovery and other operations .

Relevant concepts

Task( Mission )  stay linux In the system , The scheduling and management of the kernel itself does not distinguish between processes and threads , Just according to clone To conceptually distinguish between a process and a thread based on the difference of parameters passed in . Use here task To represent a process or thread of the system .

Cgroup( Control group ) cgroups Resource control in order to cgroup Achieve... In units .Cgroup Represents a task group divided by a resource control standard , Contains one or more subsystems . A task can be added to a certain cgroup, You can also go from one cgroup Move to another one cgroup.

Subsystem( Subsystem ) cgroups The subsystem in is a resource scheduling controller ( Also called controllers). such as CPU The subsystem can control CPU Time allocation of , Memory subsystem can limit memory usage . The author uses Ubuntu 16.04.3 For example , Its kernel version is 4.10.0, Supported by subsystem as follows ( cat /proc/cgroups):
 blkio         For block devices IO Limit .
 cpu           Limit CPU Time slice allocation , And cpuacct Mounted in the same directory .
 cpuacct     Generate cgroup Task occupation in CPU Report on resources , And cpu Mounted in the same directory .
 cpuset       to cgroup The tasks in are assigned independently CPU( Multiprocessor systems ) And memory nodes .
 devices     Allow or prohibit cgroup Task access device in .
 freezer      Pause / recovery cgroup The task .
 hugetlb     Limit the number of memory pages used .              
 memory    Yes cgroup Limit the available memory for tasks in , And automatically generate resource occupancy report .
 net_cls      Use the level identifier (classid) Tag network packets , This makes Linux Flow controller (tc Instructions ) You can identify people from specific cgroup The packet of the task , And network restrictions .
 net_prio    Allow based on cgroup Set network traffic (netowork traffic) The priority of the .
 perf_event  Allow to use perf Tools to monitor cgroup.
 pids          Limit the number of tasks .

Hierarchy( Hierarchy )  The hierarchy has a series of cgroup Arranged in a tree structure , Each level controls resources by binding corresponding subsystems . In the hierarchy cgroup A node can contain zero or more child nodes , The child node inherits the subsystem mounted by the parent node . There can be multiple levels in an operating system .

cgroups File system interface of

cgroups Provide application interface in the form of file , We can go through mount Order to see cgroups The default mount point :

$ mount | grep cgroup

First line tmpfs explain /sys/fs/cgroup The files in the directory are temporary files that exist in memory .
The mount point on the second line /sys/fs/cgroup/systemd be used for systemd System pair cgroups Support for , The author will make a special introduction to the related content in the future .
The rest of the mount points are the root level hierarchies of the subsystems supported by the kernel .

It should be noted that , In the use of systemd In the operating system of the system ,/sys/fs/cgroup The catalogue is made up of systemd Mounted during system startup , And mount it as a read-only type . let me put it another way , The system doesn't recommend that we /sys/fs/cgroup Create a new directory under the directory and mount other subsystem's . This is not the same as the previous operating system .

Now let's explore /sys/fs/cgroup What's in the directory and its subdirectories :

/sys/fs/cgroup Under the directory is the root directory of each subsystem . We use memory Subsystem as an example , have a look memory What's in the catalog ?

These documents are cgroups Of memory Root level settings in the subsystem . such as memory.limit_in_bytes The number in is used to limit the maximum available memory of the process ,memory.swappiness It's preserved in the use of swap And so on .

since cgroups It's based on these documents API Of , Then I can create or modify the contents of these files to apply cgroups. How to do it specifically ? For example, how can we limit the resources that a process can use ? Next, let's go through the simple demo To demonstrate how to use cgroups Limit the resources that a process can use .

View the... To which the process belongs cgroups

Can pass /proc/[pid]/cgroup To see which specific processes belong to cgroup:

Each row contains three columns separated by colons , What they mean are :

  • cgroup Treelike ID, and /proc/cgroups In the document ID One-to-one correspondence .
  • and cgroup All of the tree binding subsystem, Multiple subsystem Separated by commas . here name=systemd I don't have any contact with you subsystem binding , I just gave him a name systemd.
  • The process is in cgroup The path in the tree , That is, to which the process belongs cgroup, This path is relative to the mount point .

since cgroups It's based on these documents API Of , Then I can create or modify the contents of these files to apply cgroups. How to do it specifically ? For example, how can we limit the resources that a process can use ? Next, let's go through the simple demo To demonstrate how to use cgroups Limit the resources that a process can use .

cgroups Tools

In the introduction, through systemd application cgroups Before , So let's use cgroup-bin In the kit cgexec To demonstrate demo.Ubuntu Not installed by default cgroup-bin tool kit , Please install through the following command :

$ sudo apt install cgroup-bin

demo: Limit the process available CPU

When we use cgroups when , It's better not to modify the configuration file directly in the root directory of each subsystem . The recommended way is to define different nodes in the subsystem tree for different requirements . For example, we can be in /sys/fs/cgroup/cpu Create a new directory named nick_cpu The catalog of :

$ cd  /sys/fs/cgroup/cpu
$ sudo mkdir nick_cpu

Then look at the contents of the new directory :

Isn't it a little surprising ,cgroups These configuration files are automatically created when the file directory is created !

Let's set up CPU The cycle is limited to one tenth of the total :

$ sudo su
$ echo 100000 > nick_cpu/cpu.cfs_period_us
$ echo 10000 > nick_cpu/cpu.cfs_quota_us

Are the above two parameters familiar ? you 're right , The author is in 《Docker: Limit the container available CPU》 In this article "--cpu-period=100000 --cpu-quota=200000" That's what they do .

And then create a CPU Intensive program :

 Copy code

void main()
    unsigned int i, end;
    end = 1024 * 1024 * 1024;
    for(i = 0; i < end; )
        i ++;

 Copy code

Save as a file cputime.c Compile and execute in different ways :

$ gcc cputime.c -o cputime
$ sudo su
$ time ./cputime
$ time cgexec -g cpu:nick_cpu ./cputime

time Commands can report us the time spent executing the program , Among them real It's the time we really feel . Use cgexec What we can add cgroup To configure nick_cpu Apply to running cputime In the process of the program . Shown above , Just do it by default 2s about . adopt cgroups Limit CPU Resources need to run after 23s.

demo: Limit the memory available to the process

This time, let's limit the maximum memory available to the process , stay /sys/fs/cgroup/memory Create directory under nick_memory:

$ cd  /sys/fs/cgroup/memory
$ sudo mkdir nick_memory

The following settings limit the process's available memory to the maximum 300M, And don't use swap:

#  Physical memory  + SWAP <= 300 MB;1024*1024*300 = 314572800
$ sudo su
$ echo 314572800 > nick_memory/memory.limit_in_bytes
$ echo 0 > nick_memory/memory.swappiness

And then create a program that constantly allocates memory , It allocates memory five times , Every time you apply 100M:

 Copy code

#define CHUNK_SIZE 1024 * 1024 * 100
void main()
    char *p;
    int i;
    for(i = 0; i < 5; i ++)
        p = malloc(sizeof(char) * CHUNK_SIZE);
        if(p == NULL)
            printf("fail to malloc!");
            return ;
        // memset()  Function is used to put the first... Of the specified memory  n  Bytes set to a specific value
        memset(p, 0, CHUNK_SIZE);
        printf("malloc memory %d MB\n", (i + 1) * 100);

 Copy code

Save the above code as mem.c file , Then compile :

$ gcc mem.c -o mem

Execute the generated mem Program :

$ ./mem

All goes well at this time , Then add the constraint just now and try :

$ cgexec -g memory:nick_memory ./mem

Due to insufficient memory and forbidden use swap, So the process with limited resources is forcibly killed when it applies for memory .

I'll use it again stress The program tests a similar scenario ( adopt stress Procedure application 500M Of memory ):

$ sudo cgexec -g memory:nick_memory stress --vm 1 --vm-bytes 500000000 --vm-keep --verbose

stress The program can provide more detailed information , The way the process is killed is by receiving SIGKILL(signal 9) The signal .

In practical application, it is often necessary to limit a variety of resources at the same time , For example, it's not only limited to CPU Resources limit memory resources . Use cgexec It's easy to implement such a use case , Directly specify multiple -g That's the option :

$ cgexec -g cpu:nick_cpu -g memory:nick_memory ./cpumem


cgroups yes linux The functions provided by the kernel , Because there are many concepts involved , So it's not easy to understand . This paper attempts to introduce the conceptual content at the same time , Use the simplest demo demonstration cgroups Usage of . I hope it's intuitive demo Can help you understand cgroups.

Reference resources :

Linux Control Group brief introduction