logo-virtuozzo

While troubleshooting issues related to a Virtuozzo VPS, we usually come across the ‘user_beancounters’ file in the “/proc directory”. This file is of importance only if we use UBC or Combined SML+UBC memory mode for our Virtuozzo VPS. The resource control information about running virtual environments is contained in this file. So basically, ‘/proc/user_beancounters’ represents your VPS’s allocation of various system resources (memory). It thus is a main indicator of how well our VPS works, how stable it is, or whether there is a resource shortage. So, if you face any trouble while running or installing applications on your VPS, one good way to find the source of the problem is to take a look at this file.

Let’s dig deeper into the details of this file.

In Parallels Virtuozzo containers, virtualization technology resource control settings for each virtual machine are stored in the configuration file “/etc/vz/conf/XXX.conf” (where XXX is the ID of the given CT). These settings are loaded and applied to the containers during the VPS’s startup or on some events such as execution of “vzctl set CTID”. For running containers the resource control settings are exposed via “/proc/user_beancounters”. One such file “/proc/user_beancounters” exists in the node and one inside the VPS too. The file in the hardware node contains the resource control settings of all running VPSs.  A pictorial representation of the file “/proc/user_beancounters” inside a VPS is shown below:

 

#cat  /proc/user_beancouners

anvz

A brief description of the various columns are given below,

UID: Indicates the ID of the container. In Virtuozzo each container is given a unique ID for the ease of management.

RESOURCE: This field indicates the primary,secondary and auxiliary parameters in Virtuozzo. In order to get more details of these resources refer url

HELD: Indicates the current usage of the various resources.

MAXHELD: Indicates the maximum usage of the resource since VPS startup.

BARRIER & LIMIT: Gives us the values of the softlimit and hardlimit of the virtozzo resource controls. Resource requests above that particular limit gets denied.

FAILCNT: It shows the number of refused or rather denied resource allocations of VPS right from the start up stage of the VPS. A non-zero value in this column indicates resource shortage and we need to either increase that particular resource or find the process responsible for it and optimize it. Otherwise it can cause weird issues with services running inside the container. Eg: unexpected service down, intermittent website issues,etc.

The following awk script can be used to list out all the containers with a non-zero values for the column “failcnt”. This script will print out all the containers with a non-zero failcnt value, along with their resource name and the corresponding failcnt value. Save the script as “/root/failcnt.awk” or any name that you like.

 

#cat   /root/failcnt.awk
BEGIN{OFS=””;i=0;j=1;failcntflag=0;}
{
if(NF==7 && index($1,”:”) >0 ){
if(failcntflg==1){
printf “\nCTID=%s”,arr1[1];
for(j=1;j<=i;j++){ printf ” %s “,vector[j];delete vector[j];}
failcntflg=0;i=0;
}
split($1,arr1,”:”);

if($NF!=0) {
i = i+1;
failcntflg=1;
vector[i] = $2″ “$NF;
}
}

if (NF==6 && $NF!=0){
i = i+1;
failcntflg=1;
vector[i] = $1” “$NF;
}
}
END{ printf “\n” }

 

 

Now run the script from node as follows,

 

# awk    -f /root/failcnt.awk /proc/user_beancouners

CTID=101     lockedpages 13
CTID=102     kmemsize 357 shmpages 4 physpages 5 oomguarpages 1 tcprcvbuf 755
CTID=103    kmemsize 528 numothersock 1
CTID=104    kmemsize 73 numothersock 1
CTID=105    kmemsize 1 shmpages 4 tcprcvbuf 4751
CTID=106 numothersock 1986
CTID=107    kmemsize 27 physpages 7 oomguarpages 1 tcpsndbuf 295136
CTID=108    shmpages 4
CTID=109    kmemsize 2 shmpages 2 tcprcvbuf 690
CTID=110    shmpages 3
CTID=111    shmpages 1 physpages 712 oomguarpages 73 tcpsndbuf 63
CTID=112    physpages 15 oomguarpages 8
CTID=113    physpages 24 oomguarpages 11
CTID=114   numothersock 14
CTID=115   physpages 6 oomguarpages 3
CTID=116   physpages 517 oomguarpages 33
CTID=117  physpages 49 oomguarpages 40
CTID=118   physpages 87 oomguarpages 24
CTID=119   numproc 71 physpages 2250 oomguarpages 472

 

As you can see from the above output, container “102” shows the highest number of ‘failcnt’ for resources. For this VPS, “kmemsize”,”shmpages”,”physpages”,”oomguarpages” and “tcprcvbuf” show non-zero failcnt values and among them the first four resources are related to RAM. Upgrading the RAM inside that VPS is a good suggestion, but that should be considered only after finding out the resource intensive process inside the container and optimizing it.

 

You can use the following commands to list out the memory intensive processes inside the container.

* Lists top 3 memory intensive processes,

#ps -auxf | sort -nr -k 4 | head -3

OR

#wget -O   /root/ps_mem.py  http://www.pixelbeat.org/scripts/ps_mem.py  python /root/ps_mem.py |tail -3

The “/proc/user_beancounters” in the node can be monitored continuously to find out the VPSs that are short of resources and the corresponding VPS owner can be contacted for resource upgrade or optimization.