Solaris: locks & lockstat

 

Locking

Locks are used to protect critical data structures from simultaneous update access. Locks allows you to update one or a group of variables. Locks can also provide a synchronization mechanism for threads.

 

Locks are used when threads share “writable data”. However, frequent locking is required when a program writes data. There are several locking mechanism available to manage different locking situations.

Sr. #	Lock	Description
1.	Mutex	Ensures only one thread can lock a resource at a time. No other thread can lock the resource until the previous thread releases the resource. A thread performing a read, reads the data that the locking thread last stored. The state of locking thread must be visible to all processors on a multiprocessor system.
2.	Conditional Variable	Associated with a condition. Condition Variables remain in force as long as the condition exists.
3.	Counting Semaphore	Control multiple thread access to a resource. When a thread acquires the resource, the value of the semaphore is decremented.
4.	Multiple-reader, single-writer	Enables several threads to access data. The reader group and the thread that wants to write the data owns the lock.

 

A poor locking design that protects too much data by using one lock can create performance problems by serializing the threads for a long time. To solve locking problems, you must significantly “re-design” the program.

Locking problems:

Performance problems that are related to locking are difficult to identify and usually appear as programs running slower than expected for no obvious reasons.

The following programming errors cause problems with locking:

a.       Contention caused by inappropriate use of locking mechanisms

b.      A deadlock caused by two competing operations

c.       A lock control that is “lost” to the process

d.      A race for resources, which is similar to deadlock situation.

e.      An incomplete implementation of the lock control within the program.

 

“Lockstat” is the tool used to identify locking problems in an application.

 

Example:

# lockstat -H -D 10 sleep 1

 

Adaptive mutex spin: 513 events

Count indv cuml rcnt     nsec Lock                Caller

-------------------------------------------------------------------------

  480   5%   5% 1.00     1136 0x300007718e8       putnext+0x40

  286   3%   9% 1.00      666 0x3000077b430       getf+0xd8

  271   3%  12% 1.00      537 0x3000077b430       msgio32+0x2fc

  270   3%  15% 1.00     3670 0x300007718e8       strgetmsg+0x3d4

  270   3%  18% 1.00     1016 0x300007c38b0       getq_noenab+0x200

  264   3%  20% 1.00     1649 0x300007718e8       strgetmsg+0xa70

  216   2%  23% 1.00     6251 tcp_mi_lock         tcp_snmp_get+0xfc

  206   2%  25% 1.00      602 thread_free_lock    clock+0x250

  138   2%  27% 1.00      485 0x300007c3998       putnext+0xb8

  138   2%  28% 1.00     3706 0x300007718e8       strrput+0x5b8

-------------------------------------------------------------------------

[...]

 

Explanation:

In the above example, the lockstat command monitors the locks user by the “sleep” command. Within its own process as well as the locks obtained for the processes in the kernel. These locks are acquired during system call processing.

Large “indv” with a slow or fast locking time helps in identifying the problem. A module is identified that is causing a problem by referring to the lock name, which is usually prefixed with the module name.

 

Note: the “indv” column shows current event as a percentage of all events.

Solaris/Zone: Virtualized Clocks/TimeZones

Oracle Solaris Native Zones now have virtualized clocks to support applications that need to run in different times or to test specific time-related scenarios, for example, how an environment might respond to a leap second.

You can set time values in non-global zones that are different from the value in the global zone. The ability to set different time values in non-global zones is still dependent on the time changes in the global zone. If you change the time in the global zone, the non-global zone time is offset by the same amount.

For example, to set the time value in a non-global zone:

# zonecfg -z myzone

zonecfg:myzone> set limitpriv=default,sys_time

zonecfg:myzone> set global-time=false

zonecfg:myzone> exit

Solaris: Adding different types of raw volumes and disk devices to non-global zones (NGZ)

Adding file system or disk devices to a non-global zone is an integral part of creating a zone. We can add different types of file systems, raw devices and disk devices as well to a non-global zone. The post describes one of the most common ways of adding different file systems, raw and disk devices to a non-global zone.

Adding a Raw Disk device

We can either add a slice or a complete raw disk to the non-global zone. In case of a full disk use s2 slice or else use any other slice you want to add.

global # zonecfg -z zone01

zonecfg:zone01> add device

zonecfg:zone01:device> set match=/dev/rdsk/c0t0d0s6

zonecfg:zone01:device> end

zonecfg:zone01> commit

zonecfg:zone01> verify

zonecfg:zone01>exit

Adding a VxFS file system

1. Adding a VxVM file system

global # zonecfg -z zone01

zonecfg:zone01> add fs

zonecfg:zone01:fs> set type=vxfs

zonecfg:zone01:fs> set special=/dev/vx/dsk/datadg/datavol

zonecfg:zone01:fs> set raw=/dev/vx/rdsk/datadg/datavol

zonecfg:zone01:fs> set dir=/data

zonecfg:zone01:fs> end

zonecfg:zone01> commit

zonecfg:zone01> verify

zonecfg:zone01> exit

2. Adding a VxVM raw volume

global# zonecfg -z zone01

zonecfg:zone01> add device

zonecfg:zone01:device> set match=/dev/vx/rdsk/dg1/vol1

zonecfg:zone01:device> end

zonecfg:zone01> commit

zonecfg:zone01> verify

zonecfg:zone01> exit

Adding UFS file system

1. Adding UFS under SVM

global # zonecfg -z zone01

zonecfg:zone01> add fs

zonecfg:zone01:fs> set dir=/u01

zonecfg:zone01:fs> set special=/dev/md/dsk/d100

zonecfg:zone01:fs> set raw=/dev/md/rdsk/d100

zonecfg:zone01:fs> set type=ufs

zonecfg:zone01:fs> add options [nodevices,logging]

zonecfg:zone01:fs> end

zonecfg:zone01> commit

zonecfg:zone01> verify

zonecfg:zone01> exit

2. Adding UFS under VxVM volume
We can also create a UFS file system on a VxVM volume as follows.

global # vxassist -g datadg make datavol 1g

global # mkfs -F ufs /dev/vx/rdsk/datadg/datavol

global # mount -F ufs /dev/vx/dsk/datadg/datavol /zones/zone01/root/data

To add the UFS under VxVM :

global # zonecfg -z zone01

zonecfg:zone01> add fs

zonecfg:zone01:fs> set type=ufs

zonecfg:zone01:fs> set special=/dev/vx/dsk/datadg/datavol

zonecfg:zone01:fs> set raw=/dev/vx/rdsk/datadg/datavol

zonecfg:zone01:fs> set dir=/zones/zone1/root/data

zonecfg:zone01:fs> end

zonecfg:zone01> commit

zonecfg:zone01> verify

zonecfg:zone01> exit

Adding ZFS

1. Adding a ZFS file system to a non-global zone – Make sure the mount point property of the ZFS file system getting added to a zone is set to legacy, otherwise it may get assigned to multiple non-global zones simultaneously.

global # zonecfg -z zone01

zonecfg:zone01> add fs

zonecfg:zone01:fs> set type=zfs

zonecfg:zone01:fs> set special=rpool/data

zonecfg:zone01:fs> set dir=/data

zonecfg:zone01:fs> end

zonecfg:zone01> verify

zonecfg:zone01> commit

zonecfg:zone01> exit

2. Adding ZFS file system as a loopback file system (lofs) to a non-global zone :

global # zonecfg -z zone01

zonecfg:zone01> add fs

zonecfg:zone01:fs> set special=rpool/data

zonecfg:zone01:fs> set dir=/data

zonecfg:zone01:fs> set type=lofs

zonecfg:zone01:fs> end

zonecfg:zone01> commit

zonecfg:zone01> verify

zonecfg:zone01> exit

global # mkdir -p /zoneroot/zone01/root/data

global # mount -F lofs rpool/data /zoneroot/zone01/root/data

3. Delegating a dataset to a non-global zone – here you have a complete control of the dataset you delegate to the non-global zone. For example you can create your own child datasets under the dataset you delegate and set properties of the delegated dataset etc. The ZFS file system data will be available as a pool in the non-global zone.

global # zonecfg -z zone01

zonecfg:zone01> add dataset

zonecfg:zone01:dataset> set name=rpool/data

zonecfg:zone01:dataset> end

zonecfg:zone01> commit

zonecfg:zone01> verify

zonecfg:zone01> exit

3. Adding ZFS volumes to non-global zones

global # zonecfg -z zone01

zonecfg:zone1> add device

zonecfg:zone1:device> set match=/dev/zvol/dsk/rpool/datavol

zonecfg:zone1:device> end

zonecfg:zone1> verify

zonecfg:zone1> commit

zonecfg:zone1> exit

Adding a CD-ROM to non-global zone

To add a CD-ROM to the non-global zone :

global # zonecfg -z zone01

zonecfg:zone01> add fs

zonecfg:zone01:fs> set dir=/cdrom

zonecfg:zone01:fs> set special=/cdrom

zonecfg:zone01:fs> set type=lofs

zonecfg:zone01:fs> end

zonecfg:zone01> verify

zonecfg:zone01> commit

zonecfg:zone01> exit

 

OEL/Linux: Turning off or disabling SELinux

SELinux is a set of extra security restrictions on top of the normal Linux security tools. It gives the systems administrator a finer grain of control than what the kernel typically provides.

But SELINUX can sometimes get in your way. For example, I have had typical services, such as Apache, appear to start up correctly, but remain inaccessible from the outside world because I forgot to allow the apache user rights to open that port or maybe my distro forgot about it. Before you turn off SELinux make sure you know why you are turning it off and the security concerns you might be opening yourself up to.

 

Test if SELinux is running

You can test to see if SELinux is currently enabled with the following command:

Root/#> selinuxenabled && echo enabled || echo disabled

 

Turning off SELinux temporarily

Disabling SELinux temporarily is the easiest way to determine if the problem you are experiencing is related to your SELinux settings. To turn it off, you will need to become the root users on your system and execute the following command:

Root/#> echo 0 > /selinux/enforce

 

This temporarily turns off SELinux until it is either re-enabled or the system is rebooted. To turn it back on you simply execute this command:

Root/#> echo 1 > /selinux/enforce

 

As you can see from these commands what you are doing is setting the file /selinux/enforce to either '1' or '0' to denote 'true' and 'false'.

 

Configuring SELinux to log warnings instead of block

You can also configure SELinux to give you a warning message instead of actually prohibiting the action. This known as permissive mode. To change SELinux's behavior to permissive mode you need to edit the configuration file. On Fedora and RHEL systems that file is located at /etc/selinux/config. You need to change the SELINUX option to permissive like so:

SELINUX=permissive

 

Note that these changes will not take effect until the system is rebooted, which is why the first section comes in handy on a system you either cannot or do not want to reboot right now.

 

Completely turning off SELinux

To completely disable SELinux instead of setting the configuration file to permissive mode you set it to disabled like:

SELINUX=disabled

 

You will need to reboot your system or temporarily set SELinux to non-enforcing mode to create the desired effect like the example above.

OEL - Firewall not disabled on OEL 6 after running chkconfig iptables off and system reboot

To fix the problem you need to disable libvirtd

# chkconfig libvirtd off

# chkconfig libvirt-guests off

# chkconfig ip6tables off

# chkconfig iptables off

# chkconfig --list | egrep 'iptables|ip6tables|libvirt'

ip6tables 0:off 1:off 2:off 3:off 4:off 5:off 6:off

iptables 0:off 1:off 2:off 3:off 4:off 5:off 6:off

libvirt-guests 0:off 1:off 2:off 3:off 4:off 5:off 6:off

libvirtd 0:off 1:off 2:off 3:off 4:off 5:off 6:off

 

After a reboot the firewall should be disabled now

[root@grac43 ~]# service iptables status

iptables: Firewall is not running.