fbpx

Soft lockup messages from Linux kernel running in an SMP-enabled virtual machine

Details:

When running a Linux kernel in a symmetric multiprocessing (SMP) enabled virtual machine, messages similar to BUG: soft lockup detected on CPU#1! are written to the message log file. The exact format of these messages vary from kernel to kernel, and might be accompanied by a kernel stack backtrace.
Many Linux kernels have a soft lockup watchdog thread, and report soft lockup messages if that watchdog thread does not get scheduled for more than 10 seconds. On a physical host, a soft lockup message generally indicates a kernel bug or hardware bug. When running in a virtual machine, this might instead indicate high levels of overcommitment (especially memory overcommitment) or other virtualization overheads.

Solution:

The soft lockup messages are not kernel panics, and can be safely ignored.Some kernels allow you to adjust the soft lockup threshold by running the command:echo time > /proc/sys/kernel/softlockup_thresh

Where time is the number of seconds after which a soft lockup is reported. The default is generally 10 seconds.

What the Error Looks Like:

abrt_version:   2.0.8

cgroup:

cmdline:        /usr/bin/ksh ./update_archive.ksh RMUAT

executable:     /bin/ksh93

kernel:         2.6.39-400.212.1.el6uek.x86_64

last_occurrence: 1403155682

pid:            25607

pwd:            /oracle/scripts/update_archive

time:           Thu 19 Jun 2014 12:28:02 AM CDT

uid:            502

username:       oracle

 

sosreport.tar.xz: Binary file, 3959288 bytes

 

environ:

:SHELL=/bin/sh

:OLDPWD=/home/oracle

:USER=oracle

:LD_LIBRARY_PATH=/lib:/usr/lib:/oracle/product/11.2.0/dbhome_1/lib

:PATH=/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin

:PWD=/oracle/scripts/update_archive

:JAVA_HOME=/usr/java

:LANG=en_US.UTF-8

:HOME=/home/oracle

:SHLVL=2

:LOGNAME=oracle

:’NLS_DATE_FORMAT=MM-DD-YYYY HH24:MI:SS’

:_=./update_archive.ksh

 

limits:

:Limit                     Soft Limit           Hard Limit           Units

:Max cpu time              unlimited            unlimited            seconds

:Max file size             unlimited            unlimited            bytes

:Max data size             unlimited            unlimited            bytes

:Max stack size            10485760             33554432             bytes

:Max core file size        0                    unlimited            bytes

:Max resident set          unlimited            unlimited            bytes

:Max processes             16384                16384                processes

:Max open files            1024                 65536                files

:Max locked memory         65536                65536                bytes

:Max address space         unlimited            unlimited            bytes

:Max file locks            unlimited            unlimited            locks

:Max pending signals       387052               387052               signals

:Max msgqueue size         819200               819200               bytes

:Max nice priority         0                    0

:Max realtime priority     0                    0

:Max realtime timeout      unlimited            unlimited            us

 

maps:

:00400000-0055b000 r-xp 00000000 fc:00 261718                             /bin/ksh93

:0075a000-0076d000 rw-p 0015a000 fc:00 261718                             /bin/ksh93

:0076d000-00773000 rw-p 00000000 00:00 0

:0096c000-0096e000 rw-p 0016c000 fc:00 261718                             /bin/ksh93

:3e8dc00000-3e8dc20000 r-xp 00000000 fc:00 1177356                        /lib64/ld-2.12.so

:3e8de1f000-3e8de20000 r–p 0001f000 fc:00 1177356                        /lib64/ld-2.12.so

:3e8de20000-3e8de21000 rw-p 00020000 fc:00 1177356                        /lib64/ld-2.12.so

:3e8de21000-3e8de22000 rw-p 00000000 00:00 0

:3e8e000000-3e8e002000 r-xp 00000000 fc:00 1177390                        /lib64/libdl-2.12.so

:3e8e002000-3e8e202000 —p 00002000 fc:00 1177390                        /lib64/libdl-2.12.so

:3e8e202000-3e8e203000 r–p 00002000 fc:00 1177390                        /lib64/libdl-2.12.so

:3e8e203000-3e8e204000 rw-p 00003000 fc:00 1177390                        /lib64/libdl-2.12.so

:3e8e400000-3e8e58b000 r-xp 00000000 fc:00 1177360                        /lib64/libc-2.12.so

:3e8e58b000-3e8e78a000 —p 0018b000 fc:00 1177360                        /lib64/libc-2.12.so

:3e8e78a000-3e8e78e000 r–p 0018a000 fc:00 1177360                        /lib64/libc-2.12.so

:3e8e78e000-3e8e78f000 rw-p 0018e000 fc:00 1177360                        /lib64/libc-2.12.so

:3e8e78f000-3e8e794000 rw-p 00000000 00:00 0

:3e8f000000-3e8f083000 r-xp 00000000 fc:00 1177398                        /lib64/libm-2.12.so

:3e8f083000-3e8f282000 —p 00083000 fc:00 1177398                        /lib64/libm-2.12.so

:3e8f282000-3e8f283000 r–p 00082000 fc:00 1177398                        /lib64/libm-2.12.so

:3e8f283000-3e8f284000 rw-p 00083000 fc:00 1177398                        /lib64/libm-2.12.so

:3e98400000-3e98402000 r-xp 00000000 fc:00 1177423                        /lib64/libutil-2.12.so

:3e98402000-3e98601000 —p 00002000 fc:00 1177423                        /lib64/libutil-2.12.so

:3e98601000-3e98602000 r–p 00001000 fc:00 1177423                        /lib64/libutil-2.12.so

:3e98602000-3e98603000 rw-p 00002000 fc:00 1177423                        /lib64/libutil-2.12.so

:7fea21b5d000-7fea21bcd000 rw-p 00000000 00:00 0

:7fea21bcd000-7fea27a5e000 r–p 00000000 fc:00 1441433                    /usr/lib/locale/locale-archive

:7fea27a5e000-7fea27aaa000 rw-p 00000000 00:00 0

:7fea27ab8000-7fea27ab9000 rw-p 00000000 00:00 0

:7fffdfc56000-7fffdfc77000 rw-p 00000000 00:00 0                          [stack]

:7fffdfde7000-7fffdfde8000 r-xp 00000000 00:00 0                          [vdso]

:ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

 

open_fds:

:0:pipe:[122311558]

:pos: 0

:flags:     00

:1:/oracle/log/RMUAT_update_archive_140619002801.log

:pos: 199

:flags:     0100001

:2:/oracle/log/RMUAT_update_archive_140619002801.log

:pos: 199

:flags:     0100001

Leave a Reply

Your email address will not be published.