syslog loaded with these entries - performance drops

[dchester11]

Hi all -

I have been seeing some terrible performance with my sabnzbd setp and the box its on. I dug into some log sand my syslog is littered with these entries :

Code: Select all

Feb 16 04:50:32 BAILEYNAS kernel: WARNING: at net/ipv4/tcp.c:1483 tcp_recvmsg+0x88e/0xb50()
Feb 16 04:50:32 BAILEYNAS kernel: Hardware name: ReadyNAS
Feb 16 04:50:32 BAILEYNAS kernel: recvmsg bug: copied 76C6ED33 seq 76C6F2DA rcvnxt 76C7F31A fl 0
Feb 16 04:50:32 BAILEYNAS kernel: Modules linked in: pvgpio nv6vpd(P)
Feb 16 04:50:32 BAILEYNAS kernel: Pid: 2934, comm: sabnzbdplus Tainted: P      D W   2.6.37.6.RNx86_64.2.4 #1
Feb 16 04:50:32 BAILEYNAS kernel: Call Trace:
Feb 16 04:50:32 BAILEYNAS kernel:  [<ffffffff884fb66e>] ? tcp_recvmsg+0x88e/0xb50
Feb 16 04:50:32 BAILEYNAS kernel:  [<ffffffff8803432b>] warn_slowpath_common+0x8b/0xc0
Feb 16 04:50:32 BAILEYNAS kernel:  [<ffffffff880bddd0>] ? pollwake+0x0/0x70
Feb 16 04:50:32 BAILEYNAS kernel:  [<ffffffff88034459>] warn_slowpath_fmt+0x69/0x70
Feb 16 04:50:32 BAILEYNAS kernel:  [<ffffffff88505531>] ? tcp_event_new_data_sent+0x91/0xd0
Feb 16 04:50:32 BAILEYNAS kernel:  [<ffffffff885b4a51>] ? _raw_spin_lock_bh+0x11/0x30
Feb 16 04:50:32 BAILEYNAS kernel:  [<ffffffff884bb763>] ? lock_sock_nested+0x33/0x40
Feb 16 04:50:32 BAILEYNAS kernel:  [<ffffffff884fb66e>] tcp_recvmsg+0x88e/0xb50
Feb 16 04:50:32 BAILEYNAS kernel:  [<ffffffff88519003>] inet_recvmsg+0x73/0x90
Feb 16 04:50:32 BAILEYNAS kernel:  [<ffffffff884b7097>] sock_aio_read+0x147/0x150
Feb 16 04:50:33 BAILEYNAS kernel:  [<ffffffff880ae261>] do_sync_read+0xd1/0x110
Feb 16 04:50:33 BAILEYNAS kernel:  [<ffffffff880aa00b>] ? __slab_free+0x11b/0x120
Feb 16 04:50:33 BAILEYNAS kernel:  [<ffffffff880390d4>] ? timespec_add_safe+0x34/0x70
Feb 16 04:50:33 BAILEYNAS kernel:  [<ffffffff880aeafa>] vfs_read+0x15a/0x170
Feb 16 04:50:33 BAILEYNAS kernel:  [<ffffffff880aeef0>] sys_read+0x50/0x90
Feb 16 04:50:33 BAILEYNAS kernel:  [<ffffffff88024c53>] ia32_sysret+0x0/0x5
Feb 16 04:50:33 BAILEYNAS kernel: ---[ end trace 90f90117e03e356f ]—

They are logged every second! Any idea what this means?

Thanks so much

[sander]

I know nothing about tainted kernels, but here are some remarks:

SABnzbd is a python script, so SAB itself cannot produce such kernel problems. The problem is in the kernel AFAIK.

Are there "Tainted" lines without 'sabnzbdplus'? If so, it's more proof it's not SAB.
If not, I would stop SABnzbd, and start doing a lof of network traffic, and see if you get such a message.

Hopefully your post http://www.readynas.com/forum/viewtopic.php?f=7&t=75319 gives more info.

I would Google

tainted "kernel: WARNING" site:readynas.com

as the problem seems to happen to more ReadyNAS users.

What is the output of

Code: Select all

cat /proc/sys/kernel/tainted

compare against:

Code: Select all

tainted: 

Non-zero if the kernel has been tainted.  Numeric values, which
can be ORed together:

   1 - A module with a non-GPL license has been loaded, this
       includes modules with no license.
       Set by modutils >= 2.4.9 and module-init-tools.
   2 - A module was force loaded by insmod -f.
       Set by modutils >= 2.4.9 and module-init-tools.
   4 - Unsafe SMP processors: SMP with CPUs not designed for SMP.
   8 - A module was forcibly unloaded from the system by rmmod -f.
  16 - A hardware machine check error occurred on the system.
  32 - A bad page was discovered on the system.
  64 - The user has asked that the system be marked "tainted".  This
       could be because they are running software that directly modifies
       the hardware, or for other reasons.
 128 - The system has died.
 256 - The ACPI DSDT has been overridden with one supplied by the user
        instead of using the one provided by the hardware.
 512 - A kernel warning has occurred.
1024 - A module from drivers/staging was loaded.
2048 - The system is working around a severe firmware bug.
4096 - An out-of-tree module has been loaded.

and

Code: Select all

/**
 *  print_tainted - return a string to represent the kernel taint state.
 *
 *  'P' - Proprietary module has been loaded.
 *  'F' - Module has been forcibly loaded.
 *  'S' - SMP with CPUs not designed for SMP.
 *  'R' - User forced a module unload.
 *  'M' - System experienced a machine check exception.
 *  'B' - System has hit bad_page.
 *  'U' - Userspace-defined naughtiness.
 *  'D' - Kernel has oopsed before
 *  'A' - ACPI table overridden.
 *  'W' - Taint on warning.
 *  'C' - modules from drivers/staging are loaded.
 *  'I' - Working around severe firmware bug.
 *  'O' - Out-of-tree module has been loaded.
 *
 *  The string is overwritten by the next call to print_tainted().
 */

HTH

[dchester11]

Thanks for the reply. I got a 1 from /proc/sys/kernel/tainted which doesn't seem like a problem. I don't see any of these that don't reference SABnzbd.

Stopped SABnzbd and everything else. Updated RAIDiator from 4.2.25 to 4.2.26.

Just waiting now...