Intermittent error after Quick Check

[cj_]

Hey,

I am running the 0.5-beta1 release and have experienced this problem on both FreeBSD 7.2 stable and Ubuntu Jaunty using Python 2.5.

Occasionally I have a download that has finished Quick Check, but it gets caught in a loop that consumes 100% CPU and never completes.  I am only ever to fix this by shutting down and deleting all the pickle files.  Simply restarting the daemon, it will continue where it left off.  It says that Quick Check completed OK, but it never goes any farther.

I truss'ed the process and it is caught in this loop:

Code: Select all

 52044 python   CALL  getpid
 52044 python   RET   getpid 52044/0xcb4c
 52044 python   CALL  select(0,0,0,0,0xbedf16f0)
 52044 python   RET   _umtx_op 0
 52044 python   CALL  gettimeofday(0xbecf06f8,0)
 52044 python   RET   gettimeofday 0
 52044 python   CALL  read(0xb,0x2acb3000,0x5)
 52044 python   RET   read -1 errno 35 Resource temporarily unavailable

According to lsof, there is no fd 11:

Code: Select all

python  52044 cjones   10u  IPv4 0xc3cf53a0      0t0     TCP xxx.xxx.xxx.xxx:5055 (LISTEN)
python  52044 cjones   12u  IPv4 0xc48023a0      0t0     TCP xxx.xxx.xxx.xxx:50387->unknown.sj.astraweb.com:nntp (CLOSED)

The sabnzbd.log does not show anything of interest, but was not set to debug the last time it happened.  The rest of the UI is responsive, so it seems a single thread is looping.  Deadlock/race condition perhaps?

Any ideas?

[shypike]

I suspect it's a platform problem.
It looks very much like a hangup in the runtime library.
Either in freeBSD or the Python port.
We get similar complaints about freeNAS (also based on freeBSD).

I have never seen it in Ubuntu.
You could try Python 2.6 instead (standard for Ubuntu),
as far as I know, SABnzbd 0.5.0 is compatible with it.

Does it also happen when you disable the QuickCheck option (Config->Switches)
and let the par2 utility do the work?

[cj_]

Well for what it's worth, this only happens with 0.5, and I encounter it on both Linux and FreeBSD.

I restarted the server with debug set, and this is the last thing it shows before going into loop again:

Code: Select all

2009-12-01 03:03:37,121::DEBUG::[__init__:1327] Starting postprocessor
2009-12-01 03:03:37,182::INFO::[__init__:1319] Starting PostProcessing on Fringe_-_1x01_-_Pilot => Repair:False, Unpack:False, Delete:True, Script:cleanup.py
2009-12-01 03:03:37,186::DEBUG::[__init__:1327] Found TV Show - Starting folder sort (Fringe_-_1x01_-_Pilot)

When I kill the process and restart, it logs that again, so I don't think it's making it past the sorting..  So I disabled TV sorting and restarted again, it moved on fine.. I'm not sure this is really platform specific but rather NZB specific.  I copied the same NZB to another box running the beta and it also hung.   My TV sort string is set to:  {%s.n}.s%0s.e%0e.%e_n.%ext

[cj_]

Hello again, I figured it out.  The method to lower-case using {} is just broken.  The code from sabnzbd/tvsort.py:

Code: Select all

def toLowercase(path):
    ''' Lowercases any characters enclosed in {} '''
    RE_LOWERCASE = re.compile('\{([^\{]*)\}', re.I)
    while 1:
        m = RE_LOWERCASE.match(path)
        if not m:
            break
        section = path[m.start(1):m.end(1)].lower()
        folders = path[:m.start()] + section + path[m.end():]

    # just incase
    path = path.replace('{', '')
    path = path.replace('}', '')
    return path

That while loop will never break since path is never modified.  I believe that last line should say "path =" instead of "folders ="

There is another problem with this function: Due to the use of .match() instead of .search(), it will only work once in the filename, and only if it starts at the beginning.  Here is a replacement that works if you are so inclined:

Code: Select all

def toLowercase(path):
    ''' Lowercases any characters enclosed in {} '''
    RE_LOWERCASE = re.compile('{([^{]*)}')
    while True:
        m = RE_LOWERCASE.search(path)
        if not m:
            break
        path = path[:m.start()] + m.group(1).lower() + path[m.end():]

    # just incase
    path = path.replace('{', '')
    path = path.replace('}', '')
    return path

Cheers,

[shypike]

Wow, thanks for this insight!
I'll notify the author about this problem so that it can be fixed asap.

It's quite rare that we get ready made solutions.
And I apologize for more or less denying that there is a problem.

[cj_]

No problem, glad I could help.  Very pleased with the beta so far, this is the only issue I've encountered.