Potential race condition in Windows create_all_dirs

[ordeneus]

Latest Windows version.
Problem exists only when waking from sleep.

I believe it's this part of the code from create_all_dirs:

Code: Select all

        if sabnzbd.WIN32:
            if not os.path.exists(path):
                os.makedirs(path)

I think this is happening before the drive can be mounted (from a Synology DS220+). I don't believe it's a permissions issue, simply a race? It doesn't happen on restart, only on waking from sleep.

Could this resolve it?

Code: Select all

        if sabnzbd.WIN32:
            if check_mount(path) and not os.path.exists(path):
                os.makedirs(path)

It would loop for wait_ext_drive seconds which would resolve the race?

[safihre]

Could you provide a bit more context? What error are you getting and where?

You call it a race condition, but the definition of a race condition is that 2 processed try to modify the same thing at the same time. That can't be the case, because the function is locked and can only be active from 1 process.
I think you just mean a timeout problem?

[ordeneus]

Sorry, it wouldn't let me poste the error because I'm a newb.

When Windows wakes up from sleep with Sabnzbd running as an app I am seeing the following error appear in the console:

2022-10-29 09:05:02,328::INFO::[filesystem:703] Creating directories: n:\Downloads
2022-10-29 09:05:02,344::INFO::[notifier:123] Sending notification: Error - Failed making (n: \ Downloads) (type=error, job_cat=None)
2022-10-29 09:05:02,342::ERROR::[filesystem:725] Failed making (n: \ Downloads)
Traceback (most recent call last):
File sabnzbd \ filesystem . py, line 708, in create_all_dirs
File os . py , line 215, in makedirs
File os . py , line 225, in makedirs
FileNotFoundError: [WinError 53] The network path was not found:

I presumed this was because Windows had not restored the connection to the drive before sab was trying to create a directory on the drive? This only happens on waking from sleep. If I restart sab (with the machine running obviously) the error doesn't occur.

[safihre]

Yes you are right. Indeed try the wait_ext_drive, that's made for this.

[ordeneus]

I have that value set to 10 (default is 5 I believe). It happened again this morning.

The only reference to wait_ext_drive is in check_mount, which is referenced by get_unique_dir which is referenced by nzbstuff . py and postproc . py.

I may be reading this wrong of course, but does check_mount (which would look for wait_ext_drive) get called by create_all_dirs?

[safihre]

Yes, that's only for during startup of SABnzbd.
Just to be sure, you are not putting your Incomplete directory on the NAS right? Only the Complete directory?

[ordeneus]

Sorry yes, should've been clearer.

Temp download is a local SSD.
Completed directory is the NAS.

[safihre]

Good that the download is local!
Hmmm, I'm not sure how to solve this then.
It seems indeed the connection just isn't up yet when SABnzbd wants to write to it. But we can't check before every write operation if the disk is there, that would cause too much overhead.
Maybe there's a way to force Windows to reconnect quicker?
Have you tried instead of using N:\ to use the network path? \\NAS\Downloads or something like that?

[sander]

Or: keep Complete local too, and use a post-processing script to move to your NAS?

[ordeneus]

The current line from create_all_dirs is:

Code: Select all

if not os.path.exists(path):
                os.makedirs(path)

If this were changed to:

Code: Select all

if not os.path.exists(path) and check_mount(path):
                os.makedirs(path)

The "and" is a short circuit operator, so "check_mount(path)" would only be evaluated if the first part of the if statement were true? Meaning it would not be evaluated if the path does exist? You would only be adding a check for the disk if the directory doesn't exist (which is already being evaluated).

[safihre]

Yes, you highlight now the one spot where you experience a problem
But there are multiple spots throughout downloading and post-processing where we write to the Complete Directory, so all those would need to be adapted to use check_mount.

It seems that more people would have this problem, so maybe there's an easier solution? We have not had this reported before on the forums.

[ordeneus]

Ha. Always good to find a novel issue