Page 1 of 1
Potential race condition in Windows create_all_dirs
Posted: October 29th, 2022, 12:06 pm
by ordeneus
Latest Windows version.
Problem exists only when waking from sleep.
I believe it's this part of the code from create_all_dirs:
Code: Select all
if sabnzbd.WIN32:
if not os.path.exists(path):
os.makedirs(path)
I think this is happening before the drive can be mounted (from a Synology DS220+). I don't believe it's a permissions issue, simply a race? It doesn't happen on restart, only on waking from sleep.
Could this resolve it?
Code: Select all
if sabnzbd.WIN32:
if check_mount(path) and not os.path.exists(path):
os.makedirs(path)
It would loop for
wait_ext_drive seconds which would resolve the race?
Re: Potential race condition in Windows create_all_dirs
Posted: October 30th, 2022, 5:21 am
by safihre
Could you provide a bit more context? What error are you getting and where?
You call it a race condition, but the definition of a race condition is that 2 processed try to modify the same thing at the same time. That can't be the case, because the function is locked and can only be active from 1 process.
I think you just mean a timeout problem?
Re: Potential race condition in Windows create_all_dirs
Posted: October 30th, 2022, 8:05 am
by ordeneus
Sorry, it wouldn't let me poste the error because I'm a newb.
When Windows wakes up from sleep with Sabnzbd running as an app I am seeing the following error appear in the console:
2022-10-29 09:05:02,328::INFO::[filesystem:703] Creating directories: n:\Downloads
2022-10-29 09:05:02,344::INFO::[notifier:123] Sending notification: Error - Failed making (n: \ Downloads) (type=error, job_cat=None)
2022-10-29 09:05:02,342::ERROR::[filesystem:725] Failed making (n: \ Downloads)
Traceback (most recent call last):
File sabnzbd \ filesystem . py, line 708, in create_all_dirs
File os . py , line 215, in makedirs
File os . py , line 225, in makedirs
FileNotFoundError: [WinError 53] The network path was not found:
I presumed this was because Windows had not restored the connection to the drive before sab was trying to create a directory on the drive? This only happens on waking from sleep. If I restart sab (with the machine running obviously) the error doesn't occur.
Re: Potential race condition in Windows create_all_dirs
Posted: October 30th, 2022, 11:27 am
by safihre
Yes you are right. Indeed try the wait_ext_drive, that's made for this.
Re: Potential race condition in Windows create_all_dirs
Posted: October 31st, 2022, 3:03 pm
by ordeneus
I have that value set to 10 (default is 5 I believe). It happened again this morning.
The only reference to wait_ext_drive is in check_mount, which is referenced by get_unique_dir which is referenced by nzbstuff . py and postproc . py.
I may be reading this wrong of course, but does check_mount (which would look for wait_ext_drive) get called by create_all_dirs?
Re: Potential race condition in Windows create_all_dirs
Posted: October 31st, 2022, 4:53 pm
by safihre
Yes, that's only for during startup of SABnzbd.
Just to be sure, you are not putting your Incomplete directory on the NAS right? Only the Complete directory?
Re: Potential race condition in Windows create_all_dirs
Posted: October 31st, 2022, 8:08 pm
by ordeneus
Sorry yes, should've been clearer.
Temp download is a local SSD.
Completed directory is the NAS.
Re: Potential race condition in Windows create_all_dirs
Posted: November 1st, 2022, 2:00 am
by safihre
Good that the download is local!
Hmmm, I'm not sure how to solve this then.
It seems indeed the connection just isn't up yet when SABnzbd wants to write to it. But we can't check before every write operation if the disk is there, that would cause too much overhead.
Maybe there's a way to force Windows to reconnect quicker?
Have you tried instead of using N:\ to use the network path? \\NAS\Downloads or something like that?
Re: Potential race condition in Windows create_all_dirs
Posted: November 1st, 2022, 2:24 am
by sander
Or: keep Complete local too, and use a post-processing script to move to your NAS?
Re: Potential race condition in Windows create_all_dirs
Posted: November 1st, 2022, 8:20 am
by ordeneus
The current line from create_all_dirs is:
Code: Select all
if not os.path.exists(path):
os.makedirs(path)
If this were changed to:
Code: Select all
if not os.path.exists(path) and check_mount(path):
os.makedirs(path)
The "and" is a short circuit operator, so "check_mount(path)" would only be evaluated if the first part of the if statement were true? Meaning it would not be evaluated if the path does exist? You would only be adding a check for the disk if the directory doesn't exist (which is already being evaluated).
Re: Potential race condition in Windows create_all_dirs
Posted: November 1st, 2022, 8:25 am
by safihre
Yes, you highlight now the one spot where you experience a problem
But there are multiple spots throughout downloading and post-processing where we write to the Complete Directory, so all those would need to be adapted to use check_mount.
It seems that more people would have this problem, so maybe there's an easier solution? We have not had this reported before on the forums.
Re: Potential race condition in Windows create_all_dirs
Posted: November 1st, 2022, 7:39 pm
by ordeneus
Ha. Always good to find a novel issue
