Slow unpacking after upgrading to OS X 10.9

[shypike]

I am also having this issue and it's a huge pain. Is there a way to direct sabnzb to unrar to local drive and then copy to nas?

Yes, by writing a user script and associating that with the categories.

[powertowerpro]

Has anyone written a user script that I could borrow? This is killing me...

Many thanks in advance!

[pharao]

Finally I found a solution!!!

The original version of unrar on SABnzbd is 4.20.

First I tried to upgrade to version 5 with no success.
So I tried to downgrade to version 4.11 and BINGO!

I really don't know the difference between these versions.
The problem for sure is a change apple did on SMB that made this mess...

But who cares? It's working!

[shypike]

4.11 will work alright, until you hit a download that has been created with unrar 5.
The differences between 4.x versions aren't very important.

[pharao]

Has the 4.20 version any issue with rar files created under version 5?

[powertowerpro]

Has anyone checked to see if 10.9.2 fixed the issue? Supposed to be major SMB improvements. Also how does one downgrade or upgrade the unrar version in SABnzzbd?

Thanks

[shypike]

Also how does one downgrade or upgrade the unrar version in SABnzzbd?

Look in /Applications/SABnzbd.app/Content (don't know the exact path by hearth) and replace the file unrar.
This will invalidate the signature of the app though.

[powertowerpro]

For those wondering:
- 10.9.2 does not fix the slow unpacking to a network drive despite SMB improvements by Apple.
- Synology DSM 4.3-3827 which also specifies improved Max OS X SMB performance does not help either
- To upgrade or downgrade Unrar locate the file in the sabnzbd.app package: \Applications\SABnzbd.app\Contents\Resources\osx\unrar\unrar
- Download the version you want from: http://rarlab.com/download.htm

Many of the NZB's I am getting these days are using RAR v5 so I upgraded to RAR 5.01. Previously I was getting "» Unusable RAR file" errors. Good luck.

[mlogen]

Hello

I don't see any updates on this for a long time. Did a fix ever happen? I have tried all the workarounds and unpacking is still very slow compared to previous OSX builds.

Thanks

[shypike]

The problem is an issue between OSX and unrar.
Outside our scope.
Next release will probably contain unrar 5.10.
That's all we can do.

[redcell]

I also had alot of problems with unrar and copying files to a Nas on OSX...
After doing alot of research it comes down to 2 issues:
1. There is some bug in unrar or the new smb implementation in OSX since mavericks/yosemite wich affects unrar over a network (possibly also to a different filesystem)
2. There is a bug in Python in OSX when you use the "shutil.copyfile" function in code wich copies data in OSX from a HFS+ partition/drive to another filesystem or Nas, file copying goes extremely slow because of this bug, what normally takes minutes in Finder, now takes 2 to 4 hours depending on the size of a file

To bypass the unrarring problem, unrar being extremely slow, you need to unrar a file on your OSX partition and not on a Nas, and then after unrarring it needs to be copied to a Nas, unrar on a Nas takes ages.
But then a second problem occurs, the copying is done from python with the "shutil.copyfile" function, because of the bug in this function (wich as far as I know only affects OSX), the copying again takes hours instead of minutes, so if you solve/bypass the first problem, a second problem occurs lol. so because the second problem only occurs after unrarring, you wouldn't notice an improvement, the second problem can be solved like this:

You can solve this in python code, by replacing:
for example in sickbeard, when you run the sabtosickbeard script after downloading a series

When you download it directly to your Nas, the unrarring on the Nas goes slow, when you bypass this and let it unrar on your OSX drive, unrar works much faster, but then the second problem arrises, that "shutil.copyfile" in python in OSX has a bug, so then after the faster unrar the copying to the Nas goes really slow, so basicly nothing improves lol... so you wouldn't notice that anything changed.

You can solve this in sickbeard in helpers.py, wich is executed by the sabtosickbeard script, replace in the code: https://github.com/midgetspy/Sick-Beard ... helpers.py

@ line 394:

Code: Select all

ek.ek(shutil.copyfile, srcFile, destFile)

with:

Code: Select all

if sys.platform.startswith("darwin"):
    os.system('cp "%s" "%s"' % (srcFile, destFile))
else:
    ek.ek(shutil.copyfile, srcFile, destFile)

and in the beginning of helpers.py @ line 34 include:

Code: Select all

import os

this is for executing a terminal command

I also see in the sabnzbd code "shutil.copyfile" being used a couple of times: https://github.com/sabnzbd/sabnzbd/sear ... 3&q=shutil

If you use the above code, the system terminal command "cp" is used to copy a file instead of the buggy "shutil.copyfile", when you normally copy a file by dragging it in Finder to your Nas copying goes a lot faster, same with using "cp" instead of "shutil.copyfile' in python.

I have not yet studied the sabnzb code enough to see what the files config.py misc.py and newsunpack.py exactly do, but I suggest changing "shutil.copyfile" in the code to the same as the changes made in the sickbeards helpers.py
wich basicly changes the use of shutil.copyfile only when the os is OSX to use the terminal command "cp" instead, this code change only affects OSX, it will run the same with this code change as before on windows and linux wich don't seem to be affected by this bug.

So for example in misc.py

Code: Select all

shutil.copyfile(path, new_path)

It would become:

Code: Select all

if sys.platform.startswith("darwin"):
    os.system('cp "%s" "%s"' % (path, new_path))
else:
   shutil.copyfile(path, new_path)

Or something simular, but this needs to be tested! but I don't know if any of these 3 files are also used in sabnzbd to copy files to a Nas if sabnzbd is configured as such?

For to not unrar on a Nas you need to configure in sickbeard that the config => postprocessing => TV Download Dir is not on the Nas.
And also in sabnzbd settings => categories => the tv category source also must not be on the Nas, and the post script must be sabtosickbeard, so that after unrarring the file is copied to the Nas.
This way unrarring takes place on your OSX partition/drive, instead of on a Nas, bypassing the bug in unrar or the smb changes in mavericks/yosemite, so unrarring doesn't take ages.
After downloading the sabtosickbeard script will then copy the file to the Nas after unrarring, with the changed helpers.py in sickbeard this solves the slow unarring and filecopying to a Nas

[shypike]

Nice analysis. I'll see what I can do.
What ever happened to "it just works"?

[redcell]

lol, yeah "it just works" if you only stick to some basic usage of OSX, try activating apache with php in OSX, only to discover that alot of functionality is crippled and you need to compile missing php functions yourself...

[kjuojhuoh]

I had this problem and switched my unpacking destination to a physical drive rather than the NAS. But then when the file was moved to the NAS it was dreadfully slow. E.g. 1gb in 2hrs. I installed SMBup which replaces the dodgy smb implementation (I have 10.10.2) with a new one. Then I found my file transfers and unpacking to the NAS were much quicker. This might be a solution for some still experiencing this issue. Link: eduo.info/apps/smbup

[Cpuroast]

Did it occur to anyone using a NAS that bypassing this whole issue by running sab directly on the NAS might be the better way to fix this?

That way there's no network transfer during extraction or copy, it's all local to sab running on the NAS.