new in .60, changes made to way files are written

[nzbdime]

First, many thanks to the sabnzbdplus team for all their hard work. This question is for the devs.

Until now, I've been using jcfp's repo. Recently, I saw it had .60 available so I updated. First to .60beta4 and then to .60rc1.

I had a couple of issues with the upgrade and finally, a big change with a main function was made, so I wanted to confirm that these were intended.

From sitting in the IRC room, there seems to be others talking about speed display not being properly shown. That may be true. The display may be a bit weird at the moment, but I do also notice a real speed decrease and I believe I've found out why.

This may be very specific to me. I use sabnzbd to write to a sshfs mounted filesystem. In .5x, this worked really well, as it would save the files to the incomplete folder and then just move them to the complete folder. Moving a file on the same sshfs partition mounted over the network worked very much like renaming a file on a local partition. And it is for this reason, I never used post processing and only set "download" as the default post processing.

Since .60, it seems that this has changed. Articles are written to a __ADMIN__ directory and then later appended into the actual files. This changes a great deal for me as my sabnzbd now has quadruple the transfer cost as opposed to double. In .5x, it would download from the gateway and then write to the fileserver. In .6x, it would be downloading from the gateway, writing to the fileserver and everytime a file was finished downloading, it would also have to read and write to it as well. This spiked my cpu usage on the sabnzbd system and extra usage on the network from reading/writing to the filesystem. From one of those 2, I noticed a regular pattern of my active downloads being slowed.

This also pointed out another small bug, that when donwloads are in that stage of being writting from the articles to their final filenames, the download would not appear in either the queue or the history. During big downloads, this actually took too long do and one could (from not seeing their download in neither queue or history) go to the status page and see it in the "orphan" status and have the option to redownload or requeue. Either choice here is a bad one and generally leads to a headache of either starting over or manually moving stuff around. After I found out about this, I just didn't do anything and my downloads completed eventually. There were times when it took over 15 minutes (I suspect because the option to enable sfv-based checks was clicked. I've since unclicked almost every option in Switches.) and in that time, about 6 more downloads had finished and thus seem to have "disappeared". They did eventually return as I guessed the reason why and patiently waited.

So 2 questions:
Is there a reason why jcfp repo gives out betas/rcs instead of stable?

I saw something on the .60 release page about changes made so that post-processing could be better controlled or paused by the user. So it looks like if this is an intended change, I'll be reverting to .56. Any chance if this may be changed to better accommodate network users?

Thanks for your time and help.

[shypike]

The 0.5.6 suffered from a very leaky cache folder and a general lack of recovery options for any problem.
All admin of a job is now kept in the __admin__ sub folder, which removes the leakage
and gives much better options for recovery due to problems caused by corruption and failed post-processing.
Our advise is to have your "incomplete" stuff on a fast local disk.
Much of the extra disk access can be avoided by setting an article cache in Config->General.
If you make the limit larger than 120% of the largest RAR segment file, everything can be done from memory.
If you're not strapped for memory, you should be using this.

Yes, there are some display problems. Generally we put emphasis on automation and not
on supporting people that like to watch download progress
If jobs are seen as orphans during the transfer from download to history queue, than that *is* a problem
and I'll check that.
SFV checks are only done when there are no par2 file available.

The queue changes will not be reverted, since it's a better option for most users
and the downsides can be avoided by other means.

jcfp's repo contains releases that are not yet in the standard repos.
There is no stable 0.6.0 yet and 0.5.6 should have made it into the standard ones.