[RC2] Downloads slow

[Duumke]

Hi,

I'm wondering if it's just me (or one of my settings), but my downloads seem awfully slow.
I'm running SABnzbd RC2 on a Ubuntu 10.10 headless server (virtual machine in ESXi 4.1).
Configured resources:
2x vCPU @ 2.1GHz
1024mb memory.
250GB hdd.

SABnzbd config:

Code: Select all

__version__ = 19
[misc]
queue_complete = ""
https_port = 9090
folder_rename = 1
password_file = ""
rss_rate = 60
https_key = server.key
cleanup_list = ,
cache_limit = 150
ignore_unrar_dates = 0
par_option = ""
web_color = gold
replace_spaces = 0
web_color2 = ""
dirscan_speed = 60
enable_filejoin = 1
create_group_folders = 0
api_key = ***********************
never_repair = 0
no_penalties = 0
bandwidth_limit = 0
dirscan_dir = /public/Downloads/Import
disable_api_key = 0
email_to = ,
password = ""
permissions = 0775
auto_disconnect = 1
enable_date_sorting = 0
replace_dots = 0
movie_sort_extra = -cd%1
ionice = ""
enable_unzip = 1
download_free = 10
port = 8080
email_dir = ""
pre_script = None
email_full = 0
log_dir = /downloads/sabnzbd/logs
admin_dir = /downloads/sabnzbd/admin
nzb_backup_dir = /downloads/sabnzbd/nzb-backup
tv_sort_countries = 1
date_categories = tv,
ssl_type = v23
sfv_check = 1
date_sort_string = ""
schedlines = ,
host = 0.0.0.0
ignore_samples = 0
enable_tsjoin = 1
auto_browser = 0
pause_on_post_processing = 1
enable_unrar = 1
email_rss = 0
email_account = ""
email_server = ""
allow_streaming = 0
config_lock = 0
api_warnings = 1
tv_sort_string = ""
username = ""
unpack_check = 0
download_dir = /downloads/sabnzbd/incomplete
size_limit = ""
enable_par_cleanup = 1
safe_postproc = 1
cache_dir = cache
complete_dir = /downloads/sabnzbd/complete
replace_illegal = 1
par2_multicore = 0
warned_old_queue = 0
language = nl
check_new_rel = 1
enable_https = 0
no_dupes = 0
top_only = 1
quick_check = 1
movie_sort_string = ""
email_from = ""
movie_extra_folder = 0
email_endjob = 0
tv_categories = ,
pause_on_pwrar = 1
login_realm = SABnzbd
send_group = 0
refresh_rate = 0
movie_categories = movies,
script_dir = /downloads/sabnzbd/scripts
email_pwd = ""
ignore_wrong_unrar = 0
nice = ""
enable_movie_sorting = 0
https_cert = server.cert
web_dir2 = ""
fail_on_crc = 0
nzb_key = ************************
auto_sort = 0
ampm = 0
web_dir = Plush
queue_complete_pers = 0
enable_tv_sorting = 0
folder_max_length = 256
[logging]
log_new = 0
max_log_size = 5242880
log_level = 1
log_backups = 5
enable_cherrypy_logging = 0

My connection is a 120MBit connection.
At the start (first couple of hundred mb's) I'm downloading at full speed.
After that the download drops to about a tenth of the speed of my connection.
I'm also seeing the Sysload go up to 4.xx/5.xx.
Any ideas why it's hogging the CPU?
I have a feeling that's the main reason it's that slow.

Thanx for all imput.

[Duumke]

I've tried that, but it doesn't seem to make any difference.
Sysload is still high (about 2.9+)

[Tarzan]

I found the same problem.
The CPU uses was very high and the download got slow.
I paused the download for a while and let the downloaded parts in the admin folder complete.
After this was done I unpaused the download and it was at full speed again.
Also the processing of the downloaded parts was fast again.

So if the que of downloaded parts whiche are not procesed to from the ADMIN folder get to long the complete performance gets very slow.

[shypike]

Which admin folder?
The central one or the __admin__ subfolder of the job folder in the "temporary download folder"?
Any specific scenario that leads to this issue?

[Tarzan]

The _ADMIN_ folders in the "temporary download folder"
The processing can lag behind if during the download you are busy with some other processes on your computer, for instance unrarring manual of  the last download.
If the files can not be processed from this _ADMIN_ in the temporary folder to the temporary folder and you have several already downloaded but not processed and copied to the temporary folder, the CPU load goes up and the disk access goes down.
If i see this I pause the download for a while till all downloaded files are processed and restart the download.
After thiss every thing goes smooth again.

[shypike]

unrarring competes for both CPU and disk channel with the downloader.
If you have no article cache (Config->General) the competition for the disk channel will be even worse.
It's not very surprising that there's a dip in download speed and a CPU peak.
You can choose to postpone downloading until post-processing is complete.