losing files when moving to full disk (0.6.10)

Report & discuss bugs found in SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Post Reply
Yarbles
Newbie
Newbie
Posts: 36
Joined: January 25th, 2008, 6:51 am

losing files when moving to full disk (0.6.10)

Post by Yarbles »

Version: 0.6.10
OS: Win7 64
Install-type: Windows Installer
Is the issue reproducible? Yes
Disk setup: C: SABnzbd install, E: Complete, H: Incomplete

Problem: When Complete disk becomes full and unpack fails, SAB will move what files it can from the Incomplete folder to Complete and then delete the remaining files.

In previous releases if the complete folder became full then UNPACK would fail and SAB would start moving the .rar files over. That would eventually fill the disk also so I would be left with some of the files in the Complete folder and the rest in the Incomplete folder. That would allow me to free up space on the Complete disk the move the files into one folder and unpack them manually.

I plan on moving to 0.6.14 soon. Changelogs didn't seem to indicate it was fixed.

I've had this happen at least twice. Once was a "TV" download with a postscript, the other was a non-catagorized DL, no postscript. All were set to +Delete processing.

Log:

2011-12-18 11:27:25,714::WARNING::[newsunpack:582] ERROR: write error ( in the file E:\Newsbin\_UNPACK_Title/Filename.iso [R]etry, [A]bort)
2011-12-18 11:29:06,635::ERROR::[misc:748] Failed moving H:\temp\nzb_incomplete\Title\Filename.r65 to E:\Newsbin\_UNPACK_Title\Filename.r65
....


Settings INI:

Code: Select all

__version__ = 19

[misc]
    ## host we should listen on, leave "" for localhost
    host = 127.0.0.1
    
    ## port we should listen on
    port = 8080
    
    ## username for the web-interface
    username = ""
    
    ## password for the web-interface
    password = ""
    
    ## web file dir for (custom) Cheetah templates and the default.css file
    web_dir = Plush
    
    ## dir to put downloads to, won't be created automatically
    download_dir = H:\temp\nzb_incomplete
    
    ## dir to put completed downloads to, won't be created automatically
    complete_dir = E:\Newsbin\
    
    ## If specified, .nzbs fetched by postid (or added by the dirscanner) will 
    ## be backed up to this readable/writeable dir
    nzb_backup_dir = E:\Newsbin\NZB_backups
    
    ## dir to store cache and cookie files
    cache_dir = ./cache
    
    ## location of your log directory, "" to disable logging
    log_dir = ./logs
    
    ## dirscan directory
    ## SABnzbd will consume everything in that dir
    ## while trying to add it to the queue
    dirscan_dir = E:\Newsbin\NZB_blackhole
    
    ## Scheduling options
    ## Syntax: minute hour day action
    ##         1st argument must be 0-59 (minute)
    ##         2nd argument must be 0-24 (hour)
    ##         3rd argument must be 1-7 (day) or *
    ##         4th argument must be resume or pause
    ##
    ## e.g schedlines = 0 7 * pause, 0 21 * resume 
    ##     to pause SABnzbd at 7:00 and resume operation at 21:00
    schedlines = ,
    
    ## default options for dirscan added items
    ## 0 = None
    ## 1 = +Repair
    ## 2 = +Unpack
    ## 3 = +Delete
    dirscan_opts = 3
    
    ## Enable/disable filejoining
    ## 0 = Disabled
    ## 1 = Enabled
    enable_filejoin = 1
    
    ## Enable/disable unraring
    ## 0 = Disabled
    ## 1 = Enabled
    enable_unrar = 1
    
    ## Enable/disable unzipping
    ## 0 = Disabled
    ## 1 = Enabled
    enable_unzip = 1
    
    ## Enable/disable periodic queue saving
    ## 0 = Disabled
    ## 1 = Enabled
    ## Enable this on unstable systems
    enable_save = 1
    
    ## Enable/disable 
    ## 0 = Disabled
    ## 1 = Enabled
    ## Enable to cleanup par2 files 
    ## (only if verifiying/repairing succeded)
    enable_par_cleanup = 1
    
    ## should we failover on yenc crc errors
    ## 0 = no
    ## 1 = yes
    fail_on_crc = 0
    
    ## should we download to group folders? 
    ## (i.e /my/download/dir/alt.bin.whatever/somepost/)
    ## 0 = no
    ## 1 = yes
    create_group_folders = 0
    
    ## Cleanup List
    ## List of file_extensions that should be deleted
    ## Example: ".nfo," or ".nfo, .sfv"
    cleanup_list = .1, .sfv, .nzb, .srr
    
    ## Fetch old files first
    ## EXPERIMENTAL
    ## 0 = Disabled
    ## 1 = Enabled
    f_mode = 0
    
    ## Automatically sort by average age
    ## 0 = No
    ## 1 = Yes
    auto_sort = 0
    
    ## Send group command before requesting articles
    ## 0 = No
    ## 1 = Yes
    send_group = 0
    
    ## Article cache limit
    ## 0 = Disable Cache
    ## -1 = Unlimited cache
    ## >0 = Maximum memory (in bytes) to use for cache
    cache_limit = 5000000
    
    ## Umask to use for directories/files
    umask = 777
    auto_browser = 0
    permissions = ""
    check_new_rel = 1
    auto_disconnect = 1
    config_lock = 0
    download_free = 250
    dirscan_speed = 5
    refresh_rate = 60
    rss_rate = 20
    extern_proc = ""
    email_server = ""
    email_to = ,
    email_from = ""
    email_account = ""
    email_pwd = ""
    email_endjob = 0
    email_full = 0
    top_only = 0
    safe_postproc = 0
    replace_spaces = 0
    par_option = -t0
    pause_on_post_processing = 0
    ignore_samples = 2
    script_dir = C:\Apps\SABnzbD\Scripts
    dirscan_script = None
    enable_tv_sorting = 0
    tv_sort_string = ""
    web_color = ""
    web_color2 = ""
    web_dir2 = ""
    api_key = ##############################
    replace_illegal = 1
    no_dupes = 2
    ionice = ""
    disable_api_key = 0
    enable_par_multicore = 1
    bandwidth_limit = 0
    https_port = 9090
    unpack_check = 1
    https_key = server.key
    movie_sort_string = ""
    movie_sort_extra = -cd%1
    enable_movie_sorting = 0
    language = en
    movie_categories = movies,
    admin_dir = admin
    nice = ""
    movie_extra_folder = 0
    tv_sort_countries = 1
    date_categories = tv,
    email_dir = ""
    ssl_type = v23
    https_cert = server.cert
    date_sort_string = ""
    dirscan_priority = 0
    enable_tsjoin = 1
    enable_https = 0
    enable_date_sorting = 0
    quick_check = 1
    replace_dots = 0
    folder_rename = 1
    save_interval = 60
    no_penalties = 0
    par2_multicore = 1
    warned_old_queue = 0
    ignore_wrong_unrar = 0
    folder_max_length = 128
    queue_complete = ""
    size_limit = ""
    tv_categories = ,
    allow_64bit_tools = 1
    queue_complete_pers = 1
    api_warnings = 1
    login_realm = SABnzbd
    password_file = ""
    ignore_unrar_dates = 0
    never_repair = 0
    pre_script = None
    sfv_check = 1
    email_rss = 0
    allow_streaming = 0
    pause_on_pwrar = 1
    nzb_key = ############################
    ampm = 0
    rss_filenames = 0

[logging]
    ## max size of SABnzbd.log (in bytes)
    max_log_size = 5242880
    
    ## how many backups of SABnzbd.log to keep around
    log_backups = 5
    
    ## enable cherrypy logging
    ## 0 = no
    ## 1 = yes
    enable_cherrypy_logging = 1
    log_level = 0
    log_new = 0

## Fill in your servers here
## If your server doesn't need password auth set
## username and password to ''
## fillserver field must be 0 for non-fillservers (>0 otherwise)
[servers]


[categories]
    [[unknown]]
        priority = -100
        pp = ""
        name = unknown
        script = Default
        newzbin = Unknown
        dir = Unknown
    [[anime]]
        priority = -100
        pp = ""
        name = anime
        script = Default
        newzbin = Anime
        dir = Anime
    [[apps]]
        priority = -100
        pp = ""
        name = apps
        script = Default
        newzbin = Apps
        dir = Apps
    [[books]]
        priority = -100
        pp = ""
        name = books
        script = Default
        newzbin = Books
        dir = Books
    [[consoles]]
        priority = -100
        pp = ""
        name = consoles
        script = Default
        newzbin = Consoles
        dir = Consoles
    [[emulation]]
        priority = -100
        pp = ""
        name = emulation
        script = Default
        newzbin = Emulation
        dir = Emulation
    [[games]]
        priority = -100
        pp = ""
        name = games
        script = Default
        newzbin = Games
        dir = Games
    [[misc]]
        priority = -100
        pp = ""
        name = misc
        script = Default
        newzbin = Misc
        dir = Misc
    [[movies]]
        priority = -100
        pp = ""
        name = movies
        script = Default
        newzbin = Movies
        dir = Movies
    [[music]]
        priority = -100
        pp = ""
        name = music
        script = Default
        newzbin = Music
        dir = Music
    [[pda]]
        priority = -100
        pp = ""
        name = pda
        script = Default
        newzbin = PDA
        dir = PDA
    [[resources]]
        priority = -100
        pp = ""
        name = resources
        script = Default
        newzbin = Resources
        dir = Resources
    [[xxx]]
        priority = -100
        pp = ""
        name = xxx
        script = Default
        newzbin = XXX
        dir = XXX
    [[*]]
        priority = 0
        pp = 3
        name = *
        script = None
        newzbin = ""
        dir = ""
    [[tv]]
        priority = -100
        pp = ""
        name = tv
        script = TvRenamer.exe
        newzbin = TV
        dir = TV
    [[tv-autorename]]
        priority = -100
        pp = ""
        name = tv-autorename
        script = TvRenamer.exe
        newzbin = TV-rename
        dir = TV\temp
[nzbmatrix]
    username = ""
    apikey = ""
    del_bookmark = 1
[growl]
    growl_enable = 0
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: losing files when moving to full disk (0.6.10)

Post by shypike »

I'm not sure what your issue is.
Recent releases keep the original downloaded files in "incomplete"
as long as post-processing isn't succesfully completed.
Downloading will stop when the disk is almost full (when you set a limit).
Post-processing will stop when the "complete" disk is full
and you can retry when there's free space again.

It's the old behaviour that was inconsistent.
Yarbles
Newbie
Newbie
Posts: 36
Joined: January 25th, 2008, 6:51 am

Re: losing files when moving to full disk (0.6.10)

Post by Yarbles »

I know file handling was revamped in the recent releases. I thought maybe I missed a setting or something, but I haven't found anything to explain what's happening.

At least twice I've found the Complete disk full, the Complete/job folder filled with .rar files (but not all of them) and the Incomplete\job folder deleted.

Also, in the log file above the second line repeats about 15 times for .r66, .67, ...
Yarbles
Newbie
Newbie
Posts: 36
Joined: January 25th, 2008, 6:51 am

Re: losing files when moving to full disk (0.6.10)

Post by Yarbles »

I duplicated the problem with debug logging enabled, in case it is of any help.

I did a search/Replace on the JobName and FileName. Also, to replicate the same conditions I had another job downloading at the same time.

Code: Select all

2011-12-18 22:20:07,839::ERROR::[misc:748] Failed moving H:\temp\nzb_incomplete\JobName\FileName.sfv to E:\Newsbin\_UNPACK_JobName\FileName.sfv

2011-12-18 22:20:07,839::INFO::[misc:749] Traceback: 

Traceback (most recent call last):
  File "sabnzbd\misc.pyo", line 745, in move_to_path
  File "shutil.pyo", line 56, in copyfile
IOError: [Errno 28] No space left on device
2011-12-18 22:20:07,839::DEBUG::[misc:736] Moving. Old path:H:\temp\nzb_incomplete\JobName\FileName_proof.jpg new path:E:\Newsbin\_UNPACK_JobName\FileName_proof.jpg unique?:False

2011-12-18 22:20:07,868::ERROR::[misc:748] Failed moving H:\temp\nzb_incomplete\JobName\FileName_proof.jpg to E:\Newsbin\_UNPACK_JobName\FileName_proof.jpg

2011-12-18 22:20:07,868::INFO::[misc:749] Traceback: 

Traceback (most recent call last):
  File "sabnzbd\misc.pyo", line 745, in move_to_path
  File "shutil.pyo", line 53, in copyfile
  File "shutil.pyo", line 29, in copyfileobj
IOError: [Errno 28] No space left on device
2011-12-18 22:20:07,878::INFO::[postproc:627] Removing unwanted file E:\Newsbin\_UNPACK_JobName\FileName.sfv

2011-12-18 22:20:08,088::INFO::[postproc:472] Cleaning up JobName (keep_basic=False)

2011-12-18 22:20:08,096::DEBUG::[postproc:482] Removing workdir H:\temp\nzb_incomplete\JobName

2011-12-18 22:20:08,108::INFO::[postproc:83] Saving postproc queue

2011-12-18 22:20:08,108::INFO::[__init__:838] Saving data for postproc1.sab in C:\Apps\SABnzbD\SABnzbd\admin\postproc1.sab

2011-12-18 22:20:08,118::INFO::[downloader:229] Post-processing finished, resuming download
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: losing files when moving to full disk (0.6.10)

Post by shypike »

I think I have misread your post slightly.
I took it for a rant about getting the old behaviour back.
Only at the end you list your true complaint: that "move" is behaving badly when the disk is full.
I'll check this out.
Please save your logs in case I need them (should I have problems reproducing this).
Post Reply