Page 2 of 4
Re: Incorrect free space
Posted: June 27th, 2017, 8:06 am
by jcfp
Seems the values from the apple python overflow at 4*1024^4. Once you know that (and compensate for that finder thing defining a TB using decimal rather than binary) the values from sab start making sense.
Re: Incorrect free space
Posted: June 27th, 2017, 8:15 am
by safihre
@jcfp, so how do we correct the calculation then?
Also users report it below 1TB:
https://forums.sabnzbd.org/viewtopic.ph ... 98#p112391
I am starting to also suspect an macOS update to be the culprit, since users never reported this before.
Re: Incorrect free space
Posted: June 27th, 2017, 8:24 am
by jcfp
Now that part I don't know. Get apple to fix their python release? Use some alternative to get the free space (if any)?
Re: Incorrect free space
Posted: June 27th, 2017, 9:41 am
by sander
I have no problem on a MacOS with a 5588 GB (so: > 4TB) partition:
df output:
Code: Select all
server:~ sander$ df -g /Volumes/BigDataZenato
Filesystem 1G-blocks Used Available Capacity iused ifree %iused Mounted on
/dev/disk5 5588 852 4736 16% 892090 4294075189 0% /Volumes/BigDataZenato
So: >4TB partition
python script output:
Code: Select all
server:~ sander$ ./disk_space.py /Volumes/BigDataZenato
2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]
Directory /Volumes/BigDataZenato
statvfs posix.statvfs_result(f_bsize=2097152, f_frsize=8192, f_blocks=732482664, f_bfree=620759077, f_bavail=620759077, f_files=4294967279, f_ffree=4294075189, f_favail=4294075189, f_flag=0, f_namemax=255)
disk_size 5588.39923096
available 4736.01590729
So: correctly reported
python script:
Code: Select all
#!/usr/bin/env python
import os
import sys
print sys.version
_dir = '/Users/'
_dir = sys.argv[1]
print "Directory", _dir
s = os.statvfs(_dir)
print "statvfs" , s
GB = float(1024**3)
disk_size = float(s.f_blocks) * float(s.f_frsize) / GB
available = float(s.f_bavail) * float(s.f_frsize) / GB
print "disk_size", disk_size
print "available", available
Re: Incorrect free space
Posted: June 27th, 2017, 9:45 am
by .maxx
Im using an other Python Version with SAB 2.1.0 [443efb5]
Code: Select all
2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 12:39:47) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] [US-ASCII]
Re: Incorrect free space
Posted: June 27th, 2017, 2:16 pm
by sander
OK, Mac-users, here we go: is the problem in python-level statvfs() or in c-level statvfs()? Let's determine:
Do you have a C-compiler on your Mac? If so, compile and run the C-program below against your NAD drive; it does a statvfs() a the C/system level.
Compile:
Code: Select all
server:~ sander$ gcc -o statvfs_in_c statvfs_in_c.c
Run:
Code: Select all
server:~ sander$ ./statvfs_in_c /Users
Available [MB]: 306153
server:~ sander$
Tested on a Mac (thanks Dr B):
Code: Select all
server:~ sander$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.12.5
BuildVersion: 16F73
Code: Select all
#include <sys/statvfs.h>
#include <stdio.h>
int main (int argc, char **argv) {
if(argc!=2) {
printf("Error: Missing required parameter: directory.\n");
return -1;
}
struct statvfs info;
statvfs (argv[1], &info);
// Python SAB source says: available = float(s.f_bavail) * float(s.f_frsize)
long available = info.f_bavail * info.f_frsize / (1024*1024);
printf("Available [MB]: %ld\n", available);
return 0;
}
Re: Incorrect free space
Posted: June 27th, 2017, 2:30 pm
by sander
For those who can't compile: here's the compiled version:
https://www.appelboor.com/dump/MacOS/
Direct link:
https://www.appelboor.com/dump/MacOS/st ... MacOS64bit
Code: Select all
$ ll
-rwxr-xr-x 1 sander sander 8496 jun 27 21:22 statvfs_in_c_for_MacOS64bit*
Code: Select all
$ file statvfs_in_c_for_MacOS64bit
statvfs_in_c_for_MacOS64bit: Mach-O 64-bit x86_64 executable
Code: Select all
$ md5sum statvfs_in_c_for_MacOS64bit
dc24d3de514a2242a57aca4bb86bba27 statvfs_in_c_for_MacOS64bit
Re: Incorrect free space
Posted: June 28th, 2017, 2:32 am
by .maxx
@sander
can you please help me to run statvfs_in_c_for_MacOS64bit
I can't figure out how to use this file
Re: Incorrect free space
Posted: June 28th, 2017, 6:56 am
by sander
.maxx wrote: ↑June 28th, 2017, 2:32 am
@sander
can you please help me to run statvfs_in_c_for_MacOS64bit
I can't figure out how to use this file
Certainly. Glad you can test.
Download the file with your favorite webbrowser, or with curl:
Code: Select all
curl https://www.appelboor.com/dump/MacOS/statvfs_in_c_for_MacOS64bit > statvfs_in_c_for_MacOS64bit
Make it's there OK:
Code: Select all
$ ls -al statvfs_in_c_for_MacOS64bit
-rw-r-xr--+ 1 sander staff 8496 Jun 28 13:50 statvfs_in_c_for_MacOS64bit
Make it executable:
Code: Select all
chmod a+x statvfs_in_c_for_MacOS64bit
Execute it with a directory name:
Code: Select all
$ ./statvfs_in_c_for_MacOS64bit /Users/
Available [MB]: 305052
If that works, do it for your NAS mount directories and post it here.
Re: Incorrect free space
Posted: June 28th, 2017, 8:45 am
by .maxx
thanks sander
here are the results:
Correct
Code: Select all
$ /statvfs_in_c_for_MacOS64bit /Users/
Available [MB]: 435223
And I thinks here's the problem.
Wrong
Code: Select all
$/statvfs_in_c_for_MacOS64bit /Volumes/Medien
Available [MB]: 226943
df -h /Volumes/Medien/
Code: Select all
$ df -h /Volumes/Medien/
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
//admin@SERVER._smb._tcp.local/Medien 26Ti 22Ti 4.2Ti 84% 23575294178 4527357888 84% /Volumes/Medien
Re: Incorrect free space
Posted: June 28th, 2017, 9:21 am
by safihre
So the underlying C function is wrong..
While before it was correct.
MacOS update wrongly made by Apple?
Re: Incorrect free space
Posted: June 28th, 2017, 9:29 am
by .maxx
the user from the initial post got a raspi
Re: Incorrect free space
Posted: June 29th, 2017, 3:00 pm
by sander
Make sure no other program (like SAB) is saving / deleting stuff on disk.
Then do this is a terminal window. Replace '/Directory/Of/Interest' with your disk / directory, like '/Users/' or '/mnt/nas/bla':
Print disk usage via different methods:
Code: Select all
df -m
python -c "import os; s = os.statvfs('/Directory/Of/Interest'); print s; print 'free:', float(s.f_bavail) * s.f_frsize / (1024**2) "
Download & save a 1400MB file
Code: Select all
curl http://nl.cdimage.ubuntu.com/ubuntu-gnome/releases/17.04/release/ubuntu-gnome-17.04-desktop-amd64.iso > my1400MB.iso
Again print disk usage:
Code: Select all
df -m
python -c "import os; s = os.statvfs('/Directory/Of/Interest''); print s; print 'free:', float(s.f_bavail) * s.f_frsize / (1024**2) "
Post all output here.
... hopefully we can compare the outputs
Re: Incorrect free space
Posted: June 29th, 2017, 3:13 pm
by sander
https://stackoverflow.com/questions/209 ... -statvfs64 tells there is a statvfs64() ... and it seems you get that via a gcc compile option. On linux I see the difference in the executable, but not on MacOS, but can you try it anyway:
https://www.appelboor.com/dump/MacOS/ now also contains statvfs_with_LARGEFILE64_SOURCE_for_MacOS64bit
Direct link
https://www.appelboor.com/dump/MacOS/st ... MacOS64bit
Can you download & execute just like you did for the first one? And post the result here?
Re: Incorrect free space
Posted: June 30th, 2017, 2:34 am
by .maxx
Here the
LARGEFILE64 results
Correct
Code: Select all
$ /statvfs_with_LARGEFILE64_SOURCE_for_MacOS64bit /Users/
Available [MB]: 434190
Wrong
Code: Select all
$/statvfs_with_LARGEFILE64_SOURCE_for_MacOS64bit /Volumes/Medien
Available [MB]: 223562