crashes and not working (after every update)

[an3k]

Code: Select all

2023-05-16 23:29 [16/May/2023:23:29:15] HTTP 
Traceback (most recent call last):
  File "/opt/lib/python3.10/site-packages/cherrypy/_cprequest.py", line 638, in respond
    self._do_respond(path_info)
  File "/opt/lib/python3.10/site-packages/cherrypy/_cprequest.py", line 697, in _do_respond
    response.body = self.handler()
  File "/opt/lib/python3.10/site-packages/cherrypy/lib/encoding.py", line 223, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/opt/lib/python3.10/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/interface.py", line 188, in internal_wrap
    return wrap_func(*args, **kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/interface.py", line 464, in api
    return api_handler(kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/api.py", line 104, in api_handler
    response = _api_table.get(mode, (_api_undefined, 2))[0](name, kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/api.py", line 428, in _api_status
    return _api_status_table.get(name, (_api_fullstatus, 2))[0](value, kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/api.py", line 419, in _api_fullstatus
    status = build_status(
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/api.py", line 1250, in build_status
    internetspeed_future = sabnzbd.THREAD_POOL.submit(internetspeed)
  File "/opt/lib/python3.10/concurrent/futures/thread.py", line 169, in submit
RuntimeError: cannot schedule new futures after interpreter shutdown

[OneCD]

Subscribing to this topic so I can assist on the sherpa-related stuff if needed.

[safihre]

So this error occurs if something else already went wrong previously. Is there any other Traceback you got?

[an3k]

It all started with sab not starting because of missing sabctools.

Code: Select all

>>>>>>>>>> datetime:'Tue May 16 18:06:11 CEST 2023', request:'starting', QPKG:'220421', app:'4.0.1' <<<<<<<<<<
daemon: NOT active
file (/opt/bin/git): exists
update 'SABnzbd' from remote repository: OK
current git branch: master
file (/opt/bin/python3): exists
load UI ports from application: OK
start daemon: failed!
= ExecuteAndLog()
output: "Not all required Python modules are available, please check requirements.txt
Missing module: sabctools
You can read more at: https://sabnzbd.org/wiki/installation/install-off-modules
If you still experience problems, remove all .pyc files in this folder and subfolders"
result: [1]

I tried to find the culprit but couldn't get it working ... until I just did "pip install sabctools==7.0.1". Before that "pip list" told me that sabctools 7.0.2 was installed.
In both cases "python -c "import sabctools; print(sabctools.__version__, sabctools.__file__);"" resulted in

Code: Select all

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/lib/python3.10/site-packages/sabctools/__init__.py", line 2, in <module>
    from sabctools.sabctools import *
ModuleNotFoundError: No module named 'sabctools.sabctools'

Now sab was running and working. I downloaded a few files without a single error. A few hours later, without changing anything since the last interaction I opened the webinterface again and saw the error.

Code: Select all

2023-05-16 20:53:37,808::INFO::[SABnzbd:1249] Using INI file /share/CACHEDEV1_DATA/.qpkg/SABnzbd/config/config.ini
2023-05-16 20:53:37,808::INFO::[filesystem:710] Creating directories: /share/Public/Downloads/complete
2023-05-16 20:53:37,810::INFO::[notifier:123] Sending notification: Error - Failed making (/share/Public/Downloads/complete) (type=error, job_cat=None)
2023-05-16 20:53:37,809::ERROR::[filesystem:732] Failed making (/share/Public/Downloads/complete)
Traceback (most recent call last):
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/filesystem.py", line 726, in create_all_dirs
    os.mkdir(path_part_combined)
FileExistsError: [Errno 17] File exists: '/share/Public'
2023-05-16 20:53:37,810::INFO::[notifier:123] Sending notification: Error - Cannot create directory /share/Public/Downloads/complete (type=error, job_cat=None)
2023-05-16 20:53:37,810::ERROR::[filesystem:395] Cannot create directory /share/Public/Downloads/complete
2023-05-16 20:53:37,813::INFO::[postproc:134] Loading postproc queue
2023-05-16 20:53:37,814::INFO::[scheduler:190] Scheduling RSS interval task every 720 min (delay=221)
2023-05-16 20:53:37,814::INFO::[scheduler:201] Scheduling VersionCheck on day 5 at 12:53
2023-05-16 20:53:37,814::INFO::[scheduler:215] Setting schedule for midnight BPS reset
2023-05-16 20:53:37,814::INFO::[scheduler:218] Setting schedule for server expiration check
2023-05-16 20:53:37,815::INFO::[scheduler:223] Setting schedule for server quota check
2023-05-16 20:53:37,815::INFO::[__init__:329] All processes started
2023-05-16 20:53:37,815::INFO::[SABnzbd:340] Template location for Glitter is /share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/interfaces/Glitter
2023-05-16 20:53:37,815::INFO::[SABnzbd:340] Template location for Config is /share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/interfaces/Config
2023-05-16 20:53:37,816::INFO::[misc:1169] [N/A] Running external command: ['/opt/bin/unrar']
2023-05-16 20:53:37,818::INFO::[misc:1169] [N/A] Running external command: ['/opt/bin/7za']
2023-05-16 20:53:37,826::INFO::[notifier:123] Sending notification: Error - [N/A] The command in build_command is undefined. (type=error, job_cat=None)
2023-05-16 20:53:37,826::ERROR::[misc:1121] [N/A] The command in build_command is undefined.
2023-05-16 20:53:37,826::INFO::[SABnzbd:425] SABCTools module (v7.0.2)... found!
2023-05-16 20:53:37,826::INFO::[SABnzbd:426] SABCTools module is using SIMD set: AVX2
2023-05-16 20:53:37,826::INFO::[SABnzbd:427] SABCTools module is linked to OpenSSL: True
2023-05-16 20:53:37,826::INFO::[SABnzbd:446] Cryptography module (v40.0.2)... found!
2023-05-16 20:53:37,826::INFO::[notifier:123] Sending notification: Error - par2 binary... NOT found! (type=error, job_cat=None)
2023-05-16 20:53:37,826::ERROR::[SABnzbd:453] par2 binary... NOT found!
2023-05-16 20:53:37,826::INFO::[SABnzbd:458] UNRAR binary... found (/opt/bin/unrar)
2023-05-16 20:53:37,827::INFO::[SABnzbd:466] UNRAR binary version 6.10
2023-05-16 20:53:37,827::INFO::[SABnzbd:474] 7za binary... found (/opt/bin/7za)
2023-05-16 20:53:37,827::INFO::[SABnzbd:476] 7za binary version 16.02
2023-05-16 20:53:37,827::INFO::[SABnzbd:487] nice binary... found (/opt/bin/nice)
2023-05-16 20:53:37,827::INFO::[SABnzbd:491] ionice binary... found (/opt/bin/ionice)
2023-05-16 20:53:37,827::INFO::[notifier:123] Sending notification: Error - Essential modules are missing, downloading cannot start. (type=error, job_cat=None)
2023-05-16 20:53:37,827::ERROR::[SABnzbd:497] Essential modules are missing, downloading cannot start.
2023-05-16 20:53:37,828::INFO::[SABnzbd:1438] Starting web-interface on 0.0.0.0:8800
2023-05-16 20:53:37,828::INFO::[_cplogging:213] [16/May/2023:20:53:37] ENGINE Bus STARTING
2023-05-16 20:53:37,935::INFO::[_cplogging:213] [16/May/2023:20:53:37] ENGINE Serving on http://0.0.0.0:8800
2023-05-16 20:53:37,935::INFO::[_cplogging:213] [16/May/2023:20:53:37] ENGINE Bus STARTED
2023-05-16 20:53:39,198::INFO::[filesystem:710] Creating directories: /share/Public/Downloads/complete
2023-05-16 20:53:39,198::ERROR::[filesystem:732] Failed making (/share/Public/Downloads/complete)
Traceback (most recent call last):
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/filesystem.py", line 726, in create_all_dirs
    os.mkdir(path_part_combined)
FileExistsError: [Errno 17] File exists: '/share/Public'
2023-05-16 20:53:39,199::ERROR::[filesystem:395] Cannot create directory /share/Public/Downloads/complete
2023-05-16 20:53:39,199::INFO::[filesystem:710] Creating directories: /share/Public/Downloads/complete
2023-05-16 20:53:39,199::ERROR::[filesystem:732] Failed making (/share/Public/Downloads/complete)
Traceback (most recent call last):
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/filesystem.py", line 726, in create_all_dirs
    os.mkdir(path_part_combined)
FileExistsError: [Errno 17] File exists: '/share/Public'
2023-05-16 20:53:39,199::ERROR::[filesystem:395] Cannot create directory /share/Public/Downloads/complete
2023-05-16 20:53:43,929::INFO::[filesystem:710] Creating directories: /share/Public/Downloads/complete

The FileExistsError repeats approx. 50 times then stops probably because two nzb were added. I checked the web interface and saw both nzbs but nothing happened. No download.
I saw this error

Code: Select all

2023-05-16 23:28:53,981::INFO::[config:999] Writing settings to INI file /share/CACHEDEV1_DATA/.qpkg/SABnzbd/config/config.ini
2023-05-16 23:29:15,104::INFO::[notifier:123] Sending notification: Error - [16/May/2023:23:29:15] HTTP  (type=error, job_cat=None)
2023-05-16 23:29:15,098::ERROR::[_cplogging:213] [16/May/2023:23:29:15] HTTP
Traceback (most recent call last):
  File "/opt/lib/python3.10/site-packages/cherrypy/_cprequest.py", line 638, in respond
    self._do_respond(path_info)
  File "/opt/lib/python3.10/site-packages/cherrypy/_cprequest.py", line 697, in _do_respond
    response.body = self.handler()
  File "/opt/lib/python3.10/site-packages/cherrypy/lib/encoding.py", line 223, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/opt/lib/python3.10/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/interface.py", line 188, in internal_wrap
    return wrap_func(*args, **kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/interface.py", line 464, in api
    return api_handler(kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/api.py", line 104, in api_handler
    response = _api_table.get(mode, (_api_undefined, 2))[0](name, kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/api.py", line 428, in _api_status
    return _api_status_table.get(name, (_api_fullstatus, 2))[0](value, kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/api.py", line 419, in _api_fullstatus
    status = build_status(
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/api.py", line 1250, in build_status
    internetspeed_future = sabnzbd.THREAD_POOL.submit(internetspeed)
  File "/opt/lib/python3.10/concurrent/futures/thread.py", line 169, in submit
RuntimeError: cannot schedule new futures after interpreter shutdown
2023-05-16 23:29:15,105::INFO::[_cplogging:213] [16/May/2023:23:29:15] HTTP
Request Headers:
  Remote-Addr: 10.13.37.34
  HOST: 10.13.37.21:8800
  CONNECTION: keep-alive
  ACCEPT: */*
  DNT: 1
  X-REQUESTED-WITH: XMLHttpRequest
  USER-AGENT: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36
  REFERER: http://10.13.37.21:8800/
  ACCEPT-ENCODING: gzip, deflate
  ACCEPT-LANGUAGE: de,en-US;q=0.9,en;q=0.8,ru;q=0.7
  COOKIE: remeber=1; qtoken_account=AZGtxW7=; qtoken=8348ecf3e6762ca4cabe6f6731281521; nas_lang=ENG

I removed the two nzbs and a new error appeared.

Code: Select all

2023-05-16 23:32:28,034::INFO::[downloader:460] Forcing disconnect
2023-05-16 23:32:35,982::INFO::[notifier:123] Sending notification: Error - [16/May/2023:23:32:35] HTTP  (type=error, job_cat=None)
2023-05-16 23:32:35,982::ERROR::[_cplogging:213] [16/May/2023:23:32:35] HTTP
Traceback (most recent call last):
  File "/opt/lib/python3.10/site-packages/cherrypy/_cprequest.py", line 638, in respond
    self._do_respond(path_info)
  File "/opt/lib/python3.10/site-packages/cherrypy/_cprequest.py", line 697, in _do_respond
    response.body = self.handler()
  File "/opt/lib/python3.10/site-packages/cherrypy/lib/encoding.py", line 223, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/opt/lib/python3.10/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/interface.py", line 188, in internal_wrap
    return wrap_func(*args, **kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/interface.py", line 464, in api
    return api_handler(kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/api.py", line 104, in api_handler
    response = _api_table.get(mode, (_api_undefined, 2))[0](name, kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/api.py", line 428, in _api_status
    return _api_status_table.get(name, (_api_fullstatus, 2))[0](value, kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/api.py", line 419, in _api_fullstatus
    status = build_status(
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/repo-cache/sabnzbd/api.py", line 1250, in build_status
    internetspeed_future = sabnzbd.THREAD_POOL.submit(internetspeed)
  File "/opt/lib/python3.10/concurrent/futures/thread.py", line 169, in submit
RuntimeError: cannot schedule new futures after interpreter shutdown
2023-05-16 23:32:35,982::INFO::[_cplogging:213] [16/May/2023:23:32:35] HTTP
Request Headers:
  Remote-Addr: 10.13.37.34
  HOST: 10.13.37.21:8800
  CONNECTION: keep-alive
  ACCEPT: */*
  DNT: 1
  X-REQUESTED-WITH: XMLHttpRequest
  USER-AGENT: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36
  REFERER: http://10.13.37.21:8800/
  ACCEPT-ENCODING: gzip, deflate
  ACCEPT-LANGUAGE: de,en-US;q=0.9,en;q=0.8,ru;q=0.7
  COOKIE: remeber=1; qtoken_account=AZGtxW7=; qtoken=8348ecf3e6762ca4cabe6f6731281521; nas_lang=ENG

I shutdown sab and started it again, added both nzbs and everything worked fine again.

[OneCD]

With regard to the 'sabctools' issue: it's been fixed, but you must ensure you've installed/upgraded-to a recent SABnzbd QPKG to use the fix.

You will also need to have run a 'clean' action on your SABnzbd QPKG.

[an3k]

When was that issue fixed? I tried yesterday. I downloaded the latest sherpa qpkg and since it helped me last time (but I forgot exactly what I did), I searched for this https://github.com/OneCDOnly/sherpa/iss ... 1205154176 and did a clean and reinstall. I think i also did an upgrade too since sherpa itself suggested that.

[OneCD]

It was fixed at the time SABnzbd 4.0.0 was released.

sherpa is constanly evolving, as-are the QPKGs it maintains. Actions you have used previously may not be the best way to fix a similar problem, especially as I fix each of these problems as I can.

You're running commands without understanding what they're doing and hoping they will fix your issue. I'm not sure a CLI package manager is the right choice for you. Maybe try using Stephane's QSAB QPKG instead? It's a static build, and doesn't have the abilities of the sherpa-installed one, but might be a better fit for you.

[an3k]

No clue why you went that offensive now.

[OneCD]

No clue why you went that offensive now.

I re-read your linked topic. I recalled the interaction. That's why.

[an3k]

I'm seriously surprised. I remember you implementing a switch "just for me" for disabling auto-updates as well as having a nice convo about other issues you had with your script and QNAP and what not. And just because of a single convo where I was not able to help you debugging your script you go that way? Well, good to know

Ok, so you went offensive because I couldn't help you debugging your scripts?
Maybe you should rethink your script and how it works to make it easier for you and your users. I don't know who really needs or wants automatic updates whenever they restart the application but beside from that just look at what you wrote:

"sherpa upgrade sab ... will upgrade the SABnzbd QPKG only (not the SABnzbd application)."
"sherpa clean sab ... will remove all application files cloned from the SABnzbd GitHub repo, and download them again."
"sherpa reinstall sab ... will reinstall the SABnzbd QPKG over the top of the existing one."

Your script handles a) a local cache of the github repo, b) a created qpkg (which does not consist out of the locally cached github repo) and c) the actual installation.
That means that the actual installation could be 4.0.1, the qpkg could be 4.0.0 and the cached repo could be 3.6.1. You can spot the main cause of problems, can't you?!

[OneCD]

Ok, so you went offensive because I couldn't help you debugging your scripts?

I get defensive when those who can't or won't help debug are critical of another volunteer's efforts. It's quite a frustrating and thankless task trying to help a person with the mindset that "this should all be easier". If you want help, find yourself a better attitude.

Maybe you should rethink your script and how it works to make it easier for you and your users.

This is literally what I do every day.

But, I can't force anyone to understand what it does and why, if they don't want to. "You can lead a horse to water..." and so-on.

That means that the actual installation could be 4.0.1, the qpkg could be 4.0.0 and the cached repo could be 3.6.1. You can spot the main cause of problems, can't you?!

That's not the case.

Firstly, the QPKG version number is a date code indicating when that QPKG was built. It's not made for any specific application version and (in the case of SABnzbd) doesn't contain a copy of the application. The work I put into having it handle all the auto-updating from GitHub/GitLab/Codeberg, etc... means the end-user doesn't need to worry about what the latest application version is: the service script will handle it by downloading a copy of the current stable branch (whatever each repo calls stable anyway) and putting it in the cached-repo directory.

The cached-repo is updated on every start. This was something you were not happy about, so I included a switch to prevent this. The downside to using this is you then have a static application build. You're the user, so it's your choice. If your cached-repo version is out-of-date, and shows you an older SABnzbd version when you run the application, then re-enable auto-updating.

There is no "actual installation" build with a separate version number. I don't know where you got that from. There is only a single application version: it's the one in cached-repo.

I suspect your ignorance of exactly what the QPKG service-script does is causing your problems.

[an3k]

Ok, so you went offensive because I couldn't help you debugging your scripts?

I get defensive when those who can't or won't help debug are critical of another volunteer's efforts. It's quite a frustrating and thankless task trying to help a person with the mindset that "this should all be easier". If you want help, find yourself a better attitude.

Well, in the convo I clearly asked how I could debug. I showed my will to help. But all you replied was "It's not something I can teach you in 5 minutes. You'd need to be proficient in BASH scripting before attempting to debug a large scripted environment like that established by sherpa.". Telling me now I didn't wanted to help is much more than just a lie.
I'm proficient in scripting in general and I am fine with managing a large environment ... just not on Linux. And your constant "you're too stupid" but in friendly words is more than enough to make nobody want to actually help you at all. You constantly behave like you are better.

That means that the actual installation could be 4.0.1, the qpkg could be 4.0.0 and the cached repo could be 3.6.1. You can spot the main cause of problems, can't you?!

That's not the case.

Firstly, the QPKG version number is a date code indicating when that QPKG was built. It's not made for any specific application version and (in the case of SABnzbd) doesn't contain a copy of the application. The work I put into having it handle all the auto-updating from GitHub/GitLab/Codeberg, etc... means the end-user doesn't need to worry about what the latest application version is: the service script will handle it by downloading a copy of the current stable branch (whatever each repo calls stable anyway) and putting it in the cached-repo directory.

The cached-repo is updated on every start. This was something you were not happy about, so I included a switch to prevent this. The downside to using this is you then have a static application build. You're the user, so it's your choice. If your cached-repo version is out-of-date, and shows you an older SABnzbd version when you run the application, then re-enable auto-updating.

There is no "actual installation" build with a separate version number. I don't know where you got that from.

I suspect your ignorance of exactly what the QPKG service-script does is causing your problems.

Based on what you wrote I had to assume that your script has to manage three different "versions". Based on what I whitnessed I think that there's a QPKG just so that users can use the UI and QTS knows what's up and your script silently (without QTS knowing) updates sab (locally cached repo) if there are any updates. And me being required to get the new sab QPKG (there isn't one, it's a sherpa QPKG) results from the complete file structure rewrite and different packages used by sab.

However, instead of reading what I wrote ("I downloaded the latest sherpa qpkg and did a clean and reinstall. I think i also did an upgrade too since sherpa itself suggested that.") you again just told me I'm too stupid. I did what I was required ("With regard to the 'sabctools' issue: it's been fixed, but you must ensure you've installed/upgraded-to a recent SABnzbd QPKG to use the fix."I got the most recent sherpa.qpkg and ran a clean (to clean the local cache of the github repo and redownload it again) followed by a reinstall (which was wrong at this point) followed by an upgrade.
To this day you never told me (or wrote anywhere) what the correct way would be. Is it: download the new sherpa.qpkg run clean, run upgrade, run reinstall? Or is it just reinstall. Based on https://github.com/OneCDOnly/sherpa/iss ... 1193087616 the user has to run three commands.

You are not helping but you are undfriendly, insulting and arrogant. You may be good with scripting but have no clue about usability. Imagine apt-get, aptitude, pip or any other package manager would auto-update as soon as the application is restarted. Pure horeshit.

[OneCD]

You are not helping but you are undfriendly, insulting and arrogant. You may be good with scripting but have no clue about usability.

Try doing what I do with my mental-health issues, and the small handful of medications I take everyday.

Imagine apt-get, aptitude, pip or any other package manager would auto-update as soon as the application is restarted.

Hardly an apples-with-apples comparison. sherpa is something quite-new, and isn't operating in the same well-supported environments as those package managers. What do you think happened when QNAP took Ubuntu to make QTS, then removed apt?

Pure horeshit.

Then don't use sherpa. Please?