Newznab Installation Tutorial for Mac OSX

[De5perate]

This is my guide to installing Newznab on a Mac running OS-X 10.8.2 – Mountain Lion.

There were pictures included but it was far too time consuming to try to include them here.

Now I am a complete noob to all this – I don’t know what php does or Apache, I know mysql is a database but that is my limit. I have however with a lot of help from various sources through Google sources managed to install this and get it working and I am hoping my noob guide can help some others avoid some of the pain I experienced.

Step 1 – Buy Newznap Plus

Pay your £10 and get Newznab Plus (http://www.newznab.com). While you are waiting for your confirmation email to arrive install xampp. I have heard that the free version is not really worth while as you need to obtain or write the regexes required.

Step 2 – Install Xampp

Go to http://www.apachefriends.org/en/xampp-macosx.html and download and install xmapp. This is a simple drag to applications install I am not going to go into that level of detail.

Step 3 – Install Newznap Plus

You have received your confirmation email from Newznab . For me this was a painful experience as after you’ve paid your money you get rewarded with a test – not a nice download link – oh no – you get an SVN link – so I spent a while working out what to do with that. What I did was download the trial version of Versions (from http://versionsapp.com).
You don’t need to install it – just run the downloaded program:



So click New Repository Bookmark and in the dialogue box that appears enter the SVN address, username and password from your welcome email and then click create.



You should now see a screen like this – right click on the top level nnplus and select Checkout. In the next dialogue box remove ‘ (Branch)’ from the Save As box and click the down arrow icon to the right of the save as box. Now in the finder like box that appears navigate to Applications, then find your recently installed XAMPP folder then xamppfiles and finally htdocs. Now click Checkout to extract your files here.

Step 4 – Test Xampp.

Open applications and find XAMPP folder, open the folder and select XAMPP Control.app. In XAMPP Control panel click Start on all 3 options – Apache (which will request a password), MySQL and FTP (not sure if FTP is required but I am playing it safe).

When they are all showing a green ‘light’ open Safari and type localhost into the address bar. You should see a welcome screen as below – if not I suggest you resort to google as mine worked just fine.
Click on your language of choice and then you’ll be presented with the Welcome to Xampp screen where you are welcome to try out the little demos etc.

However to stay on track click on security and if you are planning to do more than access this site on your own personal network I suggest you do something more secure than the default – following their suggestion and pasting the /Applications/XAMPP/xamppfiles/xampp security into a terminal window seems a good start. I have not done that as I am keeping my access to just me. If anyone would like to alert me to any security issues with that master plan then please let me know. As I may have mentioned before I have no idea what I am doing.



Step 5 – Configure Xampp

I understand this gives read – write access to anyone – therefore you may want to look into a more secure method if you are concerned about this step.

For the next few steps you will need to edit some config files for this I have used Text Wrangler – I am certain there are other methods out there (possibly better – who knows) but this was the way I did it and so far it’s working – so head on over to http://www.barebones.com/products/textw ... nload.html and download and install Text Wrangler.

To configure Xampp – I first suggest to access the Xampp control app and stop all – Apache, My SQL and FTP.

Now open Textwrangler and click file – open. In the open dialogue box select everything in the enable box and also select Show Hidden Items check box.



Then navigate to Applications and then Xampp foler, then click on the etc shortcut and finally open httpd.conf. In this file you need to make a couple of minor changes. To make changes you first need to click on the icon showing pencil with a cross through it and accept the yes no dialog box.
Firstly in my setup it was line number 178. You need to update the Documentroot to read:

Code: Select all

DocumentRoot "/Applications/XAMPP/xamppfiles/htdocs/nnplus/www/"

Please remember the final / at the end of the structure – the absence of this caused me a lot of early problems.

Then line number 206 for me (might be different for you but the text should be the same) – update <Directory to read:

Code: Select all

<Directory "/Applications/XAMPP/xamppfiles/htdocs/nnplus/www">

In this case exclude the / at the end of the structure – I am sure there is a very good reason for it’s inclusion in the previous path but not in this one but I claim the noob so I will not attempt to explain.



That’s it for this file. So click save and close the file.

If you want you can test to see if this has worked by going to Xampp Control and starting Apache, MySQL and FTP. Then in saferi type localhost into the address bar again and press return. You should now open the Newznab install page (which should show an error).

Step 6 – Configure PHP.

In Textwrangler click file and open then navigate (if you are not already there) to Applications/xampp and the etc shortcut (see step 5 for more details). Now select php.ini and click open.

We need to make a few changes here too.

Scroll down to approx. line 226 where you will see a heading Resource Limit – the default values are shown below.



The max_execution_time needs to be increased – I suggest 120.
The memory_limit needs to be increased – I suggest 1024M



Again save the file.

Step 7 – Install Newznab (properly this time)

Open terminal and type (or copy and paste) (and press enter after each line) the following (This allows Newsnab read-write access to various folders - again I emphasise this allows read-write access to all users – there are more secure ways of doing this – however if you are the only user then proceed):

Code: Select all

chmod 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/www/lib/smarty/templates_c

Code: Select all

chmod 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/www/covers/movies

Code: Select all

chmod 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/www/covers/anime

Code: Select all

chmod 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/www/covers/music

Code: Select all

chmod 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/www

Code: Select all

chmod 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/www/install

Code: Select all

chmod -R 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/nzbfiles/

Now open XAMPP Control.app and make sure Apache, MySQL and FTP are allstarted (all green lights).

Open safari and type localhost into address bar. You should be on a Newznab Install screen with no errors now with a button at the bottom stating “Go to step: Pre flight check”:

Click That Button.

Now if you have correctly entered all the chmod lines (in this step above) and also if you have correctly modified your Resource Limits in Step 6 then you should have no errors here (if you have errors then either follow the instructions in the error or look back at step 6 of this tutorial).

Click the button marked “Go to step two: Set up the database”

Now if you haven’t changes your security settings you should simply have to type root into the Username box and click Setup Database. Otherwise you may have a different username and password.



Click the button.

It should setup the database and present you with a screen telling you the database setup is correct ……….

Click the button showing “Step three: Setup News Server connection”

Now I cannot help[ you with this step – you should have a newsgroup provider setup and have the login / connection details for it – type them in here and click test connection.

If all goes well then you should receive a message telling you “The news server setup is correct…..”

Click the button stating ”Step four: Cache Settings”

I cannot help you here – I just clicked the button without changing anything – hopefully I will learn what this does – until then I will do without.

Click “Save configuration file”

Click the button marked “Step 5: setup admin user”

Fill in this section and then click the button.

Then click the button marked “Step Seven: Set NZB File Path”

I accepted the default path and so long as you have correctly entered all the chmod commands presented earlier in this step you should get no errors – if you get an error follow the instructions in the error.

Otherwise click the button marked “Set NZB File Path”

You are done (well not quite but you are ready to start). Click on the admin home page link to begin tinkering.

Step 8 – Setup of Newznab

Now I may have mentioned before I am a noob so I have not had a long time to tinker. However I have a setup that seems to work so far and I have a plan to populate it. Therefore these are the steps I used – if you find something better – fear not I shall not be too upset.

Click on the site options link.

The first thing you should do is type (or preferably copy and paste) your newznab ID from your welcome email into the last field of the first section.

Next – in the section marked Usenet Settings I select the Use compressed headers toggle – you need to ensure your newsgroup provider supports this.

I change the Max Messages to 100,000 (one hundred thousand – for all those who use a comma as a decimal mark).

In the Where to start new groups field I change the toggle box to days and enter 1 day (initially it will go back one day to get all the headers – that is enough for starters) – if you leave the 20000 posts in there then you will not have enough posts to complete an episode or MP3.

Then scroll to the bottom and click “Save site settings”

Step 9 – Select Newsgroups.

Getting interesting – you are nearly there.

Select Admin home on the left hand links and then select the “activate some groups” link in item 2.

Choose your groups and click on the activate link. You can click on the group name to modify some settings like the backfill days.

Once you have chosen all your groups you need to head back to Terminal to populate it all.

Step 10 – Populate Newznab.

Open Terminal.

Type:

Code: Select all

cd /Applications/xampp/xamppfiles/htdocs/nnplus/misc/update_scripts

Then type

Code: Select all

/applications/xampp/xamppfiles/bin/php-5.3.1 update_binaries.php

This should start looking for headers and will take some time. If you have entered your newznab ID then it will also download the Regexes which are needed to process and index the NZBs (as far as I understand).

When that has completed you need to:

Type:

Code: Select all

cd /Applications/xampp/xamppfiles/htdocs/nnplus/misc/update_scripts

Then type

Code: Select all

/applications/xampp/xamppfiles/bin/php-5.3.1 update_releases.php

This turns all the millions of headers into nzbs and populates all the menus in Newznab.

My plan is to increase the number of days entered into the Usenet Settings in step 8 from 1 upwards to get a large backlog of available files a couple of days at a time. If anyone has a better method then please let me know.

I am also trying to automate the update scripts – I have entered the newznab.sh script into automator and edited the paths but I haven’t had time to find a way to make it visible to see if it is doing what I want. I have been toying with this for 24 hours and that is for later. I am also struggling to get my sick beard and couch potato to see the newznab server on another PC at the moment – so that is work in progress. Any help on these items would be greatly appreciated.

In the meantime I hope this tutorial helps some people out. If there are any mistakes and I am certain there are – please let me know. If you have a better way then please feel free to let the community know or take this document as a starting point and improve it.

[Tinology]

THANK YOU SO MUCH.

I can't explain how thankful I am for this post, I've been trying this set this up all week!

I wish I knew about xmapp earlier I've been trying to configure everything through terminal and editing with nano which is an absolute ballache.

If you make any other changes to your setup and make it better then please let me know.

Thanks again!

[blooboy]

Thanks to you for doing all the footwork; I got this up and going in less than 30. I am having issues with running the scripts for a second time as it keeps balking with "authentication" issues. Trying to figure that out. Otherwise it works like a dream.

Kudos to you!

Blooboy

[simonk83]

Nice, thanks. If you work out the automation do let us know

[De5perate]

I am thrilled that some of you are finding it useful - as mentioned before if you spot any mistakes or improvements then let me know I shall update.

As far as the automator is concerned I think it works, however until I have monitored it's progress over a couple of days I cannot be sure.

This is what I did (I think) -

Open Automator (located in Applications) and select Workflow as the new type of document, click Choose.
Then choose Run Shell Script from the huge list in the second column on the left, double click it.
In the Run Shell Script window that opens in Automator I deleted the word cat at the top (don't know what that does but it's gone). and then paste in the following:

Code: Select all

#!/bin/sh
# call this script from within screen to get binaries, processes releases and 
# every half day get tv/theatre info and optimise the database

set -e

export NEWZNAB_PATH="/Applications/XAMPP/xamppfiles/htdocs/nnplus/misc/update_scripts"
export NEWZNAB_SLEEP_TIME="600" # in seconds
export PHP_PATH="/Applications/XAMPP/xamppfiles/bin"
LASTOPTIMIZE=`date +%s`

while :

 do
CURRTIME=`date +%s`
cd ${NEWZNAB_PATH}
${PHP_PATH}/php-5.3.1 ${NEWZNAB_PATH}/update_binaries.php
${PHP_PATH}/php-5.3.1 ${NEWZNAB_PATH}/update_releases.php

DIFF=$(($CURRTIME-$LASTOPTIMIZE))
if [ "$DIFF" -gt 43200 ] || [ "$DIFF" -lt 1 ]
then
	LASTOPTIMIZE=`date +%s`
	${PHP_PATH}/php-5.3.1 ${NEWZNAB_PATH}/optimise_db.php
	${PHP_PATH}/php-5.3.1 ${NEWZNAB_PATH}/update_tvschedule.php
	${PHP_PATH}/php-5.3.1 ${NEWZNAB_PATH}/update_theaters.php
fi

echo "waiting ${NEWZNAB_SLEEP_TIME} seconds..."
sleep ${NEWZNAB_SLEEP_TIME}

done

This is just basically a copy of the newznab.sh file located in the folder /Applications/XAMPP/xamppfiles/htdocs/nnplus/misc/update_scripts/nix_scripts but I have edited the path to newznab scripts and added a PHP_PATH function to save me from typing it out a few times.

Then click file save... and select Application from the File Format drop down. Best to put it in the Applications folder then click save.

Then just run it from the applications folder. You should see a spinning cog at the top of the screen from where you can stop it if needed.

What I want though it to be able to see in a terminal type window what it is doing - at least to prove it works. I read somewhere that you need to rename the .app a .command but my Mac will not let me do that it just adds .command.app to the end of the file name.

It does seem to require a lot of processor power at certain times during the process, I don't know if that will ease after it has run for a while when it is finding and processing less headers.

Again - I hope this helps and I would like to hear feedback on any improvements.

[De5perate]

I have updated to tidy up the original post including some pictures and the code in code boxes.

[Tinology]

I'm currently populating my index, once it has been populated say tomorrow I run an update_binaries it won't go and find all the articles it did the day before will it and over-write everything?

[De5perate]

I believe it is cleverer than that.

I added a single binary at first and tested with 1 day backfill and it downloaded a million headers or something - I repeated it after it had finished and it found another thousand or so.

[Tinology]

That's a relief because I'm busy populating 10 groups!

[simonk83]

What I want though it to be able to see in a terminal type window what it is doing - at least to prove it works.

That's what I'd like to be able to do as well. I'm a bit OCD about these things so I'd like it to be visible at all times if possible.

[Tinology]

I believe it is cleverer than that.

I added a single binary at first and tested with 1 day backfill and it downloaded a million headers or something - I repeated it after it had finished and it found another thousand or so.

Backfill I'm a little unsure on, basically I'll populate my index with 1 day of data then select which groups I want to go further back with?

E.g. a.b.mp3 Backfill is 15 means I go back 15 days of releases?

[lokiju]

I created an account just to say a very emphatic THANK YOU!!

I've wasted a lot of time messing around with trying to get this up and running on my Mac on my own and while I'm handy with technology in general I didn't have the drive in me to figure out all the solutions to all the road blocks I was running into.

This is a HUGE help and very much appreciated.

[Andre74]

Thanks for the write up! This saved my weekend All seems to be running great. I love having my own index.

I'm also new to all this so I've got a few questions. Does the automator script ever stop spinning? Does it just run forever and the scripts do the timings of the updates?

Did you ever get a version working where you can see all the events?

[simonk83]

Does anyone know if the _threaded scripts work on OSX? They're for Linux apparently but, you know, close enough

[De5perate]

Thanks to all those who responded - I am glad to have helped.

As for the automator - I am running it and it seems to work. I haven't had time to spend fiddling with it to get a version that I can visibly track. I hope it runs indefinitely repeating the process every 12 hours but I am not convinced. I am not au fait with this code (or very au fait with any code) but I cannot see a repeating loop in the script so therefore I cannot see it looping. However the description from the original Newznab script file states:

# call this script from within screen to get binaries, processes releases and
# every half day get tv/theatre info and optimise the database

Which sounds to me like it does repeat optimising everything every 12 hours (which coincides withe the 43200 (seconds) in the script)
So I guess we should ask them. Any volunteers?

Thanks again to all the responders.