HOWTO: Improve your DL Speed

Feel free to talk about anything and everything in this board.
Post Reply
Strontium
Newbie
Newbie
Posts: 3
Joined: September 21st, 2010, 10:01 am

HOWTO: Improve your DL Speed

Post by Strontium »

Here is a way for people like me to improve DL speed if you are willing to throw money at it.

Usually people tell you to get a faster link, but that does not work if you already have the fastest link available in your country.  In my case the maximum I can get is 1Mbit/s.

So What I did was get a second DSL link at 1Mbit for a theoretical aggregate bandwidth of 2Mbit which might not sound like much but now I should have enough bandwidth to retrieve in real time, that is a 1/2 hour episode takes approximately 1/2 an hour to DL.

The problem is however SAB has no ability to spread traffic over multiple interfaces, due to the fact that the python library for handling news doesnt seem to have that ability (at least on first glance) and its a low priority feature so its not worth throwing development effort at it, and having solved the problem to my satisfaction below would agree with it, a general solution inside SAB to route over multiple interfaces would be non trivial.  The answer of course, which seems obvious to me now but took me a few days to arrive at,  is to use destination based routes (I am using Linux, how you do this on Win or Mac is unknown to me).

Basically all you need to do is:

1. Get an account on 2 DIFFERENT news servers. [2 accounts total for 2 network connections].
2. Setup your second network port so it works and you can send stuff through it by doing something like:

Code: Select all

ip route flush table 10
ip route add default via 192.168.2.254 dev eth0 table 10
ip rule add from 192.168.2.0/24 table 10

ip route flush table 20
ip route add default via 10.0.0.1 dev eth1 table 20
ip rule add from 10.0.0.0/24 table 20
Basically what this does is makes routes and rules that mean if you use a source address of 192.168.2.*** it will route through gateway 192.168.2.254 and if you use a source address of 10.0.0.* it routes through gateway 10.0.0.1 both of which are separate dsl boxes

Then you set up a route to make one of your news servers always route over your second link.  Assume eth0 is your default link that all other internet traffic goes over and you want to route giganews over eth1 all you need to do now is:

Code: Select all

sudo route add -host news.giganews.com gw 10.0.0.1
now start SAB again.

If everything is ok all traffic to and from giganews will run over eth1 and traffic to and from your other news server will be over eth0

For this to work each server needs a unique name.  This scales so you could have 4 Internet connections and route to 4 different news servers.  starts to get costly but the choice is there and it scales up perfectly.  This could even save money in some cases because sometimes its cheaper to get 2 internet connections than 1 that is twice the speed or has twice the bandwidth allocation.

To adapt to your own circumstances change server names, ip's, etc.  Hope this helps someone.

PS.  I would like to put this Howto up on the Wiki manual in the howto's but its read only :(
Last edited by Strontium on September 29th, 2010, 12:30 am, edited 1 time in total.
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: HOWTO: Improve your DL Speed

Post by shypike »

Great suggestion, if only for the happy few.
I'll be happy to put it on the Wiki for you.
User avatar
sander
Release Testers
Release Testers
Posts: 9061
Joined: January 22nd, 2008, 2:22 pm

Re: HOWTO: Improve your DL Speed

Post by sander »

shypike wrote: Great suggestion, if only for the happy few.
... or if you have a neighbour who's sharing his/her WLAN ... ;-)
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
User avatar
sander
Release Testers
Release Testers
Posts: 9061
Joined: January 22nd, 2008, 2:22 pm

Re: HOWTO: Improve your DL Speed

Post by sander »

@strontium,

your first line says "ip route flush table 20". Shouldn't that be "ip route flush table 10"?
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
User avatar
adaemox
Release Testers
Release Testers
Posts: 33
Joined: March 12th, 2009, 3:28 pm
Location: Cascadia

Re: HOWTO: Improve your DL Speed

Post by adaemox »

That's really neat. Don't think I'll ever have a need for it, but neat none the less.
Adaemox

SABnzbd+, Episode Butler, VDPAU, and Sick Beard Tutorials @ http://www.ainer.org

Ubuntu and Firefox Information at http://www.ainer.org
Strontium
Newbie
Newbie
Posts: 3
Joined: September 21st, 2010, 10:01 am

Re: HOWTO: Improve your DL Speed

Post by Strontium »

sander wrote: @strontium,

your first line says "ip route flush table 20". Shouldn't that be "ip route flush table 10"?

Indeed it should...
dualnic
Newbie
Newbie
Posts: 8
Joined: October 2nd, 2010, 8:41 am

Re: HOWTO: Improve your DL Speed

Post by dualnic »

does any one know if it possible to use two ipv6 servers ? one for each card
Jabroni
Newbie
Newbie
Posts: 7
Joined: July 27th, 2010, 8:15 pm

Re: HOWTO: Improve your DL Speed

Post by Jabroni »

I use a completly different approach :)

First of all the way I acheive the balance is thru socks proxifying. I created a lil' program that balances socks requests thru each socks proxy I have (one per each WAN connection). So basiclly you can have as many WANs as you want and on different networks (in my case I balance in 2 dsl at home, and 1 i have at my office).

Now on the other end, I have a dedicated box which runs netcat which bounces all news traffic to the news server, so all the requests the news server seems like if they were received by the same PC :) It aint a easy task to do, but now I can see ~1.5MBytes/s download from news, which I would never see on a single link here in my country
dualnic
Newbie
Newbie
Posts: 8
Joined: October 2nd, 2010, 8:41 am

Re: HOWTO: Improve your DL Speed

Post by dualnic »

hey Jabroni what software do you use to do that?
Jabroni
Newbie
Newbie
Posts: 7
Joined: July 27th, 2010, 8:15 pm

Re: HOWTO: Improve your DL Speed

Post by Jabroni »

You can use a free software called Anonymous Guest (url hxxp://www.spszone.com/anguest/) and to sockisfy the connection search the forum for the ongoing thread.
DEADBEEF
Newbie
Newbie
Posts: 4
Joined: March 1st, 2011, 9:49 am

Re: HOWTO: Improve your DL Speed

Post by DEADBEEF »

This idea is pretty intriguing as I have multiple WAN connections available and been after a way to combine their available bandwidth without going down the bonded ADSL route (which isn't available on my exchange)

Surely though, rather than paying for two separate usenet providers you can just use separate IPs if your provider supplies multiple region based news server IPs.

IE, I subscribe to bintube, they provide separate IP addresses for different regions. Could I not set my route tables so that any requests to eu.bintube.com go via eth0, while requests to us.bintube.com go via eth1?
DEADBEEF
Newbie
Newbie
Posts: 4
Joined: March 1st, 2011, 9:49 am

Re: HOWTO: Improve your DL Speed

Post by DEADBEEF »

Ok, I just had a play around with this yesterday and it was pretty simple to set up.
Here's a complete Windows/linux tutorial from start to finish.....

How to set up usenet downloading over multiple WAN connections for ludicrous speed downloading.

What you need:
  • A usenet account (duh), preferably one which allows multiple simultaneous connections from different IPs (block account), or at least one that doesn't check.
  • An installed and working binary downloader, one which supports multiple servers to be configured (such as sabnzbd).
  • Access to multiple WAN connections. One of the following will suffice:
            * Multiple LAN ports on your server machine each connected to an ADSL/cable modem.
           * Multiple WiFi adapters connected to different access points.
           * A router which has VLAN support and can be connected to multiple WANs, either via ethernet or WLAN (most DD-WRT compatible routers can do this).



Step 1: Figure out your gateway IPs.

Firstly you need to know the IPs of all the gateways you're connected to. If you know these already this will save you a bit of time. If not, do the following....

Windows: open a command prompt and type "ipconfig". This gives you a list of network adapters connected to your machine and the default gateway each one connects to the internet via. Write down the IP addresses of each gateway. If any of the gateway IP addresses clash, change them so they don't. If you can't change them see the bit at the bottom.

Linux: There's probably another way, but this is how I did it... Open a terminal session and type "nano /etc/dhcpc/*.info". Write down the gateway IP and interface name, press ctrl+X to go to the next adapter and do the same, repeat until you have the gateway IPs for each adapter.


Step 2: Get the IPs for your usenet provider.

Most usenet providers will give you a choice of domains to connect to based on your geographic location, you need the IP address each of these resolves to.
In your command prompt/terminal type "ping {domain}" where domain is one of the usenet server domains supplied by your provider. Write down the IPs they resolve to.


Step 3: Configure your reader.

This varies depending on which reader you're using, but basically you want to set up as many different server IPs as you have different gateways. On sabnzbd it's via the server tab and is pretty self explanatory.


Step 4: Route each server to a different gateway.

What we do here is instruct the networking stack to route request to different IPs via different gateways.

Windows: In your command prompt, type: "route add {server IP} {gateway IP}". Where {server IP} is the address of one of your providers usenet servers and {gateway IP} is one of your gateway IPs.
Repeat this for as many gateway/server IPs as you have available.
These will be reset when your machine reboots, as they're only temporary you can use them to check everything is working before making any permanent changes.
Once you're happy the routing is working properly repeat the command using "route -p" to add a registry entry making the rule persist between reboots.

Linux: In your console/terminal, type: "route add -host {server IP} gw {gateway IP} dev {device ID}". Where {server IP} is the address of one of your providers usenet servers, {gateway IP} is one of your gateway IPs, and {device ID} is the name of the interface associated with that gateway.
Repeat this for as many gateway/server IPs as you have available.
These will be reset when your machine reboots, as they're only temporary you can use them to check everything is working before making any permanent changes.
To make them permanent you'll need to either add the commands to an init script which runs on boot, or add some rules to your system's static route tables. The methods to do this vary depending on your distro so I'll leave that for you to figure out.


Step 5: There is no step 5

Notes:
If you're using windows and have gateway IPs which clash, you need to work out which adapter each gateway is connected to. Get a list of adapter IDs by typing "route print" and scrolling up a bit. Look under the heading of 'Interface List'. Your job now is to tie the number in the left column (the interface ID) with the gateway which uses it. Once you know which gateway is associated with each interface when you get to step 4 you can append the command with " if {interface ID}", where {interface ID} is the number in the left hand column of the interfaces list (usually 1-2 digits).


Anyhow, it seemed way more straightforward than that before I started typing it out, but there you go.

TL:DR.
  • Get IP addresses of your WAN gateways
  • Get IP addresses of your usenet servers
  • Add static routes sending each server IP to a different gateway IP.
Last edited by DEADBEEF on March 2nd, 2011, 12:24 am, edited 1 time in total.
Post Reply