3 Line ISP connection using iproute2

Hi, it’s Sami again, this time we will be setting up a 3 line ISP connection on our squid server.
This is intended for static routing only. We are working on setting it up dynamically.
As usual we will be using debian 5 (Lenny). Although this is not a fresh install! We are continuing our squid install for SSWC.

First of all install iproute2 since it is not in base.

apt-get install iproute2

Now we need a script to setup our routes.

First of all we will add the routing tables that we use.

echo "1 First_eth1" >> /etc/iproute2/rt_tables
echo "2 Second_eth2" >> /etc/iproute2/rt_tables
echo "3 Third_eth3" >> /etc/iproute2/rt_tables

Variables

P0_NET=192.168.1.0
P1_NET=192.168.16.0
P2_NET=192.168.67.0
P3_NET=192.168.40.0

P1=192.168.16.1
P2=192.168.67.1
P3=192.168.40.1

IF0=eth0
IF1=eth1
IF2=eth2
IF3=eth3

IP0=192.168.1.1
IP1=192.168.16.114
IP2=192.168.67.2
IP3=192.168.40.2

T1=First_eth1
T2=Second_eth2
T3=Third_eth3

Adding routes

ip route add $P1_NET dev $IF1 src $IP1 table $T1
ip route add default via $P1 table $T1
ip route add $P1_NET dev $IF1 src $IP1

ip route add $P2_NET dev $IF2 src $IP2 table $T2
ip route add default via $P2 table $T2
ip route add $P2_NET dev $IF2 src $IP2

ip route add $P3_NET dev $IF3 src $IP3 table $T3
ip route add default via $P3 table $T3
ip route add $P3_NET dev $IF3 src $IP3

Adding rules

ip rule add from $IP1 table $T1
ip rule add from $IP2 table $T2
ip rule add from $IP3 table $T3

ip route add $P0_NET dev $IF0 table $T2
ip route add $P1_NET dev $IF1 table $T2
ip route add 127.0.0.0/8 dev lo table $T2

ip route add $P0_NET dev $IF0 table $T3
ip route add $P2_NET dev $IF2 table $T3
ip route add 127.0.0.0/8 dev lo table $T3

ip route add $P0_NET dev $IF0 table $T1
ip route add $P3_NET dev $IF3 table $T1
ip route add 127.0.0.0/8 dev lo table $T1

Adding nexthops and weight.

ip route add default scope global \
nexthop via $P1 dev $IF1 weight 1 \
nexthop via $P2 dev $IF2 weight 1 \
nexthop via $P3 dev $IF3 weight 1

We had a lot of problems with this.
Just remember one important thing!
Running this with all eth{1,2,3} on the same network with the same gateway will NOT work.
Our gateway actually mixed up all of our interfaces and added the MAC address for eth1 to eth2 and eth3 in the arp table.

iptables is also needed for this configuration.
Something like this.

iptables -A INPUT -i eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -i eth2 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -i eth3 -m state --state ESTABLISHED,RELATED -j ACCEPT

Adding some masquerading.

iptables --table nat --append POSTROUTING --out-interface eth1 -j MASQUERADE
iptables --table nat --append POSTROUTING --out-interface eth2 -j MASQUERADE
iptables --table nat --append POSTROUTING --out-interface eth3 -j MASQUERADE