RHEL7 & CentOS7 policy based routing

speakbit Leave a comment

This page is copied from (tate.cx/rhel7-centos7-policy-based-routing/) which is offline now.

On Redhat or CentOS servers with multiple network interfaces, inbound packets will be sent to the default interface unless policy base routing is defined. Source or Policy based routing allows outbound packets from an interface to contain information that causes inbound packets to be returned to the configured interface .vs the default network interface. Additional information can be found in the RHEL7 Networking Guide.

Example

# ip addr
...
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:fc:45:9d brd ff:ff:ff:ff:ff:ff
    inet 162.253.43.134/24 brd 162.253.43.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fefc:459d/64 scope link
       valid_lft forever preferred_lft forever
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:2b:e7:89 brd ff:ff:ff:ff:ff:ff
    inet 10.13.96.161/19 brd 10.13.127.255 scope global ens4
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe2b:e789/64 scope link
       valid_lft forever preferred_lft forever
# ip route
default via 162.253.43.1 dev ens3
10.0.0.0/8 via 10.13.96.1 dev ens4
10.13.96.0/19 dev ens4  proto kernel  scope link  src 10.13.96.161
162.253.42.0/24 dev ens3  scope link
162.253.43.0/24 dev ens3  proto kernel  scope link  src 162.253.43.134
169.254.169.254 via 162.253.43.1 dev ens3

Test#1 – ping will work because the interface is the default interface

# ping -c1 162.253.43.134
PING 162.253.43.134 (162.253.43.134): 56 data bytes
64 bytes from 162.253.43.134: icmp_seq=0 ttl=60 time=5.825 ms

Test#2 – ping will fail because inbound & outbound traffic are using different interfaces

# ping 10.13.96.161
PING 10.13.96.161 (10.13.96.161): 56 data bytes
Request timeout for icmp_seq 0

Viewing the inbound ping traffic on the private (10.13.96.161) interface (ens4)

# tcpdump -n -i ens4 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens4, link-type EN10MB (Ethernet), capture size 262144 bytes
00:30:40.011105 IP 64.31.0.51 > 10.13.96.161: ICMP echo request, id 56759, seq 0, length 64

The problem and solution…
By default, Linux discards any inbound packet, if the outbound interface is not the inbound interface. To address this issue, a routing rule is needed to direct inbound traffic to the desired interface. This solution is persistent across reboots. If you have multiple interfaces that require policy based routing, repeat steps 3 to 6.

Step#1 – Install NetworkManager-config-routing-rules

# yum install NetworkManager-config-routing-rules

Step#2 – Enable NetworkManager-dispatcher

# systemctl enable NetworkManager-dispatcher.service
# systemctl start NetworkManager-dispatcher.service

Step#3 – Create a new routing table entry (ens4) for each additional interface (ens4)

# echo '500     ens4' >> /etc/iproute2/rt_tables

Step#4 – Next, specify the rules and routes desired for each additional interface

# vi /etc/sysconfig/network-scripts/rule-ens4
from 10.0.0.244/32 table ens4
to 10.0.0.244/32 table ens4

# vi /etc/sysconfig/network-scripts/route-ens4
default via 10.10.10.1 dev ens4 table ens4

Step#5 – After modifying the rule or route files

# nmcli connection reload

Step#6 – Restart network interface

# nmcli connection down ens4
# nmcli connection up ens4

To reset the routing configuration, flush the routing table

# ip route flush table ens4

Test#1 – Validate ens4 routing table

# ip route list table ens4
default via 10.13.96.1 dev ens4
10.13.96.0/19 dev ens4  proto kernel  scope link

Test#2 – Validate ens4 source rule

# ip rule
0:	from all lookup local
32765:	from 10.13.96.0/19 lookup ens4
32766:	from all lookup main
32767:	from all lookup default

Test#3 – Test source route
# ping -c1 10.13.96.161
PING 10.13.96.161 (10.13.96.161): 56 data bytes
64 bytes from 10.13.96.161: icmp_seq=0 ttl=60 time=5.511 ms

# tcpdump -n -i ens4 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens4, link-type EN10MB (Ethernet), capture size 262144 bytes
00:47:31.496909 IP 64.31.0.51 > 10.13.96.161: ICMP echo request, id 26040, seq 0, length 64
00:47:31.496969 IP 10.13.96.161 > 64.31.0.51: ICMP echo reply, id 26040, seq 0, length 64