Cfrtjryk

I'd like to override part of the (IPSec) routing table

(routing to 10.108.0.0/16 locally via eth0, not via IPSec tunnel)

my IPSEC config

conn vpc

    type=tunnel

    authby=secret

    left=172.16.0.200

    leftid=x.x.x.x

    leftsubnet=172.16.0.0/16

    leftfirewall=yes

    right=y.y.y.y

    rightsubnet=10.0.0.0/8

    #pfs=yes

    auto=start

As You can see, over the tunnel the 10.0.0.0/8 is routed

# ip r s t all

10.0.0.0/8 via 172.16.0.1 dev eth0  table 220  proto static  src 172.16.0.200 

default via 172.16.0.1 dev eth0 

10.108.0.0/16 via 172.16.0.1 dev eth0 

172.16.0.0/24 dev eth0  proto kernel  scope link  src 172.16.0.200 

broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src 127.0.0.1 

local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 127.0.0.1 

local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 127.0.0.1 

broadcast 127.255.255.255 dev lo  table local  proto kernel  scope link  src 127.0.0.1 

broadcast 172.16.0.0 dev eth0  table local  proto kernel  scope link  src 172.16.0.200 

local 172.16.0.200 dev eth0  table local  proto kernel  scope host  src 172.16.0.200 

broadcast 172.16.0.255 dev eth0  table local  proto kernel  scope link  src 172.16.0.200 

unreachable default dev lo  table unspec  proto kernel  metric 4294967295  error -101

fe80::/64 dev eth0  proto kernel  metric 256 

unreachable default dev lo  table unspec  proto kernel  metric 4294967295  error -101

local ::1 dev lo  table local  proto none  metric 0 

local fe80::52:b2ff:fe65:b0fe dev lo  table local  proto none  metric 0 

ff00::/8 dev eth0  table local  metric 256 

unreachable default dev lo  table unspec  proto kernel  metric 4294967295  error -101



# ipsec statusall

Listening IP addresses:

  172.16.0.200

Connections:

     vpc:  172.16.0.200...x.x.x.x  IKEv1/2

     vpc:   local:  [x.x.x.x ] uses pre-shared key authentication

     vpc:   remote: [y.y.y.y] uses pre-shared key authentication

     vpc:   child:  172.16.0.0/16 === 10.0.0.0/8 TUNNEL

Security Associations (1 up, 0 connecting):

     vpc[3527]: ESTABLISHED 30 minutes ago, 172.16.0.200[x.x.x.x]...y.y.y.y

     vpc{1}:   172.16.0.0/16 === 10.0.0.0/8

I've specifically added the

    #ip r a 10.108.0.0/16 via 172.16.0.1



10.108.0.0/16 via 172.16.0.1 dev eth0

I hoped it would catch 'before' the table 220, but

but traffic still goes through IPSec tunnel.
I must be missing some layer.
I know I could change rightsubnet=10.0.0.0/8 to rightsubnet=10.0.0.0/16
but I'd like to change only one route

Just checking the

# ip xfrm policy

src 10.0.0.0/8 dst 172.16.0.0/16 

    dir fwd priority 1955 

    tmpl src x.x.x.x dst 172.16.0.200

            proto esp reqid 1 mode tunnel

src 10.0.0.0/8 dst 172.16.0.0/16 

    dir in priority 1955 

    tmpl src x.x.x.x dst 172.16.0.200

            proto esp reqid 1 mode tunnel

src 172.16.0.0/16 dst 10.0.0.0/8 

    dir out priority 1955 

    tmpl src 172.16.0.200 dst x.x.x.x

            proto esp reqid 1 mode tunnel

src 0.0.0.0/0 dst 0.0.0.0/0 

    socket in priority 0 

src 0.0.0.0/0 dst 0.0.0.0/0 

    socket out priority 0 

src 0.0.0.0/0 dst 0.0.0.0/0 

    socket in priority 0 

src 0.0.0.0/0 dst 0.0.0.0/0 

    socket out priority 0 

src ::/0 dst ::/0 

    socket in priority 0 

src ::/0 dst ::/0 

    socket out priority 0 

src ::/0 dst ::/0 

    socket in priority 0 

src ::/0 dst ::/0 

    socket out priority 0 

maybe I can change something here

I'd like to route 10.108.0.0/16 via local eth0, not via IPSec tunnel

EDIT I've extended the policy with:

ip xfrm policy update dir in src 172.16.0.0/16 dst 10.108.0.0/16

ip xfrm policy update dir out src 172.16.0.0/16 dst 10.108.0.0/16

ip xfrm policy update dir fwd src 172.16.0.0/16 dst 10.108.0.0/16



# ip xfrm policy

src 10.0.0.0/8 dst 172.16.0.0/16 

    dir fwd priority 1955 

    tmpl src 54.77.116.107 dst 172.16.0.200

        proto esp reqid 1 mode tunnel

src 10.0.0.0/8 dst 172.16.0.0/16 

    dir in priority 1955 

    tmpl src 54.77.116.107 dst 172.16.0.200

        proto esp reqid 1 mode tunnel

src 172.16.0.0/16 dst 10.0.0.0/8 

    dir out priority 1955 

    tmpl src 172.16.0.200 dst 54.77.116.107

        proto esp reqid 1 mode tunnel

src 172.16.0.0/16 dst 10.108.0.0/16 

    dir fwd priority 0 

src 172.16.0.0/16 dst 10.108.0.0/16 

    dir out priority 0 

src 172.16.0.0/16 dst 10.108.0.0/16 

    dir in priority 0 

another try:

ip xfrm policy add dir out src 172.16.0.0/16 dst 172.16.0.1

ip xfrm policy add dir in src 172.16.0.0/16 dst 172.16.0.1 

ip xfrm policy add dir fwd src 172.16.0.0/16 dst 172.16.0.1



# ip xfrm policy 

src 172.16.0.0/16 dst 172.16.0.1/32 

    dir fwd priority 0 

src 172.16.0.0/16 dst 172.16.0.1/32 

    dir in priority 0 

src 172.16.0.0/16 dst 172.16.0.1/32 

    dir out priority 0 

src 10.0.0.0/8 dst 172.16.0.0/16 

    dir fwd priority 1955 

    tmpl src 54.77.116.107 dst 172.16.0.200

            proto esp reqid 1 mode tunnel

src 10.0.0.0/8 dst 172.16.0.0/16 

    dir in priority 1955 

    tmpl src 54.77.116.107 dst 172.16.0.200

            proto esp reqid 1 mode tunnel

src 172.16.0.0/16 dst 10.0.0.0/8 

    dir out priority 1955 

    tmpl src 172.16.0.200 dst 54.77.116.107

            proto esp reqid 1 mode tunnel

still it does not look like a good 'redirect'

I know this is probably not what you want to hear, but the best way to deal with IPsec and routing is to completely segregate the two out. Default IPsec in policy mode (they way Linux does it and looks like what you're using) has a tendency to "munge" layers making routing very murky. A better way to do this is to treat IPsec more like a typical logical network link: make IPsec deal with point-to-point communication, and layer GRE on top with dynamic routing protocols like OSPF and BGP (you can skip the dynamic routing if you want, but it's recommended).

In your case, instead of directly interfacing 172.16.0.0/16 with 10.0.0.8/8, you instead have the IPsec stack create a point-to-point link on a /30, or even a /31. For example, you could use 10.254.254.1/30 for the left IP and 10.254.254.2/30 for the right IP. Once that's established, create a GRE tunnel with inner IPs 10.100.100.1/30 for the local side, and 10.100.100.2/30 for the remote side (and use the aforementioned 10.254.254.0/30 IPs for the appropriate external side). Bonus: if you are able to run IPsec in transport mode, you can completely forgo the 10.254.254.0/30 part and use the actual IPs of the endpoints as the external portion of the GRE tunnel.

Now you have a bog standard ethernet interface (the GRE tunnel) and can do static routes very easily: simply point 10.0.0.8/8 via 10.100.100.2, and 172.16.0.0/16 at 10.100.100.1. Better yet: completely remove these static routes and have OSPF or BGP handle routing automatically for you (see quagga for this).

Benefits: You can easily add routes at any time without reconfiguring the underlying IPsec stack at all, since it's completely isolated out of the routing. You can run any service that expects a standard ethernet interface without weirdness (iptables is a perfect example of this). You can easily accomplish multiple divergent and redundant pathways by leveraging BGP and multiple IPsec tunnels, and completely avoid the somewhat byzantine IPsec high availability schemes. But the best benefit is that you've now placed IPsec in a very compartmentalized place (securing traffic over a link) that is easily expanded into upper network layers without any additional configuration.

IPsec is a really flexible protocol, and because of that a lot of stuff ends up getting sucked into its vortex. At a certain point it's really better to have IPsec stick to what it does best and leverage higher level network stack concepts rather than tossing the kitchen sink at it.