[tech] Beginnings of a 4G backup link, and policy-based routing

[James Arcus]

Hi all,

Good news: source-based routing is working. Any packet leaving Murasoi 
sent from a 192.168.4.0/24 address gets directed out the 4G link via 
192.168.4.1. So far that's only available to Murasoi itself.

Unfortunately, the 4G link is on CGNAT (i.e. doesn't even have 1 public 
IPv4 address) and I can't manage to get inbound working via IPv6 either. 
(Maybe it's filtered either by Telstra or the Netgear modem?) Either 
way, looks like we'll need an intermediary of some kind.

What are people's thoughts on using cloud-mooneye for that purpose? It's 
globally accessible and its reliability is untied to uni. We could set 
up a Wireguard or other VPN site-to-site tunnel.

Cheers,

James [MPT]

On 18/4/20 12:50 am, James Arcus wrote:
>
> Hi all,
>
> I was able to get to the clubroom today and hook up the 4G modem with 
> the new SIM. Here are the details so far:
>
>   * The 4G modem is 192.168.4.1/24 with DHCP off, plugged into an
>     access port of the switch Kerosene
>   * Murasoi's spare RJ45 is 192.168.4.2, static, with that port on
>     Kerosene also set to an access port
>   * Both those access ports have been added to VLAN 42, and are
>     separate from the rest of the network
>   * The spare port on Murasoi is `eth0`, with `eth1` being the regular
>     uplink and `eth2` being the 10G fibre trunk to the rest of the
>     clubroom
>   * I've firewalled off the 4G link from forwarding, so it can't
>     inadvertently leak traffic into or out of the network
>   * I've confirmed the 4G link is functional, e.g. via `ping -I eth0
>     8.8.8.8`
>   * Murasoi can talk to the modem, but not yet to the wider internet
>     until policy-based routing is set up
>
> A bit more on that last point:
>
> Right now, Murasoi knows to send any traffic destined for the rest of 
> 192.168.4.0/24 out `eth0` with a source address of 192.168.4.2. But if 
> I want to send a packet to, say, Google's 8.8.8.8, it'll hit the 
> default route and be sent via the UWA uplink. Even if my software 
> forces a source address of 192.168.4.2, that route will be hit and UWA 
> will end up with a packet it doesn't know how to deal with. That's 
> because normal routing only consideres the destination, not the source.
>
> Some software, like `ping`, can force using a specific interface as 
> well/instead of a source address. That's how I could test the link's 
> functionality. But most software can't. What we need is to make some 
> rules about which source addresses go out which routes. That's called 
> _policy-based routing_, and it's the job of commands like `ip rule`.
>
> I'll be working on getting that started tomorrow.
>
> Cheers,
>
> James [MPT]
>
> PS: PBR is a very flexible technique. UCC is lucky to have a fast, 
> low-latency and unmetered connection to the net, so we don't really 
> need anything more. This backup link is intended only in the case of 
> serious issue, and isn't meant to carry normal traffic even then. But 
> if we had multiple links with different pros and cons, PBR lets you 
> determine which traffic goes where, based on a range of criteria.
>
> PPS: If you're reading all this and wondering what the heck I'm going 
> on about, feel free to pester me on IRC/Discord or in reply. I'll see 
> if I can get it explained better.
>
>
> _______________________________________________
> List Archives: http://lists.ucc.asn.au/pipermail/tech
>
> Unsubscribe here: https://lists.ucc.gu.uwa.edu.au/mailman/options/tech/jimbo%40ucc.asn.au
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20200419/7313862d/attachment.htm