Home network schema
This series of posts describes how to set up a custom router from scratch for a home network. The router takes full advantage of multiple Internet connections, seamlessly manages Internet traffic when one of the connections drops or behaves poorly, and provides some additional functionality like firewalls, adblocking, VPN tunnels for specific clients, bandwidth shaping, network flow monitoring etc.
Motivation
My primary Internet connection at home (Comcast) sometimes stops working for minutes/hours at a time. This is super annoying when working from home due to covid-19, especially since I’m on video calls most of the day. Calling Comcast merely results in an automated voice stating that they are aware of the issue and their technicians are hard at work fixing it. (Yeah, right!) So I recently added a second Internet connection (AT&T) and this triggered the need to completely revamp the router setup I have been using so far.
Before the second Internet connection came along, the router box at the center of the above image was running pfsense. However, in order to properly and seamlessly utilize the second Internet connection, multi-wan load balancing and failover is essential. This is indeed available in pfsense, but in practice it involves compromises: Multi-wan load balancing does not play well with https and other security-minded servers, since they do not like traffic for a session coming from two different IP addresses. This is a known issue mentioned in the pfsense docs and is, strictly speaking, not a bug in pfsense. To avoid this, either traffic to https sites needs to be routed through a single WAN connection despite a load-balanced multi-wan configuration, or load-balancing needs to be foregone altogether and only failover should be utilized i.e. normally use one Internet connection only and when it fails, switch over to the other Internet connection. In both of these scenarios, full utilization of the second Internet connection would not be possible – it would merely be a backup. Why accept that? Why not find a way to fully utilize the second Internet connection and share it with all devices on the home network?
Intended solution
Key enabler: After extensive searching, the only solution I came across is a VPN service named Speedify. The Speedify client, as far as I understand it, creates separate connections – one through each Internet connection – to a single Speedify server, which then becomes your jumping off point to the rest of the Internet. Thus, it utilizes both Internet connections, providing seamless load balancing and failover when one of the Internet connections goes down or starts losing packets. The Speedify client is available on Windows, Mac OS, and Linux; it creates a virtual network interface through which Internet traffic is routed. Since the client is officially supported on Debian/Ubuntu and derivatives, I decided to install Ubuntu server on my router and configure it with all the desired functionality.
Desired functionality: Must have DNS, DHCP, Firewall, Multi-WAN Load Balancing and Failover, Blocking Advertisements and other malicious activity. Nice to have Network flows and event monitoring, creation of a guest network through a USB WiFi Dongle configured as an access point, additional VPN clients to tunnel specific traffic through specific VPN servers, advanced policies and routing to determine the interfaces through which specific devices on the home network should reach the Internet (e.g. the smart TV may need to bypass any VPNs for content services to work). Tertiary wishlist Traffic QoS and prioritization e.g. to prioritize conference calls if bandwidth is limited, intrusion detection/prevention, and a network-wide antivirus. Interestingly, pfsense provides all of the desired functionality, with the exception of effective multi-wan utilization, so I could have stuck with pfsense on the router and used the Speedify client directly on my laptop together with a USB WiFi dongle that connects to the secondary Internet provider’s wireless SSID <– But where’s the fun in that? Also, that would mean only my laptop gets access to seamless multi-wan and failover, leaving all other devices to suffer the vagaries of Comcast.
Hardware: The router box is a Protectli Vault 6 Port (Gigabit) Mini PC – Intel Dual Core i3, AES-NI, with 4GB RAM. The wireless router is an Asus RT-AC3200 device running the Asus Merlin custom firmware and is configured to be in Access Point mode. The Comcast modem is a Netgear C6300 device and the AT&T modem is an ARRIS BGW210. My default position is that an ISP modem, like the ISP, is an untrustworthy, exploitative, locked-down piece of crap. So I switch off their wireless capabilities, and run them into a custom router with a strong firewall, a privacy-friendly VPN client, and a non-ISP adblocking DNS, all of which lead to a “sanitized” Internet signal to which the home devices connect. For extra privacy, I can use a second VPN (shoutout to AirVPN and Mullvad) on a client machine if needed.
Read Part 2 for the technical software setup.