IPv4 exhaustion is real, wake up sheeple!
Kidding aside, I was utterly surprised to find out that one of my ISPs recently resorted to selling out their assigned IPv4 ranges, probably due to high market prices. I was aware that they yanked static IP addresses for residential customers some time ago, but this was something I could still work around via the DDNS daemon. However, in case when not enough IPv4 addresses are available, they switched to Carrier-grade NAT (CGNAT) and ruined my day 😢
# What the hell is “CGNAT”?
From Wikipedia (opens new window):
Carrier-grade NAT (CGN or CGNAT), also known as large-scale NAT (LSN), is a type of Network address translation (NAT) for use in IPv4 network design. With CGNAT, end sites, in particular residential networks, are configured with private network addresses that are translated to public IPv4 addresses by middlebox network address translator devices embedded in the network operator's network, permitting the sharing of small pools of public addresses among many end sites. This shifts the NAT function and configuration thereof from the customer premises to the Internet service provider network (though "conventional" NAT on the customer premises will often be used additionally).
The end result is very simple: multiple households will share an external IP address when behind CGNAT, effectively rendering port forwards on home routers obsolete. Since newly installed ISP router is not configured to forward any ports to its clients, this means that home networks cannot be reached from the outside any more. Yikes!
Obviously, as a big proponent of self-hosted services, I was devastated to find this out, especially since it was not communicated in a proper way: it just stopped working one day, without any explanation.
# VPN to the Rescue
CGNAT rendered my self-hosted VPN server useless, since it could not be accessed any more. It was reachable only within the original network (let’s call it LAN 2).
Luckily, I had a spare VPN server in another network (let’s call it LAN 1). In this case, ISP did provide (and obviously charged) a static IP address, so it could be used normally.
I had to abandon the idea of a VPN server in LAN 2, but I was able to switch to a VPN client that would connect to LAN 1 via its VPN server. OpenVPN (opens new window) provides both server/client functionality, so it was easy to disable the server service and setup a client profile.
systemctl stop openvpn@<server>
systemctl disable openvpn@<server>
Where <server> is the name of the local VPN.
New client profiles can be copied to /etc/openvpn/<vpn-name>.conf and they are immediately usable. But, first some configuration changes were in order.
# Configuring VPN Client Profile
First, I decided to disable routing of all traffic across VPN tunnel, since this was not my use case. I wanted to access local services in LAN 2, not make them see and use everything from LAN 1. This can be disabled by commenting out the following line in the conf file:
# If redirect-gateway is enabled, the client will redirect it's
# default network gateway through the VPN.
# It means the VPN connection will firstly connect to the VPN Server
# and then to the internet.
# (Please refer to the manual of OpenVPN for more information.)
#redirect-gateway def1
To guarantee that the resultant VPN server in LAN 1 was not compromised in any way, it’s a good idea to force the checks of the server certificate by turning on another option:
remote-cert-tls server
Since I had set up the authentication on LAN 1 VPN server, I had to make this process automatic so the service can work in a headless environment. To do this, first I created a separate user on the VPN server, with its own, strong password. This information can be stored in a file with /etc/openvpn/<vpn-name>.auth which has the following structure:
<username>
<password>
The authentication file is then defined back in the conf file with following directives:
auth-user-pass <vpn-name>.auth
auth-nocache
# Making VPN Connection Persistent
If everything was set up correctly, now it should be possible to start the connection to LAN 1 via systemd:
systemctl start openvpn@<vpn-name>
At this point, connection can be tested by inspecting assigned IP address from the LAN 1 subnet on the OpenVPN interface, and by trying to reach some server from it. If it works, just make sure to enable the service so the connection can be persistent:
systemctl enable openvpn@<vpn-name>
I even tried to disconnect the client right on the server, it simply reconnected automagically some seconds later. Cool!
# Routing Requests
With two LANs bridged, we just need to make sure that all requests for LAN 2 server are properly routed, if initiated from LAN 1.
In case of the LAN 1 VPN server, it can easily resolve the LAN 2 server address without any trouble. However, this will not be the case to any PCs in LAN 1, for example.
Let’s say that the VPN subnet is in the 192.168.2.0 range, with 255.255.255.0 netmask, while the LAN 1 VPN server address is 192.168.1.101. To set up a new route to the LAN 2 server, this command can be used on MacOS:
sudo route -n add -net 192.168.2.0/24 192.168.1.101
Note that this route will NOT persist on MacOS, so we need to make it permanent with the following command as well:
sudo networksetup -setadditionalroutes <interface> 192.168.2.0 255.255.255.0 192.168.1.101
Where <interface> is the correct network interface name, e.g. Ethernet. You can get all network interface names with the following command:
networksetup -listallnetworkservices
The routes are easily set up on modern OSs, with notable exception for mobile devices, e.g. Android. Since it requires root permissions, we must think out of box and set up the route in a different way. I found that the easiest way would be to connect the mobile device to the same VPN network, and the routes are set up automatically. Note that in this case VPN must NOT isolate its clients, allowing internal connections.
# Local Port Tunnelling via SSH
My first use case was trying to access a service in LAN 2 from LAN 1. Since the server in LAN 2 has an SSH service running, port tunnel was a natural choice.
First, you need to know the IP address of the target service, in my case it was a printer in LAN 2, let’s say with the 192.168.0.15 address. Next, we also need the target port number, which in my case was 631 (IPP).
On a server in LAN 1, a local port tunnel can be created with the following command:
$ ssh -o LogLevel=VERBOSE -N -L 6631:192.168.0.15:631 <user>@<server>
Authenticated to <server> ([<server>]:22).
Where:
-o LogLevel=VERBOSEactivates some logging, so you can be sure the tunnel is working-Nswitch means there will be no additional command execution, hiding the remote prompt<user>@<server>a valid SSH user for the server in LAN 2.
This command opens a port 6631 on localhost or 127.0.0.1, which forwards to the target service on port 631. Higher local port numbers are advised if you intend to run the command under normal user. Otherwise, you might need to prepend the command with sudo.
Or, in case you want the tunnel accessible by other machines in LAN 1, which do not necessarily have the route to LAN 2 (i.e. mobile devices), you can bind it to the interface IP, let’s say 192.168.1.18:
$ ssh -o LogLevel=VERBOSE -N -L 192.168.1.18:6631:192.168.0.15:631 <user>@<server>
Authenticated to <server> ([<server>]:22).
In this case, the LAN 2 service will now be available in LAN 1 via 192.168.1.18:6631, and any device in LAN 1 should be able to access it.
To close the tunnel, simply press Ctrl+C.
# Web Browsing
Second use case was trying to browse the Internet by using the external IP address of the LAN 2 (let’s call it External IP 2). This is possible via SOCKS proxy tunnel, which can also be created by utilizing the SSH service on the LAN 2 server.
First, we need to set up the system to use the SOCKS proxy on the local port, let’s say 1337. On MacOS, this can be done via Settings > Network > Advanced and then switching on the SOCKS Proxy under Proxies tab. Enter localhost for the server name, and 1337 for port.
Then, to actually create the tunnel, simply open the connection to the server in LAN 2:
$ ssh -o LogLevel=VERBOSE -N -C -D 1337 <user>@<server>
Authenticated to <server> ([<server>]:22).
Afterwards, all web requests should just try to use the system settings for proxy, most probably this is already the case in your web browser if you didn’t modify its network settings. If you try to determine the external IP address, it should be External IP 2!
To further automate this, here is a short shell script to turn on the system proxy, activate the tunnel, and then wait for Ctrl+C signal before terminating and restoring network settings. It might not work for you without setting up passwordless SSH access to LAN 2 server, but that’s out of the scope of this guide.
#!/bin/zsh
echo Turning on proxy...
echo
networksetup -setsocksfirewallproxystate <interface> on
networksetup -getsocksfirewallproxy <interface>
control_c() {
echo
echo Caught Ctrl+C, exiting cleanly...
echo
echo Turning off proxy...
echo
networksetup -setsocksfirewallproxystate <interface> off
networksetup -getsocksfirewallproxy <interface>
echo
echo Done.
exit 0
}
echo
echo Setting up Ctrl+C trap...
echo
trap control_c INT
echo Running proxy tunnel...
ssh -o LogLevel=VERBOSE -N -C -D 1337 <user>@<server>
Where <interface> is the correct network interface name, e.g. Ethernet, and <user>@<server> a valid SSH user for the server in LAN 2.
# Conclusion
I was able to successfully replicate most of the use cases for a VPN with this approach, but not everything was possible.
For example, I still don’t have an easy way for mobile devices to use the SOCKS proxy, because apparently that is not readily available in current Android?! But, other than that, most other services are usable with acceptable performance, even without direct access to the target network.
Hopefully, this guide helps you a bit if you are in a similar situation as me. Good luck!