OpenWRT Raspberry Pi Docker & VLAN Project

OpenWRT🥧 on PoE!
  • Using its single gigabit ethernet LAN port to manage/route traffic
  • Connected to a managed gigabit switch with attached devices
  • Running Docker inside OpenWRT VLANs
  • [Docker] Plex server with mapped SSD drive
  • [Docker] Ad blocking managed by PiHole
  • DNS over HTTPS upstream servers configured for PiHole
  • OpenVPN client installed
  • [Docker] Transmission torrent client with forced traffic via VPN
  • Policy-based routing for network separation (LAN via WAN, Transmission via VPN)
Overall Design Layout

Items Used

Installing the PoE Hat

The Pi 4 gets a little toasty without a heatsink or fan. There are PoE Hats which include a fan, but since I already had a case, I wanted to try and work with it. If you’ve looked at the case already, then you would notice that the hat won’t fit the case without some sort of altering. First thing needed was a GPIO header extender so that the PoE hat can access the pins, but unfortunately the holes in the case didn’t allow for a normal square 4 pin extension to fit for the PoE headers, so I had to make my own.

Download & Install OpenWRT

Initially I used this guide to install OpenWRT. It allowed me to use the entire 64GB via EXT4 partition instead of the preset 100MB. Although a problem I had was not all x64 ARM packages supported EXT4, especially Docker (which I need). So I had to revert to SquashFS and then expand the partition.

opkg update
opkg install cfdisk resize2fs
cfdisk /dev/mmcblk0
  • Set the size:
resize2fs /dev/loop0

VLAN Setup

It was this guide which really got me thinking that the Pi could be a pretty awesome home router. My problem has always been not having enough power sockets in the room for which this would be located. After reading this I figured if I could get a PoE hat, technically I don’t need an extra power socket. Hazaar!

Managed Switch

Following that guide, I decided to set up my switch as follows:

  • ports that are UnTagged
    VLAN 1: 2–7
    VLAN 2: 8
  • ports that are Blank
    VLAN 1: 8
    VLAN 2: 2–7

OpenWRT

I pretty much mirrored the config from that guide, but changed the network CIDR for my setup. Here’s a snippet of /etc/config/network:

config interface 'loopback'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
option device 'lo'
config interface 'lan'
option proto 'static'
option netmask '255.255.255.0'
option ipaddr '192.168.0.1'
option device 'eth0.1'
config interface 'wan'
option proto 'dhcp'
option device 'eth0.2'

WiFi Access Point

This part is pretty simple, I just enabled my access point to be only just that, and nothing else. Not even DHCP, since that will be handled by the Pi. Cable goes from the access point to the swtich, and I’m sure you can figure out how to set an SSID and password yourself.

Adding the USB Drive & Creating Samba Shares

In order to access the 1TB SSD drive remotely (where my media files are stored for Plex) from my Windows PC I need to install Samaba and create some file shares. This will be installed from a package inside OpenWRT that I can then configure to share out folders on my LAN via the Raspberry Pi’s IP. I used this wiki as a reference for adding a USB drive. But to sum it up:

opkg update
opkg install ntfs-3g
mkdir -p /mnt/1TB && \
ntfs-3g /dev/sda1 /mnt/1TB -o rw,lazytime,noatime,big_writes && \
ls -la /mnt/1TB
sleep 1
ntfs-3g /dev/sda1 /mnt/1TB -o rw,lazytime,noatime,big_writes
exit 0
opkg install samba4-server samba4-client luci-app-samba4
  • Use Luci to set up your shares: Luci -> Services -> Network Shares

Docker

I’m a big fan of Docker. I had been using my Pi 4 with Ubuntu running Docker, hosting a bunch of containers (such as PiHole, Plex and Transmission), and using Portainer to manage them. I still wanted to use these services, but in a much more controlled environment via split networks and routing.

Install Docker

This is pretty straight forward with the following packages:

opkg update
opkg install dockerd docker-compose luci-app-dockerman kmod-macvlan

Install Plex

This setup is pretty simple, for now. I don’t really need it to be separated from any other network, it can just sit on the device and connected via Docker bridge. On my 1TB SSD I have a subfolder called docker, for which I map all my container configurations to; it saves me a headache when having recreate everything should I need to. Also on that drive are some folders used to host music and other media. For now, my Docker command is simple, and maybe I should really turn it into a docker-compose YAML script at some point.

docker run --detach \
--name plex \
--net=host \
--restart unless-stopped \
-e PGID=1000 \
-e PUID=1000 \
-e UMASK=022\
-e VERSION=docker \
-v /mnt/1TB/dMusic:/data/dmusic \
-v /mnt/1TB/docker/plex:/config \
-v /mnt/1TB/docker/plex:/transcode \
-v /mnt/1TB/Music:/data/music \
linuxserver/plex

Install PiHole …and do the other networking stuff to support it

So this part is where it starts to really get interesting, as we’ll be using VLANs with Docker to separate the network and make things a whole lot easier for DNS setup.

config route
option interface 'macvlan'
option target '192.168.30.3'
option netmask '255.255.255.255'
#!/usr/bin/with-contenv bash
set -e
echo "fixing routes"
ip route del default
ip route add default via 172.18.0.1

OpenWRT VLAN Setup

Thus, my /etc/config/network extended to ONLY include:

config interface 'vlan20'
option proto 'static'
option ipaddr '192.168.20.1'
option netmask '255.255.255.0'
option device 'eth0.20'
config device
option type 'macvlan'
option ifname 'eth0'
option mode 'bridge'
option name 'eth0.20'
option acceptlocal '1'
option ipv6 '0'

Docker Setup

Now that my MACVLAN/VLAN was set up, I can now deploy my container. Using the docker-compose.yml file below, I run docker-compose up -d pihole.

version: "3.3"
services:
pihole:
container_name: pihole
image: pihole/pihole:latest
hostname: pihole.lan
environment:
TZ: 'Europe/London'
WEBPASSWORD: 'asdf'
volumes:
- '/mnt/1TB/docker/pihole/pihole/:/etc/pihole/'
- '/mnt/1TB/docker/pihole/dnsmasq.d/:/etc/dnsmasq.d/'
cap_add:
- NET_ADMIN
restart: unless-stopped
networks:
lan20:
ipv4_address: 192.168.20.3
networks:
lan20:
name: lan20
driver: macvlan
driver_opts:
parent: eth0.20
ipam:
config:
- subnet: 192.168.20.0/24
gateway: 192.168.20.1
iptables -t nat -A PREROUTING -i eth0 -p udp --dport 53 -j DNAT --to 192.168.20.3
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 53 -j DNAT --to 192.168.20.3

DNS over HTTPS

What you are configuring here is an upstream server that the PiHole will turn to when the domain doesn’t exist in its blocklist. The data sent to that resolver will be encrypted and returned back to the PiHole, and finally back to the original caller.

opkg update
opkg install https-dns-proxy luci-app-https-dns-proxy

Docker #2

Install Transmission …and do the other networking stuff to support it

Now that we have a better understanding of Docker, MACVLANs and VLANs, suppose we can create Docker torrent container that is separated from the rest of the network. Say, make sure it connects out via a VPN and not over normal WAN showing our home IP. For this we’re going to use the same MACVLAN/VLAN setup as we did for the PiHole, but this time we don’t put it in the same firewall zone as our LAN. The problem with that, of course, is that our LAN is set to go out the WAN, so how does the container get out to the net? Via VPN. The magic of why this actually works, I can’t tell you, it still somewhat confuses me…

version: "3.3"
services:
transmission:
container_name: transmission
image: linuxserver/transmission:latest
hostname: transmission.lan
environment:
TZ: 'Europe/London'
volumes:
- '/mnt/1TB/docker/transmission/config/:/config'
- '/mnt/1TB/docker/transmission/downloads/:/downloads'
- '/mnt/1TB/docker/transmission/watch/:/watch'
cap_add:
- NET_ADMIN
restart: unless-stopped
networks:
internal:
lan30:
ipv4_address: 192.168.30.3
networks:
internal:
name: transmission_internal
driver: bridge
lan30:
name: lan30
driver: macvlan
driver_opts:
parent: eth0.30
ipam:
config:
- subnet: 192.168.30.0/24

Install OpenVPN

There are many great tutorials out there showing you how to set up OpenVPN on OpenWRT. I’m not going to reinvent the wheel, but I do recommend you having a look at this YouTube guide. Van Tech Corner has done a lot of videos around OpenWRT that are a great starting place to look.

Firewall Zone Configurations

Now head over to Luci -> Network -> Firewall and set some new zones, where it will end up looking something like:

TORVLAN
TORVPN
LAN2TOR

Policy Based Routing

Right now if you connect to VPN your LAN is now going out via your VPN (I think, I dunno, that day confused me), where you might not want that to happen. We want the container to, but not the LAN. There are a few tutorials out there which updates the route tables, directs traffic when/when-not needed which you can look read up on. A good example of this is this guide; it initially gave me the idea of splitting my network, and running a container out a VPN, but I couldn’t adjust it for my use case for my little monkey brain to comprehend. Thankfully Van Tech Corner to the rescue, again 😁 with this YouTube video. It seemed that this add-in was to make something hard, simple. And honestly, it really does.

END

I hope you found this tutorial useful, or perhaps generated ideas for a current project of your own. I’m not claiming to be an expert at any of this, so if I have something wrong or incorrect, please tell me and I will update where need be.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store