HardKernel's ODROIDH3 as a FreeBSD Firewall

Table of Contents

I recently I replaced a 1U ThinkServer with an ODROID-H3+ to resolve some performance issues. I had the idea to use the ThinkServer as my main router and use the ODROID as a testing firewall but I was certainly wrong to even use the 1U machine in the first place. HardKernel’s ODROID-H3 is a fantastic board which boasts some impressive specs:

Up to 64GB DDR4 memory
2 x SATA3 ports
M.2 PCIe 3.0 x 4 slot (which can be booted from)
2 X Realtek RTL8125B 2.5 Gigabit LAN port
An idle power consumption of ≃1..9W, ≃15W under CPU stress and ≃18W under CPU+GPU Stress
The H3 and H3+ are only different in CPU and graphics (See H3 and H3+ specs)

Because this board has 2 ethernet ports, it can be used as a router. And as I discussed in my previous blog post, it does not suffer the performance issues of PPPoE over igb(4) (Intel gigabit) interfaces, making it perfect for FTTP connections to your home ISP. My personal list of pros and cons for this hardware are below, its worth noting that these opinions are based off my use-case as a home router / firewall:

Pros

2.5Gb interfaces make this board suitable for some of the fastest home ISP speeds (as of 2023 in the UK)
Small form factor is suitable for home usage
Low power consumption
x86 architecture supports many operating systems

Cons

No onboard wireless card means that this board cannot be configured as a WAP and that you must use another device to serve wireless
Somewhat expensive (129USD for the H3 and 165USD for the H3+ as of August 2023)
The base FreeBSD GENERIC kernel’s Realtek drivers do not support the boards Realtek RTL8125B 2.5GbE out of the box

Overall, this is one of the best pieces of hardware I’ve used for routing. If your looking for a new device for your home router, consider the ODROID-H3. The x86 architecture of this board means that most operating systems can be used on it. Personally, I use FreeBSD for routers and firewalls. The FreeBSD handbook provides a better installation guide for FreeBSD than I ever could, and the ODROID-H3 Quick-start guide’s Installation section already provides a good walk-through for initially setting up the board. Thus, the remainder of this post will be about additional setup in FreeBSD for the ODROID-H3+.

Installing the 2.5Gb Realtek Drivers

Most of the FreeBSD installation on the board is the same as any other x86 system. However, the ODROID’s Realtek RTL8125B 2.5GbE LAN ports are not supported by the GENERIC kernel out of the box. There is a driver in the ports tree (realtek-re-kmod) but we will not have any internet connectivity on the board after the initial OS installation to install it. This leaves you with the following options:

Use an adapter to provide internet connectivity to the board and install the port
Download the source to a different x86 FreeBSD host, build it and put it on the board via an external storage device
Download the source to an external storage device, compile it and install on the board

Even if you install the driver manually you should install the port after you restore internet connectivity to the board to ensure you get updates:

# pkg install realtek-re-kmod

To install the driver manually, we need to compile the driver. The build will produce a module if_re.ko, whether you compile the module on the board after installing FreeBSD or a different x86 FreeBSD host is up to you. But you must be able to get the binary onto the board somehow:

Clone the source code
cd into the source directory and build it by invoking make. The build will produce a if_re.ko kernel module in the source root directory
On the ODROID, copy the if_re.ko file to /boot/modules
Then add the following lines to /boot/loader.conf

if_re_load="YES"
if_re_name="/boot/modules/if_re.ko"

To enable DHCP on the interfaces at boot add the following to /etc/rc.conf, or configure the IP addresses statically as you wish.

ifconfig_re0="DHCP"
ifconfig_re1="DHCP"

Finally reboot the system. Two interfaces re0 and re1 should be present on the system on reboot now (Remember to install the module through the ports tree after).

Kernel Build Configuration (optional)

Now that we’ve got WAN connectivity, let’s build a new kernel image for our hardware. This step is completely optional. While there is nothing wrong with running the GENERIC kernel, it is common practice in the BSD world to build kernels that are tailored towards the hardware you’re using. This can have several benefits, for example: If the kernel is built for only the specific hardware on the system, the kernel will not probe for other types of hardware, potentially decreasing boot times.

The kernel configuration chapter in the FreeBSD handbook provides a brief introduction to building custom kernels and its benefits.

My ODROID kernel build config

This kernel build configuration is tailored towards my use case for this board as a firewall/router. Thus you may not want to remove as much from the kernel build as I do here (for instance if you also to intend to use the onboard SATA). The most notable edits are below:

I don’t intend to use this board for storage other than the OS on the NVMe and so I will not be using the onboard SATAs. I do not compile any drivers for disks other than scbus (SCSI bus) and da (Direct attach drives) as they are needed for umass so I can still attach USB drives to the board if I need to.
Nor do include:
- Drivers for RAID controllers
- Sound support
- Support for virtualisation technologies (KVM, Xen, VirtIO)
- As we install the realtek driver ourselves and the board has no wireless interface, we do not need to install any networking drivers other than the iflib framework

The full build configuration for my ODROID on FreeBSD 13.2 is here:

The following kernel build configuration has been built and tested on only an ODROIDH3+ with FreeBSD 13.2 and is based on the 13.2 GENERIC configuration.

#
# GENERIC -- Generic kernel configuration file for FreeBSD/amd64
#
# For more information on this file, please read the config(5) manual page,
# and/or the handbook section on Kernel Configuration Files:
#
#    https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html
#
# The handbook is also available locally in /usr/share/doc/handbook
# if you've installed the doc distribution, otherwise always see the
# FreeBSD World Wide Web server (https://www.FreeBSD.org/) for the
# latest information.
#
# An exhaustive list of options and more detailed explanations of the
# device lines is also present in the ../../conf/NOTES and NOTES files.
# If you are in doubt as to the purpose or necessity of a line, check first
# in NOTES.
#
# $FreeBSD$

cpu		HAMMER
ident		ODROIDH3

makeoptions	DEBUG=-g		# Build kernel with gdb(1) debug symbols
makeoptions	WITH_CTF=1		# Run ctfconvert(1) for DTrace support

options 	SCHED_ULE		# ULE scheduler
options 	NUMA			# Non-Uniform Memory Architecture support
options 	PREEMPTION		# Enable kernel thread preemption
options 	VIMAGE			# Subsystem virtualization, e.g. VNET
options 	INET			# InterNETworking
options 	INET6			# IPv6 communications protocols
options 	IPSEC_SUPPORT		# Allow kldload of ipsec and tcpmd5
options		ROUTE_MPATH		# Multipath routing support
options		FIB_ALGO		# Modular fib lookups
options 	TCP_OFFLOAD		# TCP offload
options 	TCP_BLACKBOX		# Enhanced TCP event logging
options 	TCP_HHOOK		# hhook(9) framework for TCP
options		TCP_RFC7413		# TCP Fast Open
options 	SCTP_SUPPORT		# Allow kldload of SCTP
options		KERN_TLS		# TLS transmit & receive offload
options 	FFS			# Berkeley Fast Filesystem
options 	SOFTUPDATES		# Enable FFS soft updates support
options 	UFS_ACL			# Support for access control lists
options 	UFS_DIRHASH		# Improve performance on big directories
options 	UFS_GJOURNAL		# Enable gjournal-based UFS journaling
options 	QUOTA			# Enable disk quotas for UFS
options 	MD_ROOT			# MD is a potential root device
options 	NFSCL			# Network Filesystem Client
options 	NFSD			# Network Filesystem Server
options 	NFSLOCKD		# Network Lock Manager
options 	NFS_ROOT		# NFS usable as /, requires NFSCL
options 	MSDOSFS			# MSDOS Filesystem
options 	CD9660			# ISO 9660 Filesystem
options 	PROCFS			# Process filesystem (requires PSEUDOFS)
options 	PSEUDOFS		# Pseudo-filesystem framework
options 	TMPFS			# Efficient memory filesystem
options 	GEOM_RAID		# Soft RAID functionality.
options 	GEOM_LABEL		# Provides labelization
options 	EFIRT			# EFI Runtime Services support
options 	COMPAT_FREEBSD32	# Compatible with i386 binaries
options 	COMPAT_FREEBSD4		# Compatible with FreeBSD4
options 	COMPAT_FREEBSD5		# Compatible with FreeBSD5
options 	COMPAT_FREEBSD6		# Compatible with FreeBSD6
options 	COMPAT_FREEBSD7		# Compatible with FreeBSD7
options 	COMPAT_FREEBSD9		# Compatible with FreeBSD9
options 	COMPAT_FREEBSD10	# Compatible with FreeBSD10
options 	COMPAT_FREEBSD11	# Compatible with FreeBSD11
options 	COMPAT_FREEBSD12	# Compatible with FreeBSD12
options 	SCSI_DELAY=5000		# Delay (in ms) before probing SCSI
options 	KTRACE			# ktrace(1) support
options 	STACK			# stack(9) support
options 	SYSVSHM			# SYSV-style shared memory
options 	SYSVMSG			# SYSV-style message queues
options 	SYSVSEM			# SYSV-style semaphores
options 	_KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
options 	PRINTF_BUFR_SIZE=128	# Prevent printf output being interspersed.
options 	KBD_INSTALL_CDEV	# install a CDEV entry in /dev
options 	HWPMC_HOOKS		# Necessary kernel hooks for hwpmc(4)
options 	AUDIT			# Security event auditing
options 	CAPABILITY_MODE		# Capsicum capability mode
options 	CAPABILITIES		# Capsicum capabilities
options 	MAC			# TrustedBSD MAC Framework
options 	KDTRACE_FRAME		# Ensure frames are compiled in
options 	KDTRACE_HOOKS		# Kernel DTrace hooks
options 	DDB_CTF			# Kernel ELF linker loads CTF data
options 	INCLUDE_CONFIG_FILE	# Include this file in kernel
options 	RACCT			# Resource accounting framework
options 	RACCT_DEFAULT_TO_DISABLED # Set kern.racct.enable=0 by default
options 	RCTL			# Resource limits

# Debugging support.  Always need this:
options 	KDB			# Enable kernel debugger support.
options 	KDB_TRACE		# Print a stack trace for a panic.

# Kernel Sanitizers
#options 	COVERAGE		# Generic kernel coverage. Used by KCOV
#options 	KCOV			# Kernel Coverage Sanitizer
# Warning: KUBSAN can result in a kernel too large for loader to load
#options 	KUBSAN			# Kernel Undefined Behavior Sanitizer
#options 	KCSAN			# Kernel Concurrency Sanitizer

# Kernel dump features.
options 	EKCD			# Support for encrypted kernel dumps
options 	GZIO			# gzip-compressed kernel and user dumps
options 	ZSTDIO			# zstd-compressed kernel and user dumps
options 	DEBUGNET		# debugnet networking
options 	NETDUMP			# netdump(4) client support
options 	NETGDB			# netgdb(4) client support

# Make an SMP-capable kernel by default
options 	SMP			# Symmetric MultiProcessor Kernel
options 	EARLY_AP_STARTUP

# CPU frequency control
device		cpufreq

# Bus support.
device		acpi
device		smbios
options 	IOMMU
device		pci
options 	PCI_HP			# PCI-Express native HotPlug
options		PCI_IOV			# PCI SR-IOV support

options 	COMPAT_LINUXKPI

# Enable support for the kernel PLL to use an external PPS signal,
# under supervision of [x]ntpd(8)
# More info in ntpd documentation: http://www.eecis.udel.edu/~ntp

options 	PPS_SYNC

# ATA/SCSI peripherals
device		scbus			# SCSI bus (required for ATA/SCSI)
device		da			# Direct Access (disks)

# NVM Express (NVMe) support
device		nvme			# base NVMe driver
device		nvd			# expose NVMe namespaces as disks, depends on nvme

# atkbdc0 controls both the keyboard and the PS/2 mouse
device		atkbdc			# AT keyboard controller
device		atkbd			# AT keyboard
device		psm			# PS/2 mouse

device		kbdmux			# keyboard multiplexer

device		vga			# VGA video card driver
options 	VESA			# Add support for VESA BIOS Extensions (VBE)

device		splash			# Splash screen and screen saver support

# syscons is the legacy console driver, resembling an SCO console
device		sc
options 	SC_PIXEL_MODE		# add support for the raster text mode

# vt is the default video console driver
device		vt
device		vt_vga
device		vt_efifb
device		vt_vbefb

device		agp			# support several AGP chipsets

# PCCARD (PCMCIA) support
# PCMCIA and cardbus bridge support
device		cbb			# cardbus (yenta) bridge
device		pccard			# PC Card (16-bit) bus
device		cardbus			# CardBus (32-bit) bus

# Serial (COM) ports
device		uart			# Generic UART driver

# Pseudo devices.
device		crypto			# core crypto support
device		aesni			# AES-NI OpenCrypto module
device		loop			# Network loopback
device		padlock_rng		# VIA Padlock RNG
device		rdrand_rng		# Intel Bull Mountain RNG
device		ether			# Ethernet support
device		vlan			# 802.1Q VLAN support
device		tuntap			# Packet tunnel.
device		md			# Memory "disks"
device		gif			# IPv6 and IPv4 tunneling
device		firmware		# firmware assist module
device		xz			# lzma decompression

# The `bpf' device enables the Berkeley Packet Filter.
# Be aware of the administrative consequences of enabling this!
# Note that 'bpf' is required for DHCP.
device		bpf			# Berkeley packet filter

# USB support
options 	USB_DEBUG		# enable debug msgs
device		uhci			# UHCI PCI->USB interface
device		ohci			# OHCI PCI->USB interface
device		ehci			# EHCI PCI->USB interface (USB 2.0)
device		xhci			# XHCI PCI->USB interface (USB 3.0)
device		usb			# USB Bus (required)
device		ukbd			# Keyboard
device		umass			# Disks/Mass storage - Requires scbus and da

# Netmap provides direct access to TX/RX rings on supported NICs
device		netmap			# netmap(4) support

# evdev interface
options 	EVDEV_SUPPORT		# evdev support in legacy drivers
device		evdev			# input event device support
device		uinput			# install /dev/uinput cdev

# HID support
options 	HID_DEBUG		# enable debug msgs
device		hid			# Generic HID support
options 	IICHID_SAMPLING		# Workaround missing GPIO INTR support

Building and installing the kernel

On the ODROID, copy the above configuration (or use your own) to /usr/src/sys/amd64/conf/ODROIDH3
Then cd into /usr/src and run make buildkernel KERNCONF=ODROIDH3 to start the build.
When the build has finished you can install the kernel with make installkernel KERNCONF=ODROIDH3 - which will install the kernel to /boot/kernel and move the old kernel to /boot/kernel.old
Reboot the ODROID and ensure the new kernel works. Once confirmed you can remove /boot/kernel.old