Guide

Installing pfSense 2.6 on ZimaBoard

I backed the ZimaBoard Single Board Server project on Kickstarter in early 2021, a couple of months ago it finally arrived and the first project on the todo list was to try this as a replacement for my overkill pfSense server (Dell R210 II Server) which consumed ~100W compared to ~6W of the Zimabaord, a cost reduction of over £200 per year in electricity costs too. The ZimaBoard comes pre-installed with Casa OS on the onboard 32GB eMMC storage, but that can be overwritten with whatever software we want.

ZimaBoard 432 with an additional 2.5Gbps NIC installed

When I backed the project, I also bought an extra NIC as I needed 3 connections if I was going to have a backup WAN link, however, there isn’t a way to mount the PCIe Network Card into the ZimaBoard and keep it secured. The usual riser bracket also has to be removed as it would otherwise foul the case.

Note: This isn’t the setup I finally ended up running, but this is the journey I went on.

Installing pfSense

Here are the steps I followed to get pfSense 2.6.0 up and running on the ZimaBoard:

  • Download the latest pfSense USB Image from: https://www.pfsense.org/download/
    • Architecture: AMD64
    • Installer: USB Memstick Installer
    • Console: VGA
  • Write the USB Image to a USB Flash drive using Balena Etcher (or similar)
  • Attach a display and keyboard to the ZimaBoard
  • Insert USB Flash drive now containing pfSense installing image
  • Boot the ZimaBoard and follow the wizard to install pfSense, selecting the eMMC storage#
    • The ZimaBoard supports up to 2 x SATA drives, depending on the features planned to be used on pfSense, consideration should be given to using a SATA SSD or HDD.
  • After installation completes I had to enter the BIOS and change the Boot option to MMC.

Realtek Network Drivers

The two onboard NICs are identified as Realtek PCIe GbE Family Controller and worked out of the box, though FreeBSD 12 which pfSense 2.6 is built on doesn’t include the latest drivers. The left hand NIC was re0 and the other re1

The 2.5GbE network card (Realtek RTL8125B) wasn’t detected after installation and required updated Realtek drivers to be installed.

Updated 4 Dec 2022: Realtek have released driver version 197.

Update 19 Jan 2023: Realtek have released driver version 198.

Installing Realtek version 198 driver

if_re_load="YES"
if_re_name="/boot/modules/if_re.ko"
  • Reboot pfSense

Configuration Tweaks for pfSense

If you are planning on using pfSense as a basic firewall/router you should consider minimising the writes to the eMMC storage, though if you are planning on using additional packages like ntopng it will need more space than can be provided by a RAM Disk

  • System -> Advanced -> Miscellaneous -> RAM Disk Settings:
    • Check Use RAM Disks
    • Reboot pfSense.

Enable capabilities of the CPU (Intel Celeron N3450)

  • System -> Advanced -> Miscellaneous -> Cryptographic & Thermal Hardware:
    • Cryptographic Hardware: AES-NI CPU-based Acceleration
    • Thermal Sensors: Intel Core* CPU on-die thermal sensor

It’s unclear if pfSense is correctly supporting the CPU clock boost capabilities of the CPU, as it shows the clock speed as 1.1GHz. However, to give it a chance of working:

  • System -> Advanced -> Miscellaneous -> Power Savings:
    • Check: Enable PowerD
    • Set the policies to: Hiadaptive

Problems

Networking Issues

I ran this configuration for several weeks and noticed I was having problems with my backup network connection (via Starlink) going offline, sometimes disabling and re-enabling the interface fixed it, sometimes a reboot of Starlink would fix it, sometime pfSense had to be rebooted. dmesg filled upa with a stream of errors:

arpresolve: can't allocate llinfo for 100.64.0.1 on re1

Where re1 is the onboard GbE NICs connected to the Starlink, not the 2.5GbE NIC.

DNS errors from unbound were also being constantly logged to /var/log/resolver.log

May 25 22:26:18 pfsense-zimaboard unbound[75327]: [75327:3] error: recvfrom 26 failed: Protocol not available
May 25 22:26:18 pfsense-zimaboard unbound[75327]: [75327:1] error: recvfrom 24 failed: Protocol not available

Given the move from the Dell R210 II server to the ZimaBoard was both a hardware move and an upgrade from pfSense 2.4 to 2.6, it’s unclear if one or the other was the cause of the problems.

Solution

I decided to stop using any of the Realtek NICs and instead take the Intel Quad port NIC (Intel(R) PRO/1000 ET 82576 (Quad Copper) I had in the Dell server and use that. The card is significantly larger and so I also created a new wall mount to allow the Intel NIC to be secured.

Since switching to the Intel quad port NIC, I’ve not experienced any of the issues above.

ZimaBoard and Intel quad port NIC
ZimaBoard 432 with an additional Intel quad port NIC installed

BIOS & CPU Issues

pfSense System Information
  • BIOS version information is corrupted
  • DMI table is broken (possible cause of BIOS version info)
dmidecode -t processor -t cache
# dmidecode 3.3
Scanning /dev/mem for entry point.
SMBIOS 3.0.0 present.
Invalid entry length (0). DMI table is broken! Stop.
  • The Intel N3450 CPU has a base clock frequency of 1.1GHz with a burst frequency of 2.2GHz. However, pfSense only shows the clock running at 1.1GHz, it’s unclear if this is just cosmetic or if it really isn’t bursting.
  • The speed issue seems similar to a post here on Reddit from 2019
  • The supported CPU speeds listed are:
sysctl dev.cpu.0.freq_levels
dev.cpu.0.freq_levels: 1101/0 1100/0 1000/0 900/0 800/0

Parts

18 thoughts on “Installing pfSense 2.6 on ZimaBoard

  • Thank you for sharing your work. I have always read that the intel nic’s are preferable to the cheaper Realtek ones. Curious if your setup will route your fully allocated wan download speed, and if you have a gigabit connection. Thanks Again

    Reply
    • Hi,
      My broadband speed is only ~500Mbps via my Cable provider (Virgin Media) and the backup, StarLink connection runs between 200-400Mbps. My setup has no issue running at those speeds. Obviously, there are some plugins to pfSense which can add significant extra load, but the ZimaBoard seems to have more than enough capacity.

      Since moving to the Intel Pro 1000 NIC I’ve had less frequent issues with StarLink, but it still sometimes shows the gateway status as Pending but disabling and re-enabling the interface has “fixed” it each time it occurs. I suspect both the onboard Realtek NICs and the 2.5Gbps external NIC would work fine with the latest drivers, the only issue I had was the odd behaviour with StarLink.

      In the near future I may go back and try the RealTek based solution again to see if I can narrow down the cause of the odity.

      Reply
  • I’ve read somewhere that these were using broadcom NICs not Realtek. I guess not ? I personally haven’t had issues with realtek but I’d use proxmox then virtualize pfsense using virtio those are very good drivers

    Reply
    • Martin

      Definately Realtek NICs. As for virtualising pfSense. Sure you can do that, but it’s not something I like to do, except for lab purposes. If it’s your main router then by virtualising it you’re adding an additional layer of complexity and fragility. But depends on your use case of course.

      Reply
  • I’m setting up my ZimaBoard with two 240GB SSD (ada0, ada1). The ZFS Configuration is ‘mirror’.

    With this configuration there is no reason to install anything on the ZimaBoard internal memory, correct (eMMC storage#)?

    On reboot it should then boot off the mirrored SSD.

    Thanks for the great write up.

    Reply
    • So long as you ensured you installed on to the SSD and that the boot partition isn’t on the eMMC storage. Load the BIOS and ensure the SSD is the boot disk and confirm it boots.

      Reply
  • Martin Saunders

    Hi Martin, the Realtek driver version has been updated to 198 now, so

    Reply
    • Thanks, Martin. Post updated.

      Reply
  • Any updates on your zimaboard experience with pfsense?

    Reply
  • Hello, I have a new 432 board with the latest pfsense image version 2.6.0-RELEASE (amd64) built on Mon Jan 31 19:57:53 UTC 2022 FreeBSD 12.3-STABLE. and using the default ZFS file system.
    Did you have any issues with storage issues with the internal mmc storage?
    System crash/hang after a few hours.
    I have tried reloading fresh image multiple times and using the onboard NIC’c
    Console error
    pfSense kernel: mmcsd0: Error indicated: 2 Bad CRC
    pfSense kernel: sdhci_pci0-slot0: Controller timeout
    pfSense kernel: sdhci_pci0-slot0: ============== REGISTER DUMP ==============
    pfSense kernel: sdhci_pci0-slot0: Sys addr: 0x287f6610 | Version: 0x00001002
    pfSense kernel: sdhci_pci0-slot0: Blk size: 0x00000080 | Blk cnt: 0x00000001
    pfSense kernel: sdhci_pci0-slot0: Argument: 0x00000000 | Trn mode: 0x00000012
    pfSense kernel: sdhci_pci0-slot0: Present: 0x1fff0000 | Host ctl: 0x00000025
    pfSense kernel: sdhci_pci0-slot0: Power: 0x0000000b | Blk gap: 0x00000080
    pfSense kernel: sdhci_pci0-slot0: Wake-up: 0x00000000 | Clock: 0x00000007
    pfSense kernel: sdhci_pci0-slot0: Timeout: 0x00000007 | Int stat: 0x00000000
    pfSense kernel: sdhci_pci0-slot0: Int enab: 0x00000020 | Sig enab: 0x00000020
    pfSense kernel: sdhci_pci0-slot0: AC12 err: 0x00000000 | Host ctl2:0x0000000b
    pfSense kernel: sdhci_pci0-slot0: Caps: 0x546ec881 | Caps2: 0x00000807
    pfSense kernel: sdhci_pci0-slot0: Max curr: 0x00000000 | ADMA err: 0x00000000
    pfSense kernel: sdhci_pci0-slot0: ADMA addr:0x00000000 | Slot int: 0x00000000
    pfSense kernel: sdhci_pci0-slot0: ===========================================
    pfSense kernel: sdhci_pci0-slot0: Tuning failed, using fixed sampling clock
    pfSense kernel: mmcsd0: Error indicated: 1 Timeout
    pfSense kernel: sdhci_pci0-slot0: Tuning failed, using fixed sampling clock
    pfSense kernel: mmcsd0: Error indicated: 1 Timeout
    pfSense kernel: mmcsd0: Setting erase start position failed Timeout

    Reply
    • Hi, I’m still running pfsense on my Zimaboard without issue. I also checked the logs and no sign of any errors.
      I see you’ve posted on the IceWhale discord channel (my alias: shortbloke), that’s probably the best place to get support from the IceWhale team.

      Reply
  • Hey, thanks for sharing.

    I’ve used what looks like an identical 4 port card as you but the ZimaBoard fails to boot. Did you tweak any bios settings?

    Reply
    • No bios changes needed.
      Are you saying the ZimaBoard works fine until you plug the card in, then it doesn’t boot?

      If so can you test the card in another PC?

      Reply
  • Giovanni Panozzo

    Some updates for pfSense 2.7 on ZimaBoard:
    – The realtek driver can be upgraded to version 198 with only one command: pkg install realtek-re-kmod, and a reboot. This solved the problem of Realtek interface going offline every 3-8 days (i.e.: “re1: watchdog timeout” continuously logged).
    – The CPU clock of pfSense 2.7 on Zimaboard seems to be limited to 1.1GHz:
    [2.7.0-RELEASE][[email protected]]/root: sysctl dev.cpu.0.freq_levels
    dev.cpu.0.freq_levels: 1101/0 1100/0 1000/0 900/0 800/0
    but using “Hiadaptive” powerd mode instead of “Minimum” gives improved performances and I can reach routing at 1Gbps. Maybe “Minimum” modem locks the CPU at speed slower than 1.1GHz.

    Reply
  • Hello Martin,

    Do you have the chance to share the printed cage, under the Zimaboard and the additional 4port card with us?

    Many thanks.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.