zotac

Recently we purchased a Zotac ZBOX-CI321NANO to replace our existing pfSense PC which was in a much larger case.

There are a few variations but ours has dual 1 Gbps NICs, many USB 3.0 ports, 2 GiB of RAM, a 32 GiB SSD and is powered by a Celeron 2961Y (1.10 GHz). It has plenty of horsepower to spare running as a router/firewall for an office and is positively tiny at roughly 5″ x 5″ and 2″ tall. It is the perfect pfSense box and looks great on our equipment rack.

However after a day in service we started noticing drops on one of its interfaces, the internal one to be specific, which had a lot of VLANs:

re1: watchdog timeout
re1: link state changed to DOWN
re1_vlan1: link state changed to DOWN
re1_vlan3: link state changed to DOWN
re1_vlan4: link state changed to DOWN
re1_vlan5: link state changed to DOWN
re1_vlan6: link state changed to DOWN
re1_vlan10: link state changed to DOWN
re1: link state changed to UP
re1_vlan1: link state changed to UP
re1_vlan3: link state changed to UP
re1_vlan4: link state changed to UP
re1_vlan5: link state changed to UP
re1_vlan6: link state changed to UP
re1_vlan10: link state changed to UP

Our outside interface (in our case re0) was not affected. We tried turning off MSI and MSIX via the following sysctls:

hw.re.msi_disable=1
hw.pci.enable_msix=0
hw.pci.enable_msi=0

However the timeouts persisted as did the link drops.

What did work was disabling hardware checksum offloading:

offloading

Interestingly this did not turn off the VLAN_HWCSUM option though in ifconfig:

re1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=82098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether 00:01:2e:xx:xx:xx
        nd6 options=21
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active

But it did solve the issue!

5 Responses to “FreeBSD / pfSense on a Zotac ZBox: link state down and ‘re0: watchdog timeout’ errors”

  1. Hl

    Hi,
    Thanks for this post. Disable hardware checksum offload option solve my problem 98%.
    Do you run Snort package on this box as well ? If yes , how’s the performance ?

    Reply
    • Adam Strohl

      Hl, I don’t but we do use ntop-ng and it is fine on an Intel Celeron 2961Y @ 1.10GHz even when doing 20-50 Mbps. It is hard to say how much it affects performance — for us it is a matter of it being reliable so there is no choice to turn off the HW checksumming, otherwise we lose connectivity 🙂

      Reply
  2. Chevy

    Hi Adam, I’m using a similiar Zotac box and am facing the same problem with latest pfSense (2.4.4 dev)
    I know this is 2 years old but any luck with your problem?

    Reply
    • Adam Strohl

      So even with this solution we still see timeouts under high traffic (800+ Mbit/sec) which became apparent when we upgraded our office link. So even our solution only raised the threshold for it apparently.

      Our new policy here is to avoid RealTek cards at all cost basically.

      Reply
  3. Chevy

    I updated realtek driver and was able to get ~620Mb speed. No more time out errors.

    Directly going through my ISP’s modem/router box I see 850Mb.
    I’m

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.