While working with high-load and high-lock contention NFSv4 with FreeBSD as the NFS client you may see this error and have the NFS mount hang: nfsv4 client/server protocol prob err=10026 Make sure your mount parameters do *not* include the intr keyword, per this PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=178231. We found this was especially true when using NFS appliances… Read more »
SOLVED: FreeBSD 12.2 ZFS: can only boot from disk, mirror, raidz1, raidz2, and raidz3 vdevs
After upgrading a file server to FreeBSD 12.2 this last week we ran into an interesting issue where the boot loader immediately errored with the following message: ZFS: can only boot from disk, mirror, raidz1, raidz2, and raidz3 vdevs vdev_init_from-nvlist: 5 It seemed to be a similar issue to this: https://reviews.freebsd.org/D20319 but not quite the… Read more »
Dedicated and Cloud Hosting Comparison: AWS, Azure, Google Cloud, Vultr, LiquidWeb
A-Team Systems works with dozens of different hosting providers in almost every environment conceivable. In a unique way this broad experience has given our engineers an inherent “gut feeling” on the performance of these providers and how they compare to each other. The primary goal of this benchmark and hosting review is to replace that… Read more »
SSH Pipe Speed 48 KB/sec FreeBSD 12 PF
We recently ran into an issue after upgrading a server from FreeBSD 11.2 to 12.1 where throughput via SSH pipes would crash to 48 KB/sec almost instantly after starting. The odd thing was scp (which uses SSH) was not affected. After ruling out Internet and remote host issues we discovered that the PF rule was… Read more »
FreeBSD Jenkins ERROR: Failed to parse POMs
We recently ran into an issue where the version of Maven under FreeBSD wasn’t updated enough (ports/pkg was behind) for the version of Jenkins in ports/pkg. It produced an error like the following: [INFO] Installing /usr/local/jenkins/workspace/bdd_framework_test/pom.xml to /usr/local/jenkins/.m2/repository/Cucumber_Framework/BDD_SearchLock_Safelist_Final/0.0.1-SNAPSHOT/BDD_SearchLock_Safelist_Final-0.0.1-SNAPSHOT.pom [WARNING] Attempt to (de-)serialize anonymous class hudson.maven.reporters.MavenArtifactArchiver$2; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/ java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)… Read more »
PHP-FPM Chroot “security library: invalid algorithm”
After an OS update on one of our client servers we witnessed a perplexing error in their chrooted PHP-FPM environment: security library: invalid algorithm This is produced when the chroot’s libraries were out of sync with the base system, in our case updating usr/lib64 fixed it. A simple fix but took a significant amount of… Read more »
Expand ZFS To Use Larger Disks / Free Space
One of the great features of ZFS is to be able to expand it using new, larger disks — even while it is online. To do this you’ll replace each disk, one by one with a larger disk. After each replacement you’ll rebuild the zpool, and then move on to the next disk. Once all… Read more »
Debian 8 Under Bhyve Networking Problem: Packets Over 230 Bytes Truncated
We’re a big fan of FreeBSD’s Bhyve hypervisor, including writing an open source lightweight management interface for it called BMT. While setting up a Debian 8 guest VM we ran into a perplexing issue where networking was not working correctly. After much troubleshooting we narrowed down the symptom to be that packets larger than 230… Read more »
SOLVED: Can’t Connect to Linux Server Behind NAT From Linux/FreeBSD But Can From Windows
Strange networking behavior while trying to access a Linux server via NAT? Does it work from one computer but not another? We may have the solution! We recently ran across a situation where a Linux server was behind NAT (for VPN purposes), and we could access it just fine from a Windows 10 computer, however… Read more »
SOLVED: NextCloud Admin “invalid email address” / User List Not Updating After Add Under PHP-FPM Chroot
We do a lot of NextCloud installs, and use PHP-FPM + chroot for maximum security. This gets tricky though, and one of the more obscure gotchyas we’ve run across is trying to create a new user or set a user’s email address and getting back “invalid email address” despite the address being, well, valid. Another… Read more »