setup-gadget.sh now includes: - bind-dynamic instead of bind-interfaces for dnsmasq - usb-gadget-up.service for reliable boot activation - NetworkManager dispatcher to restart dnsmasq on usb0 up setup-mesh.sh cleaned up: - Removed unused MESH_CELL and MESH_CHANNEL variables - Added inline comments explaining brcmfmac limitations README.md: - Simplified Phase 1 to single script invocation (no manual steps) - Added Lessons Learned section with 13 documented gotchas covering dwc2 overlay placement, dnsmasq race conditions, NM autoconnect reliability, brcmfmac IBSS quirks, PATH issues, and more Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|---|---|---|
| scripts | ||
| README.md | ||
RPi Zero 2W Mesh Network Setup
batman-adv mesh network across three Raspberry Pi Zero 2W devices, with USB Ethernet gadget mode for out-of-band management from a Mac.
Hardware
- 3x Raspberry Pi Zero 2W running Debian 13 (trixie)
- 3x USB data cables (micro-USB to host)
- USB hub (optional, for connecting all three simultaneously)
Node Inventory
| Node | Hostname | USB Gadget IP | Mesh IP (bat0) | USB Subnet |
|---|---|---|---|---|
| 1 | meshy01 | 10.55.1.1 | 10.99.0.1 | 10.55.1.0/24 |
| 2 | meshy02 | 10.55.2.1 | 10.99.0.2 | 10.55.2.0/24 |
| 3 | meshy03 | 10.55.3.1 | 10.99.0.3 | 10.55.3.0/24 |
Architecture
Mac (management host)
├── USB cable ── meshy01 (10.55.1.1) ──┐
├── USB cable ── meshy02 (10.55.2.1) ──┼── batman-adv mesh (10.99.0.0/24)
└── USB cable ── meshy03 (10.55.3.1) ──┘ via wlan0 IBSS ad-hoc
- USB gadget mode (dwc2 + g_ether): provides Ethernet-over-USB for SSH management
- batman-adv: layer 2 mesh routing over wlan0 in IBSS (ad-hoc) mode
- wlan0 is dedicated to the mesh — WiFi is not available for normal network access once mesh is configured
Setup Process
Prerequisites
- SSH access to all three Pis (initially over WiFi or any network)
- Passwordless sudo configured for the
aaronuser on each Pi:echo "aaron ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/010_aaron-nopasswd
Phase 1: USB Gadget Mode
This configures each Pi to present as a USB Ethernet device when connected to a host computer via the data micro-USB port.
-
Copy and run the gadget setup script on each Pi:
scp scripts/setup-gadget.sh aaron@<PI_WIFI_IP>:/tmp/ ssh aaron@<PI_WIFI_IP> 'sudo bash /tmp/setup-gadget.sh <NODE_NUM>'Where
<NODE_NUM>is 1, 2, or 3 corresponding to the node.This script handles everything in one shot:
- Enables the
dwc2device tree overlay in/boot/firmware/config.txt(under[all]) - Adds
modules-load=dwc2,g_etherto/boot/firmware/cmdline.txt - Creates
/etc/modules-load.d/usb-gadget.conffor module autoloading - Installs and configures
dnsmasqwithbind-dynamicto serve DHCP onusb0 - Creates a NetworkManager connection profile (
usb0-gadget) with a static IP - Installs
usb-gadget-up.serviceto reliably activate usb0 on boot - Installs a NetworkManager dispatcher to restart dnsmasq when usb0 comes up
- Enables the
-
Reboot all Pis, then connect them to the Mac via the data micro-USB port (the inner port, closer to the HDMI connector — not the power-only port on the edge).
-
Verify from the Mac:
# Check interfaces appeared networksetup -listallhardwareports | grep -A2 "Raspberry Pi" # Ping each node ping 10.55.1.1 # meshy01 ping 10.55.2.1 # meshy02 ping 10.55.3.1 # meshy03 # Or use the helper script bash scripts/mac-usb-check.shIf a Pi is reachable over WiFi but not USB, activate the connection manually:
ssh aaron@<PI_WIFI_IP> 'sudo nmcli con up usb0-gadget'
Phase 2: batman-adv Mesh Network
Once all three Pis are accessible over USB, deploy the mesh networking.
-
Copy and run the mesh setup script on each Pi (over USB):
scp scripts/setup-mesh.sh aaron@10.55.1.1:/tmp/ ssh aaron@10.55.1.1 'sudo bash /tmp/setup-mesh.sh 1 --start' scp scripts/setup-mesh.sh aaron@10.55.2.1:/tmp/ ssh aaron@10.55.2.1 'sudo bash /tmp/setup-mesh.sh 2 --start' scp scripts/setup-mesh.sh aaron@10.55.3.1:/tmp/ ssh aaron@10.55.3.1 'sudo bash /tmp/setup-mesh.sh 3 --start'Or use the deploy helper:
bash scripts/deploy-mesh.shThis script:
- Installs
batctl - Loads the
batman-advkernel module - Removes wlan0 from NetworkManager control
- Deletes the existing WiFi connection
- Creates
/usr/local/bin/mesh-start.shwhich:- Sets wlan0 to IBSS (ad-hoc) mode
- Joins the
rpi-meshnetwork on channel 1 (2412 MHz) - Adds wlan0 to batman-adv
- Brings up bat0 with the node's mesh IP
- Creates a
mesh-network.servicesystemd unit (enabled at boot)
- Installs
-
Verify the mesh (allow ~10 seconds for neighbor discovery):
# Check neighbors from any node ssh aaron@10.55.1.1 'sudo batctl n' # Check originator table (routing) ssh aaron@10.55.1.1 'sudo batctl o' # Ping between nodes over the mesh ssh aaron@10.55.1.1 'ping -c3 10.99.0.2' # meshy01 -> meshy02 ssh aaron@10.55.1.1 'ping -c3 10.99.0.3' # meshy01 -> meshy03 ssh aaron@10.55.2.1 'ping -c3 10.99.0.3' # meshy02 -> meshy03
Important Notes
USB Port Selection
The Pi Zero 2W has two micro-USB ports. Use the inner port (labeled USB, closer to the HDMI connector) for data. The outer port (labeled PWR) is power-only.
WiFi After Mesh Setup
Once the mesh is configured, wlan0 is dedicated to the ad-hoc mesh network. The managed WiFi connection is deleted and NetworkManager is told to ignore wlan0. USB is your only management path. To restore WiFi:
sudo systemctl stop mesh-network
sudo nmcli con add type wifi ifname wlan0 ssid "YourSSID" wifi-security.key-mgmt wpa-psk wifi-security.psk "YourPassword"
brcmfmac Driver Limitations
The Broadcom WiFi driver (brcmfmac) on the Pi Zero 2W supports IBSS mode but does not accept HT20, fixed-freq, or explicit BSSID parameters in the iw ibss join command. The mesh scripts use the simplified form:
iw dev wlan0 ibss join rpi-mesh 2412
Services
Each Pi runs these services related to the mesh:
| Service | Purpose |
|---|---|
mesh-network.service |
Starts/stops batman-adv mesh on boot |
usb-gadget-up.service |
Activates usb0 NetworkManager connection on boot |
dnsmasq.service |
Serves DHCP to the Mac on the USB interface |
Troubleshooting
# Check mesh service status
sudo systemctl status mesh-network
# View mesh startup logs
sudo journalctl -u mesh-network
# Check if wlan0 is in IBSS mode
sudo /usr/sbin/iw dev wlan0 info
# Check batman-adv interfaces
sudo batctl if
# Restart the mesh
sudo systemctl restart mesh-network
# Check USB gadget status
ip addr show usb0
sudo systemctl status usb-gadget-up
sudo systemctl status dnsmasq
Performance
Tested with all three nodes sitting next to each other on 2.4 GHz channel 1 (IBSS ad-hoc mode).
Latency (ICMP ping, 10 packets)
| Link | Min | Avg | Max | Packet Loss |
|---|---|---|---|---|
| meshy01 <-> meshy02 | 2.0 ms | 3.9 ms | 6.6 ms | 0% |
| meshy01 <-> meshy03 | 1.9 ms | 3.1 ms | 7.3 ms | 0% |
| meshy02 <-> meshy03 | 2.0 ms | 6.7 ms | 17.0 ms | 0% |
Throughput (iperf3, 10-second TCP)
| Link | Sender | Receiver |
|---|---|---|
| meshy01 -> meshy02 | 16.9 Mbps | 16.1 Mbps |
| meshy01 -> meshy03 | 17.0 Mbps | 16.1 Mbps |
| meshy02 -> meshy03 | 15.9 Mbps | 14.9 Mbps |
The Pi Zero 2W's Broadcom WiFi chip tops out around 20 Mbps in ideal conditions on 2.4 GHz. The ~15-17 Mbps observed is typical with batman-adv overhead factored in. Sufficient for IoT, sensor data, command/control, or light file transfers.
Lessons Learned / Gotchas
These are the non-obvious issues encountered during setup that you'll want to know if adapting this for another project.
USB Gadget Mode
-
dwc2 overlay must be in
[all]section of config.txt. The default Raspberry Pi OS config.txt hasdtoverlay=dwc2,dr_mode=hostunder[cm5], which only applies to Compute Module 5. A naive grep fordtoverlay=dwc2will find this and skip adding it to[all], silently breaking gadget mode on the Pi Zero 2W. -
dwc2 is built into the kernel, not a loadable module. On this kernel (6.12.75+rpt-rpi-v8),
lsmod | grep dwc2returns nothing even when dwc2 is working. Checkdmesg | grep dwc2instead to verify the driver loaded. -
dnsmasq must use
bind-dynamic, notbind-interfaces. Withbind-interfaces, dnsmasq fails at boot ifusb0doesn't exist yet (which it won't until a USB cable is connected).bind-dynamicallows dnsmasq to start and then bind to usb0 when it appears. -
NetworkManager won't reliably auto-activate the usb0 connection. Even with
connection.autoconnect yes, theusb0-gadgetconnection may not activate because usb0 appears after NM's startup scan. Theusb-gadget-up.service(withBindsTo=sys-subsystem-net-devices-usb0.device) ensures activation when the device exists. -
dnsmasq needs a dispatcher restart. Even with
bind-dynamic, if dnsmasq started before usb0 existed, it won't retroactively serve DHCP on usb0 when it appears. The NetworkManager dispatcher script (99-usb0-dnsmasq) restarts dnsmasq whenever usb0 comes up. -
Each node needs a different USB subnet. Using the same subnet (e.g., 10.55.0.0/24) for all nodes works if you only connect one at a time, but fails when multiple Pis are connected to the same Mac simultaneously. Each node uses 10.55.{1,2,3}.0/24.
-
macOS may need manual DHCP renewal. If a Pi's dnsmasq wasn't ready when the Mac first connected, macOS falls back to a 169.254.x.x link-local address and may not retry DHCP. Run
sudo ipconfig set enX DHCPon the Mac, or unplug/replug the USB cable after verifying dnsmasq is running on the Pi. -
VPNs can steal routes. If a VPN is active on the Mac, it may capture traffic destined for the 10.55.x.x subnets. Disable the VPN or add route exceptions.
batman-adv Mesh
-
brcmfmac rejects HT20/fixed-freq/BSSID in IBSS join. The Broadcom WiFi driver supports IBSS mode (confirmed via
iw list), butiw dev wlan0 ibss join rpi-mesh 2412 HT20 fixed-freq 02:12:34:56:78:9Afails with "Invalid argument (-22)". Only the simplified form works:iw dev wlan0 ibss join rpi-mesh 2412. -
/usr/sbinis not in PATH for systemd services. Theiwbinary installs to/usr/sbin/iw. Whenmesh-start.shruns via systemd,/usr/sbinmay not be in PATH. The script must explicitlyexport PATH="/usr/sbin:/usr/bin:/sbin:/bin:$PATH". -
wlan0 must be unmanaged by NetworkManager. Before putting wlan0 into IBSS mode, NetworkManager must be told to ignore it via
/etc/NetworkManager/conf.d/mesh-unmanaged.conf. Otherwise NM will fight the manual interface configuration. -
No internet after mesh setup. Once wlan0 is in ad-hoc mode for the mesh, the Pi has no internet access. Install all packages (batctl, iw, wireless-tools, iperf3, etc.) before or during mesh setup. If packages are needed later, transfer .deb files via USB from the Mac.
-
Allow ~10 seconds for neighbor discovery. batman-adv uses OGM (Originator Messages) to discover neighbors. The first
batctl ncheck may show nothing if run immediately after starting the mesh.
Scripts
| Script | Description |
|---|---|
scripts/setup-gadget.sh <node> |
Configures USB Ethernet gadget mode (run as root) |
scripts/setup-mesh.sh <node> [--start] |
Installs and configures batman-adv mesh (run as root) |
scripts/deploy-mesh.sh |
Deploys mesh setup to all nodes over USB (run from Mac) |
scripts/mac-usb-check.sh |
Checks USB Ethernet connectivity from the Mac |