OPENWRT SURGERY
After a while when I didn't have ideas for new phlog posts, now I
have ideas and no time to write them. But I did finally get around
to setting up my final OpenWRT configuration for my home router, v.
23 within my router's 32MB of RAM much better than the default.
It's final because, as noted in my post 2025-01-09Hit_and_MIPS.txt
(oh dear, this took me six month to get back to!), OpenWRT 24 has
dropped the build option for router boards in my router's hardware
family, which at least gives me the excuse to hack deeper into the
works without fearing breakage after upgrades.
I'd like to write my usual long ramble, but since allowing time for
writing this post is under the excuse of it being notes for my own
later reference, I'll go into quick dot-point mode:
* Building with OpenWRT Image Builder on x86_64 VPS (because the
image builder is a huge download so I don't want to do that over my
home internet connection)
https://openwrt.org/docs/guide-user/additional-software/imagebuilder
- Make directory named "files" in the root of the image builder
directory, pointed to with the FILES argument to make. In here go
all files to be added/changed compared with the default router file
system, with all their final directory paths and permissions (you
need root permissions to make them).
- Instructions say not to run the Image Builder Makefile as root,
but if you want to include custom files that are only readable by
root, it will fail to read them unless you run it as root. It runs
fine as root.
* OpenWRT releases supported for six months after release of next
major version:
https://openwrt.org/docs/guide-developer/security#support_status
OpenWRT 23 therefore supported until July this year. Latest
release is 23.05.5.
* The OpenWRT Wiki page for my router pointed to the "SMP" images
(until it became officially unsupported after OpenWRT 19 due to
lack of RAM). This was wrong, it only has one processor so an SMP
kernel is bloat. I should have been using the image builder
configured without SMP support, called "generic":
https://downloads.openwrt.org/releases/23.05.5/targets/bcm63xx/generic/openwrt-imagebuilder-23.05.5-bcm63xx-generic.Linux-x86_64.tar.xz
https://openwrt.org/docs/techref/targets/bcm63xx
* Openwrt comes with DHCPv6 via a separate DHCP server package. I'm
not using IPv6 so I don't need that and can just use the IPv4 DHCP
server in dnsmasq:
- Add "option dhcpv6 'disabled'" in /etc/config/dhcp.
- Also disable automatic IPv6 support for USB modem in
/etc/config/network
- Also comment-out IPv6 rules in /etc/config/firewall
- "disable_ipv6" option is not supported since OpenWrt 22.
- Remove default packages odhcp6c, odhcpd-ipv6only, and odhcpd6c,
from image builder PACKAGES
- Disabled odhcpd in /etc/config/dhcp by setting in the odhcpd
section:
option dhcpv6 'disabled'
option ra 'disabled'
* Disable other unused default services
- ntpd (NTP daemon)
- Router doesn't have a RTC so this means it won't know the
correct time anymore (will count from the date of the current
OpenWRT version's release after booting), but I don't care.
- Can be disabled in /etc/config/system or put "sysntpd" in
DISABLED_SERVICES for image builder.
- urngd (Enthropy harvester daemon)
- This might be important for encrypted WiFi, for which drivers
sometimes need a fresh enthropy source), but I'm not using that so
it gets the axe.
- Remove default urngd package from image builder PACKAGES.
- Put urngd in DISABLED_SERVICES for image builder.
- logd (syslog daemon)
- Networking wouldn't come up when I disabled this in the old
image, so I left it enabled at boot but put this at the start of
/etc/rc.local to kill logd at the end of the boot process:
# Wait until firewall confguration is finished
while killall -q -0 hotplug-call || killall -q -0 mac80211.sh || killall -q -0 fw4
do
sleep 5
done
# Kill syslog daemon
killall logd
- Use "/etc/init.d/log start" to start it after boot for
debugging purposes. Read syslog afterwards with the "logread"
command.
* Ditch Dropbear and SSH/SCP. Telnet and FTP have less overhead and
won't break when new clients demand newer server versions than the
last OpenWrt 23 dropbear package.
- Remove "dropbear" package from image builder PACKAGES.
- put "dropbear" in DISABLED_SERVICES for image builder.
- put GNU Inetutils telnetd and inetd static binaries (see
2025-01-09Hit_and_MIPS.txt) in custom files path:
/usr/bin/telnetd
/usr/bin/inetd
- Make symlinks:
/usr/local/bin -> /bin
/usr/local/var -> /var
- Disable vsftpd service, since I'll use it in inetd mode which
takes up much less RAM, by putting vsftpd in DISABLED_SERVICES for
image builder.
- Make /etc/inetd.conf:
telnet stream tcp nowait root /usr/bin/telnetd telnetd
ftp stream tcp nowait root /usr/sbin/vsftpd vsftpd
- Set "listen=NO" in /etc/vsftpd.conf
- Create this directory for vsftpd at start-up with this command
in /etc/rc.local:
mkdir -m 0755 -p /var/run/vsftpd
(/var/run is in tmpfs so can't put the directory in FILES)
- Inetd now needs a custom init script to start it at boot, in
/etc/init.d/inetd:
#!/bin/sh /etc/rc.common
# start after and stop before networking
START=19
STOP=50
start() {
SERVICE_USE_PID=1 service_start /usr/bin/inetd /etc/inetd.conf
}
stop() {
service_stop /usr/bin/inetd
}
reload() {
service_reload /usr/bin/inetd
}
- Make the script executable and owned+writable by root only.
The symlinks to this script in /etc/rc.d are created by the image
builder.
* If the router does run out of RAM, processes get killed by the
kernel "OOM Reaper", but respawned by the (OpenWRT-specific) init
system. I tweaked the "respawn" settings to be more forgiving and
better suit my usage. Following docs here:
https://openwrt.org/docs/guide-developer/procd-init-scripts#service_parameters
/etc/init.d/network - in start_service(): procd_set_param respawn 3600 20 0
- This sets a 20sec wait between respawning if the network
process is killed, and
retries indefinitely.
/etc/init.d/dnsmasq - in dnsmasq_start(): procd_set_param respawn 3600 40 0
- This sets a 40sec wait between respawning if the DNS server
process is killed, and
retries indefinitely.
/etc/init.d/log - in start_service_daemon(): # procd_set_param respawn 5 1 -1
- Disable respawning the log process if it is killed by
commenting out this line.
* Probably not an issue anymore, but before the above changes I had
trouble with running out of RAM at boot because the OpenWRT init
system starts all services in the background and therefore has
their init scripts all running in parallel. That's great for boot
speed (at least if I had multiple processors), but terrible for RAM
usage since there needs to be room for everything to run at once (a
really odd choice from the OpenWRT developers). Disabling dnsmasq
in DISABLED_SERVICES then launching it at the end of the boot
sequence before starting the mobile broadband modem avoids the
choke point, and can be achieved with this in /etc/rc.local (first
part already included under "Disable other unused default services"
above):
# Wait until firewall confguration is finished
while killall -q -0 hotplug-call || killall -q -0 mac80211.sh || killall -q -0 fw4
do
sleep 5
done
# Start these services at the end of the boot process
# to avoid running out of RAM
sleep 30
/etc/init.d/dnsmasq start
sleep 15
ifup wan
# Restart firewall if it was killed during ifup
sleep 45
/etc/init.d/firewall boot
* Finally, my idata.sh script (from 2024-03-10Off_and_On_Line.txt)
used SSH to run the data-collector script on the router. Looking at
the inetd docs, I discovered I could turn that into its own
protocol using TCPMUX and make requests using "nc" instead of
dropbear without changing the router-side script at all.
- Put these lines in /etc/inetd.conf:
tcpmux stream tcp nowait root internal
tcpmux/+idata stream tcp nowait root /root/send_idata.sh idata
- But all the best things in computing are obsolete, so I needed
to add the tcpmux protocol to the /etc/services file which is
missing it by default:
tcpmux 1/tcp
- Now in idata.sh, instead of:
# Grab data stats from router via SSH
newdata="`DROPBEAR_PASSWORD=$ROUTER_PASSWORD dbclient $ROUTER_SSH $ROUTER_SCRIPT`"
I've got:
# Grab data stats from router via TCPMUX
newdata="`echo idata | nc $ROUTER 1 | tail -n 1`"
* This is my final OpenWRT Image Builder build command. Note that
LuCI was never included in these builds by default (hence I've only
ever known OpenWRT configuration via SSH), so it doesn't need to be
excluded. I'm not sure if that's the case for other build targets:
make image FILES="files" PROFILE="brcm_bcm96358vw" PACKAGES="chat comgt hostapd-basic hostapd-common iw-full iwinfo kmod-b43 kmod-bcma kmod-cfg80211 kmod-crypto-hash kmod-mac80211 kmod-mac80211-hwsim kmod-nls-base kmod-usb2 kmod-usb-acm kmod-usb-core kmod-usb-ehci kmod-usb-ohci kmod-usb-serial kmod-usb-serial-option kmod-usb-serial-wwan kmod-usb-uhci libiwinfo librt libusb-1.0 terminfo usb-modeswitch vsftpd wireless-regdb wireless-tools coreutils-stty picocom kmod-usb-net-sierrawireless kmod-usb-serial-sierrawireless kmod-mii -odhcpd-ipv6only -odhcpd6c -odhcp6c -urng -dropbear" DISABLED_SERVICES="urngd dnsmasq sysntpd dropbear vsftpd"
- Note that the "kmod-" packages are specific to my router
hardware. No longer needing to figure out when these change
(without documentation of the changes) is another big win from not
upgrading to new major OpenWRT versions anymore.
* Still takes me a few tries to flash the firmware via TFTP (via
serial console command). Dunno why the connection always seems to
fail the first few times, but it works eventually.
The stats for this new image compared to the previous default SMP
one which I'd tried to optimise after installation are significant.
Specs: 32MB RAM, 8MB Flash
Old OpenWRT 23 (SMP) installation:
Total RAM (minus kernel size): 23504KB
RAM available after boot: 4148KB
Flash free after boot: 568KB
New OpenWRT 23 (non-SMP) installation:
Total RAM (minus kernel size): 24712KB
RAM available after boot: 9512KB
Flash free after boot: 1960KB
But most important, it now starts up in a reasonable time again (it
was taking over 5min before) and no longer has random crashes when
the mobile broadband modem signal drops out. Before it would run
out of RAM while trying to reconnect, sometimes failing, or killing
other important services in the process. Also the router I thought
was broken when I first installed OpenWRT 23 on it is actually
fine, it was just an early case of those issues causing the
firewall process to be killed which made it look like the network
interface was intermittent. It has been running solidly for days
since installing this lightweight build.
Oh and typically now that I've finally discovered the RAM/storage
advantage of the Linux kernel without SMP support, the kernel
developers are talking about removing that option:
https://www.phoronix.com/news/Linux-6.17-Maybe-SMP-Uncond
Linux seems to be looking less attractive lately, clearly aiming to
serve a different user-base than me. Also see this tally of how the
size of OpenWrt has increased over the years:
https://openwrt.org/supported_devices/432_warning#analysis_of_firmware_size_growth
I wonder if a BSD-based router OS would have done better, such as
ZRouter (
https://zrouter.org - looks dead)? Although drivers would
be a roadblock with that anyway. Anyway I don't think there's any
good reason that a basic router now needs better specs than these
ones have. Plus newer basic ISP-supplied models that are common
second-hand in Australia have little-better specs and never got any
sort of OpenWRT support to begin with.
- The Free Thinker
