HIT AND MIPS

As noted in earlier ramblings about my router running OpenWRT, with
the upgrade to OpenWRT 23 I decided to give up on new versions. In
fact the decision turns out to have been made for me because
release notes for the OpenWrt 24 release candidates reveal that my
router's board family been removed from their build targets. My
main incentive for upgrading was security, but thinking about it I
don't think an old Linux kernel that's not running any
internet-facing services is at risk of getting hacked from the
internet anyway. If they can get through the firewall then that's
basically a backdoor at the heart of Linux - even after the horror
of the XZ SSH backdoor I still don't think something that
fundamental could slip through.

But the annoying thing about computer security is that if you try
to ignore it, like a jealous cartoon robot it'll start sabotaging
everything you do just to get attention. I don't install the Web
interface to OpenWrt, so no worries about any encryption sillyness
it uses, but of course by default the command-line access is by SSH
and some day newer SSH clients will get all elitist and refuse to
talk to SSH servers as old as the Dropbear in OpenWrt 23. Telnet is
a fine unencrypted alternative that I use everywhere else on my
home network for this reason, but OpenWrt don't package a Telnet
server.

Past investigation into making a Telnet package for OpenWrt got me
lost in a documentation maze and by the time I found my way out I
had no time left to do anything. So stuff that, how about just
building a static binary on Musl libC like I did in
2023-07-30Backwards_Builds.txt for x86? Debian supports MIPS, so it
should all be easy enough, right?

Well the first hurdle is endianness (order of bytes stored in
memory). MIPS can be whichever you prefer, and hardware developers
all preferred differently. Is my MIPS CPU a big endian or a little
endian? You'd think /proc/cpuinfo would say, but nope. You'd think
the detailed OpenWrt Wiki docs about my now-obsolete platform would
mention it, but nope. It came down to me copying the busybox binary
off the router and running 'file' on it:

$ file busybox
busybox: ELF 32-bit MSB executable, MIPS, MIPS32 version 1 (SYSV),
dynamically linked, interpreter /lib/ld-musl-mips-sf.so.1, no
section header

Blink and you'll misinterpret it, but "MSB executable" means Most
Significant Byte (as opposed to LSB for Least Significant Byte),
and therefore big endian. In Debian (or Devuan in my case) terms
that's their "mips" arcitecture (LSB is "mipsel"). And guess which
of those Debian discontinued supporting since version 10? Yep,
"mips", and I've upgraded Devuan on my 'new' (ie. still a teenager)
laptop to version 12.

However it turns out the mips-linux-gnu-gcc package lives on, just
libraries including Musl aren't built for "mips" anymore. So
install that compiler then build Musl from the latest sources
(Musl defaults to building for big-endian MIPS):

musl-1.2.5$ ./configure --disable-shared --target=mips-linux-gnu \
--prefix=/usr/local/mips-linux-musl CFLAGS="-Os" \
CC=mips-linux-gnu-gcc
musl-1.2.5$ make -j2
musl-1.2.5$ sudo make install

Try a static build of the hello world program they supply:

$ /usr/local/mips-linux-musl/bin/musl-gcc -static hello.c
$ file a.out
a.out: ELF 32-bit MSB executable, MIPS, MIPS32 rel2 version 1
(SYSV), statically linked, not stripped

It's an "MSB executable", but what's that "rel2" after "MIPS32? Is
that not "mips32r1" like is listed supported in /proc/cpuinfo on
the router:
isa                     : mips1 mips2 mips32r1

The binaries from it don't say "rel1" though... Well try it anyway,
and it works! Try building Inetutils telnetd, try that, "Illegal
instruction". Huff. OK try rebuilding Musl for MIPS2 (since GCC
doesn't support a "mips32r1 option either, though it's probably
what "mips32" means):

musl-1.2.5$ make clean
musl-1.2.5$ ./configure --disable-shared --target=mips-linux-gnu \
--prefix=/usr/local/mips-linux-musl CFLAGS="-march=mips2 -Os" \
CC=mips-linux-gnu-gcc
musl-1.2.5$ make -j2
musl-1.2.5$ sudo make install

Yes, GCC is listening to me:

$ file /usr/local/mips-linux-musl/lib/crt1.o
/usr/local/mips-linux-musl/lib/crt1.o: ELF 32-bit MSB relocatable,
MIPS, MIPS-II version 1 (SYSV), not stripped

So try again with hello.c:

$ /usr/local/mips-linux-musl/bin/musl-gcc -march=mips2 -static \
hello.c

$ file a.out
a.out: ELF 32-bit MSB executable, MIPS, MIPS32 rel2 version 1
(SYSV), statically linked, not stripped

Argh! GCC will build one thing for MIPS2 but not the other!

But actually it's the linker playing tricks, because when disabling
linking with "-c" to replicate building the Musl libraries (I've
ditched the musl-gcc helper script which just adds the "-specs"
argument now):

$ mips-linux-gnu-gcc -march=mips2 -static -c -specs \
"/usr/local/mips-linux-musl/lib/musl-gcc.specs" -o hello hello.c
$ file hello
hello: ELF 32-bit MSB relocatable, MIPS, MIPS-II version 1 (SYSV),
not stripped

So why isn't the linker behaving itself? Wading through the output
of GCC in verbose ('-v') mode, I see it's linking against some
object files from the Debian mips-linux-gnu-gcc package,
crtbeginS.o and crtendS.o. What's 'file' have to say about these?

$ file /usr/lib/gcc-cross/mips-linux-gnu/12/crtendS.o
/usr/lib/gcc-cross/mips-linux-gnu/12/crtendS.o: ELF 32-bit MSB
relocatable, MIPS, MIPS32 rel2 version 1 (SYSV), not stripped

"MIPS32 rel2"! The Debian package is built for mips32r2 minimum
(turns out I missed that on the Debian Wiki MIPSPort page)! What if
I modify the specs file to remove those object files from the
linker command:

From this in /usr/local/mips-linux-musl/lib/musl-gcc.specs
(identical parts omitted):
------------------------------------------------------------------
*startfile:
%{!shared: /usr/local/mips-linux-musl/lib/Scrt1.o} /usr/local/mips-linux-musl/lib/crti.o crtbeginS.o%s

*endfile:
crtendS.o%s /usr/local/mips-linux-musl/lib/crtn.o
------------------------------------------------------------------

To this in my-musl-gcc.specs:
------------------------------------------------------------------
*startfile:
%{!shared: /usr/local/mips-linux-musl/lib/Scrt1.o} /usr/local/mips-linux-musl/lib/crti.o

*endfile:
/usr/local/mips-linux-musl/lib/crtn.o
------------------------------------------------------------------

$ mips-linux-gnu-gcc -v -save-temps -march=mips2 -static -specs \
my-musl-gcc.specs -o hello hello.c

$ file hello
hello: ELF 32-bit MSB executable, MIPS, MIPS-II version 1 (SYSV),
statically linked, not stripped

It builds a MIPS 2 executable! And in spite of the object-file
amputations, it runs on my router!

Now to try again with Inetutils:

inetutils-2.5$ make clean
inetutils-2.5$ ./configure --disable-clients --enable-year2038 \
--disable-ipv6 --disable-rpath --disable-threads \
--host=mips-linux-gnu CFLAGS="-march=mips2 -Os -static -specs \
/usr/local/mips-linux-musl/lib/my-musl-gcc.specs" \
CC=mips-linux-gnu-gcc
inetutils-2.5$ make -j2
inetutils-2.5$ sudo make install

Oh for pity's sake, now it won't compile:

In file included from ../config.h:3654,
                from logwtmpko.c:19:
logwtmp.c: In function '_logwtmp':
./confpaths.h:36:20: error: '_PATH_WTMPX' undeclared (first use in
this function); did you mean 'PATH_WTMPX'?

Seems this is a common issue with Musl LibC builds, although why it
waited until now to start fighting me I have no idea (I tried again
in a freshly-unpacked source tree with the same result):
https://busybox.busybox.narkive.com/VsufxeOv/utmp-fallback-not-complete-enough

I ended up with a hacky fix of just hardcoding paths in confpaths.h:
------------------------------------------------------------------
#endif
#ifndef PATH_WTMP
#define PATH_WTMP "/var/log/wtmp"
//#define PATH_WTMP _PATH_WTMP
#endif
#ifndef PATH_WTMPX
#define PATH_WTMPX "/var/log/wtmp"
//#define PATH_WTMPX _PATH_WTMPX
#endif
------------------------------------------------------------------

Yay it builds, now let's try 'file' (on ftpd, because I forgot I
could already get a FTP server from the vsftpd package):

$ file ftpd/ftpd
ftpd/ftpd: ELF 32-bit MSB executable, MIPS, MIPS32 rel2 version 1
(SYSV), statically linked, not stripped

Oh you evil bastard, it's gone back to "MIPS32 rel2" again. But
what the hell, try it anyway:

root@OpenWrt:~# ./ftpd --help
Usage: ftpd [OPTION...]
File Transfer Protocol daemon.

 -4, --ipv4                 restrict daemon to IPv4
 -6, --ipv6                 restrict daemon to IPv6
 -a, --auth=AUTH            use AUTH for authentication
 -A, --anonymous-only       server configured for anonymous
service only
 -d, --debug                debug mode
 -D, --daemon               start the ftpd standalone
 -l, --logging              increase verbosity of syslog messages
..

It works! No "Illegal instruction" to be seen. Actually log-in to
the FTP server fails with a "command not found" error, and no clue
as to the command, but then I rememberd vsftpd anyway and it's
smaller than the ftpd static binary (41KB vs 335KB (stripped)).
inetd and telnetd work fine after making a symlink at
/usr/local/bin/login pointing to /bin/login. Success! I have a
working Telnet server. Somehow...

So I can finally cross-compile software to run on my router, and at
least now the hoops to jump through are better documented than the
huge mess of a notes file that I originally compiled (I skipped
plenty of dead-end research and testing). Seems I really should
have just tried doing it on a much older Debian version (Debian
8 (Jessie) supported MIPS 2 apparantly) on my older laptop, or
would I then have got lots more compiler errors from older GCC?
You just can't pick the right path to take for these things
sometimes...

- The Free Thinker

You are viewing proxied material from aussies.space. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.