MIPS-X

The Next IoT Frontier
Patrick Ross - Village Idiot Labs
Zoltan Balazs - Head of Vuln Research @ CUJO AI
Hacktivity 2021
MIPS-X

INTRO
Patrick Ross
● Co-founder Village Idiot Labs (@villageidiotlab)
○ Security research in IoT
○ Content developer (labs + ctf + etc)
● Co-organizer of IoT Village
● Defcon 26 Black Badge holder

INTRO
Zoltan Balazs
Head of Vulnerability Research Lab @ CUJO AI
● Zombie Browser Toolkit
● HWFW Bypass tool (Similar stuﬀ was used in PacketRedirect in
Danderspritz FlewAvenue by EQGRP)
● Malware Analysis Sandbox Tester tool
● Played with crappy IoT devices – my RCE exploit code running on
~600 000 IP cameras via Persirai
● Invented the idea of encrypted exploit delivery via Diﬀie-Hellman key
exchange, to bypass exploit detection appliances
● Co-organizer of the Hackersuli meetup
○ this is my 12th presentation here
● Programme committee member of the Hacktivity conference
● Volunteer at IoTVillage

WHY SHOULD YOU LISTEN TO THIS?
● Are you into emulating IoT devices?
● Are you interested in the suﬀering of embedded device development?
● Have you ever debugged MIPS Assembly?
● Do you want to build a lab for IoT devices?
● stay for the memes
● Enjoy the show!
y
e
s
no
● stay for the learning opportunity to ﬁnd 0-day in your IoT devices

… somehow get root on device … because you know, this is why
you want to find bugs and exploit it
… welcome to catch 22
$ gdbserver
/bin/sh: gdbserver: not found
wget http://myserver/gdbserver
.. can’t create file … read-only filesystem
Everyone has a different definition of “fun”
Traditional debug and exploit development

The current tools available did not ﬁt our needs..
● Firmadyne
Cool, but a bit bloated
● Some outdated, incorrect blog posts on how to do this manually
● ARM-X
Also Cool, but no MIPS support :-(
WHY ARE WE DOING THIS (cont’d)
Let’s add it :)

● Architectures
● Virtualization vs Emulation
● Intro to the pain
○ MIPS-X
■ Kernel
■ Filesystem
■ NVRAM
■ QEMU VM
○ Docker
● DEMO!
TALK AGENDA

● Intel/AMD - CISC, desktops, servers
● ARM - RISC, embedded, now Macbook
● MIPS - RISC, embedded
● PowerPC - RISC, embedded, old Apple
● Sparc - RISC, mostly high-end servers, embedded
CPU Architectures

● Traditional virtualization like Virtualbox or VMware won’t help
you here...
● You have to emulate the CPU instruction set
● QEMU has the best support for CPU emulation for diﬀerent CPUs
● QEMU development boards are basically “virtual machines”,
where the HW and BIOS is “emulated”
● “FUN” FACT: kernels compiled for an IoT device won’t run in a
QEMU emulated environment, as HW, peripherals is totally
diﬀerent
VIRTUALIZATION vs EMULATION

WHICH
KERNEL?
If you want to build your own, new kernel
● “Just use Buildroot”
If you need an older, pre-built kernel
● https://people.debian.org/~aurel32/qemu/
● “apt-get install ...” works 😎
● just use archive.debian.org as repo
If you want to compile your own - 2.6.x or 3.x MIPS kernel
● Warning

Pre-built
Debian
OR
Buildroot
?
Pre-built debian hostfs ~ 280 Mbyte
Buildroot hostfs ~ 65 Mbyte
How to apt-get Internet connection
1. Start your favourite network proxy on your host
2. sudo socat UDP4-RECVFROM:53,fork UDP4-SENDTO:8.8.8.8:53
3. /etc/resolv.conf to 192.168.100.1
4. echo 'Acquire::http::Proxy "http://192.168.100.1:8080/";' >
/etc/apt/apt.conf.d/proxy.conf
5. Change the apt repositories. Replace everything http://ftp. and
http://security. to http://archive.

Building a suitable MIPS kernel undergoes a lot of trials and tribulations
● Buildroot/toolchain for compiling
● Need a MIPS compiler ... - > cross compiling
○ You can try to compile in QEMU, but it will be sloooooow
● Need legacy kernel as well as headers/modules -> to support old ﬁrmware
○ Think Linux 2.6.32 - one of the most popular Linux version in IoT
● Building a recent MIPS kernel is easy
○ https://gist.github.com/sjaeckel/94120aeb2dacfa693b1dd360de451b89
Static vs dynamically linked binaries
● Key useful ones (gdbserver, netcat, socat, telnetd) are better made from scratch
KERNEL

Choose your cross compiler toolchain wisely
Buildroot
Openwrt
Crosstool-ng
Musl
Code sourcery codebench
Dockcross
Debian cross tools
….
TOOLCHAIN HELL

● QEMU on Debian (and probably other distros) can easily run a single STATIC
or DYNAMIC binary from another CPU architecture!
● Sometimes running a single binary is enough, you don’t need full system
emulation
○ # apt install qemu-user qemu-user-static gcc-mips-linux-gnu
binutils-mips-linux-gnu binutils-mips-linux-gnu-dbg build-essential
○ # mips-linux-gnu-gcc -static -o hello64 hello64.c
○ # mips-linux-gnu-gcc -o hello64dyn hello64.c
● https://azeria-labs.com/arm-on-x86-qemu-user/
SINGLE BINARY vs FULL SYSTEM EMULATION

cd /usr/mips-linux-gnu
sudo mkdir etc
sudo ldconfig -c etc/ld.do.cache -r .

Not all binaries are created equally (MIPS isn’t necessarily MIPS.)
○ MIPS-I (register to register), MIPS-2, MIPS-3 (64 bit
datatypes), MIPS32, MIPS64, microMIPS (code compression,
16/32/64)
● CHROOT and hostfs network ﬁlesystem
● Persistent vs Non-persistent disks
FILESYSTEM

https://github.com/getCUJO/MIPS-X/blob/main/hostfs_builder/buildroot_notes.txt
Use board qemu_mips32r6_malta_defconﬁg
enable SSHD, bash, dialog, gdbserver, enable gcc 10, C++ in toolchain,
nfs
https://github.com/getCUJO/MIPS-X/blob/main/hostfs_builder/README.md
https://github.com/getCUJO/MIPS-X/blob/main/hostfs_builder/hostfs_builder.sh
Building the hostfs

● Typically needed to support the target
binaries
● Workﬂow for reverse engineering what
NVRAM parameters are needed
● Discover parameters
● (try to) Emulate them :-)
NVRAM

● Networking
○ Default QEMU nic does NOT like to do it the way WE want to
● Goal is to have the same memory layout as the IoT device
○ If this is achieved, exploits become portable
● Accessibility
○ Networking (exposing service ports, ability to ssh)
○ Debugging the target binary using GDB/gef
○ Local to the QEMU instance
○ QEMU level debugging (kernel level) FUN
QEMU VM

DOCKER
+------------------------------------------------------------+
| Host machine |
| +------------------------------------------------------+ |
| | Docker container | |
| | +------------------------------------------------+ | |
| | | QEMU | | |
| | | +------------------------------------------+ | | |
| | | | "Host" kernel + filesystem | | | |
| | | | +------------------------------------+ | | | |
| | | | | chrooted IoT firmware rootfs | | | | |
| | | | | +------------+ | | | | |
| | | | | | webserver | | | | | |
| | | | | | | | | | | |
| | | | | +------------+ | | | | |
| | | | +------------------------------------+ | | | |
| | | | +----------------+ +-----------------+ | | | |
| | | | | SSH server | | gdbserver | | | | |
| | | | +----------------+ +-----------------+ | | | |
| | | +------------------------------------------+ | | |
| | +------------------------------------------------+ | |
| +------------------------------------------------------+ |
+------------------------------------------------------------+

● Continue on NVRAM refinement
● Develop a CI/CD model where you can provide a
firmware file, and it will auto unplack/build a shell
IoT VM environment
NEXT STEPS

PROFIT
● Paying it forward to the Security community
● Continue to drive positive momentum in bug
bounty and put pressure on the vendors to
improve their secure systems development
● Open-source means anyone can contribute!

https://github.com/getCUJO/MIPS-X
https://github.com/therealsaumil/armx/pull/17
If you’re feeling lucky...

MIPS-X

More Related Content

What's hot

Similar to MIPS-X

More from Zoltan Balazs

Recently uploaded

MIPS-X