Installing software on EdgeOS devices
Introduction
When I was looking for a nice low-power router with lots of features I spent a lot of time looking at the Ubiquity EdgeRouter X. Previously I was using a whitebox router based off Ubuntu 16.04 LTS, but that machine would draw around 80W of power all the time. Admittedly, the machine was overkill for a home router—a dual core Pentium E6700, 8 GiB RAM, quad-gigabit Intel nic, and a 64 GiB SSD—but that gives you a lot of room to play with in terms of additional functionality. What really caught my eye about the EdgeRouter X was the power draw: 5W. At around $51 USD, this puts the EdgeRouter X at approximately the same level as a Raspberry Pi, in fact, the new RPi 3 draws more than 5W.
About the hardware
The EdgeRouter X rocks suprisingly capable hardware, as a little poking around shows:
coleman@hydrogen ~ $ uname -a
Linux hydrogen 3.10.14-UBNT #1 SMP Mon Nov 2 16:45:25 PST 2015 mips GNU/Linux
coleman@hydrogen ~ $ lscpu
Architecture: mips
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
coleman@hydrogen ~ $ free -h
total used free shared buffers cached
Mem: 249M 183M 65M 0B 25M 93M
-/+ buffers/cache: 64M 184M
Swap: 0B 0B 0B
coleman@hydrogen ~ $
So that’s a “quad core” (two real cores, two threads per core) MIPS32 1004Kc, 256 MB DDR3 RAM, and 256 MB NAND flash memory. Remember the fact that it has just as much RAM as NAND storage, this will be important later.
Caveat Emptor
But why do I care about the hardware? I thought this was about installing software, you ask. Well, you need to know about your hardware for a couple reasons.
The EdgeRouter X, while quite the powerful little package, has pretty weak specs compared to most of the computers we are used to working on. You need to keep in mind the hardware limitations of the device when you are installing software on these devices.
Don’t be like me and run asudo apt update && sudo apt upgrade. Yes, it worked and the EdgeRouter X started upgrading itself, but almost immediately it filled up 100% of the NAND storage and then the RAM.
Well shit. Oh well, I guess I’ll just grab a paperclip, hit the reset button on the back of the unit and everything will be fine right? Nope, after that the unit would not boot at all. Even the good old freezer trick failed to improve the situation, so I had to RMA the unit. Which incidentally was absolutely no problem, Ubiquity support was great, but annoying all the same. So be careful and know your hardware.
Another reason you need to know the hardware is to get the correct software for the architecture of your device. In our case we have a Little Endian MIPS processor, so we need to make sure that we get the right packages.
Installing the software
This part is fairly easy, the only hard part is finding the software you want on packages.debian.org. Make sure to limit your search to oldstable, as newer packages most likely won’t work since they’ll probably have dependencies on other more updated packages. For our example, I’ll be installing vnstat, a super helpful network monitoring package that lets you easily keep an eye on your network traffic stats. At the bottom of the package search results page, there will be a list of architectures that the package is available for. Be sure that you use the package for the mipsel architecture.
Now simply copy the link from your mirror of choice, and we can start downloading the package straight on the EdgeRouter X itself with a bit of help from the venerable curl:
coleman@hydrogen ~ $ curl -O http://ftp.us.debian.org/debian/pool/main/v/vnstat/vnstat_1.11-1_mipsel.deb
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 83914 100 83914 0 0 149k 0 --:--:-- --:--:-- --:--:-- 197k
Again, be sure you have enough space for the package. Lucky for us, vnstat is quite lightweight:
coleman@hydrogen ~ $ la vnstat_1.11-1_mipsel.deb
-rw-r--r-- 1 coleman users 81.9K Feb 2 04:07 vnstat_1.11-1_mipsel.deb
Now, I would reccomend su‘ing to root before we manually install the package using dpkg. You may be tempted to use sudo, but for me I ran into quite a few issues with having a messed up env due to the strange way Ubiquity configured the shell.
root@hydrogen:/home/coleman# dpkg -i vnstat_1.11-1_mipsel.deb
(Reading database ... 30413 files and directories currently installed.)
Preparing to replace vnstat 1.11-1 (using vnstat_1.11-1_mipsel.deb) ...
[ ok ] Stopping vnStat daemon: vnstatd.
Unpacking replacement vnstat ...
Setting up vnstat (1.11-1) ...
[ ok ] Starting vnStat daemon: vnstatd
And that’s it!
A little about vnstat
Now after a little while we should be able to see our network statistics for the month by running vnstat -m
coleman@hydrogen ~ $ vnstat -m
eth0 / monthly
month rx | tx | total | avg. rate
------------------------+-------------+-------------+---------------
Oct '16 983.37 GiB | 584.31 GiB | 1.53 TiB | 4.91 Mbit/s
Nov '16 800.24 GiB | 294.32 GiB | 1.07 TiB | 3.54 Mbit/s
Dec '16 172.96 GiB | 14.48 GiB | 187.44 GiB | 587.06 kbit/s
Jan '17 216.97 GiB | 21.70 GiB | 238.68 GiB | 747.52 kbit/s
Feb '17 6.57 GiB | 611.98 MiB | 7.17 GiB | 592.37 kbit/s
------------------------+-------------+-------------+---------------
estimated 156.59 GiB | 14.22 GiB | 170.80 GiB |
It even will provide nice charts of your usage over time:
coleman@hydrogen ~ $ vnstat -h
eth0 04:17
^ r
| r r
| r r r
| r r r r r r
| r r r r r r
| r r r r r r r
| r r r r r r r
| r r r r r r r r
| r r r r r r r r
| r r r t r r r r r r
-+--------------------------------------------------------------------------->
| 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 00 01 02 03 04
h rx (KiB) tx (KiB) h rx (KiB) tx (KiB) h rx (KiB) tx (KiB)
05 434601 48435 13 3151 2184 21 384276 32225
06 376106 22978 14 4823 2315 22 505504 27258
07 403603 40426 15 12619 4411 23 31003 11632
08 12892 55076 16 3588 1910 00 8660 4772
09 4538 1892 17 1789 1721 01 40174 10507
10 2924 2009 18 3493 2220 02 57586 14522
11 3525 1940 19 171649 8811 03 281204 37240
12 4643 2455 20 459976 47390 04 34236 4918
UPDATE
It has come to my attention that Ubiquity actually has a support article that shows how they recommend installing packages. Their method allows you to have dpkg manage package architectures and dependencies for you, but that comes with some additional problems. The biggest issue is that this can lead to inadvertently installing lots of software if you apt-get‘d some package that had tons of dependencies. Although my way is more tedious, it is nothing like waiting 2 weeks for an RMA.