Installing software on EdgeOS devices

Turn your EdgeOS device into a general purpose Linux machine

Introduction

When I was looking for a nice low-power router with lots of features I spent a lot of time looking at the Ubiquity EdgeRouter X. Previously I was using a whitebox router based off Ubuntu 16.04 LTS, but that machine would draw around 80W of power all the time. Admittedly, the machine was overkill for a home router—a dual core Pentium E6700, 8 GiB RAM, quad-gigabit Intel nic, and a 64 GiB SSD—but that gives you a lot of room to play with in terms of additional functionality. What really caught my eye about the EdgeRouter X was the power draw: 5W. At around $51 USD, this puts the EdgeRouter X at approximately the same level as a Raspberry Pi, in fact, the new RPi 3 draws more than 5W.

About the hardware

The EdgeRouter X rocks suprisingly capable hardware, as a little poking around shows:

[email protected] ~ $ uname -a
Linux hydrogen 3.10.14-UBNT #1 SMP Mon Nov 2 16:45:25 PST 2015 mips GNU/Linux
[email protected] ~ $ lscpu
Architecture:          mips
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    2
Core(s) per socket:    2
Socket(s):             1
[email protected] ~ $ free -h
             total       used       free     shared    buffers     cached
Mem:          249M       183M        65M         0B        25M        93M
-/+ buffers/cache:        64M       184M
Swap:           0B         0B         0B
[email protected] ~ $ 

So that’s a “quad core” (two real cores, two threads per core) MIPS32 1004Kc, 256 MB DDR3 RAM, and 256 MB NAND flash memory. Remember the fact that it has just as much RAM as NAND storage, this will be important later.

Caveat Emptor

But why do I care about the hardware? I thought this was about installing software, you ask. Well, you need to know about your hardware for a couple reasons.

The EdgeRouter X, while quite the powerful little package, has pretty weak specs compared to most of the computers we are used to working on. You need to keep in mind the hardware limitations of the device when you are installing software on these devices.

Don’t be like me and run asudo apt update && sudo apt upgrade. Yes, it worked and the EdgeRouter X started upgrading itself, but almost immediately it filled up 100% of the NAND storage and then the RAM.

Well shit. Oh well, I guess I’ll just grab a paperclip, hit the reset button on the back of the unit and everything will be fine right? Nope, after that the unit would not boot at all. Even the good old freezer trick failed to improve the situation, so I had to RMA the unit. Which incidentally was absolutely no problem, Ubiquity support was great, but annoying all the same. So be careful and know your hardware.

Another reason you need to know the hardware is to get the correct software for the architecture of your device. In our case we have a Little Endian MIPS processor, so we need to make sure that we get the right packages.

Installing the software

This part is fairly easy, the only hard part is finding the software you want on packages.debian.org. Make sure to limit your search to oldstable, as newer packages most likely won’t work since they’ll probably have dependencies on other more updated packages. For our example, I’ll be installing vnstat, a super helpful network monitoring package that lets you easily keep an eye on your network traffic stats. At the bottom of the package search results page, there will be a list of architectures that the package is available for. Be sure that you use the package for the mipsel architecture.

Now simply copy the link from your mirror of choice, and we can start downloading the package straight on the EdgeRouter X itself with a bit of help from the venerable curl:

[email protected] ~ $ curl -O http://ftp.us.debian.org/debian/pool/main/v/vnstat/vnstat_1.11-1_mipsel.deb
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 83914  100 83914    0     0   149k      0 --:--:-- --:--:-- --:--:--  197k

Again, be sure you have enough space for the package. Lucky for us, vnstat is quite lightweight:

[email protected] ~ $ la vnstat_1.11-1_mipsel.deb
-rw-r--r--    1 coleman  users      81.9K Feb  2 04:07 vnstat_1.11-1_mipsel.deb

Now, I would reccomend su‘ing to root before we manually install the package using dpkg. You may be tempted to use sudo, but for me I ran into quite a few issues with having a messed up env due to the strange way Ubiquity configured the shell.

[email protected]:/home/coleman# dpkg -i vnstat_1.11-1_mipsel.deb
(Reading database ... 30413 files and directories currently installed.)
Preparing to replace vnstat 1.11-1 (using vnstat_1.11-1_mipsel.deb) ...
[ ok ] Stopping vnStat daemon: vnstatd.
Unpacking replacement vnstat ...
Setting up vnstat (1.11-1) ...
[ ok ] Starting vnStat daemon: vnstatd

And that’s it!

A little about vnstat

Now after a little while we should be able to see our network statistics for the month by running vnstat -m

[email protected] ~ $ vnstat -m

 eth0  /  monthly

       month        rx      |     tx      |    total    |   avg. rate
    ------------------------+-------------+-------------+---------------
      Oct '16    983.37 GiB |  584.31 GiB |    1.53 TiB |    4.91 Mbit/s
      Nov '16    800.24 GiB |  294.32 GiB |    1.07 TiB |    3.54 Mbit/s
      Dec '16    172.96 GiB |   14.48 GiB |  187.44 GiB |  587.06 kbit/s
      Jan '17    216.97 GiB |   21.70 GiB |  238.68 GiB |  747.52 kbit/s
      Feb '17      6.57 GiB |  611.98 MiB |    7.17 GiB |  592.37 kbit/s
    ------------------------+-------------+-------------+---------------
    estimated    156.59 GiB |   14.22 GiB |  170.80 GiB |

It even will provide nice charts of your usage over time:

[email protected] ~ $ vnstat -h
 eth0                                                                     04:17 
  ^                                                     r                       
  |                                               r     r                       
  |  r                                            r     r                       
  |  r  r  r                                      r  r  r                       
  |  r  r  r                                      r  r  r                       
  |  r  r  r                                      r  r  r              r        
  |  r  r  r                                      r  r  r              r        
  |  r  r  r                                   r  r  r  r              r        
  |  r  r  r                                   r  r  r  r              r        
  |  r  r  r   t                               r  r  r  r           r  r        
 -+---------------------------------------------------------------------------> 
  |  05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 00 01 02 03 04    
                                                                                
 h  rx (KiB)   tx (KiB)      h  rx (KiB)   tx (KiB)      h  rx (KiB)   tx (KiB) 
05     434601      48435    13       3151       2184    21     384276      32225
06     376106      22978    14       4823       2315    22     505504      27258
07     403603      40426    15      12619       4411    23      31003      11632
08      12892      55076    16       3588       1910    00       8660       4772
09       4538       1892    17       1789       1721    01      40174      10507
10       2924       2009    18       3493       2220    02      57586      14522
11       3525       1940    19     171649       8811    03     281204      37240
12       4643       2455    20     459976      47390    04      34236       4918

UPDATE

It has come to my attention that Ubiquity actually has a support article that shows how they recommend installing packages. Their method allows you to have dpkg manage package architectures and dependencies for you, but that comes with some additional problems. The biggest issue is that this can lead to inadvertently installing lots of software if you apt-get‘d some package that had tons of dependencies. Although my way is more tedious, it is nothing like waiting 2 weeks for an RMA.