As with the previous post, please forgive this essentially non-ESP8266 related article (it is actually loosely connected, as the CuBox-i4p will be a gateway server). Normal (ESP8266) service will be resumed as soon as possible.
As you may have seen from the previous post, I’m having difficulty getting OpenBSD installed on the CuBox-i4p (four-core ARM with 2GB of memory and a Gbit Ethernet interface). As I wasn’t getting anywhere fast with OpenBSD, I decided to give FreeBSD a go, as I already have an ARM box (an RPi-B) running 10.2 quite happily. Downloading the FreeBSD release was a doddle (once you realize that you can ignore the “build using crotchet” advice scattered around on the web) and I dd’ed it to an SD card without any special, magical options. Booting wasn’t quite so trouble free, though.
Having learnt (a little) from the OpenBSD experience, I plugged the USB keyboard into the bottom port, inserted the newly minted SD card and powered on. The monitor sprang into life and the SolidRun header appeared (Yay!), but there was no countdown from U-Boot, so there was nothing to interrupt with a key-press. And there it stuck.
It seemed likely that the CuBox was squirting boot info to the semi-secret micro-USB port, so I dutifully connected up my laptop and started “screen” (and, in yet another aside, I’d just like to say a big “Thank You!” to the guys and girls at the Technical University of Berlin for a program which has been invaluable to me for more years than I care to remember) and found myself at a “loader>” prompt. Not quite what I’d expected, or hoped for, but at least there was life in the little beastie. Typing in “boot” got me a semi-sane looking initial boot sequence, but immediately after starting to load from the SD filesystem it bombed out with “No device tree blob found”. Duh!
On my previous foray into the land of U-Boot I was impressed how versatile it was and how easy it was to boot the OpenBSD installer from my tftpboot server. Having said that though, I’m definitely a U-Boot newbie and, besides, it looked rather as though the problem lay with the FreeBSD loader, rather than U-Boot itself (which rather surprised me, as I’d assumed that the 10.2 release would just work — feel free to correct me if you know that I’m pointing the finger of blame in the wrong direction). Anyway, I vaguely remembered a short thread on the FreeBSD mailing list about boot problems with the Hummingboard (another SolidRun product) and, in the course of searching for that thread, came across another, much longer one where the inimitable Ian Lepore came up with the answer:-
setenv fdt_file imx6q-cubox-i.dtb
It took me a little while to realize that this was, in fact, to be executed from the U-Boot prompt, not the loader, but once I’d restarted, interrupted the countdown from the console and typed in the magic incantation, everything sprang into life and the kernel loaded without a problem. It turns out that U-Boot passes some environmental variables to the FreeBSD loader and, for whatever reason, the initial set-up of those variables isn’t working quite right for the SolidRun hardware and, while the device-tree blob file (under /boot/dtb) is actually there, it’s not being correctly identified. The setenv command (above) sets the correct file name in U-Boot, which is then imported by the loader, leading to much joy and jubilation. The saveenv ensures that the variable remains set between reboots. Once your FreeBSD system comes up, you probably still won’t have anything on the monitor, but you will be able to access your system via the network (remember, SSH root access is disabled, so use the “freebsd” username [passwd “freebsd”] to access the system and then su to root from there).
So there you have it …even if we can’t get OpenBSD to install quite yet, we can still get FreeBSD up and running (and, for those people who are about to leave a “Why don’t you use Linux, it just works” comment, I would ask you in return to try this little experiment — boot a *BSD system and a Linux system. On each machine, type “ps ax” and then for each running process just try doing “man <process-name>” and let me know how you get on). 😛
ESP8266 readers …normal service will be resumed as soon as possible; I just wanted to get this info out somewhere where the bots/spiders would pick it up.
–Important– There are several updates appended to the bottom of this article. Even if you don’t read anything else, please read the updates. There are now so many updates that they’re longer than the original article, so here’s a mini index:-
– Getting a USB keyboard to work with the CuBox.
– Getting external USB devices to work.
– Network issues on 10/100Mbs networks during install.
– Installing OpenBSD onto an external device.
– Enabling default boot from an external device.
‡ – Note on accessing U-Boot.
I recently bought a CuBox-i4 for use as a gateway, mainly based on the fact that it would (according to the blurb) run a gazzillion different versions of Linux, Android (whatever that is! :-)) and, according to the latest info on the OpenBSD ARMV7 page, OpenBSD 5.8, too.
Well, as usual with these things, all did not go according to plan. The OpenBSD installation notes don’t have anything specific for the CuBox, but the general instructions are “dd the miniroot-cubox-58.fs to your SD card, bung it back into the slot and apply power” (I paraphrase slightly). Having connected a monitor via HDMI and a USB keyboard, I dd’ed, bunged and powered. The monitor sprang into life, came up with the SolidRun header and a very brief (3-second) countdown, flashed another 10 odd lines on the screen much too quickly for anyone to read and then …went black again. I repeated the process several times (and was never able to read the quick-as-a-flash message), but with no success. I tried several different SD cards – no change. I tried a couple of the other images – yes, something changed …the red light no longer came on and the monitor didn’t even show the initial countdown. The bottom line was though, I couldn’t seem to get OpenBSD to boot. It’s also worth noting that the “Hit any key to interrupt boot sequence” did NOT work with the USB keyboard (interesting, since there doesn’t seem to be any other way of connecting a keyboard to the system [See Update#1, below]).
Just to make sure that everything was okay with the CuBox itself, I loaded a couple of different Linux distributions and booted them successfully. I also ran SolidRun’s “Ignition” initializer/chooser, which worked fairly well, but slowly (if you can use “dd” it’s not really worthwhile waiting for Ignition to do its stuff). The bottom line was that the CuBox itself and the SD cards all worked fine with different distributions.
Back to the OpenBSD installation guide …the instructions for the BeagleBone (the only detailed instructions in the guide) were based around a serial console connection, so the next step was obviously to try to access the console on the CuBox. I had completely overlooked the micro-USB connector in the specs when I ordered the box and then, going to the SolidRun web site and browsing their forums, things started to become clear murky, very, very murky (hence this post). On the SolidRun product spec page they label this port (number “9” in the photo) as “Micro-USB to RS232” and that’s basically it. In the forums there’s a whole load of waffle about what this actually means and what voltages are available where, what sort of USB-to-Dsub adapter you need(?!?!), etc, etc. What a load of cobblers! Plug a damn USB cable into the Micro-USB port, plug the other end into a USB socket on whatever machine you have handy. Fire up minicom, screen, pooty, or whatever other comms program you have, setting the baud-rate to 115200 along the way and you’re in business. [SolidRun – The world doesn’t care if you have a fully-fledged RS232 port hidden inside your amazingly small box. Just try to de-obfuscate your specs a little and try to give the rest of us a bit of a clue …something like “Micro-USB console connector (default 115200 baud)” would do fine]
Okay, so now I plug in the same SD card, with the same miniroot image and lo, the machine boots into the OpenBSD 5.8 installer just fine. Phew!
Now I said it booted …I didn’t say I actually managed to get it to install. So far I’ve booted it from SD and across the net, using tftpboot. I’m blown away by how good U-Boot‡ is (very, very cool!), but disappointed with my installation attempts so far. It seems that the installer kernel doesn’t recognize any other device plugged into the (normal) USB ports and, in addition, throws I/O errors whenever I try to do anything with the SD card, so at the moment, I have the installer, but I can’t actually install anything. Duh! I’ll keep trying and update this post when (if) I find the answer. In the meantime, if anyone has already got this to work, please do add a comment to enlighten the rest of us.
TLDR;Summary:- You need to connect a USB cable between the micro-USB port on the CuBox and another system running a terminal-emulator/comms program to be able to interact with U-Boot‡ and to see the output from the OpenBSD kernel/installer.
Update #1 – The USB keyboard issue (not interrupting the U-Boot‡ process). It turns out that the answer to this one is quite simple …the USB keyboard must be plugged directly into the bottom of the two USB ports. This is one of those things where even just half a page of documentation in the box would save a lot of people a whole shirt-load of time.
Update #2 – This is really just an addition to #1 …as well as the keyboard, other devices will only work reliably if they are plugged into the bottom port. So, if you want to install to a USB drive (USB-thumb, or HD), you’ll need to plug it into the bottom port, or plug a powered hub into the bottom port and plug your USB drives and devices into that. This, it turns out, is a known issue with OpenBSD on the CuBox-i4.
Update #3 – When the going gets weird, the weird get …baffled. You’ll remember that I noted above that I’d successfully booted the installer using TFTP from another server on my home network. Well, after going through all of the initial set-up in the installer, I selected the HTTP install type, hit and …”Unable to retrieve file list from ftp.openbsd.org”. I changed the target server and tried again …same result. Dropping back into the shell and doing an ifconfig -a showed the status on interface imxenet0 as “no carrier” (this was the same interface which had just downloaded the tftpboot image a few minutes earlier). WTDF!?! (where “D” = Double). Trying to bring the interface back on line with ifconfig imxenet0 up had no effect. Trying to alter the media type produced weird results; the media line in the ifconfig display would show the requested media type, but followed by a completely different setting in parentheses, for example media: 100baseTX full-duplex (10baseT half-duplex). I knew for a fact that the network switch which the cable was plugged into was a 100baseT full-duplex connection, so it was logical that the 100baseTX selection should have worked, but it didn’t. I worked through the various types that ifconfig imxenet0 media showed for that interface (wet-string, paper-cups, sneaker-net, etc) and, finally, 1000baseT mediaopt full-duplex gave me a live network (with the media line of ifconfig telling me “1000baseT full-duplex (100baseTX full-duplex)“. I have absolutely no idea what’s going on with the imxenet driver, but it seems like copying the code from the U-Boot repository might be a safe bet for the next release.
Okay, so we can now get our interface initialized and working, but how do we use it inside the installer? Fairly simply, as it turns out. Boot the installer, go through the first few steps, including the network set-up using the actual values for IP addresses, gateway and DNS server that you’d like to use and then just interrupt the installer (with a C) when you get to the network “done” stage. Then, at the shell prompt, reinitialize your interface using the same IP address, but with the media type and options specified as above, so it looks something like this:-
ifconfig imxenet0 media 1000baseT mediaopt full-duplex 192.168.1.200 netmask 255.255.255.0 up
…and check that your interface is now live (you should be able to ping local IP addresses and to ping your CuBox-i4 from other systems).
You can now get back into the installer by simply typing install. When you restart using this method, the installer will give you your previously input answers as the defaults when going through the prompts, making it easy to speed through the stuff you’ve already done.
-Important- When you reach the “network interface to configure? [imxenet0]: ” you must type “done" to prevent the installer from changing the configuration you’ve just gone to such pains to get working. From this point onwards you can simply follow your normal installation procedure. Phew!
It’s worth noting here that this problem only seems to occur when installing on a 10/100Mbs link; when I plug the CuBox into a Gbe switch connection the install goes all the way through without a hiccough. The issue (on the slower network) still occurs with OpenBSD 6.0.
Update #4 – Well, there’s yet another wrinkle here that you need to work around to complete the install and that’s how to write to the SD card (which shows up as drive “sd0”) from the OpenBSD installer. As you may already have found, you can’t. Selecting “sd0” as your target disk in the installer leads to a long string of “I/O error” and “write failed” messages, with the labelling, partitioning, filesystem build and mount procedures all just ignoring the failures and speeding merrily on their way (doesn’t anyone check return values, nowadays?!?). It seems that the installer can’t write to the MMC slot on the CuBox, even though it can read it (earlier on we booted the installation image from it).
The answer to this is to use a USB card reader/writer plugged into the bottom USB socket on the CuBox . Plug a second SD card into the reader/writer and restart the installer. The second card should show up as “sd1” (it’s worth making sure that you positively identify your target card at this point …it could make a bad day even worse if you scribble over something else by accident) and should be writeable by the installer. As the OpenBSD /etc/fstab uses the drive serial identifier (rather than the old fashioned device name) nowadays, you should be able to move sd1 from the reader/writer into the SD slot on the CuBox without any issues, once the installation has successfully completed.
Once your system has rebooted into the newly installed 5.8 on the second SD card (now installed in the CuBox SD card slot) it will somehow, magically be able to write to it and you can take your CuBox out for a test drive …just don’t try to run “top” (C’mon! I just told you not to do that!).
Update #5 – Once you have OpenBSD booting and that first flush of exhilaration is gone, you realize that you need to get your OS onto something a little speedier and a lot more robust. The issue you then hit is that the CuBox itself is pretty much a bare metal device and won’t boot unless it has an SD card in the slot …and once it finds that SD card, no matter what the fstab says, it doesn’t want to let it go and will stubbornly refuse to mount root from the external disk. That kinda’ defeats the purpose of the exercise.
However, it turns out that U-Boot‡ can help us with that. Buried deep within the environmental settings is “boot_target”, which is basically the same as the old BIOS settings for a preferred boot-device list. The default list is “mmc0 sd0 pxe dhcp”, so all we need to do is to change it to boot the SCSI disk as it primary, using this command:-
setenv boot_target sd0 mmc0 pxe dhcp
The “saveenv” is critical, as it (as the name implies) makes the change permanent across power cycles. Note that there’s no “=” between the setting name “boot_target” and the list of devices and there are no quotes around the members of the list; everything on the command line is simply space-separated.
‡ Accessing U-Boot is simply a matter of pressing any key (on the USB keyboard you have attached to the -lower- USB port, or on the terminal you have connected to the mini-USB, front-panel port) within three seconds of powering on the CuBox. Type “help” to get a list of commands (-warning-, steep learning curve ahead).
§ The secret incantation to enable booting the CuBox from an external disk is in “Update #5”, above (just in case you’re a Ooleg-bot and missed it).