Category Archives: Projects

Full Raspberry Pi (Raspbian) emulation with qemu

I wanted to do some experimental hacking on my Raspberry Pi, specifically to try a bit of fun with talking to Arduino and Spark Cores. My ultimate aim was to have a go at doing something fun with the meArm robotic arm (https://www.wevolver.com/#/project/196/parts/629/document)

I started off compiling OpenCV and OpenNI on the physical pi, but quickly realised I didn’t have a big enough SD card lying around. I momentarily thought about stealing one of my wife’s pro camera SD cards, but then thought about the consequences… I then decided to emulate the whole thing and then buy an SD card when the project was done.

First off, you need a qemu environment. I’ll assume you have a basic qemu installation going, but if not, get started with

sudo apt-get install qemu-system qemu-user-static binfmt-support

Next, you will need to download the latest raspbian release image. Make a directory to use, and then grab it

mkdir ~/qemu_vms
cd ~/qemu_vms
wget http://downloads.raspberrypi.org/raspbmc_latest

You also need a kernel:

wget http://xecdesign.com/downloads/linux-qemu/kernel-qemu

XEC Design maintains a qemu kernel with the ARMhf patches already, but if you would like to build your own one, feel free to grab it at https://github.com/raspberrypi/linux

You will need to extract the zip archive that you just downloaded, and you should be left with something like:

~/qemu_vms$ ls
2014-06-20-wheezy-raspbian.img kernel-qemu

which means you are ready to start doing cool stuff! (Remember that if you are reading this, the .img file has probably changed, so keep a note of that!)

Lets boot this thing up!

qemu-system-arm -kernel kernel-qemu -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw init=/bin/bash" -hda 2014-06-20-wheezy-raspbian.img

which should start up qemu with a command prompt. Login with the default credentials (user: pi, pass: raspberry) and have a cookie for getting this far.

Now, you will notice that not everything can be emulated by qemu, so change /etc/ld.so.preload like this

nano /etc/ld.so.preload
#Comment out the libcofi_rpi object like this
#/usr/lib/arm-linux-gnueabihf/libcofi_rpi.so

Now you need to edit

/etc/udev/rules.d/90-qemu.rules

(This is a new file!)
Add the following to your new file:

KERNEL=="sda", SYMLINK+="mmcblk0"
KERNEL=="sda?", SYMLINK+="mmcblk0p%n"
KERNEL=="sda2", SYMLINK+="root"

Now you should halt/shutdown the system, and prepare for your first real boot!

Boot up again with

qemu-system-arm -kernel kernel-qemu -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" -hda 2014-06-20-wheezy-raspbian.img

Do a df -h and notice with horror that you have almost no space to work with!

Resizing the image “disk” is pretty easy though.

First close down the emulator again, then

qemu-img resize 2014-06-20-wheezy-raspbian.img +4G

This will make your partition 6GB long (do more if you like…) which should be plenty of space and will fit onto a relatively cheap 8GB SD Card.

Now boot up your emulator again and do:

sudo ln -snf mmcblk0p2 /dev/root
sudo raspi-config

Choose the first option to resize your disk, and it will tell you to reboot. Great, once everything is halted, manually restart your emulator, and do another df -h. SURPRISE! It now looks like this:

Filesystem      Size  Used Avail Use% Mounted on
rootfs          6.6G  2.1G  4.2G  33% /
/dev/root       6.6G  2.1G  4.2G  33% /
devtmpfs        125M     0  125M   0% /dev
tmpfs            25M  204K   25M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            50M     0   50M   0% /run/shm
/dev/sda1        56M  9.5M   47M  17% /boot

You are done! Great job!

Have fun!

Making a temperature monitor with Spark Core, and a bit of PHP

I have recently started making biltong at home, and coincidentally also received a Spark Core dev kit from http://spark.io as well. I then decided that a quick and simple biltong monitor would be a good starter project, so set out to do just that.

I grabbed this diagram as a starter and built a simple Spark Core temperature sensor (thermometer), straight from the Spark docs

 

breadboard-temp-sensor

 

As soon as your sensor is built, you can then proceed to the Spark Online IDE and flash some code to it

#include <math.h>

// -----------------
// Read temperature
// -----------------

// Create a variable that will store the temperature value
int temperature = 0;
double voltage = 0.0;
double tempinC = 0.0;

char vStr[10];
char tStr[10];

void setup()
{
  // Register a Spark variable here
  Spark.variable("temperature", &temperature, INT);
  Spark.variable("voltage", &voltage, DOUBLE);
  Spark.variable("tempinC", &tempinC, DOUBLE);

  // Connect the temperature sensor to A7 and configure it to be an input
  pinMode(A7, INPUT);
}

void loop()
{
  // Keep reading the temperature so when we make an API
  // call to read its value, we have the latest one
  temperature = analogRead(A7);
  voltage = (temperature * 3.3)/4095;
  tempinC = (voltage - 0.5)*100;

}

/**
 * The API request will look something like this:
 * GET /v1/devices/{DEVICE_ID}/temperature
 * # EXAMPLE REQUEST IN TERMINAL
 * # Core ID is 0123456789abcdef
 * # Your access token is 123412341234
 * curl -G https://api.spark.io/v1/devices/0123456789abcdef/tempinC \
 * -d access_token=123412341234
 */

You should then be able to test it out by sending some HTTP requests to it. Generally, I would use Curl from http://curl.haxx.se, but I am doing some Win8.1 dev and didn’t want to bother with it, so I simply used http://www.hurl.it/ to fiddle with my Spark core.

A couple of things that you need to do to set up with Hurl.it is that you need a copy of your access_token and deviceID from your Spark Core. These are both available in your online IDE sidebar.

Set the parameters in Hurl.it as access_token and then whatever you set your code vars as. In the above example it will be “tempinC”, “voltage”, or “temperature”. We will mostly be concentrating on tempinC though.

Set your URL as

https://api.spark.io/v1/devices/

with your device ID as part of the URL. You may now execute the request and it should return some JSON from your Spark core!

Right, now that we know that it works and is sending us back a valid temperature, we can proceed with a bit of PHP.

Data is not really useful until you do something with it, so let’s build a very quick and VERY dirty PHP/MySQL app to house our data. PLEASE NOTE that this code is awful, so make it better…

First up, the index.php file:

<?php
// Get cURL resource
$curl = curl_init();
// Set some options - we are passing in a useragent too here
curl_setopt_array($curl, array(
    CURLOPT_RETURNTRANSFER => 1,
    CURLOPT_URL => 'https://api.spark.io/v1/devices/1234567890abcdef/tempinC/?access_token=12470717420914290127401274',
    CURLOPT_USERAGENT => 'paulscott.co.za'
));
// Send the request & save response to $resp
$resp = curl_exec($curl);
// Close request to clear up some resources
curl_close($curl);

$data = json_decode($resp);
$temp = $data->result;
echo "<h3>Current temperature is: " . $temp . " C</h3>";

// MySQL stuff... sigh
$dbhost = 'localhost:3036';
$dbuser = 'username';
$dbpass = 'password';
$conn = mysql_connect($dbhost, $dbuser, $dbpass);
if(! $conn )
{
  die('Could not connect: ' . mysql_error());
}
$sql = 'INSERT INTO biltong '.
       '(temperature, humidity, currtime) '.
       'VALUES ( '.$temp.', "0", NOW() )';

mysql_select_db('sparkdata');
$retval = mysql_query( $sql, $conn );
if(! $retval )
{
  die('Could not enter data: ' . mysql_error());
}
//echo "Entered data successfully\n";
mysql_close($conn);

// check out the nifty graph...
echo '<h3>Temperatures in the Biltong maker over time</h3> <img src="graph.php" />';

This basically grabs the data by making a cURL request, then adds it to a MySQL database. Dead simple.
Next we want the file that this file refers to (graph.php)

<?php
include('phpgraphlib/phpgraphlib.php');
$graph = new PHPGraphLib(1100,700); 
$dataArray = array();

$dbhost = 'localhost:3036';
$dbuser = 'username';
$dbpass = 'password';

$link = mysql_connect($dbhost, $dbuser, $dbpass)
   or die('Could not connect: ' . mysql_error());
     
mysql_select_db('sparkdata') or die('Could not select database');
  
$dataArray=array();

//get data from database
$sql = "SELECT temperature, id, currtime FROM biltong LIMIT 150";
$result = mysql_query($sql) or die('Query failed: ' . mysql_error());
if ($result) {
  while ($row = mysql_fetch_assoc($result)) {
      $temperature = $row["temperature"];
      $count = $row["currtime"];
      //add to data areray
      $dataArray[$count] = $temperature;
  }
}
//configure graph
$graph->addData($dataArray);
$graph->setTitle("Temperature");
$graph->setGradient("lime", "green");
$graph->setBarOutlineColor("black");
$graph->setBars(false);
$graph->setLine(true);
$graph->createGraph();
//echo $sql;
?>

Done! You will also need PHPGraphLib from http://www.ebrueggeman.com/phpgraphlib

Now, whenever a request is sent to the index file, it will grab the data and graph it. You can add the curl request to a cron job too and get regular data etc as well.

As I say, this is a simple example, but demonstrates how easy it is to start building some very cool sensors with Spark Core!

Hardware projects of the week

As an avid backer of quite a few interesting kickstarter projects, I have early access to a number of new technologies. Two projects in particular are Bluetooth LE based, which we believe will be a multi million dollar industry in the next couple of years, and one dealing with wearable computing, which is set to explode.

The first project that we would like to talk about is Spark (http://spark.io), which is a programmable networked core that can be made to report data from numerous sensors via HTTP. With the dev kit, we get a number of useful sensors out the box, as well as a high voltage relay to control appliances in your home via REST method calls. What this means, in a simple project, would be that you can turn on your coffee machine from your phone on the way home from work, and have a fresh pot brewed as you walk in the door. There are obviously numerous other applications, which we will be exploring in the coming weeks. Spark cores can also make use of an Arduino shield shield, which will allow you to add on any Arduino compatible shield to expand capabilities.

Next up, is the MetaWear dev board from MbientLabs. See https://www.mbientlab.com/ for more information. MetaWear is a tiny dev board that can be used to power wearable computing devices, and which reports back to your phone via Bluetooth LE. It can be used to quickly create fitness bands, or any other wearable device. It is also relatively inexpensive and has API’s for Android and iOS already, as well as a few sample apps. Metawear can make use of any I2C compliant add on so the applications are almost limitless!

The third product that we would like to bring your attention to is the PowerUp3.0 device. https://www.kickstarter.com/projects/393053146/powerup-30-smartphone-controlled-paper-airplane
This device is sold as a toy, and enables you to use a Bluetooth LE receiver with your phone to create a remote controlled paper aeroplane. The interesting portion of this device is that we can see many interesting applications with it, both as a toy and not!

Another Bluetooth device that we are currently working on, with the Raspberry Pi as a back end, will allow us to transmit data of all sorts via a cheap commodity Bluetooth transmitter. This is very similar in nature to an Apple iBeacon and we are imagining a world where these things are attached to billboards at busy traffic intersections. The user will be able t receive updates on entertainment schedules, interact with Sports games or download advertising clips with special offers while they wait in traffic. Utilizing a Wifi breakout, we can then also create networks of users and information at your fingertips!

Using QEMU to emulate ARM devices

This post will show you how to set up a QEMU virtual device to play with your ARM code on an x86_64 host. It is quite simple, and you should be able to simply copy and paste into a terminal and get going relatively quickly.

As an example, we will be installing a debian build (wheezy) into your VM.

First off, we need to install the QEMU packages. I use Ubuntu/Mint, so this post will be somewhat biased towards that.

Let’s start off getting the packages we need:

sudo apt-get install qemu-kvm
sudo apt-get install qemu-system-arm
sudo apt-get install qemu-utils

Now we can check that everything is installed OK and ready to go with:

qemu -version

Make a directory to work with and then grab some files off your local debian mirror. Remember, we need the ARM based distro.

mkdir ~/arm-emul
cd ~/arm-emul
wget ftp://ftp.debian.org/debian/dists/wheezy/main/installer-armel/current/images/versatile/netboot/initrd.gz
wget ftp://ftp.debian.org/debian/dists/wheezy/main/installer-armel/current/images/versatile/netboot/vmlinuz-3.2.0-4-versatile

Remember now that depending on your board/device, you may want to check if it supports ARM EL or ARM HF. As you can probably guess from the above filenames, we are working with ARM EL. There are a number of differences between the way (and efficiency) of the two device types, but if you don’t know, then you are probably using an ARM EL device. Also, it is worth checking with your manufacturer if you haven’t built your device yourself, as ARM HF is a way better buy!

Let’s create a virtual HDD now to host the code/OS:

qemu-img create -f raw hda.img 8G

I like to create a drive as big as my devices flash ROM. In this case, it is 8GB. Yours may vary.

Now, lets get the system up and running:

qemu-system-arm -m 256 -M versatilepb -kernel ~/arm-emul/vmlinuz-3.2.0-4-versatile -initrd ~/arm-emul/initrd.gz -hda ~/arm-emul/hda.img -append “root=/dev/ram”

Should get you started with the Debian installer. Do the installation and then close your VM.

Once complete, mount your filesystem, and then copy the relevant files around. You need to do this step as debian will not be able to install the bootloader, so you kind of have to do it manually.

mkdir mount

sudo losetup /dev/loop0 hda.img
sudo kpartx -a /dev/loop0
sudo mount /dev/mapper/loop0p1 mount

cp ~/arm-emul/mount/boot/initrd.img-3.2.0-4-versatile ~/arm-emul/
sudo umount ~/arm-emul/mount

Now you can start up your brand new debian ARM VM with:

qemu-system-arm -M versatilepb -kernel ~/arm-emul/vmlinuz-3.2.0-4-versatile -initrd ~/arm-emul/initrd.img-3.2.0-4-versatile -hda ~/arm-emul/hda.img -append "root=/dev/sda1"

Great! Now off to make your custom OS and flash it to your board! Good luck!

ht_chromecast_nt_130724_16x9_992

Google Chromecast in a South African context

I had my Google Chromecast delivered from a local supplier last night, and thought that I would write a quick post on my experiences so far.

Seeing as though I have already had a large number of requests as to where I got it, I will answer that question first. I bought it from Takealot.com on the following URL http://www.takealot.com/tv-audio-video/google-chromecast-hdmi-streaming-media-player,29924636

Right, so the first thing that you need to do is plug the chromecast into a free HDMI port on your TV. Great. Easy enough. The box comes with an external power adapter as well as a USB adapter to power the device, but the external power plug is a US plug. Oops. Useless. USB power it is then! I would have liked to use the external power, simply for the fact that the TV does not have to be on for me to queue up online videos to the device, but alas, unless you are willing to screw around with clumsy power adapters and transformers, forget about it. The USB cable plugged into one of the TV USB slots and powered up the device without issue.

You are presented with a beautiful wallpaper and a device name, with the instruction to go and finish setup by connecting to your new Chromecast with an Android device or a laptop.

I tried to complete setup immediately with my Samsung Galaxy S4, having to download the Chromecast app from the Play store beforehand, which failed. I then tried to set up using my laptop computer, and was told by Google that I was using an unsupported OS (Linux), but I could try anyway. That failed too. Hum…

I then had a bit of an idea, and checked that uPnP was enabled on my (admittedly very old) wifi router, and enabled that. It would have been nice if that little caveat was covered in the intro screen, but it wasn’t. I use MAC address filtering to limit wifi access at home to specific devices, and one thing that I really did appreciate is that the Chromecast device displayed its MAC address on the setup page by default.

The setup, once started, becomes somewhat fiddly. My S4 has decided that a wifi connection is unworthy if it cannot reach the internet, and this was a big problem in completing the setup. I resorted to my laptop again, and managed to get through the steps after having to (confusingly) change wifi networks mid way to properly configure the device.

The device will not connect properly to anything but a 2.4GHz wifi network, so keep that in mind too.

I did have to do each of the steps a couple of times in order for the device to recognise my network and install itself properly, but once that was done, it was pretty plain sailing thereafter.

I immediately was able to stream a YouTube video to the TV in full HD without a glitch. The coolest part was that in the Chrome browser, I installed the Chromecast extension and was able to share not only a specific browser tab to the TV, but also my entire screen. There was some noticeable lag (about 500ms) when playing Kerbal Space Program via the network, but for a very graphics intensive app, and on a not-so-great wifi router, it was rather impressive (and still completely playable).

The apps that Google use to sell you the Chromecast (Netflix, Hulu etc.) are great, but not available in South Africa. Sure, I know that you can get a Netflix account (illegally) by spoofing a DNS service to make it look like you are in the US or UK, but the Chromecast has a trick up its sleeve. Google DNS services are hard coded into the device, so, even with a service like that, you would still need some trickery to get it right.

On the other hand, however, DStv BoxOffice works beautifully via a Chrome browser window, and you are able to enjoy the entire Online catalogue at R30 a movie, even if you are not a subscriber! To me, that is well worth the purchase price, as you are not entitled to pay a subscription, and you can rent when you want to. It is way cheaper for a family to grab a BoxOffice movie in this manner than it is to go to the cinema!

Overall, I think that the device is well worth getting (if you don’t already have an XBMC device or similar), although the price could come down a little in my opinion. There is a LOT of potential here, and I do think that we are only seeing the beginnings of something here for TV viewing.

The ultimate best awesome crazy cool killer feature of this device, though completely undocumented, is for people that do a lot of presentations. If you are a regular speaker, you will know that fiddling with projectors and screens on your laptop can quickly become a nightmare. I would suggest getting one of these things to pop into your laptop bag, and carry around with you. When you need to give a talk, simply plug it in and cast your presentation, saving a lot of time and stress!

I don’t normally do reviews like this, so I don’t really know how to end it off, but, yeah, get one of these things, they are OK tending towards good. In the future, depending on the climate, they may be awesome.

 

iBeacons and Raspberry Pi

A while back, I came across this article on Radius Networks http://developer.radiusnetworks.com/2013/10/09/how-to-make-an-ibeacon-out-of-a-raspberry-pi.html which is a set of very simple instructions to make your own iBeacon with a Raspberry Pi and a cheap bluetooth dongle.

These things should be ultra cheap to roll out, so I am expecting to see a lot of applications, such as the contextual apps that I have spoken about before, rolling out quite soon.

The possibilities of these things are huge for guerilla marketing in malls and large spaces, especially food courts and bigger shops.

Imagine contextual ads according to your preferences, that should be pretty easy actually.

Mark my words, these things will either be a complete flop (as regular bluetooth was) or huge (thanks to the iCrowd).

Time will tell!

HipsterDB

I decided to have a crack at writing a BSON based data store. I know, there are many around already, but I really wanted to see how BSON worked from a lower level, and, of course, if I could actually pull something off.

The code that I came up with today is hosted at https://github.com/paulscott56/HipsterDB It is nothing really much (yet), but I hope to be able to add more to it soon.

Basically, the way that it works is that you supply a key as a String, and a value as a document. The document itself can be just about anything, including JSON documents. The documents then get serialized to BSON and stored (for now) in an in-memory HashMap. I realise that this is not the best approach (nor the fastest), but I have limited the document sizes to a maximum of 200KB in order to make it as efficient, and as useful, as possible in the few hours that I have dedicated to it so far.

The code includes a “local” store, which you can simply include in your Java projects and use with the dead simple API, as well as a “server” mode, which spawns a socket connection (Threaded) on the specified port. You can then telnet to the server and use the server API to put and get documents by key index.

Example:

[email protected]:~$ telnet localhost 11256
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
put/1/Hello World!
Put 1 Hello World!
get/1
Getting key 1
Hello World!

Very simple!

The next steps will be to:

  • add persistent storage mechanism
  • make it faster
  • optimise the code a bit
  • write some tests
  • probably rewrite it in C