Raspbmc RC4 on Raspberry Pi

I installed the latest release candidate of Raspbmc (RC4) on my Raspberry Pi during the weekend and have been enjoying flawless video playback ever since.

Installation, as seems usual with the Raspberry Pi (and highly unusual for most Linux flavors), was a breeze. Just download the installer for your OS and follow the instructions on the site. The installer downloads a minimal OS image and writes it to the selected SD card. The next step is simply to insert the card into the Raspberry Pi and power it on. At which point it will proceed to download and install the rest of the system directly from a Raspbmc download server (the Pi obviously needs to be connected to the internet at this point). The most arduous part was making a cup of coffe to enjoy while this was going on (which took about 25 minutes).

Raspbmc in action (UI after startup).

This new RC is based on Raspbian and has support for floating point calculations in hardware. Even though video playback was ok in RC3, UI interaction would become sluggish while playing video. Thankfully this is no longer an issue. Assigning the Raspberry Pi to media center duty has the added benefit of no longer having to put up with the power guzzling and jet engine sound level of the Xbox360. In contrast the Raspberry Pi runs perfectly silent from a 5 volt generic cell phone charger (at 700 mA). Thinking for a while about where to place it I finally decided to just dump it into the cable mess behind the TV bench. It is so tiny it is hardly noticed there anyway (even cased). Interaction is via a wireless keyboard and mouse for the moment, although I might investigate the possibility of hooking it up to our remote. I highly recommend the Logitech Wireless Combo MK260 (check out this site for a list of verified peripherals).

In short, for those of you interested in a low-cost media center computer, choosing the Raspberry Pi is a no-brainer.

Quantifiable Insanity

I felt compelled to write this short piece following the most recent high frequency trading debacle at Knight Capital Group where $440 million flowed down the proverbial drain in less than one hour. I don’t think that automated trading should be banned outright but the volume of trades and the frequency at which stocks are now automatically traded is ridiculous. It seems to me that the diffentiating factor should be the acuity of the traders judgement, not whether you can get your orders in a few milliseconds faster than the next guy.

Consider the following statement:

“I believe in this company and its ability to generate profit and want to invest in it.”

Now try to apply this kind of sentiment to trades performed on a millisecond basis by hordes of autonomous agents. Agents programmed in advance with strategies for eeking out tiny profits from miniscule fluctuations in the markets, fluctuations they themselves stimulate. I would question your rationality if you came to the conclusion that this makes sense.

In a recent Wired article the weird and wonderful world of high frequency and low latency trading is covered in some detail. Even though the article is a bit sensationalist it still highlights the extremes to which these companies will go to get an edge.

In one of the more extreme examples we are told of quants seizing on the story of neutrinos possibly traveling faster than the speed of light. An experiment was being conducted as part of the OPERA experiment which is a collaboration between Cern in Switzerland and the Gran Sasso National Laboratory in Italy. The only problem was the neutrinos sent from Cern were picked up at Gran Sasso an average of 60 nanoseconds too early. After seemingly exhausting all sources of possible error the researchers reached out to the scientific community for help validating or refuting their findings. Few in the scientific establishment believed that the results would hold up, while it is at least implied in the article that some enterprising quants ran the numbers to find out just how much of an edge this would give them if true.

Even though the above example may seem far out, the underlying motivations are very real. Plans exist for laying down dedicated fiber-optic cables straight between trading sites and exchanges (instead of meandering through towns and cities) as well as plans for laying down more optimally routed cables on the ocean floor for faster transcontinental connections (to name but a few). All this proposed investment for what?

The stock markets exist for a reason, to allow companies to raise capital. It puts money that would otherwise sit idle to good use and in so doing promotes economic growth. It keeps people employed and as money invested trickle down to individual employees it will eventually flow back into the wider economy. What part does high frequency trading play in this grand cycle?

It would seem to be time to seriously reevaluate why we invest and trade. We need to return to basics or at the very least take a significant step in that direction.

Curiosity has landed!

I have to say I’m impressed. When I first saw the landing procedure intended for the lander Curiosity headed for Gale crater on Mars I was a bit skeptical. The extremely complicated landing sequence employed for the Curiosity mission seemed like asking for trouble, given that Nasa has a bit of a spotty record when it comes to delivering hardware to Mars.

Curiosity lander.

The curiosity lander was to be lowered to the surface of mars on wires suspended from a delivery vehicle autonomously balancing on thrusters. Some may remember the Mars Climate Orbiter Metric vs. English unit mixup so a bit of scepticism didn’t seem to be misplaced. The more points of potential failure you add would seem to increase the odds of something unforeseen ruining the party.

As it turned out the landing went of without a hitch. Big kudos to everyone involved in this project who accomplished something truly amazing.

Waiting several months for my Raspberry Pi to arrive meant that I have had plenty of time to come up with potential uses for it. The Raspberry Pi caught my attention because it seems to be one of those subversive technologies that come in under the radar and changes everything. Nothing about it is truly revolutionary yet it delivers fully fledged computing with virtually the entire GNU/Linux software catalog (and others) behind it, at an unbeatable price point (roughly 35$ for a bare model B board). The possibilities are literally endless. The Raspberry Pi and its successors will be used for robotics, sensors, DIY drones, multimedia centers, and home servers to name but a few. As it is an open hardware design we will probably see clones coming out of China by the shipload. In some way it also seems to fit in nicely with the whole wimpy core trend for data centers (which I admit is not uncontroversial).

This is what arrived in the mail last week:

Raspberry Pi on arrival.

My own plans for the device are slightly more modest though. Personally I’m interested in using these devices as an always-on home server, as an ultra small footprint media center (if this particular use pans out) and for work I would like to investigate using them to drive the wall screens which are proliferating all over the office.

Source code management (running Git)

I have avoided having a server running 24/7 at home for a number of reasons. They take up space, are noisy, cost money to run (admitedly not a huge deal),  and cost a not insignificant amount of money to buy. The Raspberry Pi negates all these objections handily. It takes no space, makes zero noise, pulls almost no power (idle or otherwise), and comes in under 100$ with all accessories.

Replacement for the XBox 360 as a media center

I haven’t tried this yet but it looks promising. If the codec support is there via XBMC and the performance is up to snuff it should handily beat the XBox.

Project dashboard for work

At Active Solution where I work we have begun installing wall screens to display project relevant information such as Trello boards (digital post-it replacement) and information piped to web based dashboards from our various issue tracking systems (depends on the client). Currently we have dedicated laptops serving each wall screen which is workable when we only had two. As the number of wall screens grow this approach is quickly becoming untenable. I decided to try the Raspberry Pi out as a replacement for all those power-hungry laptops.

Setup

The Raspberry Pi comes with nothing, not even a case, included (except a nice t-shirt) so you will need to get a couple of things in order to get started.

  • 4GB+ SDHC card (Class 4)
  • Micro usb charger/power supply (minimum 700 mA)
  • Mouse
  • Keyboard
  • HDMI cable
  • Screen (with HDMI input)
  • Optional: a case (I just ordered one from ModMyPi)

I will not go into great detail about how you get the Raspberry Pi up and running save to say that it was a breeze (check out the Raspberry Pi foundation quick start guide). The Debian Squeeze Linux image I opted for was ready to be written directly to the SD card (no installation needed). All you need to do is change the default password on first boot.

Raspberry Pi up and running

Raspberry Pi up and running

It took literally no more than 20 minutes to download the Linux image, write it to the SD card, assemble the Pi and hook it up to the monitor. The only snag at this point was that we had to enable a previously disabled network socket close to the intended screen. A bigger problem turned out to be the supported browsers on the Pi. As of this writing none of the browser versions available (of Midori and Chromium) supports HTML5. This is a problem for us as the dashboard software we are using (our own as well as Trello) requires HTML5 support.

Raspberry Pi on wall screen

Raspberry Pi (with Debian Squeeze and LXDE desktop) on wall mounted flatscreen.

The solution

Not one to be discouraged I soon had an idea for how to get around this limitation. Even though we might conceivably upgrade to touch screens some time in the future our wall screen dashboards are non-interactive and simply display a pre-set web page (refreshing periodically). I recently attended an in-house seminar conducted by my eminent co-workers Chris Klug and Robert Folkesson on among other things the open source HTTP service framework NancyFx. Here was a light weight solution that could be used for serving up images of web pages rendered in a browser that supports HTML5 to a wall screen connected Raspberry Pi. It was of course also a great opportunity to learn more about NancyFx.

WebSnap

I began outlining a web page snapshot service named WebSnap and soon had a prototype up and running. As our company specializes in Microsoft technology this was implemented as a WPF application in Visual Studio 2010.  At the moment of writing it is functional albeit not quite “done”. I will outline the general idea here and plan to post the full source once it has matured somewhat. For simplicity (most certainly not for scalability) I opted for simply capturing screenshots of a pages rendered in a WebBrowser control hosted in a WPF application. The ideal would of course be to render pages fully off-screen but for simple proof of concept this seemed a bit excessive. If anyone has any suggestions for solving this particular problem don’t hesitate to suggest it.

To include NancyFx (self hosting) in a Visual Studio 2010 project using the Nuget PM:

PM> Install-Package Nancy
PM> Install-Package Nancy.Hosting.Self

To start self hosting is as simple as:

NancyHost host = new NancyHost(new Uri("http://localhost"));
host.Start();
...
host.Stop();

All you really need to start to serve data over HTTP is to implement a NancyModule. Here I leaned heavily on an excellent blog post by Andre Broers on self hosting Nancy and utilizing the SuperSimpleViewEngine included with NancyFx. The full source includes view models, views and various support classes which I have not include in this post. Please refer to Andre Broers excellent post for an overview of how to do this.

SnapModule here inherits from BaseModule which in turn inherits from NancyModule. As you can se we are setting up a number of routes with different “handlers” attached.

using System;
using System.Drawing.Imaging;
using System.IO;
using System.Text;
using System.Threading;
using System.Windows;
using Nancy;
using Nancy.Responses;
using WebSnap.Models;

namespace WebSnap.Modules
{
 public class SnapModule : BaseModule
 {

  delegate void UIDelegate();

  public SnapModule()
  {
   Get["/content/{url}"] = x => Response.AsFile((string)x.url);

   Get["/load/{id}"] = x => { return Load(x); };

   Get["/images/{url}"] = x => Response.AsFile(".\\Images\\" + (string)x.url);

   Get["/"] = x =>
    {
     Model.LandingPage = new LandingPageModel();
     Model.LandingPage.Heading = "WebSnap";
     return View["LandingPage", Model];
    };

   Post["/"] = x =>
    {
     string url = (string)Request.Form.Url;
     string res = (string)Request.Form.Res;
     string hex = StringToHexString(url + "@" + res);
     return Response.AsRedirect(string.Format("/load/{0}", hex));
    };

  }

  private dynamic Load(dynamic p)
  {
   string hex = (string)p.id;
   string query = HexStringToString(hex);
   string[] parts = query.Split('@');
   string siteUrl = parts[0];
   string resolution = parts[1];
   string imageUrl = string.Format("/images/{0}.png", hex);
   string imagePath = string.Format(".\\Images\\{0}.png", hex);
   Model.SnapPage = new SnapPageModel();
   Model.SnapPage.Image = imageUrl;
   string[] dimensions = resolution.Split('x');
   Size size = new Size(double.Parse(dimensions[0]),
    double.Parse(dimensions[1]));
   // Do we need to refresh the image?
   TimeSpan imageAge = File.Exists(imagePath) ?
   GetFileAge(imagePath) : TimeSpan.MaxValue;
   if (imageAge > TimeSpan.FromMinutes(5))
   {
    // Set up delegate to navigate to page
    // in the web browser control...
    UIDelegate navigateToPage = delegate()
    {
     UI.Instance.Resize(size);
     UI.Instance.LoadCompleted = false;
     UI.Instance.WebBrowser.Navigate(siteUrl);
    };
    // and one to capture a screen shot of
    // the rendered page.
    UIDelegate captureImage = delegate()
    {
     Capture.Window(imagePath, ImageFormat.Png, UI.Instance.WebBrowser);
    };

    // Load the page in the web browser.
    Application.Current.Dispatcher.Invoke(navigateToPage);

    // Wait for the website to load on the UI thread (max 10 sec).
    WaitForLoad(TimeSpan.FromSeconds(10));

    // Capture image from the web browser.
    Application.Current.Dispatcher.Invoke(captureImage);
   }
   return View["SnapPage", Model];
  }

  private static void WaitForLoad(TimeSpan max)
  {
   DateTime start = DateTime.Now;
   do {
    if (UI.Instance.LoadCompleted) break;
    Thread.Sleep(100);
   } while ((DateTime.Now - start) < max);
  }
  ...
 }
}

The reason for marshalling all interaction with the WebBrowser controll via the dispatcher is that it is owned by the UI thread and will not take kindly to being manipulated from another thread. The NancyFx module is not going to be running on the UI thread so hence the marshalling. This of course creates a bottle (pain in) neck when we try to scale the solution up. Luckily only so many wall screens will fit in the office.

When you first navigate to the host address you will be presented with a landing page where you can enter the page to snapshot and the size of the snapshot (at the moment this is approximate).

Landing page

The landing page

When you press load the server will be instructed to load the requested page into the hosted web browser (HTML5 compatible) and will reply with a page rendering that image full window size and centered. If the same client (or another) request the same page (at the same resolution) again the same image will be returned unless the image is older than 5 minutes in which case another will be captured. The end of the url here is a hexadecimal string created by simply transforming http://www.google.com@800×600 to 687474703a2f2f7777772e676f6f676c652e636f6d4038303078363030 (each ASCII digit = two hexadecimal digits). Crude but effective (if not exactly memorable).

WebSnap

Snapshot of webpage loaded from WebSnap

At this point this is essentially a somewhat ugly hack. No consideration has been given to how this is supposed to handle increased load or whether or not it is secure. If it will be used it will be used on an internal network for the sole purpose of feeding a handful of wall screens with web page snapshots.

With this I conclude this report on my initial adventures with the Raspberry Pi. I hope to post more on this subject as my experience with the device grows and my various projects mature.

Raspberry Pi + Flatscreen + Nancy = Dashboard