Virtual Trainspotting: How we bring the railway to your desktop using Bytemark Cloud and Free software
As you might know, if you’ve read the blog before, Bytemark’s data centre in York is right next to the East Coast Main Line (ECML). The ECML is one of the busiest lines on the UK rail network carrying passengers and freight up and down the east coast of the country. This gives us a unique vantage point to view an amazingly wide range of rolling stock running up and down the line.
We’ve discussed the railway cameras project in a previous post, but this blog post will discuss the hard technical side of setting up the streaming IP cameras using a relatively small machine on Bytemark Cloud and Free software
Camera setup
Thanks again to Railcam.uk for buying the cameras, which are pretty impressive and a significant investment. The two cameras are the same model: Hikvision DS-2CD2632F-IS paired with DS-1260ZJ mounting boxes. They’re powered using Power over Ethernet which was already available at our planned installation point. However, those cameras don’t include sound, so Railcam supplied mini plug-in microphones that work surprisingly well, given the lack of an additional power source.
Selecting which cameras took a bit of careful thought given they were being installed onto the side of our data centre. Usually, Railcam would specify a single IP camera with pan-and-zoom capability. Given that the pan-and-zoom functions would be exposed via a closed source, web control panel, we agreed instead to install two static cameras with preset zoom. The two cameras look up and down the line providing a north and south aspect.
The installation was carried out by our usual contractor for security cameras. We had previously supplied digital camera photos of the view to Railcam who then gave us a suggested crop for the view from the cameras. Nathan and Pete Taphouse supervised the installation and ensured we matched this as best we could.
With those installed, we could get on with setting up the streaming.
Streaming setup
For streaming their other railcams, Railcam.uk usually use a third-party tool like CamSecure, which is a paid-for hosted service. Given the IP cameras were on the side of our building, it made sense to try and provide a reliable, in-house solution powered by Bytemark Cloud demonstrating the best of Free software.
In doing so, we had to consider how to provide high-quality streaming video to the widest range of devices with commercial levels of service. Most commercial streaming sites that you see at the moment such as Twitch.tv and CamSecure serve streaming video to a desktop web browser using a proprietary Flash applet. The applet loads a Real Time Messaging Protocol (RTMP) stream from an RTMP server, which is then fed in-turn by some kind of video encoder.
We had to identify a similar free Flash applet to display the stream: Flowplayer is one such GPL-licensed applet.
Obviously, most mobile devices these days don’t have Flash support so to serve users wanting to watch the train stream while out and about, we had to find some kind of mobile solution. For this, we decided to use the HTTP Live Streaming (HLS) protocol. HLS was originally developed by Apple, but is now supported widely on Android (in Chrome), as well as by VLC and other free video players.
Note, we did consider avoiding Flash by using HTML5 video but, at the time of writing, HTML5 has no official support for streaming video. The <video /> element only supports the display of a video file.
With these decisions made, we spun up a BigV machine with 4 cores/12GB RAM running Debian 7.0 wheezy and installed a very well-known set of applications:
libav(from Wheezy Backports) to take the video/audio stream from the cameras and encode it into the formats necessary for RTMP and HLS.nginx(compiled with the nginx-rtmp module) to serve the web pages, RTMP streams and the HLS playlists for mobile and VLC.
There are a number of commercial video encoding and RTMP stream server products, but obviously those were out of the question.
Cameras and Codecs
The Hikvision cameras support an RTSP stream with the following codecs:
- 720p h264 video.
- 8khz audio encoded as pcm_mulaw.
As above, we wanted to display the stream using a free Flash video streaming applet and through HLS. Conveniently, both the free Flash applet and HLS both support the 720p h264 video format that the cameras already provide, so it wasn’t necessary to re-encode the video (which would have potentially meant using a lot more CPU). Unfortunately, neither of the protocols support the PCM audio format, hence the need for some encoding.
For each camera, we run a single avconv process, running as follows:
avconv rtsp://host.name.of.camera -f flv -map 0:0 -map 0:1 -c:v copy -c:a:0 libmp3lame -b:a:0 128K -ar 44100 rtmp://localhost/path.to.rtmp -map 0:0 -map 0:1 -acodec libvo_aacenc -ab 96k -c:v copy -ar 44100 -f flv rtmp://localhost/path.to.hls
This takes the RTSP stream from the camera and produces two outputs, both of which are handed to nginx for re-streaming as RTMP:
- For the RTMP/Flash feed: an FLV-contained stream containing the unchanged video codec just passed-on from the camera and a 128kbps MP3 audio stream.
- For the HLS feed: an FLV-contained stream containing the unchanged video codec just passed-on from the camera and a 96kbps AAC+ audio stream.
We run one of these processes for each camera, supervised with runit (a Bytemark favourite) to make sure they’re automatically restarted if they die for any reason. They tend to use about 5-7% of one CPU on this BigV machine, so there’s plenty of room if we/Railcam wanted to add more cameras in the future (the box runs with a fairly consistent load average of about 0.75).
Conclusion
Using almost entirely Free software, we’ve been able to provide commercial levels of streaming of these two IP cameras for Railcam.uk. The server itself has been almost totally reliable since setup and it’s not uncommon to catch the sound of a friendly East Coast driver passing the York office faithfully reproduced across the Pennines in our Manchester office!
