A few months ago, when the new website was released, I hinted that it was the first in a number of stages to the site. The intention for 2020 is to have something new every month.
There’s nothing new to share today but stage two will happen towards the end of this month. Here’s what to expect.
New processor
Realtime Trains is formed of two distinct components: the front-end (website) and back-end (processor). Over Christmas, a significant amount of work has been done in rewriting the non-train service components of the processor, which is now undergoing soak testing. There a number of new changes that come with the new processor - some are interesting technically and some will have a visible impact.
New Geography Engine
Realtime Trains’ sister site, RailMiles, has had a distance calculation engine since its start in 2008 as my A level Computing project. This engine, known as the ‘RailMiles Mileage Engine’, has powered the distance display in the left column of train services in the detailed view since RTT v2 launched in 2013.
Over the years, it has been slowly subsumed into RTT. The development into the geography engine is now led by the ‘RTT project’ (if you can call it that…) and the new processor contains the first implementation of the new version. Ongoing work over the last six months has overhauled the database behind the engine modifying the locations and adding better routing capability. The most visible change will be in a large increase of trains displaying distances and the locations being displayed changing - mostly removing en-route passes of freight yards and sidings.
The RailMiles Mileage Engine beta website will be shutting down later this year. It will be merged into Realtime Trains or My RailMiles, but that decision hasn’t yet been taken.
New Train Describer Step Handler
There are two primary sources of real-time reports into Realtime Trains. One is TRUST with its automatic SMART reporting or through manual reports made by rail staff. The other is through the Train Describer and the use of our step offset tables.
The step handler effectively handles the latter and applies reports into trains. It is also responsible for the ‘approaching’, ‘arriving’, ‘at platform’ feature (location status) seen on trains in automatic reporting areas. This rewrite has been driven by two needs: adding more reporting types into the mapping tables, and the new message queue infrastructure. The main visible change will be a reduction in the number of location statuses hanging around after a train has gone past.
The Train Describer infrastructure is currently going through a larger development effort to greatly improve reporting accuracy in more challenging locations and is currently being tested on the Highland Mainline. More on that in the future.
New Message Queue Infrastructure
Our datacentre move last year gave a unique opportunity to radically improve our handling of real-time messaging streams from our data suppliers. Prior to this development, each processor connected directly to Network Rail’s Open Data platform. Over the last few years, there have been various connectivity issues to the platform resulting in the processor being blocked by firewall from their servers resulting in a loss of data.
A new, and fairly complex, system of ingesting data into our own message queue servers and cross-feeding between datacentres is now in place and has been undergoing testing for stability and reliability since November. We’re already using it inside our step offset system. There’ll be a technical post on the infrastructure itself later in the month.
The new infrastructure serves to minimise the likelihood of a loss of messaging, barring a complete loss of data out of upstream systems.
Trackernet support removed
The new processor has removed functionality to make use of London Underground’s Trackernet feeds. Chiltern Railways recently introduced GPS based reporting onto their services on the Metropolitan Line and hence the necessity for it is not as high as it once was.
There are further developments intended with the Trackernet feed in the future but this is not in immediate plans.
Status page changes
One major and one minor improvement will be made to the status page. The minor change improves the processor’s capability of updating the website about the status of the various feeds and will now actually reflect when they’re down.
The major change is the introduction of a separate box detailing the status of every Train Describer area in the country and whether we have received a message from it in the previous minute.
