Travel Analytics Blog
Trio Data Engine – the heavy lifting behind getting API data fit for delivery where it is needed
Posted by Sonja Woodman on Friday, July 27, 2018
We’ve just launched Trio Data Engine, a more agnostic way of collecting, preparing and aggregating API transaction data for extracting intelligence and insights using other BI environments including popular open source tools such as Kibana. To find out more about this powerful data processing hub and how it can benefit our data driven customers, I spoke to Jonathan Boffey, our VP of Business Development and one of the architects behind this new approach to helping customers get value from their data.
By way of a bit of background, Triometric has been working closely over a number of years with many B2B travel distributors to give them great visibility into their real-time API search traffic which comprises of millions of search requests and their replies each and every day. Everybody with a website measures and monitors their direct traffic for insights into their customer behaviour and conversion patterns. Our platform gives these travel distributors similar granular detail regarding their indirect traffic, which is mostly in XML. We capture API data, transform it, analyse it and report on it via dashboards and reports. Such insights are invaluable for managing product supply and demand, system performance, customer search and buying behaviour and so on.
What is the Trio Data Engine?
With the new Trio Data Engine we are offering a componentised approach to the delivery of value-added XML data. The platform or data engine is used as a heavy duty processing hub to take XML data inputs from various sources, carry out the transformation according to set business rules, then push or pipeline the data to the consuming system that wants it. Simply put, enterprises can exploit the power of Triometric’s high volume XML collection and processing capability to get access to the XML data they need and want for their advanced analytic systems, without taking the Trio analytics engine or reporting functionality.
How did Trio Data Engine come about?
Our industry experience tells us that we have a unique and very powerful XML data preparation capability, but there are other reporting or visualisation tools that can crunch and present the data. Our market observations tells us that more and more enterprises are looking for and implementing enterprise-wide search environments, either using commercial products or open-source products such as the Elasticsearch, Logstash, and Kibana (ELK) stack. In the latter case, while Elasticsearch is a specialised search engine and Kibana a great visualisation layer, the log analyser that takes data from various sources, is aimed at system logs rather than more complex structured data like XML. Handling complex XML data, indexing it and enriching and aggregating it in line with company business rules is what we do best. It’s our core business and our systems currently process globally over 3 billion hits per day. In working with a large hotel distributor recently, we were able to demonstrate that our XML analytic technology proved to be the ideal platform to act as the processing hub for collecting and preparing large volumes of raw real-time XML data and transforming it into a fit state for analysis and visualisation by Elasticsearch and Kibana. It is a classic coming together of respective company strengths making a formidable integrated solution to tackle the complexity of getting access to XML data and then being able to use for insights that can drive decisions.
Why is the Trio Data Engine needed?
There are three simple reasons why the Trio Data Engine delivers value to enterprises that are investing to become data-informed and data-driven: access, complexity, scale.
- Access In addition to widening the scope self-service user reporting, The Trio Data Engine helps travel companies get fast reliable access from multiple sources to large volumes of clean and aggregated data for historic analysis and feeding into AI systems for more prescriptive analysis. In other words to help travel companies analyse in great detail their API data in much the same way as they already analyse their website data.
- Complexity To add to the complexity, distributors inherently deal with a large number of variables within the context of the traveller and the trip. Then there are fluctuations such as seasonal demands, major events (or crises) and changing trends. By their nature, search requests and replies can be complex multi-layered structures containing a raft of options. The Trio Data Engine applies Triometric’s experience and expertise in preparing, enriching and aggregating XML and getting it in shape for BI tools such as Elasticsearch and Kibana to use.
- Many enterprises including travel ones are favouring open-source products such as the ELK stack as their analytics platform. However, very often, the challenge for such open source projects is transforming from a project to a real enterprise-grade solution that is capable of delivering the cross the board intelligence they are looking for. The Trio Engine takes away the complexity of preparing XML.
- Scale Today’s organisations want to be able to analyse data and gain insights in as close to real-time as possible. Travel APIs produce not only complex data but in very large quantities. When you have time-sensitive processing tasks, doing those tasks faster than the competition is important. Trio Data Engine is designed to handle high volumes of data traffic quickly and at scale. Simply put the Trio Data Engine can take raw data and transform it for analysis at very high volumes and speed.
- With Trio Data Engine we are providing a strong componentised data management and analytic foundation that helps our customers use the right tools to uncover the insights that provide vital competitive advantages.
Who is it aimed at?
We have a strong history of working with travel companies and the Trio Data Engine has been developed in collaboration with a hotel distributor and a major hotel group, so travel distribution in the first instance. That said, the Trio Engine is a data processing and transformation hub that could be adapted to take XML traffic data and corresponding business rules from other sectors, such as finance or insurance. But for now our focus is on delivering prepared XML data to IT and business departments of travel product suppliers such as hotels and airlines and intermediaries such as bed banks and hotel wholesalers.
What gap in the market is it filing?
Much of today’s travel business is still booked via indirect channels such as travel agents and distributors linked by a network of APIs. Millions of search requests and replies, bookings and cancellations pass through these APIs every hour of every day. By analysing this message stream in real-time travel companies can get answers to a raft of questions such as what are customers looking and are they getting what they requested. Is inventory availability keeping up with demand or conversely not selling as pre-contracted. How can pricing be managed to impact positive revenue flow, and so on? The best insights are generated not from answering one question but from answering a set of connected questions. Answering these connected questions often requires companies to use a connected set of data sources. This requires companies to look not at one source of data but at variety of sources and build a bigger picture from a single analytics engine and interface. The Trio Engine and its ability to give access to high volumes of search and booking data is one of those contributing sources that can help travel companies get deep meaningful insights and a 360 degree view of their customers, distribution channels, markets and more.
How does it fit in with Triometric’s portfolio of products?
The Trio Data Engine is based on Triometric’s core performance monitoring and analytics technology and forms part of our suite of product offerings that includes our on-premise platform for enterprise customers and cloud-based platform for both enterprise and smaller clients.