Microsoft Open Sources Trill Engine To Deliver A Trillion Events a Day

Recently, Microsoft has open sourced its internal project - Trill, so that more developers have access to a new temporal query language as well as “a trillion events per day” worth of aggregated data Processor.

Recently, Microsoft has open sourced its internal project - Trill, so that more developers have access to a new temporal query language, as well as, “a trillion events per day” worth of aggregated data processor.
 
Trill started as a Microsoft Research project back in 2012 and has since been featured in published research papers. Moving from the research phase to actual implementation, the company has used Trill internally for Azure Data products and other mission-critical streaming initiatives throughout the organization including Bing Ads and Halo.
 
"Trill was the first streaming engine to incorporate techniques and algorithms that process events in small batches of data based on the latency tolerated by the user. It was also the first engine to organize those batches in columnar format, enabling queries to execute much more efficiently than before. To users, working with Trill is the same as working with any .NET library, so there is no need to leave the .NET environment. Users can embed Trill within a variety of distributed processing infrastructures such as Orleans and a streaming version of Microsoft’s SCOPE data processing infrastructure." wrote the company.
MICROSOFT OPEN SOURCES  TRILL  
Source: Trill 
 
From now, open source developers will get access to the Trill enabled data and toolsets.
Trill will serve developers as a single-node engine library, any .NET application, service, or platform will be able to use Trill and start processing queries; and as a temporal query language allowing developers to express complex queries over real-time and/or offline data sets.
 
The company claims, Trill’s high performance across its intended usage scenarios will give results with incredible speed and low latency. For example, filters operate at memory bandwidth speeds up to several billions of events per second, while grouped aggregates operate at 10 to 100 million events per second.
 
Azure Stream Analytics went from the first line of code to public preview within 10 months by using Trill as the on-node processing engine. The library form factor conveniently integrates with our distributed processing framework and input/output adaptors. Our SQL compiler simply compiles SQL queries to Trill expressions, which takes care of the intricacies of the temporal semantics. It is a beautiful programming model and high-performance engine to use. In the near future, we are considering exposing Trill’s programming model through our user-defined operator model so that all of our customers can take advantage of the expressive power.” commented Zhong Chen, Principal Group Engineering Manager, Azure Data.
 
You can read the official announcement here.