CrateDB provides the backbone of Bitmovin's real-time video analytics platform

Founded in 2013, Bitmovin is a leading video streaming company that built the world’s first commercial adaptive streaming player and deployed the first software-defined encoding service that runs on any cloud platform.

Its portfolio includes services for scalable video encoding, the Bitmovin Player for playing videos on all platforms and devices, and a complete end-to-end streaming solution with Bitmovin Streams. The company offers a product for monitoring and analyzing video streaming data, which is used by the biggest media companies around the world to deliver high-quality video experiences to end users.

This function records certain metadata every time a video is played. This provides streaming providers with information on who watched the video, when it was started or stopped, and on which device it is running, among other things. Bitmovin produces billions and billions of lines of data, which streaming providers can use to monitor and analyze their customers’ user video experience.

Since the advent of video streaming, a lot has happened in this area. On the one hand, the quality of the streams is constantly increasing and, on the other hand, the number of companies and service providers offering video-on-demand is also growing. This situation increases the pressure on individual providers not only to offer their users the best content, but also to deliver customized content of the highest quality.

To meet these demands, streaming providers need to know their users, which is the only way they can optimize their offerings. This insight was the starting point for Bitmovin's video analytics solution.

Having already been very successful with its video encoding services and its streaming solutions including video players, Bitmovin started working on its new product, which aims to offer users – regardless of their video player – capacities for the collection and analysis of large and fast-moving data sets in real time.

However, the broadcast of large events, such as the World Cup, provide extremely high data throughput that traditional databases either cannot handle or can only handle at a massive cost. Moreover, Bitmovin did not want to make this solution available only to a small selection of hand-picked users but wanted to roll it out on a broad scale.

Bitmovin needed a database with certain characteristics:

A database suitable for huge volumes of data storage and real-time analytics
A database that could be operated cost-effectively
Easy to scale
Be able to process structured and semi-structured data (such as log files) in high quantities and at very high throughput speeds
SQL as query language

The databases used up to this point would have been beyond the cost of this project and would not have made sense from a business perspective, so Bitmovin began looking for an alternative database solution.

Bitmovin uses CrateDB as the backend database for its video analytics component. Unlike other CrateDB customers, the company has not opted for a Database-as-a-Service solution in the cloud. The reason is simple: Bitmovin has always managed its very complicated technology stack itself, and also handled the implementation of CrateDB. The CrateDB team provides ongoing support and the technology itself is easy to use from installation to querying. There is no need for a complex architecture or advanced knowledge of SQL to get started.

The size of Bitmovin's backend is constantly increasing, with the number of nodes doubling since implementation. This is mainly because the system is exposed to a growing data load.

Currently, the database solution includes 14 nodes and 140 terabytes of storage, where the company stores structured data such as user events and user interactions. Every day, the video analytics tool adds around 2 billions new events per day, with the largest tables comprising around 60 billion playback events. It is foreseeable that the database will continue to grow in the future.

Nodes	Terabytes of storage	Events in the largest tables	New events per day
14	140	~60 billion	~2 billion

Bitmovin works closely with the CrateDB team in times of great load peaks, to jointly integrate the insights gained from the increased data load into the product in a timely manner. With the support of CrateDB’s experts, Bitmovin is able to optimally adjust the configuration of the cluster ahead of events with high data load requirements.

The most important and essential benefit of using CrateDB is that without it, Bitmovin would not be able to offer the video analytics tool at all. Alternative solutions are either not powerful enough or too expensive, so the analytics offering would not have been profitable. Thanks to CrateDB, the company has been able to increase its number of customers, as well as, its revenue. Furthermore, the database is stable and easy to manage, which frees up Bitmovin's developers to focus on more valuable tasks than building a scalable database system.

For Bitmovin, CrateDB is better than comparable products because:

The data aggregation performance required by the video analytics component would not be possible with other comparable solutions
The capability of scaling the clusters of the database itself
Bitmovin would like to accelerate this adaptation to peak loads even further by implementing logical replication, a new feature of CrateDB
With CrateDB, real-time video analysis and filling the dashboards with data is faster

DISCUSSION - RECORDING

How Bitmovin uses CrateDB to monitor the biggest live video events

In this talk, Daniel Hölbling-Inzko explains how Bitmovin uses CrateDB to monitor the most significant live video events and especially which features they are using to address their monitoring and scalability challenges.

Bitmovin: Analyzing Large Volumes of Video Streaming Events while Reducing Cost of Analytics

Challenge

Solution

Benefits

Interested in learning more?

Company

Ecosystem

Contact