Are you struggling with scalability limits or delayed insights from your IoT data? In this session from the Data Innovation Summit, Gregor Bauer (VP Customer Engineering at CrateDB) dives deep into the architecture required to deliver real-time analytics for high-volume IoT pipelines. CrateDB is an open-source, distributed SQL database built to handle the complexities of modern data. Whether you are dealing with structured tables, dynamic JSON, time series, or text vectors, CrateDB provides a flexible and high-performance solution that scales linearly.
In this video, you will learn:
-
The Power of Distributed Architecture: Discover how a distributed storage and query engine enables high-volume concurrent reads and writes.
-
Columnar Storage Advantages: Learn why columnar storage makes ad-hoc aggregations (min, max, average) incredibly fast without the need for pre-aggregation.
-
Seamless Integration: How CrateDB fits into your ecosystem using its native PostgreSQL interface, allowing you to use existing connectors and tools like Grafana effortlessly.
-
Deployment Flexibility: From Raspberry Pi at the edge to hybrid and public clouds, see how the source stays the same everywhere.
💻 Live Demo: Real-Time Weather Data Pipeline Watch a step-by-step walkthrough of a lightweight IoT pipeline: - Ingestion: Streaming real-world Copernicus weather data.
-
Processing: Using Managed Kafka and AWS Lambda for serverless, batch-written data ingestion.
-
Visualization: Real-time insights via a Grafana dashboard with a 5-second refresh rate.
-
Advanced Querying: See live examples of geospatial polygon searches for geo-fencing and querying nested JSON data on the fly.
🛠️ Resources Mentioned:
-
Hands-on scenarios to try CrateDB: https://cratedb.com/explore
-
Try the Demo Yourself: https://github.com/crate/realtime-demo
-
CrateDB Academy: Free courses on architecture and advanced time series: https://learn.cratedb.com