RisingWave

About

RisingWave is a stream processing and management platform that allows configuring data sources, views on that data, and destinations where results are materialized. It provides both a Postgres-compatible SQL interface, like CrateDB, and a DataFrame-style Python interface.

Details

Deliver fresh, low-latency insights from real-time streams, database CDC, and time-series data. Bring streaming and batch together, let users join and analyze both live and historical data, and persist results in managed Apache Iceberg™ tables.

Managed RisingWave

RisingWave Labs offers managed products for building prototypes, production workloads, and enterprise-level, critical applications.

Synopsis

RisingWave:

Load an Apache Iceberg table, and serve it as a materialized view.

CREATE SOURCE sensors_readings
WITH (
  connector = 'iceberg',
  database.name='db.db',
  warehouse.path='s3://warehouse/',
  table.name='sensors_readings',
  s3.endpoint = '<YOUR_S3_ENDPOINT>',
  s3.access.key = '<YOUR_S3_ACCESS_KEY>',
  s3.secret.key = '<YOUR_S3_SECRET_KEY>',
  s3.region = '<YOUR_S3_REGION_NAME>'
);

CREATE MATERIALIZED VIEW average_sensor_readings AS
SELECT
  sensor_id,
  AVG(reading) AS average_reading
FROM sensors_readings
GROUP BY sensor_id;

CrateDB:

Ingest it into a CrateDB table for long-term persistence and efficient querying, even at large scale.

CREATE TABLE public.average_sensor_readings (
  sensor_id BIGINT PRIMARY KEY,
  average_reading DOUBLE
);

Learn

RisingWave Table Loader

Stream processing from Iceberg tables to CrateDB using RisingWave.

Full example tutorial, which is sourcing data from an Apache Iceberg table, and sinks it into CrateDB.

Stream processing from Iceberg tables to CrateDB using RisingWave

Interoperability

Note

The standard approach with RisingWave would be to use its CREATE SINK operation to connect to an external target. However, because this does not currently support CrateDB, a little Python event processor is needed to relay the data. An example implementation can be found in the tutorial referenced above.

Note

We are tracking compatibility issues per Tool: RisingWave and appreciate any contributions and reports.