Log Database
Log data queries with SQL
Hyper-fast. Results in milliseconds.
/*
* Based on system event logs, this query calculates:
* - a filter for specific messages using a full-text index
* - the number of entries per minute
* - the average scoring ratio for each matched row
*/
SELECT DATE_TRUNC('minute', receivedat) AS event_time,
COUNT(*) AS entries,
AVG(_score) AS avg_score
FROM "syslog"."systemevents"
WHERE MATCH(message, 'authentication failure')
USING most_fields WITH (analyzer = 'whitespace')
AND MATCH(syslogtag, 'sshd')
GROUP BY 1
ORDER BY 1 DESC
LIMIT 10;
+---------------+---------+--------------------+
| event_time | entries | avg_score |
+---------------+---------+--------------------+
| 1620220260000 | 4 | 1.5798743814229965 |
| 1620220200000 | 8 | 1.7750384211540222 |
| 1620220140000 | 10 | 1.6113891124725341 |
| 1620220080000 | 9 | 1.676726798216502 |
| 1620220020000 | 8 | 1.6908064410090446 |
| 1620219960000 | 8 | 1.690401442348957 |
| 1620219900000 | 7 | 1.7646006005150932 |
| 1620219840000 | 7 | 1.7795820917401994 |
| 1620219780000 | 10 | 1.5844267368316651 |
| 1620219720000 | 13 | 1.5637413492569556 |
+---------------+---------+--------------------+
Millions of data points per second
CrateDB allows to ingest huge volumes of log daa at very high speed on clusters of inexpensive servers.

Intuitive data modeling
CrateDB can collect any type of data (structured, unstructured, semi-structured, time-series, geospatial, BLOB) and can accept any type of log without any specific development needed.

Data indexing on the fly
CrateDB indexes all data on the fly right upon ingestion, which enables immediate analysis. It also indexes all fields by default, with no impact on performance, thanks to columnar indexing. This is very useful to perform multiple types of ad-hoc queries.

Native SQL
CrateDB implements SQL natively, making it easy for any developer to use and integrate. No additional knowledge is required.

Real-time query performance

Always on
Built-in data replication and high availability allow to run 24x7 analysis with no single point of failure.

Interested?
Log analysis consists in reviewing computer-generated event logs to proactively identify bugs, security threats or other risks. Log analysis can also be used to review user behavior or ensure compliance with regulations.
CrateDB is a database perfectly suited to log analysis thanks to the combination of a real-time SQL engine, a NoSQL foundation, and a full-text search engine. It offers the scalability, performance, and flexibility to process any type of logs and massive volumes of logs in real-time, supporting a wide variety of log analytics use cases, such as cybersecurity, network performance monitoring, video analytics.
Successful companies using CrateDB as a log database


Other resources on log databases
Tutorial
Tutorial
Would it not be great if we could have some rules inspecting log entries as they come in, extracting relevant information, and flagging anything potentially suspicious?
There are a lot of nice features in CrateDB to support this kind of setup. Learn how.