Shoott: Real-Time Insights for Product Usage and Debugging

Shoott, a photography company that offers affordable, professional photoshoots through a seamless booking process and personalized customer experience, faced a critical challenge when their analytics engine, powered by Rockset, was discontinued. Their ability to visualize product usage statistics and debug issues relied heavily on a real-time data pipeline and seamless query performance. Shoott needed an alternative solution that could replicate their workflow without introducing significant overhead, enabling them to maintain business continuity and deliver high-quality insights to their customers.

Getting real-time insights from nested fields in DynamoDB

Shoott’s operations heavily relied on Rockset’s ability to:

• Replicate data from DynamoDB without requiring a pre-defined schema.
• Support SQL-based querying of nested fields.
• Provide HTTP endpoints for saved queries, also known as query lambdas.

With Rockset being discontinued, Shoott faced several critical needs:

1. Effortless Data Synchronization: A seamless, unattended CDC (change data capture) process to replicate data from DynamoDB into a new database.
2. Query Performance: High-speed querying for reporting and visualizing product usage metrics.
3. Schema Flexibility: The ability to handle nested JSON fields without requiring constant reconfiguration.

The solution prioritized delivering real-time insights and ensuring seamless continuity in analytics processes, with minimal downtime and an intuitive user experience.

CrateDB selected as the ideal solution for real-time analytics

Shoott chose CrateDB Cloud for its unparalleled ability to address their needs and provide a robust, scalable solution for real-time analytics. Key factors that set CrateDB apart included:

• Dynamic Schema Flexibility: CrateDB’s dynamic schema eliminated the need for Shoott to redefine schemas when JSON elements changed, enabling them to adapt quickly to evolving data structures.
• Index-Everything Strategy: CrateDB’s approach to automatically indexing all data ensured fast and efficient queries, even for nested fields, without requiring manual optimization.
• Support for Nullable JSON Fields: This provided Shoott with the flexibility to query incomplete or partially structured data without disruptions.
• Views for Query Lambda Functionality: Using views combined with the CrateDB HTTP endpoint, Shoott could efficiently manage and execute complex queries, providing an easy transition from Rockset.

Using the CrateDB Toolkit, Shoott quickly set up a robust pipeline to feed changes from DynamoDB into CrateDB via change data capture (CDC). The implementation required close collaboration between the Shoott's and CrateDB's engineering teams to achieve this ambitious goal and overcome technical challenges. CrateDB’s native SQL compatibility enabled Shoott’s team to transition their existing workflows effortlessly, without the need for retraining or rewriting substantial portions of their application logic. This allowed them to focus on extracting valuable insights rather than managing infrastructure complexity.

Immediate outcomes

Since adopting CrateDB, Shoott has achieved:

1. Uninterrupted Analytics: Shoott’s reporting and visualization processes continued seamlessly, avoiding disruptions for their customers and internal teams.
2. Enhanced Query Speed: With CrateDB’s index-everything strategy, query performance exceeded expectations, delivering real-time insights for product usage and debugging.
3. Reduced Operational Overhead: The dynamic schema and robust CDC pipeline eliminated the need for manual schema management or frequent reconfigurations, saving time and resources.
4. Business Continuity: CrateDB’s alignment with Rockset’s functionality ensured that Shoott could replicate its workflows with minimal adjustments, enabling a smooth migration.

CrateDB empowered Shoott to overcome the challenges posed by Rockset’s shutdown, ensuring continuity, scalability, and efficiency in their analytics workflows. By leveraging CrateDB’s dynamic schema, high-performance querying, and built-in flexibility, Shoott not only maintained its existing capabilities but also positioned itself for future growth in delivering cutting-edge product usage insights and debugging tools.