Cloud databases face a fundamental challenge: how to remain available and durable under node failures? Modern cloud databases approach this by separating two concerns that used to be tightly coupled: compute and storage. The database engine becomes stateless, while the write-ahead log gets replicated across multiple nodes to guarantee durability. If a database server dies, another one can pick up exactly where it left off by reading from the replicated log.

Distributed log services are thus at the heart of cloud databases. In this blog post, we will explain some drawbacks of the predominant design for distributed logs to motivate a new elegant design. We will also explain why it is necessary to verify this design with formal methods.

In my previous post, I explained how a disk attached to a machine makes things difficult. Vertical scaling has its limits, and when you hit that limit, you can’t do horizontal scaling right away because of the attached disk. Mainstream databases like Postgres or MySQL don’t scale horizontally. I recently learned that BlueSky team switched from Postgres to a combination of Scylla and SQLite. One of the reasons was because (vanilla) Postgres is not horizontally scalable, but Scylla is.

❗Read recent blog post about Manticore vs Elasticsearch❗

Manticore Search is an easy to use open source fast database for search. Good alternative for Elasticsearch. What distinguishes it from other solutions is:

• It's very fast and therefore more cost-efficient than alternatives, for example Manticore is:
  • 182x faster than MySQL for small data (reproducible❗)
  • 29x faster than Elasticsearch for log analytics (reproducible❗)
  • 15x faster than Elasticsearch for small dataset (reproducible❗)
  • 5x faster than Elasticsearch for medium-size data (reproducible❗)
  • 4x faster than Elasticsearch for big data (reproducible❗)
  • up to 2x faster max throughput than Elasticsearch's for data ingestion on a single server (reproducible❗)
• With its modern multithreading architecture and efficient query parallelization capabilities, Manticore is able to fully utilize all your CPU cores to achieve the quickest response times possible.
• The powerful and speedy full-text search works seamlessly with both small and large datasets.
• Row-wise storage for small, medium and big size datasets.
• For even larger datasets, Manticore offers columnar storage support through the Manticore Columnar Library, capable of handling datasets too big to fit in RAM.
• Performant secondary indexes are automatically created, saving you time and effort.
• The cost-based query optimizer optimizes search queries for optimal performance.
• Manticore is SQL-first, utilizing SQL as its native syntax, and offers compatibility with the MySQL protocol, allowing you to use your preferred MySQL client.
• With clients available in PHP, Python, JavaScript, Java, Elixir, and Go, integration with Manticore Search becomes easy.
• Manticore also provides a programmatic HTTP JSON protocol for more versatile data and schema management.
• Built in C++, Manticore Search starts quickly and uses minimal RAM, with low-level optimizations contributing to its impressive performance.
• With real-time inserts, newly added documents are immediately accessible.
• Interactive courses are available through Interactive courses to make learning a breeze.
• Manticore also boasts built-in replication and load balancing for added reliability.
• Data can be synced from sources such as MySQL, PostgreSQL, ODBC, xml, and csv with ease.
• While not fully ACID-compliant, Manticore still supports transactions and binlog to ensure safe writes.
• Effortless data backup and recovery with built-in tools and SQL commands