Database

This article explains the challenges of scaling database connections and how connection pooling and multiplexing can address them.

This article explains and compares the CutOver algorithms of pt-osc, gh-ost, and Facebook's OnlineSchemaChange.

Exploring the design of a distributed read-write lock for PolarDB-X, this blog delves into the complexities and considerations of ensuring synchronized DDL execution in a distributed database environment.

This article compares WeScale, a MySQL-compatible database proxy, with MySQL, highlighting its read-write splitting, connection pooling, and failover features. WeScale maintains high throughput and low latency under heavy loads, with minimal additional memory use and about 1/3 CPU overhead. It scales better both horizontally and cost-effectively than scaling MySQL vertically.

The article explains WeSQL WeScale's read-write splitting, ensuring read-after-write consistency and load balancing.

The article discusses the necessity of ACID properties for database DDL operations, challenges in traditional systems, and how PolarDB-X implements transactional DDL, ensuring atomicity, consistency, isolation, and durability to enhance reliability and operational flexibility.

Database schema evolution (DDL) poses unique challenges in ensuring concurrency with transactions while maintaining ACID properties. Traditional approaches use metadata locks (MDLs) to serialize DDL and DML/DQL, but this can severely impact performance. This blog discusses the issues and explores solutions like multi-version concurrency control to enable highly concurrent non-blocking schema changes.

This blog discusses the fatal problem of distributed metadata lock (MDL) deadlocks that can occur when distributed transactions and database definition language (DDL) statements execute concurrently across multiple database nodes. It explains the causes and presents a solution for detecting these distributed MDL deadlocks.

I explore the motivation and techniques behind building a vectorized execution engine for distributed queries. Traditional tuple-at-a-time evaluation fails to utilize hardware efficiently at scale. By expressing queries as linear algebraic operations on batches of column vectors using generated kernels, significant performance gains can be achieved through improved data locality, reduced interpretation overhead and better utilization of CPU resources like SIMD units.

I explore approaches to automatically test databases for logic bugs that can severely impact applications and data integrity. Traditional testing struggles with database complexity, so I examine new techniques like Pivoted Query Synthesis, Non-Optimizing Reference Engine Construction and Ternary Query Partitioning. By generating and validating randomized queries, these methods can reveal hidden logic errors not discovered by conventional testing.

This blog post explores the fascinating interplay between Hybrid Logical Clocks (HLC) and Multi-Version Concurrency Control (MVCC) in enabling distributed transactions in CockroachDB. It covers how HLC addresses causality issues and how MVCC handles concurrency conflicts in a distributed setting.

This document analyzes the distributed transaction protocol used in Percolator, a system for large-scale incremental processing deployed by Google. Percolator extends Bigtable to provide multi-row ACID transactions through a standard two-phase commit protocol.