The article addresses the misconceptions surrounding MongoDB’s durability, emphasizing that its WiredTiger storage engine is highly robust and easily demonstrable. For over a decade, WiredTiger has provided advanced block corruption protection, often surpassing other industry-leading databases. This article showcases how to simulate write loss in a lab environment and observe how different databases detect such corruptions to prevent the return of compromised data.

PostgreSQL’s Approach to Data Integrity

To illustrate the challenges when a database fails to detect lost writes, PostgreSQL was chosen for this demonstration. While PostgreSQL version 18 and later enable checksums by default, a lab experiment using a Docker instance of PostgreSQL 18rc1 revealed limitations. After setting up a demo table, inserting data, triggering a checkpoint, and flushing the buffer cache, an update to a specific row was performed. Subsequently, simulating disk corruption by overwriting the updated data block with an older, valid block did not trigger an error in PostgreSQL. This means that if a storage failure writes a correct, but misplaced, block, PostgreSQL’s checksums might not detect the inconsistency, potentially returning incorrect data. However, if the corruption is not block-aligned or is structurally invalid, PostgreSQL’s checksums successfully identify the issue, leading to an error.

Oracle Database and Lost Write Protection

Oracle Database employs a different strategy to detect lost writes. It compares block checksums with those on standby databases, significantly reducing the probability of corruption occurring simultaneously across multiple environments. This robust “Lost Write Protection” mechanism has been demonstrated in previous studies.

WiredTiger (MongoDB Storage Engine): Advanced Corruption Prevention

MongoDB’s WiredTiger storage engine excels in preventing lost writes and detecting disk failures that could lead to incorrect page retrieval. WiredTiger achieves this by embedding a checksum directly into the “address cookie” of each page address within the BTree pointers. This opaque set of bytes includes the offset, size, checksum, and object ID of the referenced block.

A lab setup involving a MongoDB container and the wt command-line utility (for direct interaction with WiredTiger files) was used. After creating a demo table and inserting records, an update to a record was performed. WiredTiger’s copy-on-write mechanism ensures that updates write new blocks instead of overwriting existing ones, inherently reducing corruption risks. When attempting to simulate corruption by replacing the new data block with an old, valid one, WiredTiger immediately detected the mismatch. The embedded checksum in the BTree pointer did not align with the checksum of the block it was referencing (the old block), triggering an error. This demonstrates that even with direct file system access, it is incredibly difficult to corrupt WiredTiger data undetected, as any change requires updating checksums that are intricately linked across BTree pointers.

Conclusion: Superior Durability and Detection

PostgreSQL’s checksums help detect data corruption where a page’s checksum doesn’t match its content. However, it can falter if a system writes a valid but incorrect block, or misses a write, potentially returning erroneous results. Oracle Database utilizes checksums and, with Data Guard, can verify data integrity by transmitting checksums across the network to standby databases. MongoDB’s WiredTiger, with checksums enabled by default, offers a highly sophisticated and robust solution. By embedding the expected checksum within the BTree address cookie, every internal BTree pointer effectively carries the checksum for its referenced page. This design, coupled with its copy-on-write architecture, enables WiredTiger to detect swapped or obsolete pages without requiring communication with replicas, establishing it as a leader in data durability and corruption detection.