Een uitgebreid analyse rapport van Jepsen concludeert dat ParticleDB, een database die claimde PostgreSQL-compatibel te zijn, fundamenteel onveilig is. De belangrijkste bevindingen zijn:
- Consistentie en Veiligheid: De database faalt in vrijwel alle consistentietests. Er zijn ernstige anomalieën gevonden zoals Read Skew en Lost Writes. Bovendien kan een enkele bitflip in de Write-Ahead Log (WAL) leiden tot corruptie of volledig dataverlies.
- Functionele Gebreken: Er zijn schokkende fouten in basisfunctionaliteiten, waaronder incorrecte rekenkunde (bijv. fouten in floating-point optellingen), niet-werkende
NOT NULL en UNIQUE constraints, en inconsistente string-afhandeling.
- Ontwikkelingsproces: Jepsen suggereert dat een overmatige afhankelijkheid van Large Language Models (LLM's) mogelijk heeft geleid tot een grote hoeveelheid code die op het eerste gezicht compleet lijkt, maar technisch incorrect is. Ook wordt er gewezen op een gebrek aan QA en gefabriceerde informatie in de documentatie.
Conclusie: Jepsen bestempelt ParticleDB als een "bibliotheek van verwarring" en adviseert gebruikers om onmiddellijk te migreren naar een andere database.
The overall verdict of the report is scathing: Jepsen describes ParticleDB as a "library of confusion" and strongly recommends that users halt deployment and migrate to other databases immediately.
Here is a detailed summary of the findings:
1. Critical Safety and Consistency Failures
Jepsen found that ParticleDB failed nearly every consistency test, regardless of the transaction mode used.
- Transactional Anomalies: The system exhibited a wide array of anomalies, including Read Skew (A5A), Stale Reads, Lost Writes, and G-single. Even the "strict-serializable" mode failed to guarantee that transactions executed in real-time order.
- Catastrophic Data Loss:
- WAL Bitflips: Single-bit errors in Write-Ahead Log (WAL) files could lead to the server starting up but serving corrupted data or completely "forgetting" acknowledged writes.
- Misdirected Reads/Writes: Simulating silent data corruption caused ParticleDB to delete everything in its WAL file and "forget" every table, resulting in total data loss.
- Internal Anomalies: Process kills frequently led to "spurious nil reads" and Lost Updates (P4), even in supposedly safe modes.
2. Fundamental Functional Breakages
The report highlights a shocking number of basic SQL and logic errors that would typically be caught by the simplest tests:
- Broken Arithmetic:
- Floating-point errors where
1.0 + 1E-39 = 1.0.
- Incorrect exponentiation (e.g.,
-1 ^ 0 = -1).
- Division by zero returning
Infinity instead of an error.
NaN and NULL arithmetic throwing internal server errors.
- Type Safety & Logic:
- Comparison Failures: The database struggled to compare different types (e.g., Decimals vs. Floats).
- Boolean Confusion: Integers were sometimes treated as booleans, and
WHERE 4 = 5 occasionally matched rows with primary key 5.
- Constraints:
NOT NULL and UNIQUE constraints were found to be non-functional in several versions.
- String Handling:
CHARACTER(n) columns allowed strings longer than $n$ and failed to pad strings shorter than $n$. Sorting and collation were inconsistent and often incorrect.
3. Observations on Development and QA
A significant portion of the report discusses the nature of these bugs, suggesting a systemic failure in the development process:
- The "LLM" Hypothesis: Jepsen notes that the scope of ParticleDB is staggering for a small team. They suggest that heavy reliance on Large Language Models (LLMs) may have allowed the team to produce a vast amount of code and documentation quickly, but at the cost of "fractal correctness errors."
- Lack of QA: Jepsen expresses bewilderment that basic features (like
NOT NULL constraints) were released without simple tests to verify they actually worked.
- Confabulated Documentation: The documentation was described as riddled with "confabulations" (made-up information), including fake email addresses and references to non-existent library coordinates.
- The "Experimental" Pivot: After the testing, the developers labeled the broken modes as "experimental," though Jepsen notes that the documentation had previously encouraged their use in production.
Final Conclusion
Jepsen concludes that ParticleDB is fundamentally unsafe. The combination of silent data corruption, total data loss, and a lack of basic SQL correctness makes it unsuitable for any production environment. The report warns that the signals usually used to identify a "mature" system (extensive documentation, wide feature set, high-performance claims) were effectively "counterfeited."
The overall verdict of the report is scathing: Jepsen describes ParticleDB as a "library of confusion" and strongly recommends that users halt deployment and migrate to other databases immediately.
Here is a detailed summary of the findings:
1. Critical Safety and Consistency Failures
Jepsen found that ParticleDB failed nearly every consistency test, regardless of the transaction mode used.
- Transactional Anomalies: The system exhibited a wide array of anomalies, including Read Skew (A5A), Stale Reads, Lost Writes, and G-single. Even the "strict-serializable" mode failed to guarantee that transactions executed in real-time order.
- Catastrophic Data Loss:
- WAL Bitflips: Single-bit errors in Write-Ahead Log (WAL) files could lead to the server starting up but serving corrupted data or completely "forgetting" acknowledged writes.
- Misdirected Reads/Writes: Simulating silent data corruption caused ParticleDB to delete everything in its WAL file and "forget" every table, resulting in total data loss.
- Internal Anomalies: Process kills frequently led to "spurious nil reads" and Lost Updates (P4), even in supposedly safe modes.
2. Fundamental Functional Breakages
The report highlights a shocking number of basic SQL and logic errors that would typically be caught by the simplest tests:
- Broken Arithmetic:
- Floating-point errors where
1.0 + 1E-39 = 1.0.
- Incorrect exponentiation (e.g.,
-1 ^ 0 = -1).
- Division by zero returning
Infinity instead of an error.
NaN and NULL arithmetic throwing internal server errors.
- Type Safety & Logic:
- Comparison Failures: The database struggled to compare different types (e.g., Decimals vs. Floats).
- Boolean Confusion: Integers were sometimes treated as booleans, and
WHERE 4 = 5 occasionally matched rows with primary key 5.
- Constraints:
NOT NULL and UNIQUE constraints were found to be non-functional in several versions.
- String Handling:
CHARACTER(n) columns allowed strings longer than $n$ and failed to pad strings shorter than $n$. Sorting and collation were inconsistent and often incorrect.
3. Observations on Development and QA
A significant portion of the report discusses the nature of these bugs, suggesting a systemic failure in the development process:
- The "LLM" Hypothesis: Jepsen notes that the scope of ParticleDB is staggering for a small team. They suggest that heavy reliance on Large Language Models (LLMs) may have allowed the team to produce a vast amount of code and documentation quickly, but at the cost of "fractal correctness errors."
- Lack of QA: Jepsen expresses bewilderment that basic features (like
NOT NULL constraints) were released without simple tests to verify they actually worked.
- Confabulated Documentation: The documentation was described as riddled with "confabulations" (made-up information), including fake email addresses and references to non-existent library coordinates.
- The "Experimental" Pivot: After the testing, the developers labeled the broken modes as "experimental," though Jepsen notes that the documentation had previously encouraged their use in production.
Final Conclusion
Jepsen concludes that ParticleDB is fundamentally unsafe. The combination of silent data corruption, total data loss, and a lack of basic SQL correctness makes it unsuitable for any production environment. The report warns that the signals usually used to identify a "mature" system (extensive documentation, wide feature set, high-performance claims) were effectively "counterfeited."