Improve concurrent database access by JanJakes · Pull Request #361 · WordPress/sqlite-database-integration

JanJakes · 2026-04-13T14:59:33Z

Summary

Reduces lock contention under concurrent access by avoiding unnecessary write locks for read-only queries.

Skip the wrapper transaction for SELECT statements. A standalone SELECT translates to a single SQLite query, so the implicit autocommit transaction provides sufficient consistency — no explicit BEGIN/COMMIT needed.
Use deferred BEGIN for other read-only wrapper transactions. SHOW and DESCRIBE queries still use a wrapper transaction (they may translate to multiple SQLite queries), but now use a deferred BEGIN (SHARED lock) instead of BEGIN IMMEDIATE (RESERVED lock).

Background

The wrapper transaction previously used BEGIN IMMEDIATE for all queries, including read-only SELECTs. This acquired a RESERVED (write) lock on every query — but SQLite only allows one RESERVED lock at a time. Under concurrent load (e.g., multiple PHP-FPM workers handling requests simultaneously), all processes competed for the single write lock. When the 10-second busy timeout was exceeded, SQLite returned SQLITE_BUSY ("database is locked", error 5).

SHARED locks, acquired by a deferred BEGIN, are compatible with RESERVED locks — multiple readers can proceed concurrently alongside an active writer.

Fixes #318

1. Use deferred BEGIN (SHARED lock) instead of BEGIN IMMEDIATE (RESERVED lock) for read-only wrapper transactions (SHOW, DESCRIBE). 2. Skip the wrapper transaction entirely for SELECT statements. Fixes: #318

bgrgicak · 2026-04-16T04:18:13Z

+			 * [GRAMMAR]
+			 * simpleStatement: selectStatement | showStatement | utilityStatement | ...
+			 */
+			$statement_node = $child_node->get_first_child_node();


Should we wrap this in if ( null === $child_node ) because it seems like $child_node can be null?

👍 At this point in the AST, there is probably always a child node, but it's a good idea to add it, also for consistency with the code above. Fixed in 14dc77a.

bgrgicak · 2026-04-16T04:22:34Z

+/**
+ * Tests for concurrent access to the same SQLite database file.
+ */
+class WP_SQLite_Driver_Concurrency_Tests extends TestCase {


Should we add a test for showStatement queries to confirm that it uses the correct BEGIN?

👍 Added tests for SHOW and DESCRIBE in c559b3a.

JanJakes · 2026-04-16T10:32:02Z

@bgrgicak Thanks! Addressed both issues.

Pin BEGIN IMMEDIATE for write statements (INSERT, UPDATE, DELETE, REPLACE, CREATE/ALTER/DROP/TRUNCATE TABLE) so a future regression that accidentally flags a write as read-only — downgrading the lock to a deferred BEGIN — is caught immediately. Mirror the existing SELECT two-connection contention test for SHOW and DESCRIBE, proving they succeed under a concurrent RESERVED lock instead of only asserting the logged BEGIN string. Factor out a couple of small helpers to remove the repeated driver and query-logger setup boilerplate.

## Release `3.0.0-rc.1` Version bump and changelog update for release `3.0.0-rc.1`. **Changelog draft:** * Improve concurrent database access ([#361](#361)) * Remove legacy SQLite driver ([#358](#358)) **Full changelog:** v2.2.23...release/v3.0.0-rc.1 ## Next steps 1. **Review** the changes in this pull request. 2. **Push** any additional edits to this branch (`release/v3.0.0-rc.1`). 3. **Merge** this pull request to complete the release. Merging will automatically build the plugin ZIP and create a [GitHub release](https://github.com/WordPress/sqlite-database-integration/releases). > [!NOTE] > This is a **pre-release**. It will not be deployed to [WordPress.org](https://wordpress.org/plugins/sqlite-database-integration/).

@Unlink

…" on connect) (#443) Related to STU-1821 ## What changed Every WordPress request runs `SET SESSION sql_mode = …` at bootstrap (via `wpdb::set_sql_mode()`). This statement changes only session state and writes nothing to the database, but the driver acquires an SQLite write lock for it. Under concurrent access, this makes an otherwise read-only request fatal with `database is locked` (`SQLITE_BUSY`) the moment any other connection holds the write lock. This PR fixes the two places the `SET` path takes a write lock (`WP_PDO_MySQL_On_SQLite`): 1. **The wrapper transaction.** `query()` opens `BEGIN IMMEDIATE` (a RESERVED/write lock) for every non-`SELECT` statement, and `SET` was treated as a write. → `SET` is now detected as read-only, so it opens a deferred `BEGIN` (SHARED) instead — the same treatment `SHOW`/`DESCRIBE` already receive (#361). 2. **The empty-result builder.** `create_result_statement_from_data()` fabricates a 0-column result with a no-op `INSERT … WHERE FALSE` against a regular table. SQLite acquires a write lock for *any* DML, even one that inserts zero rows, so this re-took a write lock right after the wrapper committed. → The 0-column statement is now built with a no-op `INSERT` into a connection-private **`TEMP` table**, whose temp database has no shared lock. Both changes are required: with only (1), the fatal simply moves from the `BEGIN IMMEDIATE` to the no-op `INSERT`. ## Why This is a direct follow-up to #361 (which stopped `SELECT`/`SHOW`/`DESCRIBE` from taking write locks and closed #318). `SET` was left on the write path. Because `set_sql_mode()` runs on every connection, the impact is broad: a single slow request holding a write transaction (a background WooCommerce/Jetpack write, an import, a long render on a heavy site) makes every concurrent request fail at connect, not only concurrent writers. SQLite is single-writer by design, but statements that perform no writes should never contend for the write lock. Journal mode does not address this on its own: it reproduces identically under both `DELETE` and `WAL` (WAL still serialises writers). ## Reproduction Deterministic, single process, two driver connections to one file: ```php $path = tempnam( sys_get_temp_dir(), 'sqlite' ); @Unlink( $path ); $make = fn() => new WP_SQLite_Driver( new WP_SQLite_Connection( array( 'path' => $path ) ), 'db' ); // Build both connections first, so B's schema setup doesn't run under the lock. $a = $make(); $a->query( 'CREATE TABLE t ( id INTEGER PRIMARY KEY )' ); $b = $make(); $b->execute_sqlite_query( 'PRAGMA busy_timeout = 1500' ); // fail fast on a lock // A holds a write lock. $a->query( 'START TRANSACTION' ); $a->query( 'INSERT INTO t ( id ) VALUES ( 1 )' ); // B runs the connect-time statement. It writes nothing, so it must not block. $b->query( "SET SESSION sql_mode = 'NO_ENGINE_SUBSTITUTION'" ); // Before this PR: throws "database is locked". After: returns immediately. ``` **Before:** `WP_SQLite_Driver_Exception: … database is locked`, first in `begin_wrapper_transaction()`, or after fixing only (1), in `create_result_statement_from_data()`. **After:** the `SET` returns immediately while A still holds its write lock. This also reproduces end-to-end in a multi-worker WordPress install (eg. PHP-FPM with more than one workers) under concurrent requests, where one slow request holding a write transaction makes concurrent page loads fatal at `set_sql_mode`. ## Tests Two regression tests added to `WP_SQLite_Driver_Concurrency_Tests`: - `testSetQueryOpensReadOnlyTransaction`: `SET` opens `BEGIN`, not `BEGIN IMMEDIATE`. - `testSetQuerySucceedsWhileAnotherConnectionHoldsWriteLock`: with another connection holding `BEGIN IMMEDIATE` and `timeout = 0`, `SET` completes instead of throwing `SQLITE_BUSY`. Both fail on `trunk` and pass with this change. ## Notes / risk - **The 0-column contract is preserved.** `WP_SQLite_Driver::query()` returns `fetchAll()` when `columnCount() > 0`, otherwise `rowCount()`. The replacement empty-result statement keeps `columnCount() === 0`, so writes and `SET` still return their affected-row count (not an empty array). Verified: running a representative `CREATE`/`INSERT`/`SELECT`/`COUNT`/`UPDATE`/`DELETE`/`SET` sequence produces byte-identical results before and after. - **Bonus:** real writes (`INSERT`/`UPDATE`/`DELETE`) also fabricate their empty result through this path, which previously meant a second, redundant write-lock acquisition after the real write. That redundant lock is now gone, slightly reducing lock pressure for all writes. - **Temp-table lifetime.** The `TEMP` table is created once per connection (guarded by a flag); `TEMP` tables live for the connection's lifetime. A reviewer preferring maximum robustness over the micro-optimisation could drop the flag and always issue the idempotent `CREATE TEMP TABLE IF NOT EXISTS`. --------- Co-authored-by: Adam Zieliński <adam@adamziel.com>

Improve concurrent database access

752a8c4

1. Use deferred BEGIN (SHARED lock) instead of BEGIN IMMEDIATE (RESERVED lock) for read-only wrapper transactions (SHOW, DESCRIBE). 2. Skip the wrapper transaction entirely for SELECT statements. Fixes: #318

JanJakes marked this pull request as ready for review April 13, 2026 15:29

JanJakes requested review from a team and bgrgicak April 14, 2026 14:42

bgrgicak reviewed Apr 16, 2026

View reviewed changes

JanJakes added 2 commits April 16, 2026 12:21

Guard against null child node in read-only detection

14dc77a

Add tests for SHOW and DESCRIBE using a deferred BEGIN transaction

c559b3a

JanJakes requested a review from bgrgicak April 16, 2026 10:31

bgrgicak approved these changes Apr 17, 2026

View reviewed changes

JanJakes merged commit 2224d1a into trunk Apr 17, 2026
16 checks passed

JanJakes deleted the db-locked-error branch April 17, 2026 12:36

JanJakes mentioned this pull request Apr 17, 2026

Release 3.0.0-rc.1 #364

Merged

gcsecsey mentioned this pull request Jun 29, 2026

Don't take a write lock for SET statements (fixes "database is locked" on connect) #443

Merged

coderabbitai Bot mentioned this pull request Jul 24, 2026

auto-recover: orphaned worker branch for #2321 Ultimate-Multisite/superdav-ai-agent#2324

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve concurrent database access - #361

Improve concurrent database access#361
JanJakes merged 4 commits into
trunkfrom
db-locked-error

JanJakes commented Apr 13, 2026

bgrgicak Apr 16, 2026

JanJakes Apr 16, 2026

bgrgicak Apr 16, 2026

JanJakes Apr 16, 2026

JanJakes commented Apr 16, 2026

Uh oh!

Labels

2 participants

Uh oh!

Conversation

JanJakes commented Apr 13, 2026

Summary

Background

bgrgicak Apr 16, 2026

Choose a reason for hiding this comment

JanJakes Apr 16, 2026

Choose a reason for hiding this comment

bgrgicak Apr 16, 2026

Choose a reason for hiding this comment

JanJakes Apr 16, 2026

Choose a reason for hiding this comment

JanJakes commented Apr 16, 2026

Uh oh!

Labels

2 participants