Implement `INSERT` and `UPDATE` value type casting by JanJakes · Pull Request #276 · WordPress/sqlite-database-integration

JanJakes · 2025-10-23T16:39:06Z

This PR originates from the following error (#268):

SQLSTATE[23000]: Integrity constraint violation:
19 cannot store TEXT value in BLOB column wp_wfconfig.val

However, there isn't a simple fix. It requires correctly casting values for INSERT, REPLACE, and UPDATE statements, and that's what this PR implements. Type casting logic, together with "implicit defaults", was already implemented for MySQL's "non-strict" mode. This PR extends the type casting part to be used with both strict and non-strict modes.

Generally, it does the following:

-- Rewrite INSERT statement from:
INSERT INTO t (col1, col2, ...) VALUES (v1, v2, ...);

-- To:
INSERT INTO t (col1, col2, ...)
SELECT <apply-type-casting-to-select-list> FROM (VALUES (v1, v2, ...)) WHERE true;

-- Rewrite UPDATE statement from:
UPDATE t SET col1 = value;

-- To:
UPDATE t SET col1 = <apply-type-casting-to-value>;

The PR comes with a comprehensive test suite, but it doesn't implement all tiny MySQL nuances and acknowledges some differences in TODO comments. It should increase the type compatibility when saving values significantly. We can address more edge cases separately, and it should also enable us checking for column lengths and trimming or enforcing the values based on the SQL mode.

Fixes #268.

Copilot

Copilot reviewed 5 out of 5 changed files in this pull request and generated 11 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-10-29T16:23:56Z

@JanJakes I've opened a new pull request, #278, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot · 2025-10-29T16:25:10Z

@JanJakes I've opened a new pull request, #279, to work on those changes. Once the pull request is ready, I'll request review from you.

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

adamziel · 2025-10-30T11:13:02Z

+		$this->assertQuery( "INSERT INTO t VALUES ('3.0')" );
+
+		// TODO: These are supported in MySQL:
+		$this->assertQueryError( "INSERT INTO t VALUES ('4.5')", 'SQLSTATE[23000]: Integrity constraint violation: 19 cannot store REAL value in INTEGER column t.value' );


Good call describing known limitations of the system with tests

adamziel · 2025-10-30T11:15:12Z

+	 * @throws Exception The error message.
+	 * @return void
+	 */
+	public function throw( $message ): void {


I'm surprised this is not a reserved keyword and makes a valid method name. PHP docs confirms it's safe:

They are, however, allowed as property, constant, and method names of classes, interfaces and traits, except that class may not be used as constant name.

TIL

adamziel · 2025-10-30T11:16:08Z

+		if ( 0 === count( $columns ) ) {
+			throw $this->new_driver_exception(
+				sprintf(
+					"SQLSTATE[42S02]: Base table or view not found: 1146 Table '%s' doesn't exist",


MySQL error codes! Nice

adamziel · 2025-10-30T11:17:57Z

-					$this->cast_value_in_non_strict_mode( $column['DATA_TYPE'], $identifier ),
-					$identifier
-				);
+				$fragment  .= $this->cast_value_for_insert_or_update( $column['DATA_TYPE'], $identifier );


cast_value_for_insert_or_update – Any other statements that need casting? E.g. REPLACE INTO?

@adamziel Yeah, REPLACE as well. Maybe we could use a more generic name like cast_value_for_saving or something along those lines?

adamziel · 2025-10-30T11:18:54Z

 	 *
-	 * When the strict mode is not enabled, executing an UPDATE statement that
-	 * sets a NOT NULL column value to NULL saves an IMPLICIT DEFAULT instead.
+	 * This method applies relevant type casting and emulates IMPLICIT DEFAULT


Let's drop a couple of links to the relevant MySQL doc page in a few places as a reference:

https://dev.mysql.com/doc/refman/8.4/en/data-type-defaults.html

adamziel · 2025-10-30T11:19:38Z

 		string $translated_value
 	): string {
-		$sqlite_data_type = self::DATA_TYPE_STRING_MAP[ $mysql_data_type ];
+		// TODO: This is also a good place to implement checks for maximum column


adamziel · 2025-10-30T11:19:54Z

 				} elseif ( 'year' === $mysql_data_type ) {
-					$function_call = sprintf( "STRFTIME('%%Y', %s)", $translated_value );
+					/*
+					 * The YEAR type in MySQL only uses 1 byte and therefore


good comment

adamziel · 2025-10-30T11:22:27Z

 	 * @return string                   The translated value.
 	 */
-	private function cast_value_in_non_strict_mode(
+	private function cast_value_for_insert_or_update(


Do we need to consider any potential interferences with SQLite's type affinity?

sqlite> create table t(a int); sqlite> insert into t(a) values (22), ('33'); sqlite> select a from t where a = '22' or a = 33; 22 33

https://www.sqlite.org/datatype3.html

@adamziel This particular example behaves the same in MySQL, but generally, it can be a problem when an expression evaluates types differently, or comparisons are automatically type casted in one DB but not in the other one.

CREATE TABLE t(a DATE); INSERT INTO t(a) VALUES ('2021-01-01'); SELECT * FROM t WHERE a = '20210101'

This returns one row in MySQL but no rows in SQLite. It's probably difficult to solve—it would require inferring the correct type of every part of every expression. Fortunately, the most common numeric string cast seems to work in both DBs.

@adamziel Checking further, the situation is not that good:

SELECT 1 = '1'; -- true in MySQL, false in SQLite SELECT 1 WHERE 1 = '1'; -- one row in MySQL, no rows in SQLite

So it appears that the numeric string type cast is only applied when comparing against a numeric column.

We can probably come up with a sqlite expression equivalent to MYSQL_TYPECAST( value, column_type ) => value_in_column_type that pattern-matches and transforms the value in the same way as mysql-server would. Maybe we can even source the logic from the actual mysql-server codebase

@adamziel I think that's basically what the cast_value_for_saving() method does. It knows the column type, and it has a value expression, and then it wraps it in a casting expression. We can make it more exhaustive and handle all edge cases, but the core logic is there.

What will be a problem, though, is expressions where we have no column type at all—things like WHERE '1' = 1, but with arbitrary complexity. Anyway, that's probably a topic for another PR.

Separate pr sounds good! The transform there would be more like (lhs value, lhs type, comparison, rhs value, rhs type) -> expression that evaluates to true or false using mysql rules.

@adamziel Oh, right. Seems like it's definitely better to tackle this separately when needed. Rewriting all expressions on every nesting level seems like a bigger topic and it could possibly cause some performance issues (like index not being used due to inline casting, etc.).

Otherwise, do you think this is good to merge?

adamziel · 2025-10-30T11:23:39Z

+				// In non-strict mode, they get an IMPLICIT DEFAULT value.
+				if ( $is_strict_mode ) {
+					$fallback = sprintf(
+						"THROW('Incorrect %s value: ''' || %s || '''')",


Such a nice use of callbacks 🤯 ! At the same time – could we just throw in place? Do we need to delay it? What's the benefit of doing this vs just throwing a new exception right here?

@adamziel We need to throw the error from SQLite, because in PHP we don't know the value. You can insert known values (INSERT INTO ... VALUES (1, 2, 3)), as well as unknown ones INSERT INTO ... SELECT .... For that reason, we need SQLite to evaluate this for every inserted/updated row, as we can't do that in PHP. In PHP the "value" can be a column name, an expression, etc.

JanJakes · 2025-10-30T16:51:54Z

@adamziel I added two commits with small fixes.

… clause (#282) A small follow-up on #276. We emulate MySQL type casting for `INSERT`, `REPLACE`, and `UPDATE` values, but we need to do so for the `ON DUPLICATE KEY UPDATE` clause as well. This fixes some errors in the top 100 plugin activation tests.

JanJakes force-pushed the insert-update-type-casting branch 4 times, most recently from 955402f to d04ff75 Compare October 29, 2025 11:00

JanJakes added 3 commits October 29, 2025 13:07

Fix database name used for INSERT and UPDATE in non-strict mode

5a61725

Use case-insensitive comparison for INSERT column names

ab00d0a

Improve docs and code readability

f8c56eb

JanJakes force-pushed the insert-update-type-casting branch 2 times, most recently from ff98a29 to a1eb12e Compare October 29, 2025 14:03

Apply INSERT type casting also in STRICT SQL mode

51daf2b

JanJakes force-pushed the insert-update-type-casting branch from a1eb12e to 8bd98ae Compare October 29, 2025 15:53

JanJakes marked this pull request as ready for review October 29, 2025 16:13

JanJakes requested a review from Copilot October 29, 2025 16:13

Copilot AI reviewed Oct 29, 2025

View reviewed changes

Copilot AI mentioned this pull request Oct 29, 2025

No changes needed - code already uses count() > 0 per feedback #278

Closed

Copilot AI mentioned this pull request Oct 29, 2025

Acknowledge polyfill support for PHP 8.0+ string functions #279

Closed

JanJakes added 6 commits October 29, 2025 18:04

Fix YEAR type behavior

c72af18

Apply UPDATE type casting also in STRICT SQL mode

503071c

Fix query tests by MySQL syntax to create schema, not SQLite

a18901b

Improve INSERT and UPDATE error handling

951fc45

Add a fix for a PDO SQLite bug in PHP < 7.3 (#79664)

69e1189

See also: php/php-src#5654

Fix typos

b7f7921

JanJakes force-pushed the insert-update-type-casting branch from 8bd98ae to b7f7921 Compare October 29, 2025 17:07

JanJakes requested a review from Copilot October 29, 2025 17:14

Copilot AI reviewed Oct 29, 2025

View reviewed changes

Comment thread wp-includes/sqlite-ast/class-wp-sqlite-driver.php

Comment thread wp-includes/sqlite-ast/class-wp-sqlite-driver.php

Comment thread wp-includes/sqlite-ast/class-wp-sqlite-driver.php

Comment thread wp-includes/sqlite-ast/class-wp-sqlite-driver.php

JanJakes requested a review from adamziel October 30, 2025 08:28

adamziel reviewed Oct 30, 2025

View reviewed changes

JanJakes added 2 commits October 30, 2025 17:44

Improve method naming and docs

8bc21a0

Add links to MySQL docs

2d1f6b6

JanJakes requested a review from adamziel October 30, 2025 16:51

adamziel merged commit def2400 into develop Oct 31, 2025
14 checks passed

JanJakes deleted the insert-update-type-casting branch October 31, 2025 08:29

JanJakes mentioned this pull request Nov 3, 2025

Apply type casting and implicit defaults to ON DUPLICATE KEY UPDATE clause #282

Merged

Uh oh!

Conversation

JanJakes commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI commented Oct 29, 2025

Copilot AI commented Oct 29, 2025

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adamziel Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JanJakes Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adamziel Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JanJakes commented Oct 30, 2025

Uh oh!

Labels

4 participants

JanJakes commented Oct 23, 2025 •

edited

Loading

adamziel Oct 30, 2025 •

edited

Loading

JanJakes Oct 30, 2025 •

edited

Loading

adamziel Oct 30, 2025 •

edited

Loading