Skip to content

direct: Fix state serialization for zero-value fields on deeply-embedded SDK specs#5782

Open
pietern wants to merge 2 commits into
mainfrom
enable-pg-native-login-bug
Open

direct: Fix state serialization for zero-value fields on deeply-embedded SDK specs#5782
pietern wants to merge 2 commits into
mainfrom
enable-pg-native-login-bug

Conversation

@pietern

@pietern pietern commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

The dyn→typed ForceSendFields routing only handled one level of struct embedding. When a field declared in an SDK spec embedded two levels deep (e.g. PostgresProjectPostgresProjectConfigProjectSpec) was set to its zero value, its name was recorded in the wrong struct's ForceSendFields. The direct engine then failed to serialize the plan state with field X cannot be found in struct Y.

Fix routes each field's ForceSendFields entry to the struct that actually declares it, at any embedding depth. This affected postgres_projects (enable_pg_native_login: false), postgres_branches, and postgres_endpoints (replace_existing: false). Terraform was unaffected (it serializes via dyn, not the SDK marshaler).

Adds a unit regression test for the deep-embedding case and a systemic guard that round-trips every registered resource type's zero-value fields, so any future resource using the embedded-spec wrapper pattern is covered automatically.

This pull request and its description were written by Isaac.

The dyn->typed ForceSendFields routing only handled one level of struct
embedding. When a field declared in an SDK spec embedded two levels deep
(e.g. PostgresProject -> PostgresProjectConfig -> ProjectSpec) was set to
its zero value, its name was recorded in the wrong struct's
ForceSendFields. The direct engine then failed to serialize the plan
state with "field X cannot be found in struct Y".

Route each field's ForceSendFields entry to the struct that actually
declares it, at any embedding depth. This affected postgres_projects
(enable_pg_native_login: false), postgres_branches, and postgres_endpoints
(replace_existing: false). Terraform was unaffected (it serializes via
dyn, not the SDK marshaler).

Add a unit regression test for the deep-embedding case and a systemic
guard that round-trips every registered resource type's zero-value
fields, so any future resource using the embedded-spec wrapper pattern is
covered automatically.

Co-authored-by: Isaac
@pietern pietern requested a review from denik June 30, 2026 18:47
@pietern pietern temporarily deployed to test-trigger-is June 30, 2026 18:47 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 30, 2026 18:47 — with GitHub Actions Inactive
@github-actions

github-actions Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/bundle/ - needs approval

Files: acceptance/bundle/invariant/configs/postgres_project.yml.tmpl, acceptance/bundle/resources/postgres_projects/basic/databricks.yml.tmpl, acceptance/bundle/resources/postgres_projects/basic/out.requests.json
Suggested: @denik
Also eligible: @janniklasrose, @anton-107, @andrewnester, @shreyas-goenka, @lennartkats-db

/bundle/ - needs approval

Files: bundle/config/resources_types_test.go
Suggested: @denik
Also eligible: @janniklasrose, @anton-107, @andrewnester, @shreyas-goenka, @lennartkats-db

General files (require maintainer)

4 files changed
Based on git history:

  • @denik -- recent work in ./, libs/dyn/convert/, bundle/config/

Any maintainer (@andrewnester, @anton-107, @denik, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.
See OWNERS for ownership rules.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 29cab63

Run: 28522520649

Env 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 7 3 13 230 1039 5:16
🟨​ aws windows 7 2 3 13 230 1037 10:31
💚​ aws-ucws linux 10 13 314 957 4:53
💚​ aws-ucws windows 10 13 316 955 4:03
💚​ azure linux 4 15 230 1038 4:21
💚​ azure windows 4 15 232 1036 3:43
💚​ azure-ucws linux 4 15 316 954 5:35
🔄​ azure-ucws windows 2 2 15 318 952 5:59
💚​ gcp linux 4 15 229 1040 3:51
💚​ gcp windows 4 15 231 1038 3:30
25 interesting tests: 13 SKIP, 7 KNOWN, 4 flaky, 1 RECOVERED
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/replace_existing 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestSecretsPutSecretBytesValue ✅​p 🔄​f 🙈​s 🙈​s ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestSecretsPutSecretStringValue ✅​p 🔄​f 🙈​s 🙈​s ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
💚​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestFetchRepositoryInfoAPI_FromRepo/root 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R
🔄​ TestFetchRepositoryInfoAPI_FromRepo/subdir 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R
Top 5 slowest tests (at least 2 minutes):
duration env testname
3:10 aws windows TestSecretsPutSecretStringValue
3:00 azure-ucws windows TestAccept
2:47 azure windows TestAccept
2:44 gcp windows TestAccept
2:43 aws-ucws windows TestAccept
Comment thread libs/dyn/convert/struct_info.go Outdated
ForceEmpty: make(map[string]bool),
GolangNames: make(map[string]string),
ForceSendFieldsStructKey: make(map[string]int),
ForceSendFieldsStructKey: make(map[string]string),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why can't we keep precalculated index? the structs are static, so it should always resolve to the same index, right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good point, fixed on latest.

my_project:
project_id: test-pg-proj-$UNIQUE_NAME
display_name: "Test Postgres Project"
enable_pg_native_login: false

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we also add this to invariant tests?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Included this in the project resource config for the invariant tests.

… at runtime

Store the full reflect index path to each field's governing ForceSendFields
slice, keyed by JSON name, instead of a single struct index. The previous
map[string]int could only address a ForceSendFields one embed level down, so
the deep-embedding fix walked the value at conversion time to relocate them.
The location is static per type, so resolve it once in buildStructInfo and
FieldByIndex into it from both consumers.

Co-authored-by: Isaac
@pietern pietern temporarily deployed to test-trigger-is July 1, 2026 13:50 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is July 1, 2026 13:50 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants