Skip to content

docs(storage): add zonal bucket pre-warmed writer pool sample - #5770

Open
NickGoog wants to merge 7 commits into
GoogleCloudPlatform:mainfrom
NickGoog:docs-zonal-bucket-writer-pool
Open

NickGoog wants to merge 7 commits into
GoogleCloudPlatform:mainfrom
NickGoog:docs-zonal-bucket-writer-pool

Conversation

@NickGoog

Copy link
Copy Markdown

Adds optimizeWriteLatencyPool sample (region tag: storage_optimize_write_latency_pool) demonstrating a pre-warmed pool of zonal bucket writers with FinalizeOnClose = false and Flush() to avoid object creation and finalization metadata overhead on the critical write path.

Verified with both mock unit tests and live integration testing against a Rapid (zonal) bucket in us-central1-a:

Running live Go test against bucket=, prefix=live_go_pool_1790087532323
Go 1. Init pool (3 writers): 1.075960737s
Go 2. Write+flush: 26.459652ms
Go 4. Read back: "0123456789", pool size after refill: 3
--- PASS: TestWriterPoolGoSample (1.40s)
PASS

Description

Fixes #

Ask me for bug ID

Checklist

  • I have followed Contributing Guidelines from CONTRIBUTING.MD
  • Tests pass: go test -v ./... (see Testing)
  • Code formatted: gofmt (see Formatting)
  • Vetting pass: go vet (see Formatting)
  • These samples need a new API enabled in testing projects to pass (N/A - uses existing Cloud Storage API)
  • These samples need a new/updated env vars in testing projects set to pass (N/A - uses existing testing env vars)
  • This sample adds a new sample directory, and I updated the CODEOWNERS file with the codeowners for this sample (N/A - added to existing storage/rapid directory)
  • This sample adds a new Product API, and I updated the Blunderbuss issue/PR auto-assigner with the codeowners for this sample (N/A - Cloud Storage is an existing product)
  • Please merge this PR for me once it is approved
@NickGoog
NickGoog requested review from a team as code owners September 24, 2026 14:16
@snippet-bot

snippet-bot Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Here is the summary of changes.

You are about to add 1 region tag.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment
@product-auto-label product-auto-label Bot added api: storage Issues related to the Cloud Storage API. samples Issues that are directly related to samples. labels Sep 24, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces optimizeWriteLatencyPool in storage/rapid/optimize_write_latency_pool.go to optimize write latency using a pre-warmed pool of writers for zonal buckets, along with corresponding unit tests in storage/rapid/rapid_test.go. The review feedback recommends handling and logging errors in the background goroutine when pre-warming a replacement writer fails, rather than silently ignoring them, to prevent debugging difficulties.

Comment thread storage/bidi/optimize_write_latency_pool.go Outdated
@NickGoog
NickGoog force-pushed the docs-zonal-bucket-writer-pool branch 2 times, most recently from 07184d5 to 0ca534e Compare September 24, 2026 15:55
Adds optimizeWriteLatencyPool sample (region tag: storage_optimize_write_latency_pool) demonstrating a pre-warmed pool of zonal bucket writers with FinalizeOnClose = false and Flush() to avoid object creation and finalization metadata overhead on the critical write path.

Verified with both mock unit tests and live integration testing against a Rapid (zonal) bucket in us-central1-a:

  Running live Go test against bucket=<zonal-bucket>, prefix=live_go_pool_1790087532323
  Go 1. Init pool (3 writers): 1.075960737s
  Go 2. Write+flush: 26.459652ms
  Go 4. Read back: "0123456789", pool size after refill: 3
  --- PASS: TestWriterPoolGoSample (1.40s)
  PASS
@NickGoog
NickGoog force-pushed the docs-zonal-bucket-writer-pool branch from f7750d4 to c8055a8 Compare September 25, 2026 18:42
Comment thread storage/bidi/optimize_write_latency_pool.go Outdated
Comment thread storage/bidi/optimize_write_latency_pool.go Outdated
Comment thread storage/bidi/optimize_write_latency_pool.go Outdated
Comment thread storage/bidi/optimize_write_latency_pool.go Outdated
- Use a buffered channel for the writer pool instead of a mutex-guarded slice.
- Take a writer with select and a timeout.
- Remove redundant FinalizeOnClose = false (it defaults to false).
- Clarify the TODO on the pre-warm Flush.
Comment thread storage/bidi/optimize_write_latency_pool.go Outdated
Comment thread storage/bidi/optimize_write_latency_pool.go Outdated
@tritone

tritone commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

@cpriti-os would you mind reviewing/approving this one?

Comment thread storage/bidi/optimize_write_latency_pool.go Outdated
Comment thread storage/bidi/optimize_write_latency_pool.go Outdated
Comment thread storage/bidi/optimize_write_latency_pool.go Outdated
Comment thread storage/bidi/optimize_write_latency_pool.go Outdated
@NickGoog
NickGoog requested a review from cpriti-os October 1, 2026 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: storage Issues related to the Cloud Storage API. samples Issues that are directly related to samples.

3 participants