Skip to content

fix: validate storage name and alias values#1950

Merged
vdusek merged 3 commits into
masterfrom
fix/validate-storage-name-alias
Jun 9, 2026
Merged

fix: validate storage name and alias values#1950
vdusek merged 3 commits into
masterfrom
fix/validate-storage-name-alias

Conversation

@vdusek

@vdusek vdusek commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

Storage aliases were not validated the same way storage names are. This change:

  • Adds validation for storage aliases (previously unvalidated), allowing the same simple identifier characters used for names plus the reserved __default__ alias.
  • Ensures a storage name or alias always maps to a single subdirectory inside the configured storage directory, both via the high-level API and when the file-system storage clients are used directly.

Includes unit tests for the new validation and for the file-system clients.

Add validation for storage aliases (previously unvalidated) and ensure storage names and aliases map to
a single subdirectory inside the storage directory, including when the file-system storage clients are
used directly. Add tests.
@vdusek vdusek added t-tooling Issues with this label are in the ownership of the tooling team. adhoc Ad-hoc unplanned task added during the sprint. labels Jun 5, 2026
@vdusek vdusek self-assigned this Jun 5, 2026
@github-actions github-actions Bot added this to the 142nd sprint - Tooling team milestone Jun 5, 2026
@github-actions github-actions Bot added the tested Temporary label used only programatically for some analytics. label Jun 5, 2026
@codecov

codecov Bot commented Jun 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.03%. Comparing base (76927d7) to head (7870a85).
⚠️ Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
src/crawlee/storages/_utils.py 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1950      +/-   ##
==========================================
+ Coverage   92.98%   93.03%   +0.04%     
==========================================
  Files         167      167              
  Lines       11712    11722      +10     
==========================================
+ Hits        10890    10905      +15     
+ Misses        822      817       -5     
Flag Coverage Δ
unit 93.03% <95.00%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
@vdusek vdusek marked this pull request as ready for review June 5, 2026 07:44
@vdusek vdusek requested review from Pijukatel and janbuchar and removed request for janbuchar June 5, 2026 07:45

@Pijukatel Pijukatel left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing confuses me.

This alias: with/slash

  • It is not a valid alias at the high-level API. (rejected by validate_storage_alias)
  • It is a valid alias when using the storage client directly. (approved by validate_subdirectory)

Is there a specific reason for it?

@vdusek

vdusek commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator Author

One thing confuses me.

This alias: with/slash

  • It is not a valid alias at the high-level API. (rejected by validate_storage_alias)
  • It is a valid alias when using the storage client directly. (approved by validate_subdirectory)

Is there a specific reason for it?

Oh, good catch, it should be rejected by both of course, I'll fix it.

@vdusek vdusek requested a review from Pijukatel June 8, 2026 10:20
@vdusek vdusek merged commit 0cac092 into master Jun 9, 2026
33 checks passed
@vdusek vdusek deleted the fix/validate-storage-name-alias branch June 9, 2026 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

3 participants