Skip to content

fix: extend from IntEnum in RequestState to fix serialization#556

Merged
vdusek merged 6 commits into
masterfrom
fix-request-state-serialization
Oct 1, 2024
Merged

fix: extend from IntEnum in RequestState to fix serialization#556
vdusek merged 6 commits into
masterfrom
fix-request-state-serialization

Conversation

@vdusek

@vdusek vdusek commented Sep 30, 2024

Copy link
Copy Markdown
Collaborator

Closes: #551

@vdusek vdusek requested a review from janbuchar September 30, 2024 15:51
@github-actions github-actions Bot added this to the 99th sprint - Tooling team milestone Sep 30, 2024
@github-actions github-actions Bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Sep 30, 2024

@janbuchar janbuchar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. let's consider if we want to keep this compatible with crawlee-js, where we store numbers
  2. we could just use IntEnum or (str, Enum) - https://docs.pydantic.dev/dev/api/standard_library_types/#enum - I don't want to maintain duplicates of standard library code
@vdusek

vdusek commented Oct 1, 2024

Copy link
Copy Markdown
Collaborator Author

let's consider if we want to keep this compatible with crawlee-js, where we store numbers

I see a slight advantage in serializing it as something more readable, like {"state": "Unprocessed"}, instead of using just numbers, such as {"state": 1}.

we could just use IntEnum or (str, Enum) - https://docs.pydantic.dev/dev/api/standard_library_types/#enum - I don't want to maintain duplicates of standard library code

Oh, that's good, thanks.

@vdusek vdusek requested a review from janbuchar October 1, 2024 07:32
@github-actions github-actions Bot added the tested Temporary label used only programatically for some analytics. label Oct 1, 2024
@janbuchar

Copy link
Copy Markdown
Collaborator

let's consider if we want to keep this compatible with crawlee-js, where we store numbers

I see a slight advantage in serializing it as something more readable, like {"state": "Unprocessed"}, instead of using just numbers, such as {"state": 1}.

Yeah, I agree with that. I guess it boils down to whether we want the two formats to be compatible. Frankly, I don't see any advantage in that.

@vdusek

vdusek commented Oct 1, 2024

Copy link
Copy Markdown
Collaborator Author

Yeah, I agree with that. I guess it boils down to whether we want the two formats to be compatible. Frankly, I don't see any advantage in that.

Ok, let's use IntEnum then.

@janbuchar

Copy link
Copy Markdown
Collaborator

Yeah, I agree with that. I guess it boils down to whether we want the two formats to be compatible. Frankly, I don't see any advantage in that.

Ok, let's use IntEnum then.

Just so we're clear, we're both OK with string values, but we're keeping the integers for the sake of compatibility, right?

@janbuchar

Copy link
Copy Markdown
Collaborator

Could you also incorporate the test changes from cadlagtrader@02a5c6b?

@vdusek

vdusek commented Oct 1, 2024

Copy link
Copy Markdown
Collaborator Author

Could you also incorporate the test changes from cadlagtrader@02a5c6b?

I already implemented a new test covering this.

@janbuchar

Copy link
Copy Markdown
Collaborator

Could you also incorporate the test changes from cadlagtrader@02a5c6b?

I already implemented a new test covering this.

Yeah, I mean just to make it clear that the reported issue is fixed 😄

@B4nan

B4nan commented Oct 1, 2024

Copy link
Copy Markdown
Member

We can change the enum in JS version to store numbers, no problems with that. Generally speaking, numbers will be more performant and less memory sensitive than strings, but we already store a lot of stuff in the request object that I doubt this can be measurable.

@janbuchar

Copy link
Copy Markdown
Collaborator

We can change the enum in JS version to store numbers, no problems with that. Generally speaking, numbers will be more performant and less memory sensitive than strings, but we already store a lot of stuff in the request object that I doubt this can be measurable.

image

...apparently, that's already the case. The performance impact is most likely non-existent, yes.

@vdusek vdusek merged commit 6bf35ba into master Oct 1, 2024
@vdusek vdusek deleted the fix-request-state-serialization branch October 1, 2024 11:34
@vdusek vdusek changed the title fix: extend from StrEnum in RequestState to fix serialization Oct 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

3 participants