Skip to content

fix: Graceful exit in BasicCrawler.run#224

Merged
janbuchar merged 3 commits into
masterfrom
graceful-exit
Jun 25, 2024
Merged

fix: Graceful exit in BasicCrawler.run#224
janbuchar merged 3 commits into
masterfrom
graceful-exit

Conversation

@janbuchar

Copy link
Copy Markdown
Collaborator

Before this, we got a traceback when CTRL+C was pressed during a crawl. Now we get this:

... history ...
INFO:httpx:HTTP Request: GET https://crawlee.dev/docs/3.7/quick-start "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: GET https://crawlee.dev/docs/3.6/quick-start "HTTP/1.1 200 OK"
^CINFO:crawlee.basic_crawler.basic_crawler:Pausing... Press CTRL+C again to force exit.
INFO:crawlee.autoscaling.autoscaled_pool:Waiting for remaining tasks to finish
INFO:crawlee.basic_crawler.basic_crawler:The crawl was interrupted. To resume, do: CRAWLEE_PURGE_ON_START=0 python some_crawling_script.py

@janbuchar janbuchar added the t-tooling Issues with this label are in the ownership of the tooling team. label Jun 24, 2024
@janbuchar janbuchar requested a review from vdusek June 24, 2024 16:28
@github-actions github-actions Bot added this to the 92nd sprint - Tooling team milestone Jun 24, 2024

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just tests do not pass

@janbuchar

Copy link
Copy Markdown
Collaborator Author

To be continued with #228

@janbuchar janbuchar merged commit 337286e into master Jun 25, 2024
@janbuchar janbuchar deleted the graceful-exit branch June 25, 2024 09:53
@janbuchar janbuchar mentioned this pull request Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

2 participants