Skip to content

fix: add explicit err msgs for missing pckg extras during import#165

Merged
vdusek merged 8 commits into
masterfrom
err-msg-when-missing-extra
May 31, 2024
Merged

fix: add explicit err msgs for missing pckg extras during import#165
vdusek merged 8 commits into
masterfrom
err-msg-when-missing-extra

Conversation

@vdusek

@vdusek vdusek commented May 31, 2024

Copy link
Copy Markdown
Collaborator

Description

  • Add explicit error messages for missing package extras during import.
  • It applies to BrowserPool, BeautifulsoupCrawler, and PlaywrightCrawler.
  • Also until now the Playwright was mandatory for even BasicCrawler, fixing it.

Related issues

Testing

PlaywrightCrawler

ImportError                               Traceback (most recent call last)
Cell In[3], line 1
----> 1 from crawlee.playwright_crawler import PlaywrightCrawler

File ~/Projects/crawlee-py/src/crawlee/playwright_crawler/__init__.py:5
      3     from .types import PlaywrightCrawlingContext
      4 except ImportError as exc:
----> 5     raise ImportError(
      6         'To use this module, you need to install the "playwright" extra. Run "pip install crawlee[playwright]".',
      7     ) from exc

ImportError: To use this module, you need to install the "playwright" extra. Run "pip install crawlee[playwright]".

BeautifulsoupCrawler

ImportError                               Traceback (most recent call last)
Cell In[1], line 1
----> 1 from crawlee.beautifulsoup_crawler import BeautifulSoupCrawler

File ~/Projects/crawlee-py/src/crawlee/beautifulsoup_crawler/__init__.py:5
      3     from .types import BeautifulSoupCrawlingContext
      4 except ImportError as exc:
----> 5     raise ImportError(
      6         'To use this module, you need to install the "beautifulsoup" extra. Run "pip install crawlee[beautifulsoup]".',
      7     ) from exc

ImportError: To use this module, you need to install the "beautifulsoup" extra. Run "pip install crawlee[beautifulsoup]".

BrowserPool

ImportError                               Traceback (most recent call last)
Cell In[2], line 1
----> 1 from crawlee.browsers import BrowserPool

File ~/Projects/crawlee-py/src/crawlee/browsers/__init__.py:5
      3     from .playwright_browser_plugin import PlaywrightBrowserPlugin
      4 except ImportError as exc:
----> 5     raise ImportError(
      6         'To use this module, you need to install the "playwright" extra. Run "pip install crawlee[playwright]".',
      7     ) from exc

ImportError: To use this module, you need to install the "playwright" extra. Run "pip install crawlee[playwright]".

Checklist

  • Changes are described in the CHANGELOG.md
  • CI passed
@github-actions github-actions Bot added this to the 90th sprint - Tooling team milestone May 31, 2024
@github-actions github-actions Bot added the t-tooling Issues with this label are in the ownership of the tooling team. label May 31, 2024
@vdusek vdusek changed the title fix: better error message when missing package extra May 31, 2024
@vdusek vdusek marked this pull request as ready for review May 31, 2024 10:36
@vdusek vdusek requested a review from janbuchar May 31, 2024 10:39
Comment thread src/crawlee/beautifulsoup_crawler/__init__.py Outdated
Comment thread src/crawlee/basic_crawler/basic_crawler.py
@vdusek vdusek requested a review from janbuchar May 31, 2024 12:07
@vdusek vdusek requested review from janbuchar and removed request for janbuchar May 31, 2024 14:05
Comment thread src/crawlee/browsers/browser_pool.py Outdated
@vdusek vdusek requested a review from janbuchar May 31, 2024 14:52
@vdusek vdusek merged commit 200ebfa into master May 31, 2024
@vdusek vdusek deleted the err-msg-when-missing-extra branch May 31, 2024 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

2 participants