Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Add More Options to Crawlee CLI for Crawler Creation #414

Open
siddiqkaithodu opened this issue Aug 9, 2024 · 0 comments · May be fixed by #538
Open

Feature Request: Add More Options to Crawlee CLI for Crawler Creation #414

siddiqkaithodu opened this issue Aug 9, 2024 · 0 comments · May be fixed by #538
Assignees
Labels
enhancement New feature or request. t-tooling Issues with this label are in the ownership of the tooling team.

Comments

@siddiqkaithodu
Copy link
Contributor

At the moment, the Crawlee CLI provides two options for creating a crawler in the CLI:

  1. Beautiful Soup
  2. Playwright

It would be great if we could add more alternatives, such as selecting the request module (like curl cffi,httpx) and parser modules (like parsel). This would allow the project to be bootstrapped with the chosen dependencies.

Additionally, it would be super helpful to include the ability to specify the domain name in the CLI we want to start with, similar to how it’s done in Scrapy.

@janbuchar janbuchar added enhancement New feature or request. t-tooling Issues with this label are in the ownership of the tooling team. labels Aug 9, 2024
@janbuchar janbuchar linked a pull request Sep 20, 2024 that will close this issue
@janbuchar janbuchar self-assigned this Sep 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request. t-tooling Issues with this label are in the ownership of the tooling team.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants