web-poet

web-poet is a Python 3.7+ implementation of the page object pattern for web scraping. It enables writing portable, reusable web data extraction code.

See the documentation.

Developing

Setup your local Python environment via:

1. pip install -r requirements-dev.txt
2. pre-commit install

Now everytime you perform a git commit, these tools will run against the staged files:

• black
• isort
• flake8

You can also directly invoke pre-commit run --all-files or tox -e linters to run them without performing a commit.