tpeng / Googlesearch
Scrape the Google search result with Scrapy.
Stars: ✭ 83
Programming Languages
julia
2034 projects
Googlesearch
Scrape the google advanced search result with scrapy bootstraped from given queries. This spider can be used to collect the HTML pages to prepare for a corpus.
Usage
scrapy crawl googlesearch -a queries='xxx' -a region='xxx' replace the 'xxx' to the keywords you want search with Google
and the region (e.g. ie for Ireland) you wish to limit to.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].