All Projects → VAllens → CrawlerSamples

VAllens / CrawlerSamples

Licence: MIT license
This is a Puppeteer+AngleSharp crawler console app samples, used C# 7.1 coding and dotnet core build.

Programming Languages

C#
18002 projects

Projects that are alternatives of or similar to CrawlerSamples

Taiko
A node.js library for testing modern web applications
Stars: ✭ 2,964 (+8133.33%)
Mutual labels:  headless, headless-browsers, headless-chrome, headless-chromium
puppeteer-lambda
Module for using Headless-Chrome by Puppeteer on AWS Lambda.
Stars: ✭ 117 (+225%)
Mutual labels:  headless, headless-chrome, puppeteer
headless-chrome-alpine
A Docker container running headless Chrome
Stars: ✭ 26 (-27.78%)
Mutual labels:  headless, headless-browsers, headless-chrome
Pdf Bot
🤖 A Node queue API for generating PDFs using headless Chrome. Comes with a CLI, S3 storage and webhooks for notifying subscribers about generated PDFs
Stars: ✭ 2,551 (+6986.11%)
Mutual labels:  headless, headless-chrome, headless-chromium
Url To Pdf Api
Web page PDF/PNG rendering done right. Self-hosted service for rendering receipts, invoices, or any content.
Stars: ✭ 6,544 (+18077.78%)
Mutual labels:  headless, headless-chrome, puppeteer
Mocha Chrome
☕️ Run Mocha tests using headless Google Chrome
Stars: ✭ 66 (+83.33%)
Mutual labels:  headless, headless-chrome
Puppeteer Walker
a puppeteer walker 🕷 🕸
Stars: ✭ 78 (+116.67%)
Mutual labels:  headless, puppeteer
Chrome Devtools Protocol
Chrome Devtools Protocol client for PHP
Stars: ✭ 112 (+211.11%)
Mutual labels:  headless, headless-chrome
Phpchrometopdf
A slim PHP wrapper around google-chrome to convert url to pdf or to take screenshots , easy to use and clean OOP interface
Stars: ✭ 127 (+252.78%)
Mutual labels:  headless, headless-chrome
Gowitness
🔍 gowitness - a golang, web screenshot utility using Chrome Headless
Stars: ✭ 996 (+2666.67%)
Mutual labels:  headless, headless-chrome
Marionette
Selenium alternative for Crystal. Browser manipulation without the Java overhead.
Stars: ✭ 119 (+230.56%)
Mutual labels:  headless, puppeteer
Google Meet Scheduler
😴 Attends classes for you.
Stars: ✭ 150 (+316.67%)
Mutual labels:  headless, puppeteer
Crawlergo
A powerful dynamic crawler for web vulnerability scanners
Stars: ✭ 1,088 (+2922.22%)
Mutual labels:  headless, headless-chrome
Daily Signin
网站签到脚本
Stars: ✭ 52 (+44.44%)
Mutual labels:  headless, puppeteer
macaca-puppeteer
Macaca puppeteer driver
Stars: ✭ 39 (+8.33%)
Mutual labels:  headless, puppeteer
Ferrum
Headless Chrome Ruby API
Stars: ✭ 1,009 (+2702.78%)
Mutual labels:  headless, headless-chrome
Wendigo
A proper monster for front-end automated testing
Stars: ✭ 121 (+236.11%)
Mutual labels:  headless, puppeteer
Ppspider
web spider built by puppeteer, support task-queue and task-scheduling by decorators,support nedb / mongodb, support data visualization; 基于puppeteer的web爬虫框架,提供灵活的任务队列管理调度方案,提供便捷的数据保存方案(nedb/mongodb),提供数据可视化和用户交互的实现方案
Stars: ✭ 237 (+558.33%)
Mutual labels:  headless, puppeteer
Serverless Puppeteer Layers
Serverless Framework + AWS Lambda Layers + Puppeteer = ❤️
Stars: ✭ 247 (+586.11%)
Mutual labels:  headless, puppeteer
Navalia
A bullet-proof, fast, and reliable headless browser API
Stars: ✭ 950 (+2538.89%)
Mutual labels:  headless, headless-chrome

Crawler Samples

This is a PuppeteerSharp + AngleSharp crawler console app samples, used C# 7.1 coding and dotnet core build.

Useful links

Usage

Open this sample project in the Visual Studio 2017 and press F5 key run it.

Or

Open this sample project in the Visual Studio 2017 and enter Build(B) --> Publish(H) in the top toolbar.

Go to build\win\netcoreapp2.0\x64 or build\win\netcoreapp2.0\x86 folder, run CrawlerSamples.ConsoleApp.exe here.

Result picture

Sample Snapshoot

Note

For the first time, it will download the download-Win64-536395.zip portable installation package from the network to the .local-chromium directory of the current program.

It will some time to wait here.

If a network problem occurs during downloading, which results in download failure, it throws an exception and exits the program.

Author

Author: Allen

License

MIT

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].