chairco / 2017_PyConTW_Talk Licence: other
tw.pycon.org/2017/events/talk/314386410792550475/
Programming Languages javascript 184084 projects - #8 most used programming language
python 139335 projects - #7 most used programming language
Projects that are alternatives of or similar to 2017 PyConTW Talk sse-option-crawler SSE 50 index options crawler 上证50期权数据爬虫
Stars : ✭ 17 (-5.56%)
Mutual labels: crawler
tasuku ✅ タスク — The minimal task runner for Node.js
Stars : ✭ 1,488 (+8166.67%)
Mutual labels: task
Sharingan We will try to find your visible basic footprint from social media as much as possible - 😤 more sites is comming soon
Stars : ✭ 13 (-27.78%)
Mutual labels: crawler
domfind A Python DNS crawler to find identical domain names under different TLDs.
Stars : ✭ 22 (+22.22%)
Mutual labels: crawler
timer.cljs Scheduling async operations in Clojurescript
Stars : ✭ 22 (+22.22%)
Mutual labels: task
WeiboCrawler 无cookie版微博爬虫,可以连续爬取一个或多个新浪微博用户信息、用户微博及其微博评论转发。
Stars : ✭ 45 (+150%)
Mutual labels: crawler
vania A module which fairly distributes a list of arbitrary objects among a set of targets, considering weights.
Stars : ✭ 75 (+316.67%)
Mutual labels: task
lets CLI task runner for developers - a better alternative to make
Stars : ✭ 50 (+177.78%)
Mutual labels: task
medium-stat-box Practical pinned gist which show your latest medium status 📌
Stars : ✭ 29 (+61.11%)
Mutual labels: crawler
arachnod High performance crawler for Nodejs
Stars : ✭ 17 (-5.56%)
Mutual labels: crawler
BilibiliCrawler 🌀 crawl bilibili user info and video info for data analysis | BiliBili爬虫
Stars : ✭ 25 (+38.89%)
Mutual labels: crawler
flink-crawler Continuous scalable web crawler built on top of Flink and crawler-commons
Stars : ✭ 48 (+166.67%)
Mutual labels: crawler
spiderable-middleware 🤖 Prerendering for JavaScript powered websites. Great solution for PWAs (Progressive Web Apps), SPAs (Single Page Applications), and other websites based on top of front-end JavaScript frameworks
Stars : ✭ 29 (+61.11%)
Mutual labels: crawler
resc A task orchestrator using redis, written in rust
Stars : ✭ 27 (+50%)
Mutual labels: task
concurrent-tasks A simple task runner which will run all tasks till completion, while maintaining concurrency limits.
Stars : ✭ 27 (+50%)
Mutual labels: task
用 Django 建立一個可以設定爬蟲排程任務與監控的網站
範例環境設定
建立 log 資料夾和 .log 檔案
cd code/pycontw/
mkdir -p logs
cd logs
touch django.log ghostdriver.log project.log
建立環境變數 local.env, local.py
cp pycontw/settings/local_sample.env pycontw/settings/local.env
cp pycontw/settings/local_sample.py pycontw/settings/local.py
設定環境變數
在執行終端機模式執行下面指令取得 Secret_key:
$ python -c ' import random; import string; print("".join([random.SystemRandom().choice(string.digits + string.ascii_letters + string.punctuation) for i in range(100)]))'
vim local.env
將剛剛產生的 key 填入 SECRET_KEY= 不用加引號 ''
確認 requirments.txt 內套件都安裝,執行 migrate 就完成了。
因為 lazy_logger 最新版本 0.1.2 尚未上傳到 PyPi 所以請先到 lazy-logger 資料夾內 python setup.py install
安裝最新版本 lazy_logger 確認 lazy_logger 的版本是 0.1.2 萬一低於 0.1.2 版本會造成使用 @logger.patch 的 hook 回傳值是 None。
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at
[email protected] .