All Projects → abbeyokgo → Tumblr_crawler

abbeyokgo / Tumblr_crawler

Licence: mit
tumblr解析网站

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Tumblr crawler

Annie
👾 Fast and simple video download library and CLI tool written in Go
Stars: ✭ 16,369 (+19621.69%)
Mutual labels:  tumblr, crawler
Media Scraper
Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok
Stars: ✭ 206 (+148.19%)
Mutual labels:  tumblr, crawler
Tumblthree
A Tumblr Blog Backup Application
Stars: ✭ 923 (+1012.05%)
Mutual labels:  tumblr, crawler
Tumblthree
A Tumblr Backup Application
Stars: ✭ 211 (+154.22%)
Mutual labels:  tumblr, crawler
TumblTwo
TumblTwo, an Improved Fork of TumblOne, a Tumblr Downloader.
Stars: ✭ 57 (-31.33%)
Mutual labels:  crawler, tumblr
Tumblr crawler
This is a Multi-thread crawler for Tumblr.
Stars: ✭ 258 (+210.84%)
Mutual labels:  tumblr, crawler
Tumblr Crawler
Easily download all the photos/videos from tumblr blogs. 下载指定的 Tumblr 博客中的图片,视频
Stars: ✭ 1,118 (+1246.99%)
Mutual labels:  tumblr, crawler
Jd Autobuy
Python爬虫,京东自动登录,在线抢购商品
Stars: ✭ 1,174 (+1314.46%)
Mutual labels:  crawler
Poopak
POOPAK - TOR Hidden Service Crawler
Stars: ✭ 78 (-6.02%)
Mutual labels:  crawler
Scrapy Examples
Some scrapy and web.py exmaples
Stars: ✭ 71 (-14.46%)
Mutual labels:  crawler
Arachnid
Powerful web scraping framework for Crystal
Stars: ✭ 68 (-18.07%)
Mutual labels:  crawler
Goscraper
Golang pkg to quickly return a preview of a webpage (title/description/images)
Stars: ✭ 72 (-13.25%)
Mutual labels:  crawler
Puppeteer Walker
a puppeteer walker 🕷 🕸
Stars: ✭ 78 (-6.02%)
Mutual labels:  crawler
Skraper
Kotlin/Java library and cli tool for scraping posts and media from various sources with neither authorization nor full page rendering (Facebook, Instagram, Twitter, Youtube, Tiktok, Telegram, Twitch, Reddit, 9GAG, Pinterest, Flickr, Tumblr, IFunny, VK, Pikabu)
Stars: ✭ 72 (-13.25%)
Mutual labels:  tumblr
Tumblr backup
A Python script for saving your Tumblr blog to your hard drive as HTML or CSV.
Stars: ✭ 80 (-3.61%)
Mutual labels:  tumblr
Spider
python crawler spider
Stars: ✭ 70 (-15.66%)
Mutual labels:  crawler
Acm Statistics
An online tool (crawler) to analyze users performance in online judges (coding competition websites). Supported OJ: POJ, HDU, ZOJ, HYSBZ, CodeForces, UVA, ICPC Live Archive, FZU, SPOJ, Timus (URAL), LeetCode_CN, CSU, LibreOJ, 洛谷, 牛客OJ, Lutece (UESTC), AtCoder, AIZU, CodeChef, El Judge, BNUOJ, Codewars, UOJ, NBUT, 51Nod, DMOJ, VJudge
Stars: ✭ 83 (+0%)
Mutual labels:  crawler
Work crawler
Download comics novels 小说漫画下载工具 小説漫画のダウンローダ 小說漫畫下載:腾讯漫画 大角虫漫画 有妖气 知音漫客 咪咕 SF漫画 哦漫画 看漫画 漫画柜 汗汗酷漫 動漫伊甸園 快看漫画 微博动漫 733动漫网 大古漫画网 漫画DB 無限動漫 動漫狂 卡推漫画 动漫之家 动漫屋 古风漫画网 36漫画网 亲亲漫画网 乙女漫画 comico webtoons 咚漫 ニコニコ静画 ComicWalker ヤングエースUP モアイ pixivコミック サイコミ;アルファポリス カクヨム ハーメルン 小説家になろう 起点中文网 八一中文网 顶点小说 落霞小说网 努努书坊 笔趣阁→epub.
Stars: ✭ 1,224 (+1374.7%)
Mutual labels:  crawler
Anticrawlersolution
It covers the blockade principle of most anti-climbing strategies and corresponding solutions.👽👽👽👽(涵盖了大部分的反爬策略的封锁原理以及对应的解决方案。)
Stars: ✭ 77 (-7.23%)
Mutual labels:  crawler
Tumblr Liked Photos Export
Export the photos from your liked posts in tumblr
Stars: ✭ 74 (-10.84%)
Mutual labels:  tumblr

升级版:https://github.com/tangrela/ojbk_jiexi


Tumblr解析网站搭建教程

  1. 首先安装Python。linux自带了python,windows请自行下载python。推荐Centos7/Python2.7
  2. 这时候,pip应该可以用了。如果不行,linux请按下面的命令安装pip: wget https://bootstrap.pypa.io/get-pip.py && python get-pip.py
  3. 安装依赖库:pip install -r requirement.txt
  4. 创建数据库:python rebuildDB.py
  5. 运行:gunicorn -w4 -b 0.0.0.0:5000 run:app

然后访问 ip:5000 试试 如果不能访问,看看防火墙是否开了5000端口?


以上都是基本的安装。

如果你需要使用MySQL

修改config.py:注释第六行 --> 第五行开头#去掉,修改userpasswddatabase

配置自启动

  1. 修改supervisord.conf,将directory修改为脚本根目录
  2. echo "supervisord -c 网站根目录/supervisord.conf" >> /etc/rc.d/rc.local
  3. chmod +x /etc/rc.d/rc.local

配置nginx

修改nginx配置文件,添加server

server {
        listen       80;
        server_name t.v4s0.us; #域名
        charset utf-8;

        access_log  /www/wwwlogs/t.v4s0.us.log;

        location / {
        proxy_pass http://127.0.0.1:5000;
        proxy_redirect off;
        proxy_set_header Host $host:80;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
        location /(images|javascript|js|css|flash|media|static)/ {
                root /root/tumblr_clawer/app/static; #目录修改好
                expires 1d;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
   }
   
  }

其他需求请加qq群


示例网站:http://t.3kk.me

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].