All Projects → Evil0ctal → TikTokDownloader_PyWebIO

Evil0ctal / TikTokDownloader_PyWebIO

Licence: MIT license
🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音|TikTok数据爬取工具,支持API调用,在线批量解析及下载。

Programming Languages

python
139335 projects - #7 most used programming language
javascript
184084 projects - #8 most used programming language
CSS
56736 projects

Projects that are alternatives of or similar to TikTokDownloader PyWebIO

Spidr
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (-28.62%)
Mutual labels:  scraper, spider, web-scraping
fastAPI-aiohttp-example
How to use and test fastAPI with an aiohttp client
Stars: ✭ 69 (-92.49%)
Mutual labels:  aiohttp, asgi, fastapi
scraper
图片爬取下载工具,极速爬取下载 站酷https://www.zcool.com.cn/, CNU 视觉 http://www.cnu.cc/ 设计师/用户 上传的 图片/照片/插画。
Stars: ✭ 64 (-93.04%)
Mutual labels:  scraper, spider
Scrape Linkedin Selenium
`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
Stars: ✭ 239 (-73.99%)
Mutual labels:  scraper, web-scraping
lopez
Crawling and scraping the Web for fun and profit
Stars: ✭ 20 (-97.82%)
Mutual labels:  scraper, web-scraping
Goribot
[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (-79.33%)
Mutual labels:  scraper, spider
Querylist
🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
Stars: ✭ 2,392 (+160.28%)
Mutual labels:  scraper, spider
cashews
Cache with async power
Stars: ✭ 204 (-77.8%)
Mutual labels:  aiohttp, fastapi
Not Your Average Web Crawler
A web crawler (for bug hunting) that gathers more than you can imagine.
Stars: ✭ 107 (-88.36%)
Mutual labels:  scraper, spider
BookingScraper
🌎 🏨 Scrape Booking.com 🏨 🌎
Stars: ✭ 68 (-92.6%)
Mutual labels:  scraper, web-scraping
crawler-chrome-extensions
爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer
Stars: ✭ 53 (-94.23%)
Mutual labels:  scraper, spider
ByteDanceOpen
Bytedance(TikTok, Toutiao)third party open platform backend SDK, provides easy-to-use authorization, template management, code package management and other APIs.字节跳动(抖音, 头条)第三方开放平台小程序sdk, 提供了简单易用的授权,模版管理,代码包管理等api.
Stars: ✭ 67 (-92.71%)
Mutual labels:  douyin, tiktok
Linkedin Profile Scraper
🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (-81.39%)
Mutual labels:  scraper, spider
Phpscraper
PHP Scraper - an highly opinionated web-interface for PHP
Stars: ✭ 148 (-83.9%)
Mutual labels:  scraper, web-scraping
Colly
Elegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+1590.42%)
Mutual labels:  scraper, spider
Zillow
Zillow Scraper for Python using Selenium
Stars: ✭ 141 (-84.66%)
Mutual labels:  scraper, web-scraping
blinkist-m4a-downloader
Grabs all of the audio files from all of the Blinkist books
Stars: ✭ 100 (-89.12%)
Mutual labels:  scraper, spider
Sillynium
Automate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements
Stars: ✭ 100 (-89.12%)
Mutual labels:  scraper, web-scraping
Rod
A Devtools driver for web automation and scraping
Stars: ✭ 1,392 (+51.47%)
Mutual labels:  scraper, web-scraping
fastapi-project
FastAPI application without global variables(almost) =)
Stars: ✭ 26 (-97.17%)
Mutual labels:  asgi, fastapi

Douyin_TikTok_Download_API(抖音/TikTok API)

English | 简体中文

🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音|TikTok数据爬取工具,支持API调用,在线批量解析及下载。

GitHub license Release Version GitHub Star GitHub Fork GitHub issues GitHub closed issues GitHub Repo size

PyPI v PyPI wheel PyPI dm PyPI pyversions

API-V1 status API-V2 status

爱发电 Kofi Patreon

👻介绍

🚨如需使用私有服务器运行本项目,请参考部署方式[Docker部署, 手动部署]

本项目是基于 PyWebIOFastAPIAIOHTTP,快速异步的抖音/TikTok数据爬取工具,并通过Web端实现在线批量解析以及下载无水印视频或图集,数据爬取API,iOS快捷指令无水印下载等功能。你可以自己部署或改造本项目实现更多功能,也可以在你的项目中直接调用scraper.py或安装现有的pip包作为解析库轻松爬取数据等.....

一些简单的运用场景:

下载禁止下载的视频,进行数据分析,iOS无水印下载(搭配iOS自带的快捷指令APP 配合本项目API实现应用内下载或读取剪贴板下载)等.....

🖥公共站点: 我很脆弱...请勿压测(·•᷄ࡇ•᷅ )

API-V2: 支持输入Douyin|TikTok用户主页爬取该作者[主页视频数据(去水印链接, 已点赞视频列表(权限需为公开), 视频评论数据, 背景音乐视频列表数据, 等等...), 详细信息请查看V2文档, 服务器响应时间有时会变长, 使用时请将timeout值设高.

🍔Web APP: https://douyin.wtf/

🍟API-V1: https://api.douyin.wtf/docs

🌭API-V2: https://api-v2.douyin.wtf/docs

💾iOS Shortcut(快捷指令): Shortcut release

📦️桌面端下载器(仓库推荐):

⚗️技术栈

scraper.py:

  • 向[Douyin|TikTok]的API提交请求并取回数据,处理后返回字典(dict),支持异步。

web_api.py:

  • 获得请求参数并使用Scraper()类处理数据后以JSON形式返回,视频下载,配合iOS快捷指令实现快速调用,支持异步。

web_app.py:

  • web_api.py以及scraper.py制作的简易Web程序,将网页输入的值进行处理后使用Scraper()类处理并配合web_api.py的接口输出在网页上(类似前后端分离)

以上文件的参数大多可在config.ini中进行修改

💡项目文件结构

.
└── Douyin_TikTok_Download_API/
    ├── /static -> (PyWebIO static resources)
    ├── web_app.py -> (Web APP)
    ├── web_api.py -> (API)
    ├── scraper.py -> (Parsing library)
    ├── config.ini -> (configuration file)

功能:

  • 抖音(抖音海外版: TikTok)视频/图片解析
  • 网页端批量解析(支持抖音/TikTok混合提交)
  • 网页端解析结果页批量下载无水印视频(V3.0.0暂时移除,请自行部署V2.X版本)
  • API调用获取链接数据
  • 制作pip包方便快速导入你的项目
  • iOS快捷指令快速调用API实现应用内下载无水印视频/图集
  • 解析作者主页内所有视频(API-V2 支持抖音/TikTok)
  • 解析视频内所有评论信息(API-V2 支持抖音/TikTok)

🤦‍待办清单:

💡欢迎提出新的建议或将你的思路在issue中与我分享,或直接提交PR至Development分支 ♪(・ω・)ノ)

  • 编写一个桌面端的下载器实现本地批量下载
  • API-V2添加对hash_tag页面的数据爬取 #101
  • 对其他短视频平台添加支持,如:抖音火山版,快手,西瓜视频,哔哩哔哩

📦调用解析库:

💡PyPi:https://pypi.org/project/douyin-tiktok-scraper/

安装解析库:pip install douyin-tiktok-scraper

import asyncio
from douyin_tiktok_scraper.scraper import Scraper

api = Scraper()

async def hybrid_parsing(url: str) -> dict:
    # Hybrid parsing(Douyin/TikTok URL)
    result = await api.hybrid_parsing(url)
    print(f"The hybrid parsing result:\n {result}")
    return result

asyncio.run(hybrid_parsing(url=input("Paste Douyin/TikTok share URL here: ")))

🗺️支持的提交格式:

💡提示:包含但不仅限于以下例子,如果遇到链接解析失败请开启一个新 issue

  • 抖音分享口令 (APP内复制)
7.43 pda:/ 让你在几秒钟之内记住我  https://v.douyin.com/L5pbfdP/ 复制此链接,打开Dou音搜索,直接观看视频!
  • 抖音短网址 (APP内复制)
https://v.douyin.com/L4FJNR3/
  • 抖音正常网址 (网页版复制)
https://www.douyin.com/video/6914948781100338440
  • 抖音发现页网址 (APP复制)
https://www.douyin.com/discover?modal_id=7069543727328398622
  • TikTok短网址 (APP内复制)
https://www.tiktok.com/t/ZTR9nDNWq/
  • TikTok正常网址 (网页版复制)
https://www.tiktok.com/@evil0ctal/video/7156033831819037994
  • 抖音/TikTok批量网址(无需使用符合隔开)
https://v.douyin.com/L4NpDJ6/
https://www.douyin.com/video/7126745726494821640
2.84 nqe:/ 骑白马的也可以是公主%%百万转场变身https://v.douyin.com/L4FJNR3/ 复制此链接,打开Dou音搜索,直接观看视频!
https://www.tiktok.com/t/ZTR9nkkmL/
https://www.tiktok.com/t/ZTR9nDNWq/
https://www.tiktok.com/@evil0ctal/video/7156033831819037994

🛰️API文档

💡提示:也可以在web_api.py的代码注释中查看接口文档

API-V1文档: 本地:http://localhost:8000/docs 在线:https://api.douyin.wtf/docs

API-V2文档: 在线:https://api-v2.douyin.wtf/docs

API演示:

  • 爬取视频数据(TikTok或Douyin混合解析) https://api.douyin.wtf/api?url=[视频链接/Video URL]&minimal=false
  • 下载视频/图集(TikTok或Douyin混合解析) https://api.douyin.wtf/download?url=[视频链接/Video URL]&prefix=true&watermark=false
  • 替换域名下载视频/图集
[抖音]
原始链接:
https://www.douyin.com/video/7159502929156705567
替换域名:
https://api.douyin.wtf/video/7159502929156705567
# 返回无水印视频下载响应
[TikTok]
original link:
https://www.tiktok.com/@evil0ctal/video/7156033831819037994
Replace Domain:
https://api.douyin.wtf/@evil0ctal/video/7156033831819037994
# Return No Watermark Video Download Response

更多演示请查看文档内容......

💻部署(方式一 Linux)

💡提示:最好将本项目部署至美国地区的服务器,否则可能会出现奇怪的BUG。

  • 首先要去安全组开放8080(Web)和8000(API)端口。
  • 在宝塔面板应用商店内搜索进程守护或手动安装supervisord
[宝塔面板]
https://www.bt.cn/new/download.html
[aapanel]
https://www.aapanel.com/new/download.html
[Supervisor]
http://supervisord.org/installing.html
  • 配置项目config.ini文件
  • 安装依赖文件pip install -r requirements.txt
  • 设置supervisord守护进程
  • 启动命令:
[Web]
python3 web_app.py
[API]
python3 web_api.py
  • 程序入口:
[Web]
http://localhost:8080
[API]
http://localhost:8000

💽部署(方式二 Docker)

💡Docker Image repo: Docker Hub

  • 安装docker
curl -fsSL get.docker.com -o get-docker.sh&&sh get-docker.sh &&systemctl enable docker&&systemctl start docker
  • 留下config.int和docker-compose.yml文件即可
  • 运行命令,让容器在后台运行
docker compose up -d
  • 查看容器日志
docker logs -f douyin_tiktok_download_api
  • 删除容器
docker rm -f douyin_tiktok_download_api
  • 更新
docker compose pull && docker compose down && docker compose up -d

❤️ 贡献者

📸截图

API速度测试(对比官方API)

🔎点击展开截图

抖音官方API:

本项目API:

TikTok官方API:

本项目API:


项目界面

🔎点击展开截图

Web主界面:

Web main interface:


📜 Star历史

Star History Chart

MIT License

Start: 2021/11/06 GitHub: @Evil0ctal Contact: [email protected]

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].