All Projects → slow-but-steady → yt-videos-list

slow-but-steady / yt-videos-list

Licence: Apache-2.0 license
Create and **automatically** update a list of all videos on a YouTube channel (in txt/csv/md form) via YouTube bot with end-to-end web scraping - no API tokens required. Multi-threaded support for YouTube videos list updates.

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to yt-videos-list

scrapetube
Get all videos from a youtube channel, get all videos from a playlist, get all videos that match a search
Stars: ✭ 120 (+87.5%)
Mutual labels:  scraper, youtube-api, youtube-playlist, youtube-channel
easyYoutubeDL
A manager for automating download lists based on youtube-dl
Stars: ✭ 21 (-67.19%)
Mutual labels:  youtube-dl, youtube-downloader, youtube-playlist, youtube-channel
pyderman
Install Selenium-compatible Chrome/Firefox/Opera/PhantomJS/Edge webdrivers automatically.
Stars: ✭ 24 (-62.5%)
Mutual labels:  selenium, chromedriver, geckodriver, operadriver
Fast Youtube To Mp3 Converter Api
Very Fast YouTube to MP3 & MP4 Converter API
Stars: ✭ 69 (+7.81%)
Mutual labels:  youtube-api, youtube-dl, youtube-downloader
node-youtube-music
Unofficial YouTube Music API for Node.js
Stars: ✭ 34 (-46.87%)
Mutual labels:  youtube-api, youtube-playlist, youtube-channel
ytqck.github.io
YouTube quick ⚡ Search and Download Music for Free.
Stars: ✭ 18 (-71.87%)
Mutual labels:  youtube-api, youtube-dl, youtube-downloader
Sillynium
Automate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements
Stars: ✭ 100 (+56.25%)
Mutual labels:  scraper, selenium, chromedriver
android-youtube-dl
The android library that wraps Python 2.7 and youtube-dl python scripts
Stars: ✭ 35 (-45.31%)
Mutual labels:  youtube-api, youtube-dl, youtube-downloader
kick-off-web-scraping-python-selenium-beautifulsoup
A tutorial-based introduction to web scraping with Python.
Stars: ✭ 18 (-71.87%)
Mutual labels:  scraper, csv, selenium
Zillow
Zillow Scraper for Python using Selenium
Stars: ✭ 141 (+120.31%)
Mutual labels:  scraper, selenium, chromedriver
Laravel-Youtube-API
A way to add an API to your Laravel app that converts youtube video's to mp3/mp4 and returns a download/stream link and information.
Stars: ✭ 19 (-70.31%)
Mutual labels:  youtube-api, youtube-dl, youtube-downloader
TikTok
Download public videos on TikTok using Python with Selenium
Stars: ✭ 37 (-42.19%)
Mutual labels:  scraper, selenium, chromedriver
Webdrivermanager
WebDriverManager (Copyright © 2015-2021) is a project created and maintained by Boni Garcia and licensed under the terms of the Apache 2.0 License.
Stars: ✭ 1,808 (+2725%)
Mutual labels:  selenium, chromedriver, geckodriver
Youtub.ly Android
An android app to download 📹 videos and songs from youtube to phone internal storage
Stars: ✭ 17 (-73.44%)
Mutual labels:  youtube-api, youtube-dl, youtube-downloader
pymusicdl
Download songs based on song name/ YouTube playlist/ Spotify playlist /album
Stars: ✭ 41 (-35.94%)
Mutual labels:  youtube-dl, youtube-downloader, youtube-playlist
jest-selenium
This project shows how to drive your selenium tests with Jest.
Stars: ✭ 22 (-65.62%)
Mutual labels:  selenium, chromedriver, geckodriver
youtubly-android
An android app to download 📹 videos and 🎶 songs from youtube to phone internal storage. In a nutshell NewPipe for just audio.
Stars: ✭ 24 (-62.5%)
Mutual labels:  youtube-api, youtube-dl, youtube-downloader
youtube-json-server
Your personal Youtube API server to get Youtube API responses without needing credentials.
Stars: ✭ 32 (-50%)
Mutual labels:  youtube-api, youtube-playlist
youtube-playlist
❄️ Extract links, ids, and names from a youtube playlist
Stars: ✭ 73 (+14.06%)
Mutual labels:  scraper, youtube-api
youtube-dl-batch
Simple batch files for simplifying basic usage of https://github.com/rg3/youtube-dl Windows .exe releases
Stars: ✭ 55 (-14.06%)
Mutual labels:  youtube-dl, youtube-downloader

General Overview

See the releases page to see new additions/modifications for each release!

See this comparison page to see new additions/modifications that will be available in the NEXT release!

See sister YouTube-Channels repository for a list of interesting channels!
  • The YouTube-Channels sister repository is a separate repository that uses this package to create a list of videos uploaded by every channel supported by the repository.
  • The sister repository will update the lists of videos once a week.
  • NOTE: In order to minimize the size of the sister repo, the repo contains the list of videos in ONLY the csv format, and not in txt or md format.

GitHub stars GitHub forks

GitHub license PyPI version
GitHub repo size GitHub code size in bytes Lines of code
GitHub Open Issues GitHub closed issues GitHub pull requests GitHub closed pull requests

GitHub most recent (Pre-)Release Date
GitHub release (latest SemVer including pre-releases) version
GitHub commit activity/week GitHub commit activity/month GitHub commit activity/year
GitHub last commit GitHub last commit to main (branch) GitHub commits since latest release (by SemVer including pre-releases)

This package provides a quick, simple way to create a list of all videos posted to any YouTube channel by providing just the URL to that channel's videos. Follow the link to the API in the language you want to work in from the API guides section below.

This program relies on Selenium so it has built-in dependecy setup for all Selenium webdrivers, but you can also explicitly use submodules of this package to download Selenium webdriver dependencies for your own projects. This makes it much easier to work with Selenium even if you update your browser constantly! See the Explicitly downloading all Selenium dependencies for directions.

If you already scraped a channel and the channel uploaded a new video, simply rerun this program on that channel and this package updates your files to include the newer video(s)!

API guides

python

Other Information

Dependencies
  • The first time you run this package the automated downloader should install everything you need, but in case it doesn't, refer to the link below and/or file an issue here.
    • Manual Selenium downloads here
  • The Selenium drivers are all pretty similar but differ in subtle ways, so play around with them and see what's different :)

NOTE that you also need the corresponding browser installed to properly run the selenium driver.

Interesting information about YouTube

There are two types of YouTube channels: one type is a user channel and the other is a channel channel.

Future Features

Main Features

  • take url and scrape the video name and url for every video for that user
  • create txt, csv, md files

Additional Features

  • enable custom name for output file
  • update previously created file with new videos uploaded since file creation
    • put all urls from file in a (hash) set and check to see if any urls on page (next time program runs on same page) are missing from the (hash) set
  • create web interface
    • input box for channel url
    • generate static HTML page with video URLs for output
Technical Specifications

This python3 package supports multi-platform, mutli-driver use and is currently under active development.

Currently supported operating systems include MacOS, Linux32, Linux64, Windows32, and Windows64. Currently supported drivers include Firefox, Opera, Safari (MacOS), Chrome, Brave, and Edge (Windows).

This package provides built-in support for common errors and exceptions, along with helpful hints, including

  • checking to see if the file to be created already exists
  • checking to see if the user explicitly specified a driver to use, and in case they didn't specify a driver, running the program using Firefox as default and showing the user the available driver options in the terminal output
    • explicitly specifying driver using
      • ListCreator(driver='firefox')
      • ListCreator(driver='opera')
      • ListCreator(driver='safari')
      • ListCreator(driver='chrome')
      • ListCreator(driver='brave')
      • ListCreator(driver='edge')
  • running the program in headless mode if using the geckodriver (Firefox) or chromedriver
    • ListCreator(headless=True)
  • checking to see if the user has the correct Selenium dependency installed
    • installs and shows the user the commands they can run to install the correct dependency if the user has an incorrect dependency (or hasn't downloaded the dependency at all)
      • program checks the operating system of the user's machine and returns a curl command piped into a tar command to download the correct dependecy into a directory the program can access without having to add the executable to PATH manually
      • user still needs to pick the correct command to run by following the directions (if autoamted download doesn't complete)
  • checking to see if the result of the scraping returns anything, and prompts the user to verify the url argument if nothing is found

Overview of package structure

__init__.py

__main__.py

script.py

execute.py

program.py

file submodule

download submodule

notifications.py

Stargazers Over Time

Stargazers over time

Usage Statistics

PyPI - Daily Downloads PyPI - Weekly Downloads PyPI - Monthly Downloads
PePY Weekly Downloads PePY Monthly Downloads PePY Total Downloads
GitHub release (latest by SemVer including pre-releases) downloads

If you found this interesting or useful, please consider starring this repo so other people can more easily find and use this. Thanks!

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].