All Projects → lobstrio → Shadow Useragent

lobstrio / Shadow Useragent

Licence: mit
Pick the most common user-agents on the Internet 👻

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Shadow Useragent

bots-zoo
No description or website provided.
Stars: ✭ 59 (-59.86%)
Mutual labels:  scraping, useragent
Educative.io Downloader
📖 This tool is to download course from educative.io for offline usage. It uses your login credentials and download the course.
Stars: ✭ 139 (-5.44%)
Mutual labels:  scraping
Laravel Bank Statements
Laravel package to collect your bank statements history. Currently support for parsing statements history from BCA, Mandiri, BNI, and MUAMALAT e-banking websites.
Stars: ✭ 105 (-28.57%)
Mutual labels:  scraping
Browscap Java
A blazingly fast and memory efficient (thread-safe) Java client on top of the BrowsCap CSV source files.
Stars: ✭ 123 (-16.33%)
Mutual labels:  useragent
Scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
Stars: ✭ 42,343 (+28704.76%)
Mutual labels:  scraping
Torchbear
🔥🐻 The Speakeasy Scripting Engine Which Combines Speed, Safety, and Simplicity
Stars: ✭ 128 (-12.93%)
Mutual labels:  scraping
Languagepod101 Scraper
Python scraper for Language Pods such as Japanesepod101.com 👹 🗾 🍣 Compatible with Japanese, Chinese, French, German, Italian, Korean, Portuguese, Russian, Spanish and many more! ✨
Stars: ✭ 104 (-29.25%)
Mutual labels:  scraping
Fantasy Basketball
Scraping statistics, predicting NBA player performance with neural networks and boosting algorithms, and optimising lineups for Draft Kings with genetic algorithm. Capstone Project for Machine Learning Engineer Nanodegree by Udacity.
Stars: ✭ 146 (-0.68%)
Mutual labels:  scraping
Search Engine Google
🕷 Google client for SERPS
Stars: ✭ 138 (-6.12%)
Mutual labels:  scraping
Htmlsql
htmlSQL is a experimental PHP library which allows you to access HTML values by an SQL like syntax.
Stars: ✭ 120 (-18.37%)
Mutual labels:  scraping
Od Database
Distributed crawler, database and web frontend for public directories indexing
Stars: ✭ 121 (-17.69%)
Mutual labels:  scraping
Seleniumcrawler
An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
Stars: ✭ 117 (-20.41%)
Mutual labels:  scraping
Mobile Device Detect
Detect mobile device and its type.
Stars: ✭ 132 (-10.2%)
Mutual labels:  useragent
Webmagic
A scalable web crawler framework for Java.
Stars: ✭ 10,186 (+6829.25%)
Mutual labels:  scraping
Embed
Get info from any web service or page
Stars: ✭ 1,808 (+1129.93%)
Mutual labels:  scraping
D4n155
OWASP D4N155 - Intelligent and dynamic wordlist using OSINT
Stars: ✭ 105 (-28.57%)
Mutual labels:  scraping
Awesome Puppeteer
A curated list of awesome puppeteer resources.
Stars: ✭ 1,728 (+1075.51%)
Mutual labels:  scraping
Scan For Webcams
scan for webcams on the internet
Stars: ✭ 128 (-12.93%)
Mutual labels:  scraping
Phpscraper
PHP Scraper - an highly opinionated web-interface for PHP
Stars: ✭ 148 (+0.68%)
Mutual labels:  scraping
Sqrape
Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)
Stars: ✭ 144 (-2.04%)
Mutual labels:  scraping

shadow-useragent

shadow-useragent gives you access to the most commonly used UserAgents on the Internet, safe from outdated data.

Behold, the power of UserAgent:

>>> import shadow_useragent
>>> ua = shadow_useragent.ShadowUserAgent()
>>> ua.percent(0.05)
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.1.1 Safari/605.1.15'
>>> ua.most_common
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36'

When you use our library, we register your User Agent, and update the frequency of use. The percent of use is thus dynamically updated, and never outdated. No more unused header that reveals your true identity, you are protected by the group.

Besides, you can rely on traditional features, like picking an header from outstanding IE browser family.

Feature Support

shadow-useragent is the only safe, and updated user-agent package.

  • UserAgent Percentage of Use
  • Day-to-day Update
  • Exhaustive UserAgents Family
  • Elegant @property Methods
  • Community-Based Package

shadow-useragent supports Python 2.7 & 3.4–3.7, and runs great on PyPy.

Installation

To install shadow-useragent, simply use pipenv (or pip, of course):

$ pip install shadow-useragent
👻  

Never disappointed.

Documentation

Documentation is not available for now. Unlimited delivery coming soon.

Usage


import shadow_useragent 
ua = shadow_useragent.ShadowUserAgent()

# Access user-agent per Percentage of Use
ua = ua.percent(0.03) 
ua = ua.most_common

# Access user-agent per Browser Family
ua = ua.firefox 
ua = ua.chrome 
ua = ua.safari 
ua = ua.edge 
ua = ua.ie
ua = ua.android
ua = ua.ipad

# Random Access ;)
ua = ua.random
ua = ua.random_nomobile # Mobile-UA excluded

# If you want an Exhaustive list, you can play with
uas = ua.get_uas()
uas = ua.get_sorted_uas()  # Sorted List per Percentage of Use

>>> uas[0]
{
  'id':1,
  'scraping_time':'2019-07-31T17:05:15Z',
  'percent':9.6,
  'useragent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36',
  'system':'Chrome 75.0 Win10',
  'browser_family':'Chrome',
  'browser_version_string':'75.0.3770',
  'os_family':'Windows',
  'os_version_string':'10',
  'device_family':'Other',
  'device_brand':None,
  'device_model':None
}

Notes

Contact for any issues or request : [email protected]

shadow-useragent stores collected UAs in the folder of your pip packages: site-packages/shadow_useragent/data.

The UserAgents' list is automatically updated with a shadow call (mm) to the shadow-useragent API every 24 hours. If you want to force the update :

from shadow_useragent import ShadowUserAgent
shadow_useragent = ShadowUserAgent()
shadow_useragent.force_update()

Run on Repl.it

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].