amoudgl / Short Jokes Dataset
Licence: gpl-2.0
Python scripts for building 'Short Jokes' dataset, featured on Kaggle
Programming Languages
python 139335 projects - #7 most used programming language
Projects that are alternatives of or similar to Short Jokes Dataset
Trump LiesTutorial: Web scraping in Python with Beautiful Soup
Stars: ✭ 201 (-6.51%)
Mutual labels: dataset, web-scraping
CharlatanCreate fake data in R
Stars: ✭ 209 (-2.79%)
Mutual labels: dataset
Bet On SibylMachine Learning Model for Sport Predictions (Football, Basketball, Baseball, Hockey, Soccer & Tennis)
Stars: ✭ 190 (-11.63%)
Mutual labels: web-scraping
Tech.ml.datasetA Clojure high performance data processing system
Stars: ✭ 205 (-4.65%)
Mutual labels: dataset
HdltexHDLTex: Hierarchical Deep Learning for Text Classification
Stars: ✭ 191 (-11.16%)
Mutual labels: dataset
Covid19zaCoronavirus COVID-19 (2019-nCoV) Data Repository and Dashboard for South Africa
Stars: ✭ 208 (-3.26%)
Mutual labels: dataset
Fifa18 All Player StatisticsA complete catalog of all the players in Fifa 18 and their complete statistics.
Stars: ✭ 185 (-13.95%)
Mutual labels: dataset
Pynasa Stars: ✭ 212 (-1.4%)
Mutual labels: dataset
R Web Scraping Cheat SheetGuide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.
Stars: ✭ 207 (-3.72%)
Mutual labels: web-scraping
Semantic Segmentation SuiteSemantic Segmentation Suite in TensorFlow. Implement, train, and test new Semantic Segmentation models easily!
Stars: ✭ 2,395 (+1013.95%)
Mutual labels: dataset
Awesome Json DatasetsA curated list of awesome JSON datasets that don't require authentication.
Stars: ✭ 2,421 (+1026.05%)
Mutual labels: dataset
Data Setstate driven all in one data process for data visualization
Stars: ✭ 191 (-11.16%)
Mutual labels: dataset
Dynamic Training BenchSimplify the training and tuning of Tensorflow models
Stars: ✭ 210 (-2.33%)
Mutual labels: dataset
Dataset loadersA collection of dataset loaders
Stars: ✭ 187 (-13.02%)
Mutual labels: dataset
Split Folders🗂 Split folders with files (i.e. images) into training, validation and test (dataset) folders
Stars: ✭ 203 (-5.58%)
Mutual labels: dataset
Ava downloader⏬ Download AVA dataset (A Large-Scale Database for Aesthetic Visual Analysis)
Stars: ✭ 214 (-0.47%)
Mutual labels: dataset
OmnianomalyKDD 2019: Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network
Stars: ✭ 208 (-3.26%)
Mutual labels: dataset
Mini Imagenet ToolsTools for generating mini-ImageNet dataset and processing batches
Stars: ✭ 209 (-2.79%)
Mutual labels: dataset
short-jokes-dataset
This repository contains all the python scripts used to build Short Jokes dataset. The dataset contains 231,657 short jokes scraped from various websites.
All the web scraper scripts are present in the /scripts/scrapers/
folder. These scripts are written for specific websites (website link mentioned in the header of each file) and they generate csv files of jokes in /data/
folder with the fixed format: ID, Joke
.
Scrapers were written only for those websites which allowed scraping and there were no CAPTCHA or any blocking javascripts for information gathering bots.
Jokes from subreddits /r/jokes
and /r/cleanjokes
are extracted using scripts/scrapers/subredditarchive.py
. The script uses PRAW, a Reddit API wrapper which uses timestamp search for finding posts between given timestamps on a particular subreddit. Following Reddit's API terms of use, request for data is made every 2 seconds. For each subreddit, posts are downloaded from the day of creation of subreddit to 31st January, 2017. The script generates a json file for each post in a separate folder. Json dumps for both the subreddits can be accessed from here (Uncompressed 2.3GB). Jokes from all the json files are extracted and written to a csv file using scripts/json_to_csv.py
.
scripts/merge_csvs.py
removes the duplicates from all the csv files and merges the jokes into a single csv to get the final dataset shortjokes.csv
.
Contributions
- If you are aware of any resource (preferably large) of good clean jokes, feel free to suggest or send a pull request with scraper script and csv file in the above format.
- Any other positive suggestions for the dataset are welcome.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at
[email protected].