Frisbee
Frisbee is a small utility to collect email addresses from search engines and other free-form text sources. Frisbee makes it simple to find email addresses posted on the web by taking user-fed input and translating it into an automated search query. Users can extend frisbee by adding modules for new search engines or other obscure data sources.
Quick Start
Install the library:
pip install frisbee or python setup.py install
Run a search
frisbee search -e bing -d bnpparibas.com -l 50 --greedy --save
Search in bulk
frisbee search -e bing -f domains -l 50 --save
Sample Code
This sample code shows some of the range of functionality within the module:
from frisbee import Frisbee
# Create an instance
frisbee = Frisbee(save=True)
# Describe your job
jobs = [{'engine': 'bing', 'modifier': 'site:github.com',
'domain': 'foo.bar', 'limit': 50}]
# Execute the jobs
frisbee.search(jobs)
# Get the results
results = frisbee.get_results()
Example Output
Below is an example job result:
[{
"engine": "bing",
"modifier": "site:github.com",
"domain": "blockade.io",
"limit": 50,
"results": {
"start_time": "2018-12-13 16:54:15",
"end_time": "2018-12-13 16:54:19",
"emails": [
"[email protected]"
],
"duration": "4",
"processed": 44
},
"project": "zealous_kirch"
}]
Features
- Ability to search for email addresses from search engine results
- Modular design that can be extended easily to include new sources
- Modifier options that can filter or target search query
- Limit option to reduce the number of results parsed
- Greedy option to learn from collected results and fuzzy to find related
- Save output describing job request and results
- Individual or bulk look-ups using the command line utility
Changelog
05-30-19
- Feature: Added a bulk option to the command line tool to ease usage
- Change: Replaced multiprocessing with concurrent.futures to simplify logic
- Change: Split logic of dynamic module loading and future work outside of the Frisbee class
- Change: Reverted back to the BS4 parsing versus raw text
- Change: Replaced the regular expression processing to be more efficient
- Change: Progressively save results as they come in to avoid any losses from a deadlock
- Change: Randomize the top-level directory to avoid conflicts
05-24-19
- Feature: Clean SERPs to remove files or other formats we can't inspect
- Change: Use text extraction instead of BS4 HTML parsing to get body of websites (ensures clean email extraction)
- Change: Increased logging and timeout parameters
12-20-18
- Feature: Added typing to the core code
- Feature: Added a fuzzy flag to find related domains
12-14-18
- Feature: Activated greedy option to save and output to screen
- Bugfix: Wrapped loading of HTML for cases where data is dirty
12-13-18
- Initial push!