freeproxy
Get http proxies from some free proxy sites.
| Site | Accessibility |
|---|---|
| pachong.org | China Only |
| cn-proxy.com | Exclude China |
| txt.proxyspy.net | Exclude China |
| free-proxy-list | Exclude China |
| www.gatherproxy.com | Exclude China |
| www.hide-my-ip.com | Exclude China |
| www.getproxy.jp | Exclude China Account Required |
| www.xicidaili.com | World Wide |
| www.cybersyndrome.net | World Wide |
Installing
$ pip install freeproxyRequirements
Usage
Collect proxies
Run freeproxy from Terminal:
usage: freeproxy [-h] [-l] [-t] URL
Get http proxies from some free proxy sites
positional arguments:
URL The url for testing proxies. Like "https://www.google.com"
optional arguments:
-h, --help show this help message and exit
-l Logging debug messages to a file
-t Test availability of proxies stored in dbFetch latest proxies from above sites and test them using URL.
$ freeproxy "https://www.google.com"Testing results are stored in ~/.freeproxy/proxy.db (sqlite). Below is the peewee model:
class Proxy(Model):
"""
Database Model
"""
proxy = CharField(primary_key=True) # "ip:port"
check_time = DateTimeField(null=True) # time of testing
response_time = FloatField(null=True) # response time(seconds)
status_code = IntegerField(null=True) # status codeIf the testing URL was unreachable:
status_codewould beNoneresponse_timewould beNone
Test proxies
You can use the default testing method(Test all the proxies stored in database) or define your own strategy.
freeproxy -t URL uses the default one.
Read proxies
The default method freeproxy.read_proxies() reads all the proxies from database whose status_code is 200.
You can implement your own one.
Customize
See peewee doc for querying, or write your own SQL.
from freeproxy import from_pachong_org, from_xici_daili, from_cyber_syndrome
from freeproxy import Proxy, test_proxies, init_db
def your_fetching():
proxies = from_pachong_org() + from_xici_daili() + from_cyber_syndrome()
test_proxies(proxies, timeout=8, single_url='http://www.baidu.com')
def your_reading():
query = Proxy.select().where(~(Proxy.status_code >> None))
return [p.proxy for p in query]
def your_testing():
test_proxies(your_reading(), single_url="http://www.baidu.com")
if __name__ == '__main__':
init_db() # If you have never successfully run `freeproxy` on Terminal
your_fetching()
your_testing()ScreenShot
