All Projects → layershifter → Tldextract

layershifter / Tldextract

Licence: apache-2.0
[DEPRECATED] Library for extraction of domain parts e.g. TLD. Domain parser that uses Public Suffix List

Labels

Projects that are alternatives of or similar to Tldextract

Lion
Lion is a fast HTTP router for building modern scalable modular REST APIs in Go
Stars: ✭ 750 (+244.04%)
Mutual labels:  subdomain
Subdomains
互联网公司子域名收集
Stars: ✭ 89 (-59.17%)
Mutual labels:  subdomain
Massdns
A high-performance DNS stub resolver for bulk lookups and reconnaissance (subdomain enumeration)
Stars: ✭ 2,093 (+860.09%)
Mutual labels:  subdomain
Reconftw
reconFTW is a tool designed to perform automated recon on a target domain by running the best set of tools to perform scanning and finding out vulnerabilities
Stars: ✭ 974 (+346.79%)
Mutual labels:  subdomain
Nager.publicsuffix
.NET publicsuffix domain parser
Stars: ✭ 67 (-69.27%)
Mutual labels:  subdomain
Subtake
Automatic finder for subdomains vulnerable to takeover. Written in Go, based on @haccer's subjack.
Stars: ✭ 104 (-52.29%)
Mutual labels:  subdomain
K8cscan
K8Cscan大型内网渗透自定义插件化扫描神器,包含信息收集、网络资产、漏洞扫描、密码爆破、漏洞利用,程序采用多线程批量扫描大型内网多个IP段C段主机,目前插件包含: C段旁注扫描、子域名扫描、Ftp密码爆破、Mysql密码爆破、Oracle密码爆破、MSSQL密码爆破、Windows/Linux系统密码爆破、存活主机扫描、端口扫描、Web信息探测、操作系统版本探测、Cisco思科设备扫描等,支持调用任意外部程序或脚本,支持Cobalt Strike联动
Stars: ✭ 693 (+217.89%)
Mutual labels:  subdomain
Multi Domain Laravel
An example of multi-domain/subdomain app in Laravel.
Stars: ✭ 171 (-21.56%)
Mutual labels:  subdomain
Subjack
Subdomain Takeover tool written in Go
Stars: ✭ 1,194 (+447.71%)
Mutual labels:  subdomain
Pigat
pigat ( Passive Intelligence Gathering Aggregation Tool ) 被动信息收集聚合工具
Stars: ✭ 140 (-35.78%)
Mutual labels:  subdomain
Synologyddnscloudflaremultidomain
Synology DDNS Cloudflare service provider with multidomains and subdomains
Stars: ✭ 51 (-76.61%)
Mutual labels:  subdomain
Spoodle
A mass subdomain (Subbrute) + poodle vulnerability scanner
Stars: ✭ 66 (-69.72%)
Mutual labels:  subdomain
A scan framework
Network Security Vulnerability Manage
Stars: ✭ 116 (-46.79%)
Mutual labels:  subdomain
Ksubdomain
无状态子域名爆破工具
Stars: ✭ 976 (+347.71%)
Mutual labels:  subdomain
Gsdf
A domain searcher named GoogleSSLdomainFinder - 基于谷歌SSL透明证书的子域名查询工具
Stars: ✭ 155 (-28.9%)
Mutual labels:  subdomain
Anubis
🔓Subdomain enumeration and information gathering tool
Stars: ✭ 722 (+231.19%)
Mutual labels:  subdomain
Acamar
A Python3 based single-file subdomain enumerator
Stars: ✭ 89 (-59.17%)
Mutual labels:  subdomain
Can I Take Over Xyz
"Can I take over XYZ?" — a list of services and how to claim (sub)domains with dangling DNS records.
Stars: ✭ 2,808 (+1188.07%)
Mutual labels:  subdomain
Aspnetcoresubdomain
Simple usage lib for subdomain routing in ASP.NET Core/Framework MVC
Stars: ✭ 157 (-27.98%)
Mutual labels:  subdomain
Amass
In-depth Attack Surface Mapping and Asset Discovery
Stars: ✭ 1,693 (+676.61%)
Mutual labels:  subdomain

DEPRECATED

Consider to use https://github.com/jeremykendall/php-domain-parser as maintained alternative.

TLDExtract

TLDExtract accurately separates the gTLD or ccTLD (generic or country code top-level domain) from the registered domain and subdomains of a URL, e.g. domain parser. For example, say you want just the 'google' part of 'http://www.google.com'.

Latest Version on Packagist Software License Build Status Coverage Status Total Downloads


Everybody gets this wrong. Splitting on the '.' and taking the last 2 elements goes a long way only if you're thinking of simple e.g. .com domains. Think parsing http://forums.bbc.co.uk for example: the naive splitting method above will give you 'co' as the domain and 'uk' as the TLD, instead of 'bbc' and 'co.uk' respectively.

TLDExtract on the other hand knows what all gTLDs and ccTLDs look like by looking up the currently living ones according to the Public Suffix List. So, given a URL, it knows its subdomain from its domain, and its domain from its country code.

$result = tld_extract('http://forums.news.cnn.com/');
var_dump($result);

object(LayerShifter\TLDExtract\Result)#34 (3) {
  ["subdomain":"LayerShifter\TLDExtract\Result":private]=>
  string(11) "forums.news"
  ["hostname":"LayerShifter\TLDExtract\Result":private]=>
  string(3) "cnn"
  ["suffix":"LayerShifter\TLDExtract\Result":private]=>
  string(3) "com"
}

Result implements ArrayAccess interface, so you simple can access to its result.

var_dump($result['subdomain']);
string(11) "forums.news"
var_dump($result['hostname']);
string(3) "cnn"
var_dump($result['suffix']);
string(3) "com"

Also you can simply convert result to JSON.

var_dump($result->toJson());
string(54) "{"subdomain":"forums.news","hostname":"cnn","suffix":"com"}"

This package is compliant with PSR-1, PSR-2, PSR-4. If you notice compliance oversights, please send a patch via pull request.

Does TLDExtract make requests to Public Suffix List website?

No. TLDExtract uses database from TLDDatabase that generated from Public Suffix List and updated regularly. It does not make any HTTP requests to parse or validate a domain.

Requirements

The following versions of PHP are supported.

  • PHP 5.5
  • PHP 5.6
  • PHP 7.0
  • PHP 7.1
  • PHP 7.2
  • PHP 7.3
  • HHVM

Install

Via Composer

$ composer require layershifter/tld-extract

Additional result methods

Class LayerShifter\TLDExtract\Result has some usable methods:

$extract = new LayerShifter\TLDExtract\Extract();

# For domain 'shop.github.com'

$result = $extract->parse('shop.github.com');
$result->getFullHost(); // will return (string) 'shop.github.com'
$result->getRegistrableDomain(); // will return (string) 'github.com'
$result->isValidDomain(); // will return (bool) true
$result->isIp(); // will return (bool) false

# For IP '192.168.0.1'

$result = $extract->parse('192.168.0.1');
$result->getFullHost(); // will return (string) '192.168.0.1'
$result->getRegistrableDomain(); // will return null
$result->isValidDomain(); // will return (bool) false
$result->isIp(); // will return (bool) true

Custom database

By default package is using database from TLDDatabase package, but you can override this behaviour simply:

new LayerShifter\TLDExtract\Extract(__DIR__ . '/cache/mydatabase.php');

For more details and how keep database updated TLDDatabase.

Implement own result

By default after parse you will receive object of LayerShifter\TLDExtract\Result class, but sometime you need own methods or additional functionality.

You can create own class that implements LayerShifter\TLDExtract\ResultInterface and use it as parse result.

class CustomResult implements LayerShifter\TLDExtract\ResultInterface {}

new LayerShifter\TLDExtract\Extract(null, CustomResult::class);

Parsing modes

Package has three modes of parsing:

  • allow ICANN suffixes (domains are those delegated by ICANN or part of the IANA root zone database);
  • allow private domains (domains are amendments submitted to Public Suffix List by the domain holder, as an expression of how they operate their domain security policy);
  • allow custom (domains that are not in list, but can be usable, for example: example, mycompany, etc).

For keeping compatibility with Public Suffix List ideas package runs in all these modes by default, but you can easily change this behavior:

use LayerShifter\TLDExtract\Extract;

new Extract(null, null, Extract::MODE_ALLOW_ICANN);
new Extract(null, null, Extract::MODE_ALLOW_PRIVATE);
new Extract(null, null, Extract::MODE_ALLOW_NOT_EXISTING_SUFFIXES);
new Extract(null, null, Extract::MODE_ALLOW_ICANN | Extract::MODE_ALLOW_PRIVATE);

Change log

Please see CHANGELOG for more information what has changed recently.

Testing

$ composer test

Contributing

Please see CONTRIBUTING and CONDUCT for details.

License

This library is released under the Apache 2.0 license. Please see License File for more information.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].