GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → hashlookup → hashlookup-forensic-analyser

hashlookup / hashlookup-forensic-analyser

Licence: other
Analyse a forensic target (such as a directory) to find and report files found and not found from CIRCL hashlookup public service - https://circl.lu/services/hashlookup/

Programming Languages

python
139335 projects - #7 most used programming language

Labels

bloom-filterdfirnsrlforensic-analysisforensics-investigationsdfir-automationnsrllookuphashlookup

Projects that are alternatives of or similar to hashlookup-forensic-analyser

Detectionlab
Automate the creation of a lab environment complete with security tooling and logging best practices
Stars: ✭ 3,237 (+7427.91%)
Mutual labels:  dfir, dfir-automation
dcfldd
Enhanced version of dd for forensics and security
Stars: ✭ 27 (-37.21%)
Mutual labels:  forensic-analysis, forensics-investigations
Awesome Forensics
A curated list of awesome forensic analysis tools and resources
Stars: ✭ 1,775 (+4027.91%)
Mutual labels:  dfir, forensic-analysis
Packrat
Live system forensic collector
Stars: ✭ 16 (-62.79%)
Mutual labels:  dfir, forensics-investigations
Judge-Jury-and-Executable
A file system forensics analysis scanner and threat hunting tool. Scans file systems at the MFT and OS level and stores data in SQL, SQLite or CSV. Threats and data can be probed harnessing the power and syntax of SQL.
Stars: ✭ 66 (+53.49%)
Mutual labels:  forensic-analysis, forensics-investigations
Palmprint-Recognition-in-the-Wild
No description or website provided.
Stars: ✭ 22 (-48.84%)
Mutual labels:  forensic-analysis, forensics-investigations
BlueCloud
Cyber Range including Velociraptor + HELK system with a Windows VM for security testing and R&D. Azure and AWS terraform support.
Stars: ✭ 88 (+104.65%)
Mutual labels:  dfir, dfir-automation
ForensicsTools
A list of free and open forensics analysis tools and other resources
Stars: ✭ 392 (+811.63%)
Mutual labels:  forensic-analysis, forensics-investigations
calamity
A script to assist in processing forensic RAM captures for malware triage
Stars: ✭ 24 (-44.19%)
Mutual labels:  dfir
fastfinder
Incident Response - Fast suspicious file finder
Stars: ✭ 116 (+169.77%)
Mutual labels:  dfir
pyaff4
The Python implementation of the AFF4 standard.
Stars: ✭ 37 (-13.95%)
Mutual labels:  forensic-analysis
bloomfilter
Simplistic (but fast) java implementation of a bloom filter.
Stars: ✭ 35 (-18.6%)
Mutual labels:  bloom-filter
truehunter
Truehunter
Stars: ✭ 30 (-30.23%)
Mutual labels:  dfir
catalyst
Catalyst is an open source SOAR system that helps to automate alert handling and incident response processes
Stars: ✭ 91 (+111.63%)
Mutual labels:  dfir
decwindbx
A sort of a toolkit to decrypt Dropbox Windows DBX files
Stars: ✭ 22 (-48.84%)
Mutual labels:  dfir
IRScripts
Incident Response Scripts
Stars: ✭ 29 (-32.56%)
Mutual labels:  dfir
LevelDBDumper
Dumps all of the Key/Value pairs from a LevelDB database
Stars: ✭ 23 (-46.51%)
Mutual labels:  dfir
lsrootkit
Rootkit Detector for UNIX
Stars: ✭ 53 (+23.26%)
Mutual labels:  forensic-analysis
ntHash
Fast hash function for DNA sequences
Stars: ✭ 66 (+53.49%)
Mutual labels:  bloom-filter
ir scripts
incident response scripts
Stars: ✭ 17 (-60.47%)
Mutual labels:  dfir
View All Similar Projects ➔

hashlookup-forensic-analyser

lint_python

Analyse a forensic target (such as a directory) to find and report files found and not found from CIRCL hashlookup public service or the Bloom filter from CIRCL hashlookup. This tool can help a digital forensic investigator to know the context, origin of specific files during a digital forensic investigation.

Usage

usage: hashlookup-analyser.py [-h] [-v] [--extended-debug] [--progress] [-d DIR] [--report] [--print-all] [--print-unknown] [--include-stats] [--format FORMAT] [--cache] [--bloomfilter BLOOMFILTER]

Analyse a forensic target to find and report files found and not found in hashlookup CIRCL public service.

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         Verbose output.
  --extended-debug      Debug file processed along with the mode and type.
  --progress            Pring progress of the file lookup on stderr.
  -d DIR, --dir DIR     Directory to analyse.
  --report              Generate a report directory including a summary and all the results.
  --print-all           Print all files result including known and unknown.
  --print-unknown       Print all files unknown to hashlookup service.
  --include-stats       Include statistics in the CSV export.
  --format FORMAT       Output format (default is CSV).
  --cache               Enable local cache of known and unknown hashes in /tmp/hashlookup-forensic-analyser.
  --bloomfilter BLOOMFILTER
                        Specify filename of a bloomfilter in DCSO bloomfilter format.

Sample report

If you want to analyse a specific directory (in this case, a Kernel module directory), the following command be executed:

python3 hashlookup-analyser.py --bloomfilter=../hashlookup/hashlookup-full.bloom.1 --report -d /usr/lib/modules/5.11.0-41-generic/

When executed with --report, a summary report in Markdown and a full JSON file report are generated.

A sample summary report is available.

Sample output

python3 hashlookup-analyser.py --print-all -d /usr/local/bin/ --include-stats --cache
hashlookup_result,filename,sha-1,size
known,/usr/local/bin/scss,171B3B77F16894C33ADAC884F162A531FB3A3F62,517
known,/usr/local/bin/sass-convert,555BCFEF91F023E634594C5CC46A41B2A0173742,533
known,/usr/local/bin/listen,758A6526DF120C118FB4309B9AC6AB5DAE21FE6F,529
known,/usr/local/bin/nokogiri,C516EF9E51867E1615DA3FDABDAC55B99E16B6A3,541
known,/usr/local/bin/dumppdf.py,9D4596DE611EF8840383DF4E7682378498B13329,12851
known,/usr/local/bin/sass,C92705012D7CE8BC7BEBFA13D7154720B7D19A08,517
unknown,/usr/local/bin/octopress,F3939C055B533201ED2C65CD5E113A3644CF52FC,547
unknown,/usr/local/bin/pyrsa-verify,0ECC95CACCFE73F100B62306B0EC73214B07A4EF,210
unknown,/usr/local/bin/safe_yaml,4376A50A4C705F6AAB9D7B6A10D292C40AE58CC3,547
unknown,/usr/local/bin/gemoji,672785F37185A4154C9159C1A151A7440007FD2A,529
unknown,/usr/local/bin/bayes.rb,DEF8945D48E0557C2EDA5F506BDDA52CB6F870F2,577
unknown,/usr/local/bin/redcarpet,40E0C2E0D33FF42A6A69DA76BAD24EF5FBC3EF5B,547
unknown,/usr/local/bin/shodan,848E102C87A084C1401BB3FE85BF07E2A6EA984A,214
unknown,/usr/local/bin/pyrsa-priv2pub,3C37A0507F51A35EA5969559C051132B94012CEE,233
unknown,/usr/local/bin/jwt,733BE10A7BFCB3CEE5F593C680542ACFB050C902,211
unknown,/usr/local/bin/github-pages,2FA4D8E77E4D92B11C488EBD3BF46E3C9CF28727,565
unknown,/usr/local/bin/f2py,4DD0FD5C23DAB47B2B9F0BC602EA0E7380E0E1AB,216
unknown,/usr/local/bin/f2py3.8,4DD0FD5C23DAB47B2B9F0BC602EA0E7380E0E1AB,216
unknown,/usr/local/bin/tabulate,676B359E35C10404C7D96D3FEC51E1168ED2F5F3,209
unknown,/usr/local/bin/vba_extract.py,5B8C38C90140D15EA0D57DE5FD0BC6C9DF297E78,1813
unknown,/usr/local/bin/jekyll,6569B0B04E681E2FDCE84516336CAB10117F9D1D,529
unknown,/usr/local/bin/pdf2txt.py,C1D9785041216CBC3831D6A5F826B080EC481F32,8338
unknown,/usr/local/bin/rougify,D5F5C89269E1A7F84A983141C5C22AD50CE14F45,527
unknown,/usr/local/bin/summarize.rb,F0E1C3E5DAB10190C1B1E9D6E4665DD33A8FA4C9,585
unknown,/usr/local/bin/camelot,10CC9D468BE06F880F9065D7A34968701D6F2121,208
unknown,/usr/local/bin/kramdown,7DC8F2F6BD1D685817F2795EEC3AAABB2FBE4803,541
unknown,/usr/local/bin/pyrsa-encrypt,956BA954D79A29F1878375639E8F4D392F63B66C,212
unknown,/usr/local/bin/flask,4BCCBDCC6797E5B48D84207B2E71F74F545443E9,208
unknown,/usr/local/bin/jirashell,A7867C26CCD9EE5ABD1E6ED47A84F0A54A88480F,213
unknown,/usr/local/bin/posix-spawn-benchmark,4393D19B57AF71972AAB9D64159C249290806CF7,579
unknown,/usr/local/bin/pyrsa-sign,145A250476DA9D78ADA74E2F5975C3CC34BD3E10,206
unknown,/usr/local/bin/f2py3,4DD0FD5C23DAB47B2B9F0BC602EA0E7380E0E1AB,216
unknown,/usr/local/bin/pyrsa-keygen,4FEC09E57C767CEBEBC9D1F74FC9FC6B056DA0B2,210
unknown,/usr/local/bin/wsdump.py,01D4947AE9CE873A08EC94A910F7928441685B5E,6902
unknown,/usr/local/bin/emailrep,A73D10ED93DA2AA0A5A2A5282CE1BE14A8F6D769,211
unknown,/usr/local/bin/pyrsa-decrypt,57CB2C91C75701E67D734794363BA523E3DCD952,212
unknown,/usr/local/bin/__pycache__/dumppdf.cpython-38.pyc,7F7332186257CB043570296A13F30E43749C79A8,10562
unknown,/usr/local/bin/__pycache__/vba_extract.cpython-38.pyc,BABF747254BED4881368C1148265538F5FE0C756,1202
unknown,/usr/local/bin/__pycache__/pdf2txt.cpython-38.pyc,0C22717A2D2C6676005B99EB6CFF03BF73DAD5A1,6684
unknown,/usr/local/bin/__pycache__/wsdump.cpython-38.pyc,ADD28D31B88E5995A0725A424B415C350726CFD5,6449
stats,Analysed directory /usr/local/bin/ on kolmogorov running Linux-5.10.0-1045-oem-x86_64-with-glibc2.29 at 2021-10-17 15:50:07.299694+00:00- Found 6 on hashlookup.circl.lu - Unknown files 34 - Excluded files 0

Bloom filter

If you don't want to share your lookups online and do faster lookup, hashlookup provides a bloom filter to download.

The file is around 700MB and can be stored locally in your home directory. hashlookup-analyser works in the same way, --bloomfilter option allows to specify the filename location of the bloom filter.

python3 bin/hashlookup-analyser.py --bloomfilter /home/adulau/hashlookup/hashlookup-full.bloom --include-stats -d /bin

License

The software is an open source software released under the "Simplified BSD License".

Copyright 2021-2022 Alexandre Dulaunoy

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].