This is a tool used to collect various intelligence sources for a given file. Fileintel is written in a modular fashion so new intelligence sources can be easily added. Files are identified by file hash (MD5, SHA1, SHA256). The output is in CSV format and sent to STDOUT so the data can be saved or piped into another program. Since the output is in CSV format, spreadsheets such as Excel or database systems will easily be able to import the data. This works with Python v2, but it should also work with Python v3. If you find it does not work with Python v3 please post an issue.
This code has been tested on Windows 7 and Mac OSX El Capitan.
Modular application to look up file intelligence information. Outputs CSV to
ConfigurationFile Configuration file
InputFile Input file, one hash per line (MD5, SHA1, SHA256)
-h, –help show this help message and exit
-a, –all Perform All Lookups.
-v, –virustotal VirusTotal Lookup.
-n, –nsrl NSRL Lookup for SHA-1 and MD5 hashes ONLY!
-o, –otx OTX by AlienVault Lookup.
-t, –threatcrowd ThreatCrowd Lookup for SHA-1 and MD5 hashes ONLY!
-r, –carriagereturn Use carriage returns with new lines on csv.
First, make sure your configuration file is correct for your computer/installation. Add your API keys and usernames as appropriate in the configuration file. Python and Pip are required to run this tool. There are modules that must be installed from GitHub, so be sure the git command is available from your command line. Git is easy to install for any platform. Next, install the python requirements (run this each time you git pull this repository too):
If you are using the NSRL database lookups, download the NSRL “Minimal” data set as a zip file. Put it in a directory you can access and point your configuration file to that zip file. There is no need to unzip the NSRL data.
If you want to use 7Zip (fast) rather than the internal Python zip library (slow) to read the large NSRL zip file, you will need to install 7Zip. Windows installation of 7Zip is quite simple, but Mac OX X or Linux will need to install p7zip, the command line tool. For Mac OS X, you can install this tool with Brew. Once in install 7Zip you will need to point your configuration file appropriate to wherever the 7z executable lies.
$ python fileintel.py myconfigfile.conf myhashes.txt -a > myoutput.csv
You should be able to import myoutput.csv into any database or spreadsheet program.
Note that depending on your network, your API key limits, and the data you are searching for, this script can run for a very long time! Use each module sparingly! In return for the long wait, you save yourself from having to pull this data manually.
VirusTotal (Public API key and network I/O required, throttled when appropriate)