Malicious File Intelligence

This is a tool used to collect various intelligence sources for a given file. Fileintel is written in a modular fashion so new intelligence sources can be easily added. Files are identified by file hash (MD5, SHA1, SHA256). The output is in CSV format and sent to STDOUT so the data can be saved or piped into another program. Since the output is in CSV format, spreadsheets such as Excel or database systems will easily be able to import the data.  This works with Python v2, but it should also work with Python v3. If you find it does not work with Python v3 please post an issue.

This code has been tested on Windows 7 and Mac OSX El Capitan.

Help Screen:

$ python fileintel.py -h
usage: fileintel.py [-h] [-a] [-v] [-n] [-o] [-t] [-r]
ConfigurationFile InputFile

Modular application to look up file intelligence information. Outputs CSV to
STDOUT.

positional arguments:
ConfigurationFile     Configuration file
InputFile             Input file, one hash per line (MD5, SHA1, SHA256)

optional arguments:
-h, –help            show this help message and exit
-a, –all             Perform All Lookups.
-v, –virustotal      VirusTotal Lookup.
-n, –nsrl            NSRL Lookup for SHA-1 and MD5 hashes ONLY!
-o, –otx             OTX by AlienVault Lookup.
-t, –threatcrowd     ThreatCrowd Lookup for SHA-1 and MD5 hashes ONLY!
-r, –carriagereturn  Use carriage returns with new lines on csv.

Install:

First, make sure your configuration file is correct for your computer/installation. Add your API keys and usernames as appropriate in the configuration file. Python and Pip are required to run this tool. There are modules that must be installed from GitHub, so be sure the git command is available from your command line. Git is easy to install for any platform. Next, install the python requirements (run this each time you git pull this repository too):

$ pip install -r requirements.txt

There have been some problems with the stock version of Python on Mac OSX (http://stackoverflow.com/questions/31649390/python-requests-ssl-handshake-failure). You may have to install the security portion of the requests library with the following command:

$ pip install requests[security]

NSRL

If you are using the NSRL database lookups, download the NSRL “Minimal” data set as a zip file. Put it in a directory you can access and point your configuration file to that zip file. There is no need to unzip the NSRL data.

7Zip

If you want to use 7Zip (fast) rather than the internal Python zip library (slow) to read the large NSRL zip file, you will need to install 7Zip. Windows installation of 7Zip is quite simple, but Mac OX X or Linux will need to install p7zip, the command line tool. For Mac OS X, you can install this tool with Brew. Once in install 7Zip you will need to point your configuration file appropriate to wherever the 7z executable lies.

Virtualenv

Lastly, I am a fan of virtualenv for Python. To make a customized local installation of Python to run this tool, I recommend you read:
http://docs.python-guide.org/en/latest/dev/virtualenvs/

Running:

$ python fileintel.py myconfigfile.conf myhashes.txt -a > myoutput.csv

You should be able to import myoutput.csv into any database or spreadsheet program.

Note that depending on your network, your API key limits, and the data you are searching for, this script can run for a very long time! Use each module sparingly! In return for the long wait, you save yourself from having to pull this data manually.

Intelligence Sources:

Resources:

Malicious File Intelligence: fileintel Download