TheTHE (The Threat Hunting Environment) · Malware Defense / Network Defense / Incident Response...
12
TheTHE (The Threat Hunting Environment) Simple, shareable, team-focused and expandable Threat Hunting Environment Malware Defense / Network Defense / Incident Response Abstract TheTHE (or thethe) is an application intended to help analysts and hunters over the early stages of their work in an easier, unified and quicker way. One of the major drawbacks when dealing with a hunting is the collection of information available on a high number of sources, both public and private. All this information is usually scattered and sometimes even volatile. Perhaps at a certain point there is no information on a particular IOC (Indicator of Compromise), but that situation may change within a few hours and become crucial for the investigation. Based on our experience on Threat Hunting, we have created a free and open source framework to make the early stages of the investigation simpler from: - Server-client architecture. Investigation may be shared among your team. - APIkeys are stored in a database and may be shared by a team from a single point. - Results are cached; so not repeated API calls are used. - Better feeds your Threat Intelligence Platform. TheTHE allows to better perform a prior investigation of your assets. - Easy plugins: Whatever you need, it is easily embedded within the system. - Ideal for SOCs, CERTS and or Law Enforcement any team. - Automation of tasks and searches. - Rapid API processing of multiple tools.
Transcript of TheTHE (The Threat Hunting Environment) · Malware Defense / Network Defense / Incident Response...
TheTHE (The Threat Hunting Environment)
Simple, shareable, team-focused and expandable
Threat Hunting Environment
Malware Defense / Network Defense / Incident Response
Abstract
TheTHE (or thethe) is an application intended to help analysts and hunters over the
early stages of their work in an easier, unified and quicker way. One of the major drawbacks when dealing with a hunting is the collection of information available on a high number of sources, both public and private.
All this information is usually scattered and sometimes even volatile. Perhaps at a certain point there is no information on a particular IOC (Indicator of Compromise), but that situation may change within a few hours and become crucial for the investigation. Based on our experience on Threat Hunting, we have created a free and open source
framework to make the early stages of the investigation simpler from:
- Server-client architecture. Investigation may be shared among your team. - APIkeys are stored in a database and may be shared by a team from a single
point. - Results are cached; so not repeated API calls are used. - Better feeds your Threat Intelligence Platform. TheTHE allows to better perform
a prior investigation of your assets.
- Easy plugins: Whatever you need, it is easily embedded within the system. - Ideal for SOCs, CERTS and or Law Enforcement any team. - Automation of tasks and searches. - Rapid API processing of multiple tools.
- Unification of information in a single interface, so that screenshots, spreadsheets, text files, etc. are not scattered.
- Enrichment of collected data. - Periodic monitoring of a given IOC in case new information or related
movements appear.
TheTHE has a web interface where the analyst starts its work by entering IOCs that will be sent to a backend, where the system will automatically look up for such
resource on the various configured platforms in order to obtain unified information from different sources and access related reports or data existing on them. Furthermore, any change in the resources to be analyzed will be monitored.
Everything is executed on a local system, without needing to share information with third parties until such information is not organized, linked, complete and synthesized. This allows that, in case the information must be analyzed later on any other platform (such as a Threat Intelligence Platform), it can be done in the most enriching possible
manner.
Tool Details
TheTHE is an open source and modular framework developed in Python 3 and VueJS
that allows to locally consolidate and analyze information on a MongoDB database,
without sharing such information with other platforms until it is not appropriately
organized, linked and synthesized. It is a unique tool within its category that makes it
possible to help analysts and hunters, as well as to perform their investigation tasks in a
more agile and practical manner.
TheTHE is a framework that runs locally in your own system or local server. Currently it
has passive modules for information collection as well as active modules, which in turn
allow to:
- Obtain information automatically from multiple public and private sources (by configuring users’ own accounts and configurable APIs) such as: Hunter.io, Maltiverse, Shodan, Sherlock, etc.
- Execute tests and consolidate information from other tools such as cansina.
Future work: - Monitor specific IOCs programmatically under platforms in case new data may
appear in the future. - Monitor changes in the infrastructures under investigation in case of failure or if
new threats appear within. - Keep a local history of the investigations performed.
- Access information in a consolidated way on a local DB from a web interface. - Store securely the various API Keys and the pre-configuration of queries from
dozens of public and private platforms. - More plugins to come!
Download and Installation
A complete set of instructions for installation is on https://github.com/ElevenPaths/thethe
We advise you to install it on a GNU/Linux, MacOS or similar UNIX derivative. Client side is compatible with all mayor browsers, such Chrome or Firefox.
How-to’s
TheTHE is based on projects. A project is a container of related IoCs, for example. Create a project for a set of users or for a specific research.
In each project, there are four main menus based on the initial IoC you are working with. According to the IoC entered, TheTHE will try to classify it into the appropriate menu:
- Network: Basically, IP addresses. - Domain: Only domains, any TLD. - URL: If your domain has a path, then is an URL. - Hash: Any hash, MD5, SHA1 and SHA256
- Emails. - Usernames: Any string not in any other category will be treated as a username. You can enter a list of IOCs:
Thethe will try to match each IOC to a cathegory automatically:
Aren’t you happy with the auto-cathegorization? You are free to change its type:
Within each menu, the minimum information required to process the information will be loaded. Within it, you can choose the appropriate plugins for each category that may
be applied to each IoC.
For Network:
For Domains:
For URL:
For Hashes:
For emails:
For Usernames:
When a plugin is being used the task will be queued, and results will be displayed when the necessary information is retrieved. All tasks will be queued asynchronously, and work can continue while the results are calculated. In case of using third-party services
on the network that need to consume APIs with API keys, these will be stored on the main server and all users will be able to use them remotely. The results will be cached indefinitely so as not to use requests if they have already been made by another team member. The results may be refreshed on request.
Depending on the plugin output, a new tag will be created when the IoC is selected. Tags may be used to better categorize your work; with colors and names you can
choose. Tags will be available once created for the same project.
API Keys management
Certain plugins require an API Key. There is a dialog to manage all the keys which are
stored in a database.
Examples
HaveIBeenPwn View
DIARIO View
Sherlock view
GeoIP View
Phishtank view
Available Plugins (completed)
abuseipdb
basic_ip
binaryedge
botscout
DIARIO
dns
emailrep
geoip
haveibeenpwned
hunterio
maltiverse
metagoofil
onyphe
otx
pastebin
phishtank
pulsedive
robtex
sherlock
shodan
tacyt
threatcrowd
threatminer
urlscan
verifymail
virustotal
vt_domain
whois
