In today’s dynamic and interconnected digital landscape, organisations face an ever-increasing number of cyber security threats. Whether it’s a sophisticated malware attack, a data breach, or a rapidly spreading ransomware campaign, the need for efficient and swift incident response has become paramount. Timely detection and immediate action can make all the difference between a minor security incident and a data disaster.
To tackle these challenges, cyber security teams and SOC (Security Operations Centre) analysts rely on advanced tools and technologies to enhance their incident detection and response capabilities. Among these tools, TheHive stands out as an open-source, powerful security incident response platform designed to streamline the handling of cyber security incidents. When combined with ELK Stack (Elasticsearch, Logstash, Kibana), AWS Elastic Beanstalk, and Elasticsearch Docker, it becomes an even more potent force in incident management.
In this blog post, we will delve into the World of Elastic Search and TheHive integration, exploring how this powerful combination of the ELK Stack and AWS services empowers organisations to bolster their security posture. We’ll dive into the key features and benefits of using Elastic Search to trigger alters in TheHive, and how this seamless integration enables SOC teams to make more informed decisions, collaborate effectively, and respond to incidents with unprecedented efficiency.
Security Incident Response Platforms (SIRP) (Also known as SOAR – Security Orchestration, Automation and Response) provide integrated and real-time countermeasures against security breaches. As SIRP usually generates incident reports after a breach, security analysts can use reports for further triage and investigation.
Key features of SIRP include:
- Cyber Data collected from SIEM, endpoints and other sources
- Prebuilt knowledge base of threats and vulnerabilities
- Attack behaviour analysis, including major observables in the breach
- Integrated work process in security case analysis and handling
- Forensic data retention for post-incident reporting and analysis
- Tasks assignment and tracking
The following article introduces a typical SIRP, named TheHive, and how it receives security alerts generated by Elasticsearch for SOC analysts to investigate.
What is Elasticsearch for TheHive?
TheHive, as stated on its website, is “a scalable, open source and free Security Incident Response Platform, tightly integrated with MISP (Malware Information Sharing Platform), designed to make life easier for SOCs, CSIRTs, CERTs and any information security practitioner dealing with security incidents that need to be investigated and acted upon swiftly.” In other words, it collects alerts from different sources, with which security analysts will manage the security cases from creation to closure.
Elasticsearch previously acted as a log repository. With the introduction of Elastic SIEM, it is also powerful and versatile for threat hunting and anomaly detection. SIEM enables analysis of host-related and network-related security events as part of alert investigations or interactive threat hunting. It uses pre-built rules and machine learning jobs to periodically search for documents and detect anomalies that meet the criteria. All the detected signals will be stored in certain indices for management and retrieval.
Elasticsearch for TheHive
While Elasticsearch can ingest and store all types of data, it can be used to pre-process the security logs and send detected signals to TheHive for further analysis. Think of it as a connector that allows data to flow bi-directionally so the Elasticseatch engine capabilities can be leveraged.
Use Elasticsearch for TheHive to Generate Alerts
The Flow Chart:
The above chart describes the workflow of using Elasticsearch to send alerts to TheHive.
Components to be included:
Beats are open-source data shippers that are installed as agents on users’ systems. Beats send security events and other data to Elasticsearch. In the 7.9 version, a single and unified solution called Elastic Agent is introduced. It means that users don’t need to install multiple beats on hosts and can centrally manage a fleet of agents in Ingest Manager.
What is Elasticsearch? Elastic search is a real-time, distributed storage, search, and analytics engine. Elasticsearch can index streams of semi-structured data, such as logs or metrics.
What is Kibana? Kibana is an open-source analytics and visualisation platform designed to work with Elasticsearch. It is used to search, view, and interact with data stored in Elasticsearch indices. Users can perform advanced data analysis and visualise their data in a variety of charts, tables, and maps. The watcher UI in Kibana is used to configure alerts sent to TheHive. Watcher is an Elasticsearch feature that can be used to create actions based on conditions, which are periodically evaluated using data queries. Specifically, the webhook service can be used to send a request to any web service; thus, it enables communication between Elasticsearch and TheHive.
The Hive is an alert management platform that analysts can use to triage and investigate security breaches.
What is Included in the Alert
When alerts are generated and sent to TheHive, the contents need to follow particular templates in order to be accepted.
After the alerts are imported into TheHive, analysts can go to the UI for further investigation. Usually, they can get access to information including title, source, reference, type and description. Some alerts also include the severity level and observables so analysts can merge similar cases and speed up the investigation process.
Benefits of Using the Open Source SIRP with Elasticsearch
Gain more Security Insights from Data
Elasticsearch takes all kinds of data collected from users’ systems and stores them in structured JSON documents, with a higher performance result. Its scalability, let alone the capability to enrich security data with, for example, GeoIP information, allows for an integrated view of data from which deeper insights can be obtained.
Leverage SIEM to Conduct Threat Detection
Elastic SIEM is built with pre-defined rules and machine learning jobs, which comply with frameworks, such as The MITRE ATT&CK, or other best practices. With known malicious executables or anomalies, Elastic SIEM is able to find signals of possible breaches and largely reduce false positives according to historical patterns.
Based on signals detected by the Elastic SIEM, watchers can use customised criteria, such as the severity level or the number of events leading to trigger alerts in TheHive. It streamlines the threat-hunting and alert management process.
Watchers that query the data and send alerts to TheHive can be customised to evaluate the condition on a real-time basis. This allows security analysts to collaborate on investigations in the live stream and respond quickly to detected threats. Not surprisingly, the sources, references, observables and other enriched contents sent in an ongoing fashion enable analysts to quickly assign and trace alerts when they are created in TheHive.
In conclusion, we can see that the fusion of Elastic Search and TheHive opens up a realm of possibilities for organisations seeking to bolster their incident response and collaboration efforts.
By harnessing the potential of this comprehensive integration, cyber security teams can optimise their incident response efforts, proactively detect threats, and collaboratively mitigate risks more effectively. The power of real-time alerts, precise searching, and correlation between data sets ensures that incidents are dealt with promptly, minimising the potential impact of security breaches.
Author: Astrid Liu
- Arnaudloos.com. 2020. Open Source SIRP with Elasticsearch and TheHive – Overview
- Elastic.co. 2020. Get Up And Running | SIEM Guide [7.8] | Elastic
- TrustRadius. 2020. List Of Top Incident Response Platforms 2020.
- Thehive-project.org. 2020. Thehive Project.
- GitHub. 2020.Thehive-Project/Thehivedocs.
Skillfield is a Melbourne-based Cyber Security and Data Services consultancy and professional services company. We provide solutions that help our customers discover, protect and optimise big data in a way that works for them.