THANK YOU FOR SUBSCRIBING
Editor's Pick (1 - 4 of 8)
Big Data Analytics in Cyber Security
By Vahid Behbood, Chief Data Scientist, BT Security
Considering the high number of devices and entities that normally operate in a modern organisation and, more importantly, the massive amount of data that they produce in the form of log events, it is apparent that big data is hugely helpful in this scenario. The data volume is in the order of terabytes per day, originating from many sources generating millions of events per second. Finding the malicious events in this digital environment is analogous to finding a needle in haystack, which makes the big data platform and framework invaluable for generating actionable insights into malicious behaviour patterns. Investigation Another layer of security for an organisation is the investigation layer which is sometimes referred to as threat hunting. In the investigation stage, security experts dig into data lakes of past events produced by digital entities to explore and find historical malicious events and incidents. In typical practice, the analyst performs retrospective investigation spanning from 6 months’ worth of data up to a couple of years. Considering that timeline, a cyber analyst could deal with unimaginable amount of data, and it would be near impossible to carry out an investigation without acknowledgement of the big data ecosystem and architecture. The investigative analytics on this massive amount of data span from basic descriptive analytics to advanced analytics like Artificial Intelligence. Descriptive analytics can include simply understanding general behaviour of traffic – such as the amount of web traffic during the normal business hours and the geographical destination of web connections. Advanced analytics such as Deep Learning leverage massive amounts of data to mine the historical data and explore interesting patterns of adversaries that have been unnoticed and undetected by cyber experts(e.g. due to the novelty and complexity of attack or just simply too much data to be monitored). These advanced analytics are a great example of human experts being augmented by big data, rather than being replaced. Response The last security layer, but definitely not the least, is the response layer. It focuses on triage, analysis and response to detected malicious events and incidents. The response layer is the most critical, intensive and resource-consuming stage o the security monitoring process. It requires numerous cyber experts with different levels of security expertise to examine every single alert supplied by multiple security tools within an organisation. For example, Security Operation Centres (SOCs) monitoring environments for large, frequently-targeted organisations such as banks could receive in the order of thousands of alerts per day. These events need to be analysed by cyber analysts to find malicious incidents and respond as quickly as possible in order to contain and/or remediate damage done by the threat. The Response layer is the most resource intensive part of security monitoring process and it also represents the key performance indicator for cyber security departments. Therefore, many companies are attracted by the idea of introducing smart automation, which should be able to learn and adapt the response process when receiving different types of alerts. This allows it to act promptly and with minimum human intervention when action needs to be taken. To be smart and adaptive, it requires access to the big data environment that contains the historical data of case incidents including alerts and responses. Utilising this data, smart automation forms robust learning of the relation between alerts and the relevant response actions. Smart automation unlocks the potential of security experts by reducing the time spent on menial tasks and increasing the time spent on advanced tasks. Moreover, it considerably reduces the time to detection and response and improves the productivity of security departments. Cyber Security Platform Big data technology has successfully been imbedded into analytics products for various business sectors such as finance, retail, health and telecom. Its now also plating a crucial role in today’s cyber security industry, due to the volume, velocity and variety of the data in this domain. In fact, it’s now almost impossible to unravel the actual view of security information without using big data technologies and frameworks in the core of security architecture. Big data analytics empower companies to offer holistic, accurate and efficient security monitoring mechanism and continually maintain the standard security posture.BT is pioneering in this space with its Cyber Security Platform; combining big data engineering and advanced analytics. Leveraging huge volumes of data and innovative analytics capabilities is actively helping to protect BT as well as our partners and customers, from today's advanced threat landscape.