The first generation of SIEM gave security professionals visibility. Then, SIEM 2.0 took it away by presenting more data than they could possibly cope with. Now, machine learning is revolutionising the game once more, and CISOs should take note
To understand where we are today with security information and event management (SIEM), it’s important to first understand exactly how we got here. Like many modern security technologies, the history of SIEM isn’t a particularly long one. In fact, the first commercial SIEM products were introduced less than twenty years ago, around the start of the century.
My personal initiation into SIEM came around 1999, when me and my team at Deutsche Bank tried to build a SIEM-like tool of our own from scratch. Fast forward to 2006 and I found myself at Barclays Capital with budget to spend and only a few SIEM vendors to choose from. Even with the limited information available to me at the time, I knew enough to base my choice at least partially on the ability of a SIEM platform to scale effectively, something which remains critical today.
SIEM 1.0 circa 2006 – A revolutionary new approach to security
The arrival of the first generation of SIEM platforms heralded a new dawn in the data security industry, combining security event management with security information management for the first time. However, these early SIEM 1.0 platforms had an Achilles heel in that they only scaled vertically, which quickly became a major limiting factor. As bigger and bigger hardware was needed to handle loads it eventually became I/O bound, at which point scaling came to an end.
SIEM 2.0 circa 2011 – A significant improvement, but still flawed
The second generation of SIEM arrived just in time. Perhaps unsurprisingly, the main difference to SIEM 1.0 was in the way it scaled, doing away with the centralised database and instead using big data to allow horizontal scaling.
At Barclays, this allowed us to quickly break the previous ingestion barrier and we soon found ourselves comfortably processing 2.5 billion events per day. Furthermore, horizontal scaling meant we could add lower spec hardware to the system to scale without limits. Double the number of servers meant roughly double the data or twice the performance.
SIEM 2.0 also allowed for better reporting and dashboards, as well as the ability to query historical data for the first time. With SIEM 1.0, long data retention periods were irrelevant as the technology could not effectively query the data back more than a few weeks. SIEM 2.0’s big data architecture meant data could be queried over much longer periods and have responses returned in a reasonable amount of time.
However, if scalability was SIEM 2.0’s greatest gift, it eventually became its greatest curse. Not because it didn’t work, but because it simply moved the problem further down the operational pipeline. Before SIEM, security professionals were essentially blind, unable to see what was happening in their own IT environments. The first generation of SIEM gave them sight but the second generation took it away again by presenting more data than they could possibly cope with. It also failed to innovate around the alerting aspects of SIEM, leaving teams reliant on pre-configured alerts which at best correlated against only a few elements.
SIEM 3.0 (circa 2015 – Present) – Revolution rather than evolution
Enter the third generation of SIEM, a radical iteration focussing on operational capability rather than technology. SIEM 3.0 introduced analytics into the mix for the first time, through the application of machine learning.
In a UEBA system, a baselining period allows for the establishment of ‘normal behaviour’ in an environment. Any future deviations from this baseline then attribute risk to users and systems, allowing security teams to focus on actual risks rather than false alarms.
SIEM 3.0 also removes the final scalability problem in SIEM, the operational process piece. Of course, this does require security teams to take what is often a more difficult step than replacing a security technology – changing an existing operational process, many of which have been embedded for years. Political resistance to change can make this final step the hardest of all, but few CISOs can look ahead and see a rosy future if they stick with second generation SIEM technology.
Aside from the fact that security operations are far less likely to spot security breaches before the kill-chain is complete, when relying on alert based SIEM technology, the reality is that such operational teams are increasingly full of over worked and poorly trained security analysts doing little more than hitting the numbers of false positive dismissal.
No technology which sustains such inefficient and ineffective working practices can last long in a world where both the breaches and fines get bigger every year. Who knows what SIEM 4.0 will bring, but until it’s arrival, SIEM 3.0 is undoubtedly the enabling technology of a modern effective security operations centre (SOC).
