Fortinet Blog | News and Threat Research

(This is a continuation from last week, in which we discussed the distinction between big data and security and how they intersect. )

Most know that the there is often a big disparity between theory and reality. And nowhere is that more evident than when big data and security solutions conflict. In fact, big data concepts often translate to challenges when given practical applications. Let’s look at a few examples provided by Ryan Potter, Fortinet director of security strategy.

The Challenges of Security Information and Event Management (SIEM)

Starting out, the volume of data amassed in security solutions provides the most obvious big data headache. That said, many security solutions that utilize and convert big data tend to focus on the size and volume. Security Information and Event Management (SIEM) solutions, for example, require decisions and actions in real time from a wide variety of sources. In most cases, the volume of analyzed data is relatively small. But big data challenges emerge, however, when attempting to process large volumes data in almost real-time.

Real-time requirements aren’t the only big data obstacles that SIEM presents. SIEM also creates big data hurdles in terms of the volume of data needed for security. Specifically, SIEM systems serve as a repository for security alerts and logging systems, necessary for organizations that want to collect all possible data containing information applicable to its security posture. This includes copious reporting logs detailing user-access – network-level firewall and IPS, endpoint security, proxy-related data and even deep packet inspection records. And it all adds up pretty quickly to generate major big data challenges. Subsequently, organizations will need to begin thinking more about what data is absolutely relevant and necessary in the security context. That means asking ‘how much is enough,” Potter says.

The SIEM Conflict

Conversely, SIEM can also get in the way of big data technology when analyzing security threat data. As with other areas, leveraging big data to analyze security events creates almost limitless opportunity for threat evaluation. Specifically, big data enables IT professionals to scrutinize activity and monitor for aberrations crucial for detecting evasive threats and reining in potential data breaches.

Big data analytics often requires actionable response plans, as well as dedicated logging, analysis and reporting mechanisms designed to aggregate log data from security devices and then analyze, report and archive all related activity to measure policy compliance. And that’s where SIEM can clearly come into play. However, advanced SIEM solutions can also be overkill for a company that is either understaffed or has overestimated security needs. In fact, Potter maintains that smaller firms with simpler requirements may still have after-the-fact big data needs that don’t necessarily require in-depth technology provided by SIEM solutions.

Access is another consideration when leveraging big data technology in security solutions – and it’s a pretty important one. In order to successfully combat increasingly stealthy attacks, high-performance firewalls and other network security solutions need to be able to handle increased throughput, as well as connections and application traffic. In most cases, traditional secure access is often sufficient when analytics and data storage are collocated together. However, higher performance security devices are required when compute resources are remote from the data-store. That, in turn, poses a big data challenge when attempting to encrypt at that scale.

Undoubtedly big data represents the answer to a slew of security problems. But users also need to keep in mind that big data will inevitably create more than a few unforeseen challenges to security as adoption reaches its stride. For one, the need to capture, manage and process information quickly in big data environments will inevitably make security an afterthought. And new threat vectors created with the rise of portable storage mechanisms and increased bandwidth will also put new big data sets at higher risk of compromise or attack.

Subsequently, in order for the marriage of big data and security to remain functional, big data assets can’t go by the wayside when it comes to security. For now, security gets shelved in the name of new efficiencies and productivity created by big data. But it’s not sustainable, Potter maintains.

Security isn’t anything new. But neither is big data. And the inevitable amalgam of the two arenas isn’t a passing fad. As such, organizations will need to find a satisfactory balance between the two disciplines in order to optimize performance and create efficiencies in a way that is also secure and compliant. Because, if anything, big data analytics will be integral in resolving next-generation security problems of tomorrow.