I suggested in a previous post that assuring user experience should be big data job #1 https://techtonicsadvisors.com/assuring-user-experience-is-big-data-job-one/. In conjunction with this thesis, security should be big data job #2. The point here is that if an organization cannot assure the user experience and secure the data then all of its big data ambitions are for naught.
The threat landscape is growing ever more hazardous as cloud, mobile and social gain wide acceptance. These overlapping trends are overburdening legacy security technologies, processes and staff. As a result, Chief Security Officers (CSOs) must evolve toward a holistic unified data protection strategy. This strategy makes security a big data project, encompassing all users, systems and applications.
It means adopting a platform that collects massive amounts of data from both internal and external sources in real-time, indexes the data with time stamps so it is readily searchable, and applies algorithms to correlate the data into events for analysis. These algorithms may be prepackaged by a vendor, developed by a third party or custom built.
A big data security platform should have the following characteristics:
- It must aggregate internal data sources, such as logs, packets, systems, applications, and devices, as well as external sources, including threat intelligence and cloud services incorporating Hadoop and other open sources technologies;
- It must be able to ingest this data in real-time at terabyte or greater scale in massively parallel processing environments;
- Best suited for distributed architecture, the platform should allow for deep-dive correlation and various analytics models to identify threats and vulnerabilities at any layer of the technology stack;
- Finally, the platform should be tightly integrated with security policies and rules to facilitate adjustments and automate remediation.
Such a platform touches endpoint, network, data, content and cloud. It allows the security team to quickly identify and troubleshoot systems, investigate security incidents and demonstrate compliance efficiently and cost effectively. This information provides visibility at all layers of the technology stack and across the enterprise. It allows CSOs to prioritize actions, adjust policies and rules, and speed and improve workflows around incident response.
But as organizations collect, store and analyze more data from a greater number of sources and keep that data online for longer periods of time, this platform need not be exclusively on internally managed infrastructure. More enterprises are turning toward cloud-based solutions or managed service providers as a component of a big data security strategy. Splunk and SumoLogic are good performance monitoring solutions that have strong appeal to security professionals. Customers have the flexibility to deploy these capabilities either on-premise or in the cloud.
The Status Quo Won’t Do
Security tools from the endpoint to the network will remain important pieces of data protection. But preventive, signature-based point solutions have outlived their usefulness against more sophisticated malware, advanced persistent threats and other targeted attacks. These disparate systems waste time and money. Having specialized security analysts manually sift through false positives extends incident detection/resolution time and risk exposure.
Discrete security analytics point products that are narrowly focused on specific threats results in a patchwork quilt of technologies from different vendors that do not often work well together. A piecemeal approach to information across servers, networks, storage, operating systems, applications and databases leaves the CSO with an inaccurate and inefficient depiction of the enterprise’s security posture. Meanwhile, first-generation Security Information and Event Management (SIEM) platforms, which were built on SQL databases or proprietary data stores cannot scale to big data requirements.
So while hardened server configurations, next-generation firewalls, signatures to scan for known malware and software vulnerability patches are basic defensive measures, their shortcomings underscore the need for a holistic unified big data security strategy.
How to get there
- Establish a holistic unified big data security strategy and architecture. The strategy should have clearly defined goals, including roles, metrics and time lines. It should be consistent across all security initiatives for data at rest and in motion. The CSO should gain buy-in from the CEO, CFO, CIO and other C-level executives in the company to assure understanding and facilitate implementation. One key objective should be to provide maximum protection with minimum user intrusion.
- Integrate security intelligence into enterprise security dashboards. This enables the collection of security data from within and outside of the enterprise. It also facilitates GRC (governance, regulatory and compliance) mandates.
- Train and deploy security staff on the highest value assets. With limited resources and skills sets, CSOs should tactically use their budget to make certain that human capital is utilized where it is most useful to the organization. Security staff should be focused on the platform that delivers the most intelligence.
- Communicate the strategy throughout the organization. This raises awareness about risks and policies. It also helps maintain adherence with GRC initiatives.
A properly executed big data strategy and platform can serve as the heart of a company’s risk management, incident detection/response and GRC activities. A best practices approach to big data security lowers enterprise and IT risk while providing a better ROI through faster remediation and lower total cost of ownership. All the better if the platform can double in the role of performance monitoring to assure user experience.
