AEGIS Lab

Harnessing the Power of AI Aegis Lab Cyber Data Lake for Advanced Cybersecurity Analytics and Insights

Cyber-Data-Lake-Edited

Challenges

Detecting cybersecurity insider threats requires a vast array of data from a variety of unrefined activity logs, assets, agents, metrics, web apps and services and cloud infrastructures (AWS, Azure, etc). You may hold these in a Security Information & Event Management (SIEM), or they are scattered across your physical and digital IT infrastructure. Integrating and structuring these diverse data in uniform formats suitable for harvesting their valuable insights for cyber threat detection represent a significant challenge for any enterprise or organization.

Features

AI Aegis Lab Cyber Data Lake streamlines system and data integration and processing workflows. The service’s robust data transformation capabilities ensure that raw and complex logs are transformed into a clear and structured format, facilitating analysis and insights extraction.

Solution

AI Aegis Lab Cyber Data Lake is a powerful and comprehensive microservice data management platform for all log and systems activities:

First Layer

The first layer collects, transforms, and consolidates unstructured and complex raw log data, as well as other tool-generated data, and enacts their efficient ingestion and management in a specialized database.

Second Layer

The second layer consists of a specialized database management system for all user behavior features extracted from the raw event log data.

AI Aegis Lab Cyber Data Lake

AI Aegis Lab Cyber Data Lake represents a crucial nexus within your organization’s Cybersecurity behavioral data processing landscape. By leveraging a range of modules a microservices, it offers you a robust, efficient and flexible data management platform for handling all the events, processes, and logs data within your digital ecosystem:

Log Fetching and Conversion

  • 1- Connectors are a set of configurable data fetchers that integrate and extract a range of log data systems, tools, and repositories. We support SIEMs, popular security software and third-party applications.
  • 2- Converters are a set of routines that transform all unstructured and structured logs and monitoring and system data into our universal format and structure.
  • 3- Lightweight Agents gather all user interactions with other entities (end-points, apps, and systems) via a lightweight application which is easily installed on users’ machines.

Normalization

Normalization is a critical process that transforms certain attributes of log data to facilitate accurate user and entity behavior modeling in subsequent stages. This process involves various steps, such as anonymizing personally identifiable information (PII) by excluding sensitive fields. Additionally, normalization includes mapping and consolidating multiple data identifiers of the same nature throughout the data pipeline. For instance, if a user has multiple IDs across different domains and applications, normalization ensures that these identifiers are properly mapped and consolidated for comprehensive analysis and modeling purposes. By performing normalization, organizations can enhance data consistency, integrity, and privacy, laying the foundation for robust cybersecurity analytics and insights extraction.

Log Enrichment

Log Enrichment is a process that enhances the data object by incorporating additional information as new fields. For example, this enrichment may involve adding a corresponding geotag to an IP address. By enriching logs with supplementary data, organizations can gain deeper context and insights into the events and activities recorded in the logs. Log Enrichment plays a crucial role in augmenting the data object with valuable details, enabling organizations to extract meaningful insights and make informed decisions based on the enriched log data.

Event Database

Event Database is an optimized database specifically designed to store and manage converted cybersecurity event logs and records in a converted and normalized format.

Feature Database

Feature Database is an optimized database specifically designed to store and manage all behavioral features extracted from events’ logs.

API

API is a versatile service responsible for publishing and transporting all data (captured logs or computed features) across different processing layers.

Cyber Security Data Lake Diagram

AI Aegis Lab Cyber Data Lake supports

AI Aegis Lab Cyber Data Lake supports a wide range of common SIEMs, network and application logs:

Internet services.

EMails, FTP, Ftpd, Proftpd, Pureftpd. Proxy, NGINX, Squid. Connection,VPN, IPSec, SSH (SCP). HTTP Server, Apache, NGINX. CICD, puppet, jenkins, gitlab.

Security services.

Firewalls, UTMs, like checkpoint, fortigate, imperva, Sophos. LDAP. IDS, e.g., dragon nids, Snort, Suricata, OSSEC. Antiviruses, like Symantec, ClamAV

Operating systems.

Windows, Linux, MacOS, BSD

Databases:

MySQL, MongoDB, Oracle, SQL Server, Postgresql, Redis

Web applications.

CMS, WordPress, any other in-house web apps

Cloud Logs:

Includes popular clouds, AWS, Azure, GCP, and more

Agent-based logs.

File monitoring (Access/Open/Copy/Move/Share). Process and app monitoring. Monitoring of web surfing through browser extension.

Network.

Network traffic packets from endpoints, switches, routers, and other network devices.

Dive into Advanced Analytics with AI Aegis Lab Cyber Data Lake

Elevate your cybersecurity data management, unlock powerful insights, and fortify your defences. Begin your enhanced cyber analytics journey today!