SonarC Architecture

SonarC is a system for storing, managing and providing access to security data and events. It is a cloud-based Big Data system running on clouds or on-prem utilizing object stores for long term retention. The system has built-in analytics, search, and machine learning capabilities. Data can be ingested using a variety of methods including syslog, Logstash, CSV files and more.

SonarC includes the following components:

  • The SonarW NoSQL Data Warehouse.
  • SonarC cloud infrastructure allowing you to store data efficiently and transparently on cloud storage and on-prem object stores.
  • The SonarGateway ingestion layer.
  • The SonarCollector ETL layer.
  • The SonarC GUI.
  • The SonarK discovery GUI (based on Kibana).
  • SonarSQL, providing SQL access to data stored within SonarW.
  • JSON Studio providing a GUI for advanced analytic query building and visualization.
_images/arch1.png

SonarC is a software package that is installed on a RHEL Linux server. SonarC can run on a physical server or as a virtual machine. SonarC can be installed as the only application on the server or co-located with other applications. However, due to the nature of the SonarC Big Data workloads, SonarC is a resource-intensive application and consumes all resources available to it - compute, memory and I/O. It is therefore recommended to run SonarC on its own server.

System Sizing

A single SonarC node is usually used for up to 30TB of compressed data. You can store more than 30TB on a single node and reporting times may still be reasonable but you can also cluster multiple SonarC nodes to provide faster response times. Consult your SonarC account manager for additional sizing guidelines.

Each SonarC node should have the following specs:

  • Two Intel Xeon processors, at least 6 cores per socket, at least 2.4Ghz each.
  • At least 64GB of memory.
  • Either HDD or SSD drives. In both cases, and especially when using HDDs, the drives should be striped using RAID0 or RAID10. For example, you can choose to use SATA drives. In this case you should create a single RAID array using at least four such disks to give you the ability to read at a rate nearing 500MB/s. The system has been optimized to allow leveraging low-cost SATA drives in order to achieve the most cost-effective large data store using inexpensive drives.
  • At least one SSD of size ~400MB used for temporary storage for SonarW.

If you are deploying SonarC on an Amazon AWS EC2 instance, SonarC recommends using a m4.4xlarge instance or an m4.10xlarge instance if workloads are expected to be very large. An io2 Amazon Elastic Block Store (EBS) volume with at least 10K-12K IOPS is recommended since this will allow you to grow the volume as your data size grows with no changes to the SonarC application or to RHEL. If you choose general purpose EBS then you should use a RAID0 configuration.