Skip to main content

​
Overview

Managed Logging, also known as LaaS (Logging as a Service), is a service that collects and stores logs from Virtual Machines and Bare Metal servers. It works with any log source — Gcore infrastructure or external servers — and provides access to collected logs through OpenSearch Dashboards.

​
Setup overview

  1. Install and configure a log shipper that supports Kafka: Fluent Bit, Filebeat, Logstash, or any other source that exports logs to Kafka (including a Python application with Kafka logger output).
  2. Access the collected logs via OpenSearch Dashboards.

​
Advantages

Logging compiles logs from different services into one system. To find the cause of an error or collect data for analytics, there is no need to open logs of each service or machine individually — all data is available in OpenSearch Dashboards with customizable views and filters. Logs remain available even when their source is gone — for example, when a server goes down or a Kubernetes pod is deleted. The Logging tab shows what happened to a machine before its failure or to an object before its deletion.

​
Use cases

The use cases of Managed Logging include:
  1. Error detection and troubleshooting. Enter an error message into the OpenSearch Dashboards search bar to identify when and which server experienced an issue.
  2. Investigation of security incidents. Monitoring and analyzing logs enables detection of suspicious activities, unauthorized access attempts, or other potential security breaches.
  3. Server connectivity check. Open logs of two servers for the same time interval and see whether all data has reached an end user.
  4. Data speed measurement. Open logs and compare the time when one server sent the data with the time when another one received it.

​
Log storage

Logs are exported to Kafka servers, which then send them to OpenSearch servers for permanent storage. Logs are accessible through OpenSearch Dashboards. Kafka servers act as a buffer for reliability. If an OpenSearch server fails to receive logs, they are saved in Kafka and forwarded to OpenSearch when the connection is restored. This architecture also protects data integrity during transmission.

​
Restrictions

Managed Logging enforces the following restrictions to maintain system performance:
  1. Message format: text/JSON
  2. Maximum fields in a JSON log: 225 fields
  3. Maximum message size: 1 MB