Skip to main content
Fluent Bit collects logs from a device and sends them to an external storage. The Fluent Bit configuration file defines a log source — such as a specific service, internal storage, TCP port, or OS events — and a destination for log export (Gcore Managed Logging servers). Once configured, Fluent Bit works automatically: logs recorded in the system are transferred to Gcore Managed Logging and appear on OpenSearch Dashboards.

​
Install

Fluent Bit’s official documentation provides up-to-date installation guides for different operating systems. Open this link and select your OS. Simply follow the steps in the relevant installation manual.

​
Configure

The Fluent Bit configuration file consists of two sections: INPUT and OUTPUT. INPUT determines the log source, and OUTPUT sets the destination. To set up the configuration file:
  1. Open the fluent-bit.conf file and add the following data:
[INPUT]  
    Name        tail
    Path        /var/log/syslog
[OUTPUT]   
    name        kafka    
    brokers       laas-example.gcore.com:443  
    topics        yourlogin.yourtopic   
    rdkafka.security.protocol        SASL_SSL    
    rdkafka.sasl.mechanism        SCRAM-SHA-512   
    rdkafka.sasl.username       yourlogin
    rdkafka.sasl.password       yourpassword
Replace the placeholder values:
  • tail: Data source type
  • /var/log/syslog: File path to monitor
  • laas-example.gcore.com:443: Kafka endpoint — find this on the Managed Logging page in the Gcore Customer Portal
  • yourlogin.yourtopic: Username from the Managed Logging page and topic name, separated by a dot (.)
  • yourlogin: Username from the Managed Logging page
  • yourpassword: Password from the Managed Logging page
For other log sources, see the “Input” section of the Fluent Bit documentation and select the relevant input type. For example, to collect from a specific file, follow the Tail input guide and configure the INPUT section accordingly.
  • [OUTPUT]: Type of process (export).
    • name: Servers where logs will be delivered (Kafka servers).
    • brokers: Server(s) where logs will be exported.
    • topics: Topic(s) where logs will be exported.
    • rdkafka.security.protocol: Security protocol that encrypts data in transit.
    • rdkafka.sasl.mechanism: Authentication mechanism used to verify the username and password for the logs storage.
    • rdkafka.sasl.username: Username used to verify the sender.
    • rdkafka.sasl.password: Password used to verify the sender.
  1. Save the changes in the fluent-bit.conf file.
  2. Restart Fluent Bit, and it will begin sending logs to the Gcore Managed Logging.

​
No timestamps

Some services (for example, nginx) transmit timestamps of logs in a non-standard format. In this case, OpenSearch Dashboards will show logs without their date and time. If this happens, add the following line inside the [OUTPUT] section. This instructs Fluent Bit to forward timestamps in ISO 8601 format, which is compatible with services like nginx. 
timestamp_format iso8601