Skip to main content
Logstash is an open-source data collection and processing tool. It’s used to collect, parse, enrich, and transform log data from various sources, including log files, event streams, and databases.

​
Install Logstash

Download Logstash from the official Elastic website and install it per the instructions.

​
Configure Logstash

A standard Logstash configuration consists of three sections: input, filter, and output. The input and filter sections depend on the sources of logs. To send logs to Gcore Managed Logging, configure Logstash with Kafka output and enable the Kafka Integration Plugin in your Logstash installation. 1. Configure Logstash with Kafka output by adding the following data to the logstash.conf file: 
output {
        kafka {
          codec => json
          topic_id => "yourlogin.yourtopic"
          bootstrap_servers => "endpoint"
          sasl_mechanism => "SCRAM-SHA-512"
          security_protocol => "SASL_SSL"
          sasl_jaas_config => "org.apache.kafka.common.security.scram.ScramLoginModule required username='yourlogin'password='yourpassword'; "
          key_serializer => "org.apache.kafka.common.serialization.StringSerializer"
          value_serializer => "org.apache.kafka.common.serialization.StringSerializer"
        }
    }
2. Customize the highlighted values:
  • yourlogin.yourtopic: Username from the Managed Logging page and topic name, separated by a dot (.)
  • endpoint: Kafka endpoint — find this on the Managed Logging page in the Gcore Customer Portal
  • yourlogin: Username from the Managed Logging page
  • yourpassword: Password from the Managed Logging page
Username, password, and topic name are available in the Gcore Customer Portal on the Managed Logging page. For configuration details, see Configure logging and view logs.
For more settings, check out the Kafka output plugin documentation.
  • topic_id: ID of the Kafka topic to which Logstash will publish messages.
  • bootstrap_servers: A comma-separated list of Kafka broker addresses.
  • sasl_mechanism: Authentication mechanism used to verify the username and password for the logs storage.
  • security_protocol: Security protocol used to encrypt data in transit.
  • sasl_jaas_config: JAAS configuration required for SASL authentication.
  • key_serializer: Serialization format used to convert message keys sent to Kafka.
  • value_serializer: Serialization method applied to message content before transmission to Kafka.
3. Save the changes in the Logstash configuration file. 4. Restart Logstash. Logstash will start sending logs to Gcore Managed Logging.