The pipeline management feature centralizes the creation andmanagement of Logstash configuration pipelines in Kibana.
Centralized pipeline management is an X-Pack feature that is not includedwith the basic license. If you want to try all of the features, you can start a30-day trial. At the end of the trial period, you can purchase a subscription tokeep using the full functionality of the X-Pack components. For moreinformation, see https://www.elastic.co/subscriptions andLicenseManagement.
You can control multiple Logstash instances from the pipeline management UI inKibana. You can add, edit, and delete pipeline configurations. On the Logstashside, you simply need to enable configuration management and register Logstashto use the centrally managed pipeline configurations.
After you configure Logstash to use centralized pipeline management, you canno longer specify local pipeline configurations. The pipelines.yml
file andsettings such as path.config
and config.string
are inactive when centralizedpipeline management is enabled.
Before using the pipeline management UI, you must:
logstash_admin
role as well as the logstash_writer
role that you createdwhen you configured Logstash to use basic authentication. Additionally,in order to view (as read-only) non-centrally-managed pipelines in the pipeline managementUI, make sure your Kibana user has the monitoring_user
role as well.To manage Logstash pipelines in Kibana:
Open Kibana in your browser and go to the Management tab. If you’ve set upconfiguration management correctly, you’ll see an area for managing Logstash.
To add a new pipeline, click Create pipeline and specify values.
Pipeline ID |
A name that uniquely identifies the pipeline. This is the ID that you used whenyouconfigured centralized pipeline managementand specified a list of pipeline IDs in the |
Description |
A description of the pipeline configuration. This information is for your use. |
Pipeline |
The pipeline configuration. You can treat the editor in the pipeline managementUI like any other editor. You don’t have to worry about whitespace or indentation. |
Pipeline workers |
The number of parallel workers used to run the filter and output stages of the pipeline. |
Pipeline batch size |
The maximum number of events an individual worker thread collects beforeexecuting filters and outputs. |
Pipeline batch delay |
Time in milliseconds to wait for each event before sending an undersizedbatch to pipeline workers. |
Queue type |
The internal queueing model for event buffering. Options are memory forin-memory queueing, or persisted for disk-based acknowledged queueing. |
Queue max bytes |
The total capacity of the queue. |
Queue checkpoint writes |
The maximum number of events written before a checkpoint is forced whenpersistent queues are enabled. |
apache
) in Kibana, Logstash willattempt to stop the pipeline. Logstash waits until allevents have been fully processed by the pipeline. Before you delete a pipeline,make sure you understand your data sources. Stopping a pipeline maylead to data loss.