Data Resiliency

As data flows through the event processing pipeline, Logstash may encountersituations that prevent it from delivering events to the configuredoutput. For example, the data might contain unexpected data types, orLogstash might terminate abnormally.

To guard against data loss and ensure that events flow through thepipeline without interruption, Logstash provides the following data resiliencyfeatures.

  • Persistent Queues protect against data loss by storing events in aninternal queue on disk.
  • Dead Letter Queues provide on-disk storage for events that Logstash isunable to process. You can easily reprocess events in the dead letter queue byusing the dead_letter_queue input plugin.

These resiliency features are disabled by default. To turn on these features,you must explicitly enable them in the Logstash settings file.