Data Processing with Logstash (and Filebeat)

About This Course

Process data with Logstash, which is a key part of the ELK stack (Elasticsearch, Logstash, Kibana) and Elastic Stack.

Want to learn how to process events with Logstash? Then you have come to the right place; this course is by far the most comprehensive course on Logstash here at Udemy! This course specifically covers Logstash, meaning than we can go into much more detail than if this course covered the entire Elastic Stack. So if you want to learn Logstash specifically, then this course is for you!

This course assumes no prior knowledge of or experience with Logstash. We start from the very basics and gradually transition into more advanced topics. The course is designed so that you can follow along the whole time step by step, and you can find all of the configuration files within a GitHub repository. The course covers topics such as handling Apache web server logs (both access and error logs), data enrichment, sending data to Elasticsearch, visualizing data with Kibana, along with covering a number of popular use cases that you are likely to come across. Upon completing this course, you will know all of the most important aspects of Logstash, and will be able to build complex pipeline configurations and process many different kinds of events and data.

What is Logstash?
In case you don't know what Logstash is all about, it is an event processing engine developed by the company behind Elasticsearch, Kibana, and more. Logstash is often used as a key part of the ELK stack or Elastic Stack, so it offers a strong synergy with these technologies. You can use Logstash for processing many different kinds of events, and an event can be many things. You can process access or error logs from a web server, or you can send events to Logstash from an e-commerce application, such as when an order was received or a payment was processed. You can ingest data from files (flat files, JSON, XML, CSV, etc.), receive data over HTTP or TCP, retrieve data from databases, and more. Logstash then enables you to process and manipulate the events before sending them to a destination of your choice, such as Elasticsearch, OpenSearch, e-mail, or Slack.

Why do we need Logstash?
Because by sending events to Logstash, you decouple things. You effectively move event processing out of the web application and into Logstash, representing the entire data pipeline, or perhaps just a part of it. This means that if you need to change how events are processed, you don’t need to deploy a new version of a web application, for instance. The event processing and its configuration is centralized within Logstash instead of every place you trigger events. This means that all the web application needs to do, is to send an event to Logstash; it doesn’t need to know anything about what happens to the event afterwards and where it ends up. This improves your architecture and lets Logstash do what it does best; process events.

Let's get started!
I hope that you are ready to begin learning Logstash. Have a look around the curriculum if you want to check out the course content in more details. I look forward to seeing you inside the course!

• Understand the fundamental concepts of Logstash

• Build pipelines that process and manipulates thousands of events

• Send data to Logstash from numerous sources and to several destinations