You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
Eduardo Silva ff2507be7c storage: increase default memory limit from 5M to 100M 1 day ago
.github github: workflows: add build/release workflows (#3107) 3 days ago
.gitlab ci: improve GitLab CI/CD Pipeline (#1372) 1 year ago
bin Initial import. 6 years ago
build build: reintroduce build/.empty for convenience 2 years ago
ci ci: disable 'fstore' test on Travis 1 month ago
cmake build: Disable FLB_HTTP_SERVER on Windows build 1 month ago
conf conf: add a parser for IP Tables firewall rules (#3108) 1 day ago
cpack build: CMake: update package info 4 years ago
debian debian: fixes for dpkg-buildpackage (#1634) 9 months ago
doc-reports doc: added fuzzing report 2 months ago
documentation doc: add users: DigitalOcean 4 months ago
examples Dockerfile: Windows container image support (#1801) 1 year ago
gen_static_conf build: conf: fix target definition for generated headers (#1242) 1 year ago
include storage: increase default memory limit from 5M to 100M 1 day ago
init init: systemd: do not prefix configuration file path (#1610) 1 year ago
lib lib: monkey: sync event epoll changes on closing timeout fd 1 day ago
plugins in_winlog: split an identifier into "EventID" and "Qualifiers" (#3121) 2 days ago
scripts scripts: add scripts to override timestamp with float/integer timestamp (#1653) 9 months ago
snap snap: bump to v1.8.0 1 week ago
src input_chunk: honor 'down' state of a forced 'up' chunk 1 day ago
tests tests: internal: fuzzers: engine: fix broken calls. (#3132) 1 day ago
tools/xxd-c tools: xxd-c: include getopt.h through <mk_getopt.h> for portability (#1026) 2 years ago
.dockerignore dockerfile: switch to distroless as base image for container, add .dockerignore (#874) 2 years ago
.gitignore gitignore: add chunkio/cio_version.h (#1984) 1 year ago
.gitlab-ci.yml ci: gitlab: update Dockerfile path (#1414) 1 year ago
.travis.yml ci: travis: exclude always failing job (#3008) 2 weeks ago
CMakeLists.txt build: update description in CMakeLists.txt options (#3104) 4 days ago
CODEOWNERS doc: add CODEOWNERS 10 months ago
CODE_OF_CONDUCT.md docs: add code of conduct (#2018) 11 months ago
CONTRIBUTING.md tests: runtime: out_stackdriver: add test parameter 8 months ago
DEVELOPER_GUIDE.md developer_guide.md : added documentation for config maps 10 months ago
Dockerfile dockerfile: bump to v1.8.0 1 week ago
Dockerfile.debug dockerfile: fix busybox in debug docker image (#1581) 1 year ago
Dockerfile.windows dockerfile: windows: update to v1.7.0 2 weeks ago
Dockerfile.windows.devel dockerfile: Simplify Windows Dockerfiles (#1993) 1 year ago
GOLANG_OUTPUT_PLUGIN.md doc: GOLANG: fix some typos (#1360) 1 year ago
GOVERNANCE.md doc: add GOVERNANCE.md 10 months ago
LICENSE Doc: add reference to Apache License v2.0 6 years ago
MAINTAINERS.md doc: fix maintainer full name 11 months ago
README.Dockerfile.md Dockerfile: Windows container image support (#1801) 1 year ago
README.md doc: update README.md 1 week ago
appveyor.yml ci: download WinFlexBison from GitHub instead of MSYS (#2740) 3 months ago
debian.sh Debian: add packager script 5 years ago
fluent-bit_1.8.0.bb bitbake: bump to v1.8.0 1 week ago
fluent-bit_git.bb bitbake: fix compile error with wrong DEPENDS and fix packaging error with wrong systemd service file (#1606) 1 year ago
fluentbit_logo.png Doc: Add logo to README 4 years ago
update_version.sh scripts: add script to update version 2 months ago
valgrind.supp valgrind: suppress multi-thread (false positive) condition 1 month ago

README.md

logo

Travis CI: Build Status

Fluent Bit is a fast Log Processor and Forwarder for Linux, Windows, Embedded Linux, MacOS and BSD family operating systems. It's part of the Graduated Fluentd Ecosystem and a CNCF sub-project.

Fluent Bit allows to collect log events or metrics from different sources, process them and deliver them to different backends such as Fluentd, Elasticsearch, Splunk, DataDog, Kafka, New Relic, Azure services, AWS services, Google services, NATS, InfluxDB or any custom HTTP end-point.

Fluent Bit comes with full SQL Stream Processing capabilities: data manipulation and analytics using SQL queries.

Fluent Bit runs on x86_64, x86, arm32v7 and arm64v8 architectures.

Features

  • High Performance
  • Data Parsing
  • Reliability and Data Integrity
  • Networking
    • Security: built-in TLS/SSL support
    • Asynchronous I/O
  • Pluggable Architecture and Extensibility: Inputs, Filters and Outputs
  • Monitoring: expose internal metrics over HTTP in JSON and Prometheus format
  • Stream Processing: Perform data selection and transformation using simple SQL queries
    • Create new streams of data using query results
    • Aggregation Windows
    • Data analysis and prediction: Timeseries forecasting
  • Portable: runs on Linux, MacOS, Windows and BSD systems

Fluent Bit in Production

Fluent Bit is used widely in production environments. In 2020 Fluent Bit was deployed more than 220 Million times, and continues to be deploy over 1 million times a day. The following is a preview of who uses Fluent Bit heavily in production:

If your company uses Fluent Bit and is not listed, feel free to open a Github issue and we will add the logo.

users

Documentation

Our official project documentation for installation, configuration, deployment and development topics is located here:

Quick Start

Build from Scratch

If you aim to build Fluent Bit from sources, you can go ahead and start with the following commands.

cd build
cmake ..
make
bin/fluent-bit -i cpu -o stdout -f 1

If you are interested into more details, please refer to the Build & Install section.

Linux Packages

We provide packages for most common Linux distributions:

Linux / Docker Container Images

Our Linux containers images are the most common deployment model, thousands of new installation happen every day, learn more about the available images and tags here.

Windows Packages

Fluent Bit is fully supported on Windows environments, get started with these instructions.

Plugins: Inputs, Filters and Outputs

Fluent Bit is based in a pluggable architecture where different plugins plays a major role in the data pipeline:

Input Plugins

name title description
collectd Collectd Listen for UDP packets from Collectd.
cpu CPU Usage measure total CPU usage of the system.
disk Disk Usage measure Disk I/Os.
dummy Dummy generate dummy event.
exec Exec executes external program and collects event logs.
forward Forward Fluentd forward protocol.
head Head read first part of files.
health Health Check health of TCP services.
kmsg Kernel Log Buffer read the Linux Kernel log buffer messages.
mem Memory Usage measure the total amount of memory used on the system.
mqtt MQTT start a MQTT server and receive publish messages.
netif Network Traffic measure network traffic.
proc Process Check health of Process.
random Random Generate Random samples.
serial Serial Interface read data information from the serial interface.
stdin Standard Input read data from the standard input.
syslog Syslog read syslog messages from a Unix socket.
systemd Systemd read logs from Systemd/Journald.
tail Tail Tail log files.
tcp TCP Listen for JSON messages over TCP.
thermal Thermal measure system temperature(s).

Filter Plugins

name title description
aws AWS Metadata Enrich logs with AWS Metadata.
expect Expect Validate records match certain criteria in structure.
grep Grep Match or exclude specific records by patterns.
kubernetes Kubernetes Enrich logs with Kubernetes Metadata.
lua Lua Filter records using Lua Scripts.
parser Parser Parse record.
record_modifier Record Modifier Modify record.
rewrite_tag Rewrite Tag Re-emit records under new tag.
stdout Stdout Print records to the standard output interface.
throttle Throttle Apply rate limit to event flow.
nest Nest Nest records under a specified key
modify Modify Modifications to record.

Output Plugins

name title description
azure Azure Log Analytics Ingest records into Azure Log Analytics
bigquery BigQuery Ingest records into Google BigQuery
counter Count Records Simple records counter.
datadog Datadog Ingest logs into Datadog.
es Elasticsearch flush records to a Elasticsearch server.
file File Flush records to a file.
flowcounter FlowCounter Count records.
forward Forward Fluentd forward protocol.
gelf GELF Flush records to Graylog
http HTTP Flush records to an HTTP end point.
influxdb InfluxDB Flush records to InfluxDB time series database.
kafka Apache Kafka Flush records to Apache Kafka
kafka-rest Kafka REST Proxy Flush records to a Kafka REST Proxy server.
nats NATS Flush records to a NATS server.
null NULL Throw away events.
s3 S3 Flush records to s3
stackdriver Google Stackdriver Logging Flush records to Google Stackdriver Logging service.
stdout Standard Output Flush records to the standard output.
splunk Splunk Flush records to a Splunk Enterprise service
tcp TCP & TLS Flush records to a TCP server.
td Treasure Data Flush records to the Treasure Data cloud service for analytics.

Contributing

Fluent Bit is an open project, several individuals and companies contribute in different forms like coding, documenting, testing, spreading the word at events within others. If you want to learn more about contributing opportunities please reach out to us through our Community Channels.

If you are interested in contributing to Fluent bit with bug fixes, new features or coding in general, please refer to the code CONTRIBUTING guidelines. You can also refer the Beginners Guide to contributing to Fluent Bit here.

Community & Contact

Feel free to join us on our Slack channel, Mailing List or IRC:

License

This program is under the terms of the Apache License v2.0.

Authors

Fluent Bit is originally made and currently sponsored by Treasure Data among other contributors.