Filebeat is part of the Elastic Stack, and is used to parse and ship logs to Logstash, Elasticsearch, and Kibana. If you are like me you may have a multitude of Raspberry Pis running, and doing things where monitoring logs in something like Elastic would be cool. This guide aims to provide a walk-through on compiling filebeat on 32bit arm architectures.

Elastic provides Filebeat packages for different platforms and architectures, but sadly not the armhf/armv7l architecture that Raspberry Pis use. No worries, we’ll build our own! Compiling this on rpi is challenging so you can download the package(s) that I build if you want.

TLDR You can do some tricks to compile beats to work on 32 bit raspberry pis. I compile this regularly for my lab, feel free to use my latest copies:

General steps:

  1. Install latest version of go
  2. Clone source
  3. build
  4. Copy armv7 elf into a package that elastic provides
  5. zip it up
  6. ….
  7. profit!

Install latest go version locally.

NOTE: The golang version that comes stock with raspbian is olllld. Purge previous go versions or you will have a bad time. I like to use SNAP, because it is simple to install and auto updates.

sudo apt install snapd
# reboot system
sudo init 6
# log back into system
sudo snap install core
sudo snap install go --classic

download source

go get -d -v github.com/elastic/beats
mkdir -p $GOPATH/src/github.com/elastic
# If not already cloned
cd  $GOPATH/src/github.com/elastic
git clone https://github.com/elastic/beats.git
#if already cloned
cd  $GOPATH/src/github.com/elastic/beats
git pull origin master

compile

This step when run on a raspberry pi using 32bit raspian compiles a arm7l binary we will use to “patch” an official package with a binary which will run on our pis. On my raspberry pi 3b+ it takes about 5-10 minutes.

export VERSION=7.15.1
cd $GOPATH/src/github.com/elastic/beats/filebeat
git checkout 7.15
git checkout tags/v${VERSION}
cd filebeat
make

package it up

Since arm7l (armhf) isn’t officially supported by Elastic, we need to copy our freshly built executable to a nice clean distro from another architecture. After that we can package it up.

cd ~ && mkdir tmp
cd ~/tmp
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-${VERSION}-darwin-x86_64.tar.gz
tar xzvf filebeat-${VERSION}-darwin-x86_64.tar.gz
mv filebeat-${VERSION}-darwin-x86_64 filebeat-${VERSION}-rpi-arm7l
cp $GOPATH/src/github.com/elastic/beats/filebeat/filebeat filebeat-${VERSION}-rpi-arm7l
#Filebeat wants files to be owned by root
sudo chown -R root:root filebeat-${VERSION}-rpi-arm7l
sudo tar -cJf filebeat-${VERSION}-rpi-arm7l.tar.xz filebeat-${VERSION}-rpi-arm7l

install on your rpi

I like to install in /opt and create symlinks to newer versions this way I don’t stomp over old configurations and can revert if I want. You can also copy the xzipped tarball over to your other raspberry pis and unzip it.


cp filebeat-${VERSION}-rpi-arm7l.tar.xz /tmp
sudo su
export VERSION=7.15.1
cd /opt
tar -xJvf /tmp/filebeat-${VERSION}-rpi-arm7l.tar.xz
#copy filebeat configs from old version if it exists (optional)
cp filebeat/filebeat.yml filebeat-${VERSION}-rpi-arm7l
cp filebeat/modules.d/*.yml filebeat-${VERSION}-rpi-arm7l/modules.d/
# remove symlink if it exists
rm filebeat
# create new symlink to new version.
ln -s  filebeat-${VERSION}-rpi-arm7l filebeat

edit filebeat.yml with your particulars

Assuming you setup a local DNS entry for your elastic cluster called “elastic.local”, you will need to change that domain to whatever you have setup in your environment.

enabled: true
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: [elastic.local:9200"]
setup.kibana:
  # Array of hosts to connect to.
  hosts: ["elastic.local:5601"]

enable modules

./filebeat modules list
./filebeat modules enable system nginx

You will have to load ingest pipelines for some modules

You only need to do this one system after upgrading or installing fresh. The following command takes care of updating/installing index management and dashboards as well. The example below loads pipelines, dashboards and modules in oneshot.

sudo ./filebeat setup -e --index-management \
  --modules system,nginx,apache,threatintel \
	--pipelines \
  --dashboards \
	-E output.logstash.enabled=false \
	-E 'output.elasticsearch.hosts=["elastic.local:9200"]' \
	-E 'setup.kibana.host="elastuc.local:5601"'

setup a systemd filebeat service

You will likely want to have filebeat running as a service to ensure it comes back up on reboot. Here is how to set it up in systemd.

touch /etc/systemd/system/filebeat.service
chmod 644 /etc/systemd/system/filebeat.service
vi /etc/systemd/system/filebeat.service

And paste the following contents into the filebeat.service file.

#
# filebeat systemd service
#

[Unit]
Description=Filebeat
Documentation=https://www.elastic.co/guide
After=network.target

[Service]
Type=simple
Restart=always
WorkingDirectory=/opt/filebeat
ExecStart=/opt/filebeat/filebeat -c /opt/filebeat/filebeat.yml

[Install]
WantedBy=multi-user.target

Start filebeat service

systemctl daemon-reload
systemctl enable filebeat.service