Sending logs from Docker to centralised log server

Docker logs are not handled through rsyslog (at least not by default), since I am using Graylog, I use a version of logspout which has been built with a GELF adapter.

Step 1 - Add container to docker-compose

For my purposes I am using logspout to send logs from my home assistant docker deployment, but the same configuration can be used along with any other docker setup.

  logspout: # chose to use this instead of setting logging levels on each container
    container_name: logspout
    image: vincit/logspout-gelf
    hostname: ha-docker
    restart: unless-stopped
    environment:
      - TZ=Europe/Dublin
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command:
      - "gelf://${LOGSPOUT_SERVER}:12201"

hostname will be the name that will show up as the source in graylog.

LOGSPOUT_SERVER should be the address of the graylog server. Alternitively it can be set in .env as I have here. Note that if Graylog is on the internal proxmox network behind PfSense, this should be the PfSense WAN address and the port should be forwarded to the graylog server <guide to come>

Step 2 - Add Input to Graylog

Log in to Graylog and select Systems/Inputs -> Inputs

Select GELF UDP -> Launch new input

Ensure Bind address is 0.0.0.0 (so logs can be received from any server) and that the port is set to 12201

Docker logs should now show up in the search dashboard, to test it you can do something like turn a light on and off (which will trigger zigbee2mqtt) or enter a password wrong to generate a log message and check that it is showing up.

Step 3 - Searching docker logs

Since all the logs will come from the same source, we can search with the tag container_name: <name> to see logs from individual containers.

We can also create a saved search that will show extra columns relevant to docker.

Enter in a new search

source: <docker hostname>

Click the 'Fields' icon on the left bar and select 'container_name' and select 'add to all tables' and 'Show top values'

 Select 'Save' to the right of the search bar and give it a name.

The search can now be loaded when needed, the page can also be bookmarked for easy access.

Step 4 - Extracting log levels

<to add>