Category: Security
Centralised logging with Graylog or OpenSearch: searchable, retained logs
Logs from servers, applications, firewalls and clouds in a single place, with normalised fields, fast searches and a retention period you can justify in an audit.
- Where it runs
- In your account, in your name
- Backups
- With tested restores
- Updates
- Planned and documented
- Licence
- Graylog Open: SSPL. OpenSearch: Apache 2.0
In short
Centralising logs means bringing the logs of all your systems to a single place where they can be searched, retained and monitored. We set it up with Graylog, which adds streams, processing rules and alerts on top of OpenSearch, or with OpenSearch and its dashboards directly. We take care of collection, normalisation, retention, backups, ingestion monitoring and upgrades, in your account.
What do we look after?
Having logs scattered across each server works until the day you have to reconstruct what happened: the server has already rotated its files, the cloud kept seven days or the attacker deleted whatever gave them away. A central system prevents that, as long as someone maintains it:
Collection
Agents and encrypted syslog inputs from servers, containers, network and cloud.
Normalisation
Rules that extract the same fields from different sources so you can search them all at once.
Storage
Properly sized OpenSearch indices, with rotation and retention per type of log.
Monitored ingestion
Alerts when a source goes quiet, when the volume spikes or when the input queue grows.
Backups and archive
Snapshots to object storage and test restores of old indices.
Access
Single sign-on, roles per team and a record of searches.
Which architecture do we recommend?
In Graylog, the piece that receives and processes is the Graylog server, which keeps its configuration in MongoDB, and the piece that stores is OpenSearch, which the Graylog Data Node installs and manages for you. Without Graylog, the agents send to OpenSearch and processing happens in them or in an intermediate layer. In both cases, ingestion goes through a point with a queue so that no logs are lost if storage is slow.
| Situation | What we set up | Why |
|---|---|---|
| Dozens of servers and a small team | Graylog and Data Node on one node, with snapshots elsewhere | Everything in one product, simple to run |
| High volume or long retention | Graylog on two nodes and OpenSearch on three or more | Storage is what grows most |
| You already have OpenSearch or a managed service | Fluent Bit or Vector into OpenSearch and its dashboards | Makes use of what is there, with no extra layer |
Running the search cluster itself, with memory, shards and upgrades, is covered in managed Elasticsearch and OpenSearch.
Which problems do we come across most often?
- Sources that stopped sending weeks ago without anyone noticing.
- Everything in one index: debug and audit logs with the same retention and the same permissions.
- Different fields for the same thing: the source IP is named in five ways and no search brings them together.
- Full disk: with no retention, the store stops accepting logs at exactly the worst moment.
- Unencrypted, unauthenticated syslog crossing networks that are not yours.
- Personal data or secrets in the logs that nobody reviewed before keeping them for months.
How do we handle backups and upgrades?
Graylog’s configuration lives in MongoDB, which we back up separately; the rules and streams are also exported to your repository. The indices are backed up with snapshots to object storage, and from time to time we restore an old one in a separate environment to check that it can be searched. To upgrade, we follow the order and the compatible versions that Graylog publishes for the server, MongoDB and OpenSearch, rehearse first in testing and check that the processing rules still produce the same fields.
Who does it make sense for?
- A company preparing for an audit that needs to show it retains and protects access and change logs.
- A technical team with many servers and services that wastes time logging into each machine over SSH to read logs.
- Anyone who already has a SIEM and wants to keep the full context for longer and at a lower cost.
For threat detection on those same systems, see managed Wazuh. If what you need is application logs alongside metrics, monitoring with Grafana and Prometheus, with Loki, is a better fit.
How do we protect the logs themselves?
A logging system stores exactly what an attacker would want to delete or read. The agents send over TLS, the dashboard is not exposed to the internet and access is through single sign-on. Each team sees only its own streams or indices, and searches are recorded. Snapshots go to object storage in another account, with object lock if the provider offers it, so that they cannot be deleted from the platform. Before storing anything, we review with your team which fields carry personal data or secrets and filter or mask them at input.
- SourcesServers, network, cloud
- AgentsFluent Bit, Vector or syslog TLS
- GraylogInputs, rules and streams
- OpenSearchIndices with retention
- ArchiveSnapshots on S3
Where do we set it up?
In your account, with the provider you already use or the one that fits best. If the data has to stay in the EU, Hetzner and OVHcloud are the European options.
Providers we set it up on
- Hetzner
- AWS
- Google Cloud
- Azure
- OVH
- Own server
| Provider | How we set it up | Things to bear in mind |
|---|---|---|
| Hetzner | Cloud or dedicated servers for Graylog and the data nodes, with NVMe disks and snapshots on Hetzner Object Storage. | Plenty of disk for little cost, which is what a logging system needs most. |
| AWS | Graylog on EC2 or Amazon OpenSearch Service as the store, reading CloudTrail, VPC Flow Logs and CloudWatch Logs. | If there are only AWS logs, well-organised CloudWatch Logs is sometimes enough; we assess that first. |
| Google Cloud | Compute Engine with Graylog or OpenSearch and Cloud Logging export through Pub/Sub. | |
| Azure | Virtual machines with Graylog or OpenSearch, reading the activity logs and Microsoft Entra ID logs through Event Hubs. | |
| OVH | Public Cloud instances or dedicated servers with Graylog and OpenSearch, or OVHcloud managed OpenSearch as the store. | |
| Own server | Graylog and OpenSearch on your servers or virtual machines, receiving from the whole internal network. | The logging system should not live on the same machine it monitors: if that machine is compromised, the evidence is lost. |
What is included
- Graylog with its Data Node, or OpenSearch with OpenSearch Dashboards, in your account and sized for your daily volume.
- Collection from Linux and Windows servers, containers, network devices and your cloud logs, with agents or syslog over TLS.
- Field normalisation (source, user, IP, action, result) so that logs from different sources are searched the same way.
- Separate streams or indices per type of log, each with its own retention and permissions.
- Retention agreed in writing: a few months searchable and longer archived in object storage.
- Alerts on ingestion itself: sources that stop sending, abnormal spikes, disk and input queues.
- Backups of the configuration and index snapshots, with a tested restore.
- Access through single sign-on, roles per team and a record of who searches what.
Frequently asked questions
Is this the same as Wazuh?
No. Wazuh is a SIEM: its agents look for threats, review configurations and monitor file integrity. A centralised logging system stores all logs and makes them searchable, including those that are not about security, such as application or network logs. They often coexist: Wazuh detects and Graylog or OpenSearch keep the full context.
Graylog or OpenSearch with its dashboards?
Graylog provides what is needed to work with logs every day: inputs for almost any format, processing rules, streams with permissions and alerts. OpenSearch with OpenSearch Dashboards is more flexible and fully Apache 2.0, but you have to handle processing beforehand, in the agents. If you want a product ready for the team, Graylog; if you already work with OpenSearch, its dashboards.
What about Graylog's licence?
Graylog Open is published under the SSPL licence, which is not approved by the OSI as open source. Using it inside your company is not a problem; its restriction affects those who offer Graylog as a service to third parties. Some features, such as built-in archiving, belong to the paid editions, and we tell you beforehand if you would need any of them.
How long do logs have to be kept?
It depends on your legal and contractual obligations and on your sector, and it is not our decision. The usual approach is to separate a short period that can be searched instantly from a longer one in cheaper object storage. Whatever we agree is written down and applied with index policies, so that the disk does not decide for you.
Would Loki not be simpler?
For application logs that are queried alongside metrics, often yes. Loki indexes only labels, takes up little space and fits with Grafana. For free-text searches on any field, audit reports or permissions per type of log, Graylog or OpenSearch do a better job.
Tell us about your case
In a free 20-minute call we look at where you are and where to start. No commitment.
Official documentation consulted
The technical details on this page are based on:
Reviewed on