Are you still using plain text files to collect logs for your application? I was too until recently, but no more. While plain text logging is simple, it has a number of limitations. The good news is there are a number of superior logging alternatives available, including two that I have used personally—Loggly and Graylog.
No one likes dealing with infrastructure like logs, but it is an important part of any production system. And when the time comes that you have to dig into your logs to work through a hairy issue, you’ll be massively happier if it doesn’t involve digging through a dozen text files spread across four different machines.
Plain text logs are dead simple, but they have a number of inherent issues.
Searchability
Plain text logs are minimally searchable, requiring command line tools like grep. This is tedious, and it is hard to get a complete picture by searching this way because you have to look at events one at a time. Both Loggly and Graylog support robust, full text searching of log messages, allowing you to easily filter logs by both time and subject.
This is invaluable, as it enables you to quickly target a particular log entry you want and to easily find other messages like that.
Centralization
Plain text logs are generally recorded separately per machine and application rather than in one place. In an even mildly complex system with a number of different components and servers, this can make finding and diagnosing logs a nightmare of piecing together different files and cross-referencing timestamps. Loggly and Graylog both solve this through their separate, centralized systems that collect logs from all of your different subsystems.
Access and Permissions
This ties into a lot of the same problems as decentralization. If logs are spread out among different systems, then every user who might need access to those logs needs to have access to those systems. This can be a security management nightmare. It is very common for logs to be useful to developers for fixing reported bugs, but often, those same developers should not have direct access to sensitive production systems and data.
Having centralized logs allows you to manage those roles separately from the systems that are generating the logs. Both Loggly and Graylog also support even more fine-grained permission management internally, even allowing you to restrict which subsets of logs a certain user can see.
Active Notifications
Text logs are passive. You can root-cause and post-mortem issues from them, but they don’t actively notify you when issues start to come up. Active logs are a massive advantage here, serving as another whole level of system metrics. Instead of relying on users to tell you that a system is giving an error, you can be actively notified by your logging system in real time as unexpected issues come up. You can also use this feature to identify things like higher than expected load or response times, and any other traditional metrics.
Analytics
You can’t look at a text log file and easily gain insights like the average number of errors you get per day. Loggly and Graylog make insights like this trivial, and they even have a number of them built in or easily configurable. Metrics like this help you use logs to actively, rather than passively, manage your system.
Graylog vs. Loggly
I’ve mentioned the two centralized logging systems I’ve used—Loggly and Graylog—throughout this post, but I haven’t gone into what makes those two systems different.
Loggly is a SaaS provider of centralized log management. They offer various tiers of service and features from a free plan all the way up to custom enterprise plans. Graylog is a free and open-source log management system that is designed to be run and managed yourself. Honestly, not much comparison is needed because the systems are so similar in capabilities. The main differences are how much time and money you are willing to invest in managing logs, and whether you trust log data to a third-party provider.
Loggly has a far lower barrier of entry. It just requires integrating a Loggly client library or http logger into your application and setting a few options. The downsides are that you have to pay for that convenience and accept limitations like low retention lengths. Depending on your organization, hosting logs on an external service may also be an issue.
Graylog is free and open-source, which instantly gives it more flexibility than a paid proprietary service. In general, Graylog is much more flexible and feature-complete than any other offering out there, and it has a better interface. However, this comes at the cost of having to deploy, host, and manage it all yourself, which can be a big ask for a small team. And Graylog is not the easiest system to deploy either. Even the minimal setup requires a number of different dependent subsystems to be set up properly, including Elasticsearch and MongoDB.
Don’t Settle for Plain Text Logs
There is no reason for most projects to use plain text logs. There are easily available systems out there that offer orders of magnitude more value at a comparably low cost.
Another great open-source, run-it-yourself option is what’s known as the ELK stack — Elasticsearch, Logstash, and Kibana. We recently got this trio running for our products where I work, and it has been a revolution for our operations staff!