When I started on my current project, I noticed a few failing iOS builds on TeamCity that needed to be fixed. Since I was still on-boarding to the project and wanted to get my feet wet, I took it upon myself to resolve these issues as soon as possible.
B
efore I could run off fixing a bunch of tests, I needed to figure out which tests were failing. This was going to be difficult since the build wasn’t configured to collect any sort of test summary. No big deal, I could always check the build logs to figure out which tests were faili… Wait, how did this build log 115MB of output?!
The build configuration in question was set up to run over 3000 unit, integration, and acceptance tests using Apple’s xcodebuild tool. According to the man page, xcodebuild doesn’t appear to have a -verbose flag, which means you get ALL of the logs, ALL of the time, whether you want them or not.
This was the case for the failing builds I mentioned. Both Chrome and Firefox had issues loading the contents of a 115MB file, which made parsing through the logs quite difficult.
Build Logs Should Be Efficient
If the process or the result of your logging system is harmful to the performance of the system where it operates, you’re going to have a bad time. Build logs should strike a healthy balance between giving you the information you need at a given time and drowning you in so much information that you can’t find the logs you want. At the very least, your build logs should tell you if there was a problem and provide the information you need to resolve it.
In my case, these builds logged so much output that it made the process of finding the few failing tests much more difficult. This wasn’t an efficient use of our build tools or my time.
Build Logs Should Be Effective
In addition to being efficient, build logs still need to serve a “greater purpose” in order to be effective. The types of logs that are important/useful while developing an application aren’t the same as the type of logs that are important/useful once that application is running in a production environment.
The greater purpose for the builds I was using was to report the results of the tests that were executed. Anything other than that could potentially clutter the results and get in the way.
To achieve this, I ended up adding the xcpretty Ruby Gem to the project. This greatly improved the efficiency of the project’s build logs, dropping the previous 115MB log file to a shocking ~1.5MB for subsequent builds!
xcpretty continued to prove its value by providing options to add custom formatters for xcodebuild’s output. This allowed us to format the logs in a way that helped TeamCity parse and generate test summaries for each build, including detecting flaky tests.
Verbose Is Not Always Best
Verbose build logs can be helpful at times, especially when you’re debugging a failing build and need all the information about the system to resolve the issue.
With that being said, verbose mode should not be the default in every case. Your logging mechanisms should be efficient and effective in providing the required information. They should serve a greater purpose and serve that purpose well.
What greater purpose do your logs serve, and what tools/practices have you used to achieve it?