Why think about logging?

Reading about "How to log" is like watching paint dry. Get over it - doing this right can make a huge difference in your application.

Typically, software developers write to log files to help themselves debug or troubleshoot problems in their code. When a problem occurs, the developer can see what was going on before, during, and after the problem -- but their logs are only useful to them.

Semantic Logging

On the other side of the spectrum, there are instances when developers log data for a much different purpose:

• Web server clicks
• Financial trades
• Cell-phone connect/disconnects
• Audit trails

These are all instances of "semantic" level events. These events record the state of business processes, which are critical to the company running the software. Typically the data contained in these events are statistically analyzed by applications that must be written, generally from scratch. Then that analysis is displayed on hand-built dashboards, useful for historical comparisons, future projections, performance tuning, billing, security, etc.

Don't reinvent the wheel

With Splunk, you can use the logs generated from your application to achieve the same powerful analytics without building special-purpose software. Logging semantic data in addition to debugging data can significantly enrich virtually all applications. Developers can give their customers a whole new level of insight into their business, with very little work. It's really amazing what important information you can derive from seemingly simple event logs.

Take the following simple pseudo-code as an example:

    void submitPurchase(purchaseId)
    {
       log.info("action=submitPurchaseStart, purchaseId=%d", purchaseId)
       //these calls throw an exception on error
       submitToCreditCard(...)
       generateInvoice(...)
       generateFullfillmentOrder(...)
       log.info("action=submitPurchaseCompleted, purchaseId=%d", purchaseId)
    }

With Splunk, the following analytics can be easily performed using just the two events generated in the above pseudo-code:

• Graph purchase volume by hour, by day, by month.
• How long are purchases taking during different times of the day and different days of the week?
• Are purchases taking longer than they did last month?
• Are my systems getting slower and slower, or are they ok?
• How many purchases are failing? Graph the failures over time.
• Which specific purchases are failing?

Each one of these results and questions would be a simple search, which can be saved, scheduled, and emailed out, if desired.

In fact, the more you log like this, the more analytics you can get without really even thinking about it too much. If you do put thought into it, you could really take it to the next level.

The best part about this is that there is no need to "architect" any complex solution. The more data you log, the more you get out of it. It's really that simple. Use Splunk instead of complex and inflexible architectures like RDBMS/SQL.