The problem of how you call something is not something I think about conciously very often, but it became really obvious to me that it is important to name things and define what they really mean every so often. In my daily work I use the words Event and Log Entry all the time. While talking to developers and other geeks, it has never been a problem, but I was talking to some other groups lately, outside of my company and when I mentioned the word event it took me a while to understand that they did not think about an event the way I did. An event for them was an incident, a physical event, the constellation of things coming together and causing something to happen. For me an event is something I use very loosely. An event gets generated by a device. It’s the same as a log entry. It’s a “string” that describes what happened. Windows for example generates events. They get collected in the event log. But again, I am using the term very loosely. What’s a log entry then in contrast to an event? Hmm… And is a tcpdump record a log entry or an event or what is it? Hard to say. I guess it takes the effort of someone to define all that. I might…

I am scribbling on another book chapter. This time it’s for a visualization book. I am writing about how to analyze firewall and IDS logs. I am using line graphs and treemaps to do so. Guess what tool I am using to generate all the graphs. Yes. AfterGlow.
I am not quite done with writing, but am pretty happy with the way it shapes out. The chapter is not going to be highly technical. I am not going into how to configure AfterGlow and parse log messages and such. I focus more on the process-level. It is quite an interesting experience to put something into words that you intuitively do all the time.
I am not sure when the book is actually going to come out, but I will post here when it’s available.

There is a lot of talk around interoperability standards lately.Following these discussions, it seems to me that people are intermixing a lot of different topics:

a) Log format (syntax)
b) Event transport
c) Event classification (also called taxonomy, categorization, grammar)
d) Logging recommendations (what events specific devices should report AND what fields they should contain as a minimum

I would really like to see future discussions broken up into these four groups!

Yes, I am still playing around with visualization. In fact, I am writing a Chapter for a book. It’s going to be about using line graphs and treemaps to visualize firewall and IDS logs.

I just realized something that you have to be careful about when using AfterGlow’s fan-out filtering!

If you have a graph where you are going to filter based on the source-node’s fan-out, you will still see all the eventNode->targetNode pairs! It won’t restrict them to only that ones that are from the remaining source nodes! Here is an example:

My log contains thousands of line, but only these three entries should remain after doing a (-f 2) filtering:

12.108.194.82,nounce,111.222.69.44|113
12.108.194.82,block,111.222.69.44|113
12.108.194.82,pass,111.222.69.44|25

However, the graph will look as this:

What you can do is using the -p 1 option: afterglow.pl -p 1 -f 2 which will result in the desired graph:

I just came accross this review on DefCon presentations. Is published on unixreview.com. I don’t know the author, but I certainly like what she had to say about my presentation:

I’ll begin with Afterglow, by Raffael Marty, which is a visual log analysis tool. Martyýs tool runs either via Perl or Java and does some cool stuff by making text log files into understandable graphs. This type of tool can help you truly understand the big picture of what your logs are telling you without making you go cross-eyed trying to read them all. This was the second year Marty has discussed this tool, and I was very impressed by his dedication to this work.

There is actually another person out there who wrote a summary of DefCon 13 (so last year) and put some graphs online. I am impressed 😉