While I am on a roll, talking about normalization and log standards, let me have a look at a publication from Gartner. It is a bit dated already (May 2006), but people are probably still referring to it. There are a couple of things that I want to make sure people understand.
While I like the fact that someone like Gartner is trying to dive into a technical topic, I am not too certain that this is very productive. The Gartner publication I am looking at is “Define Application Security Log Output Standards” by Amrit Williams. I must say, the publication is not horribly wrong or bad, however, there are some interesting problems that I want to address:

• The publication outlines what fields should be contained in an “account access event”. Most of the fields make sense. However, there are two fields: “login success” and “login failure”. These fields should be normalized. There shouldn’t be two fields, one for success and one for failure. Just have one that indicates success or failure. That way you can correlate those two events against each other. Otherwise you can’t because you have two different fields. Well, you can, but it’s much more difficult.
• Another field in the account access event is “access rights”. If you include this field in an event, you need a system which can deal with sets or lists of values. This is not simple and I don’t think any of the SIMs really take care of that. Not that they shouldn’t, but it’s really really expensive to build that into a correlation engine. Now, in this specific instance, for access rights, they should not be in an event anyways. This is static information that should be read into the correlation engine asynchronously or looked up on a need to know bases.
• The publication further indicates that the access events have additional variables, called “Variable 1”, “Variable 2”, etc. I have no idea what these fields would be used for. But that’s not even important. The important part is that having generic variables without a fixed meaning is not very useful for later consumption in reports or correlation rules. You need a semantic associated with every field. That’s exactly why there is a common event language to start with!
• The same mistake with splitting out the same type of events into multiple event fields is done in the “account /role management events”. Make one field tat talks about “creation”, “modification”, etc. One of the things to mention in this context is an event taxonomy. I am working on a generic taxonomy right now for CEE, the common event exchange format. CEE is an effort that I pushed Mitre to address a long time ago. Finally, there is a small working group and we should soon have the <A href=”http://cee.mitre.org”>Web presence</A> up and running.
• I don’t agree with the “Log Output Formats” discussion at all. Sorry. Gartner (or Amrit?) recommends syslog as output format. While I am quite a fan of syslog, it’s definitely not my transport of choice. Read that again: TRANSPORT! Syslog is not a log format. It’s a transport. I am not going to roll-up my rant about formats and transports again. Read my older blog entry about the <a href=”http://raffy.ch/blog/2007/04/19/standard-logging-format-common-event-exchange-cee/”>format vs. transport</A> issue.
• It seems really interesting to me that syslog is pushed as the “log format” (again, it’s a transport, but whatever). The publication even mentions all the RFCs associated with syslog, but not a single sentence about the draw backs. Unstructured, reliability (okay TCP is mentioned), poor timestamp, etc.

Again, I think it’s great that Gartner picked this topic up. It’s incredibly important, but it takes a fair amount of work and experience to get a decent log standard put together. Stay tuned and check back for more information about <a href=”http://raffy.ch/blog/2007/04/23/common-event-expression-cee/>CEE</A>.

[tags]log standard,syslog,cee,event fields[/tags]