I launched a new portal that deals with visualization of log files:

http://secviz.org

The portal can only survive if people- you – take an active part in contributing content.

There are multiple resources available where community input is most welcome:

* Graph Exchange: The idea is that people can submit their graphs, explain why they think the graphs are useful, and how they generated them.
* Parser Exchange: To generate graphs, you need to parse your data. This is a place where you can submit your parsers.
* Links: A whole bunch of links around data analysis and visualization.
* Discussions: A free forum where you can start discussions around the topics of log visualization and analysis.

Let me know what you think and most importantly, submit your graphs

I keep running into these little annoyances in Linux. (And as I said here before, I love Linux, but there are some things which are just bad.) This time I was trying to see what happens if you lock an accound. You didn’t even know you could do that?

passwd -l 

Do you know what syslog has to say about this?

Nov 14 16:35:12 zurich passwd[21226]: password for `test' changed by `root'

And even worse, if you unlock:

passwd -u 

Linux says:

Nov 14 16:35:12 zurich passwd[21226]: password for `test' changed by `root'

Great! What am I supposed to do with this? Is a password change really the same as a lock out of a user?

To continue on the path of auditing and such, have you tried to configure an automatic lock-out after a certain amount of failed logins? Good luck. After a while you might find pam_telly. You have to use this PAM module to achive that lockout. You can configure after how many failed passwords an account gets locked. Again, why is this in such a hidden module? Why not built-in? Is anyone going to rebuild the authentication sub-system? Please? And if you are at it, rethink the whole logging infrastructure too! Don’t forget to use a common log format, a specific fixed format that enforces certain information and is parsable! Stop logging copyright messages into syslog (Ok: dhclient?).

There is all this talk about event interoperability standards or logging standards. Don’t we have enough of them? IDMEF, IDXP, SDEE, WELF, CBE, RDEP, OPSEC. All of them are approaches to solve the same problem: Simplify or enable the interoperability of devices and applications. Does anyone support these standards? No! The question is why? Here is my answer:
Have you ever looked at these standards? Noticed anything? These guys are all trying to solve many problems at once. I already blogged about the four different types of log standards that we need. One important things it that the transport needs to be separated from the format! SDEE for example requires SOAP as a transport. Have you implemented SOAP messaging ever? What an effort. I don’t want to do it in my applications. I want something easy! Why not using simple transports? What about files or syslog. And when I say syslog, I don’t mean the gibberish you can log in the message, but I mean the transport. Very simple! Very easy to implement!
Some standards are using XML. It’s just too much work to implement XML messages. You need to keep track of the elements, the hierarchy, the attributes, validate against the DTD, the Schema, etc. And you need a transport that can support it. Nevertheless, there are a few advantages to XML: You can express lists and you can enforce a very well defined format. But that’s it.
So my point being, use a text-based format. Do we have any standards in that arena? Well, there is CEF (Common Event Format). And that’s it. I don’t konw of any others. The standard is very well designed. And not by academics or people that have never seen a log file before, but by people that have seen hundreds of different log formats. A log standard needs some other considerations. Things like event IDs or severities. Things that an event consumer is interested it! But that’s a topic for another entry.
There is a second point that you can make agains text-based formats (the first point being that lists are hard to express), which is speed. I completely agree, if you want speed, you need to go binary! Period. Use NetFlow as an example where you send some kind of a template first and then you send the messages in that format. However, there are other drawbacks: it’s harder to implement (you need preprocessing), not every transport is suited for it, etc.
So to conclude: We really need three logging standards: