Log analysis has shifted fairly significantly in the last couple of years. It is not about reporting on log records (e.g., Web statistics or user logins) anymore. It is all about pinpointing who is responsible for certain actions/activities. The problem is that the log files do oftentimes not communicate that. There are instances of logs (mainly from network centric devices), which contain IP addresses that are used to identify the subject. In other instances, there is no subject that can be identified in the log files at all (database transactions for example).
What I really want to identify is a person. I want to know who is to blame for deleting a file. The log files have not evolved to a point where they would contain the user information. It generally does not help much to know what machine the user came from when he deleted the file.
This all is old news and you probably are living with these limitations. But here is what I was wondering about: Why has nobody built a tool or started an open source project which looks at network traffic to extract user to machine mappings? It’s not _that_ hard. For example SMB traffic contains plain-text usernames, shares, originating machines, etc. You should be able to compile session tables from this. I need this information. Anyone? There is so much information you could extract from network traffic (even from Kerberos!). Most of the protocols would give you a fair understanding of who is using what machine at what time and how.
Technorati Tags: identify correlation, user, log analysis, user mapping
Arbor Networks Peakflow/X, Q1 Labs Q1Radar, and Lancope StealthWatch Xe all do this, and have for ages.
In terms of network telemetry, I highly recommend you check out NetFlow from routers – it’s layer-4 information, extraordinarily useful for looking at network behaviors. Flexible NetFlow, a new feature built atop NetFlow v9, allows you to get packet header and payload information, as well.
Comment by Roland Dobbins — May 26, 2007 @ 11:28 pm
Yes. Commercial tools. What about the open source world? I can’t affort anything like that for my own purposes. On top of that, last time I checked, these tools did not export that information in events such that it could be used for correlation purposes. I might be wrong. Guess I have to check on Tuesday when I’m back in the office 😉
NetFlow is not giving me what I need. I have used NetFlow for a lot of things, but it’s layer 4, as you say, so no users and such. NetFlow is, to be honest, quite boring. I find myself more and more interested in application layer information.
Comment by Raffy — May 27, 2007 @ 2:26 am