Another release of AfterGlow. It’s mainly a feature release where I introduce fan-out filtering. I also fixed some minor bugs around property file parsing and some error messages. Nothing major.

The fan-out filtering is really cool. So far you could only filter based on node counts. Saying for example you wanted to eliminate all the nodes which had a count of 5 or less. Now you can say you want to filter out all the nodes which have a fan out (outgoing neighbors) of 4 or less. This way you can for example identify scans, as here:

This was generated with:

afterglow.pl -c color.properties -p 1 -d -o 7 -g 8

Where -g is the filter for the fan-outs. The graph clearly shows that some external machines were scanning all the internal machines on one and the same port.

I am still waiting for people to send me their graphs!!

I was playing with AfterGlow and visualizing netflow data lately. I realized that some of the configurations in AfterGlow are a bit tedious. So I added a new property to address coloring independent of the node: color=”blue” for example. I also added a feature to turn off labels on a per-node type. For example label.source=0 turns off source node labels. And finally I started playing with clustering of nodes. A lot of times you end up with a huge amount of nodes and you don’t really need to know all the details of that cluster, you just want to know that there is something going on around there. So for example for IP addresses, oftentimes you want to cluster by subnets (e.g., A, B, C classes):

cluster.source=regex_replace(“(\\d\+\\.\\d+)”).”/16″ if (!match(“^(212\.254\.110|195\.141\.69)”))

This looks complicated, but it’s not that bad. Basically there are a few functions I introduced. Read it from right to left. First you check whether there is a match on the source field (we are doing cluster.source) for either of these two IP ranges. If so, you take that value and you do a regex_replace on it. Basically getting the two first octets and adding “/16″ to it. An easier example would be:

cluster.target=”bigger than 1024” if ($fields[1]>1024)

AfterGlow 1.4, which supports clusters and all the other things is now available via sourceforge.

While working on some firewall log analysis, I started working on AfterGlow 1.2, which is going to have the capability to turn of the node labels on a per node type basis. In addition, if you turn the labels off, the node also becomes much smaller.

But what was really interesting is that while I was browsing the man pages for graphviz, I discovered that fdp has a node attribute called pin. This got me quite excited. If pin is true, that node will remain at its initial position. Don’t ask me how that would work in reality? What is the initial position? How do you define that? Not sure, but I’ll figure it out. This would get me closer to the animated graphs!

Digging a bit deaper in the graphviz documentation, I realized that neato can also pin a node, by adding a ! to the pos argument of a node:
node [ pos = "2,2!" ];

To solve the animation, one probably has to generate a graph with the -Tplain option, then get the x and y coordinates (second and third argument to the node entries) of the graph and use them as input for the next graph.

I just released AfterGlow 2.0. In addition, I released AfterGlow 1.1.6, a bug fix release to the 1.x releases.

The new release is AfterGlow 2.0, which is no more written in Perl, but in Java. It generates a new kind of graph output. Whereas AfterGlow 1.x generates link graphs, AfterGlow 2.0 generates TreeMaps.

This afternoon I will be talking at the EuSecWest 2006 Conference in London. The topic is “Visual Security Event Analysis” (what else ;)). The presentation has quite a lot of examples on how you can use both AfterGlow 1.x and AfterGlow 2.0.

I am looking forward hearing back from you with screenshots and use-cases of how you are using graphs to do security analysis.

Wow. I just found this pretty awesome TreeMap tool. The data format it reads is pretty easy and I quickly built a file with some of my firewall data. Well, fake firewall data 😉

What you see in here is first the color: green are firewall passes, red are blocks, then the hierarchy is such that the target system is top, then the target IP and then you see the date inside of the boxes, when the access happened.

Well, the tool is pretty awesome. Lots of interactivity. You define the hierarchies manually, on the fly it updates the graph. Then you can color and filter and all kinds of nifty things. Try it out.

It’s one of those afternoons again where I am loosing myself in doing some research on graph tools. Maybe I can spare some others an afternoon of browsing around and summarize some things I found:

• kgraphviewer – this is a tool to show graphics. The exciting thing is that you can open .dot files. However, only dot is used to render them, not neato or twopi. Too bad.
• ZGRViewer – looks very promising. You can open .dot files and render them with anything (neato, twopi) the output is zoomable and you can move around. Pretty nice. It’s written in Java. Now the question is whether I could use this framework to build my animated viewer that reads the next .dot file and updates the graph nicely. I will get it done someday!
• tinfu – Another java library that draw .dot files. It had trouble with some of my .dot files. I guess the tool’s not very robust.
• Grappa – Another Java tool. The feature list looks promising. The Web demo is interesting. Need to download it and have a closer look.

I guess one thing that I realized is that you could principally build a graph with, let’s say twopi, and generate an SVG. Then you can manipulate that, show it to the user and so on. The question is still how to periodically update a graph without totally re-layouting it. Well, I am working on this. I think I just haven’t really understood the problem yet 😉

While playing with the AT&T graphviz library again, I was trying to get nodes to show up as pictures, which is fairly simple:

node [peripheries=0, fontsize=10, shapefile="computer.gif"]

However, what I really want is a transparent gif (computer.gif) and then using background color to color that node. That way I could programmatically change the color of the nodes and still use icons to make the nodes a bit nicer. This is what I tried:

node [peripheries=1, fontsize=10, shapefile="computer.gif", fillcolor=blue]

Unfortunately it did not work. Anyone?

I haven’t looked at Shoki in a while. Today I downloaded a version again and tried to compile it on my Fedora Core 4 installation, just to find out that the thing would not compile. Well, I dug around in the code for a bit and after some searches on the Web, I realized that gcc 4 is stricter about the C conventions and Shoki was written with some declarations being non-standard. What fixed it was to define the CC flag in the Makefile to use gcc32 instead of gcc.
Playing with this tool, I somehow have the impression that I just don’t get it. I can redefine the axes and play with that, but even zomming into a certain selection I can’t seem to accomplish. And then there is all this extra stuff like fast fourier transformations etc. While I know what that is, I just don’t quite understand how all that works in Shoki. Maybe I have to spend an afternoon with the documentation 😉 Or maybe there are people out there who have some tips or hints for me?
What I am really interested in is if someone managed to analyze a dataset and can show me what he found with what feature. Do all the bells and whistles (some of the advanced features) really help? Help me out!

Conference season is kicking in again. It looks like this year will be a busy year for me. I will be speaking at the RSA conference mid February in San Jose. Then I just got notice that I got accepted to EuSecWest06. Both places I will be talking about Security Event Visualization. The EuSecWest presentation is going to be more technical and AfterGlow driven, while the RSA presentation is more on the higher level of visualizing security data and attaching a workflow to that.
I have been interested in the workflow aspect of security monitoring for a long time already. It kind of started about three years back with a presentation on Intrusion Management I gave at ETH Zurich. There I tried to outline that the incident response and security event monitoring processes need to be tightly integrated into the other IT process. I guess over time this has become quite apparent, but I still don’t see it completely implemented at many places.

I will keep posting the answers to my Focus-IDS post where I asked people what they use to visualize their log files. Here is some other home grown solution to visualize pf logs: Fireplot. It’s basically a scatter plot over time where the x-axis shows the port.