It’s been a while since I wrote “Applied Security Visualization“. Here is an older video that I just came about. A good overview of the book. Enjoy!
November 11, 2010
Applied Security Visualization – Book Video
September 4, 2010
Logging Formats and Standards
I have discussed the topic of logging standards multiple times on this blog. Some recent developments in the logging space urged me to give an update and provide my opinion:
Yet another vendor just released a “standard” log format (note the quotes around standard). It’s called UCF, the Universal Collection Framework™ (UCF). This is how the vendor describes it:
UCF is the first WAN-aware, store-and-forward, encrypted, compressed IT data transport. It allows customers to gather IT data, increase resilience, reduce network chatter and encrypt from almost any device, anywhere, quickly and easily. UCF leverages a new transport and store protocol that LogLogic intends to open source in the near future.
Sounds a whole lot like syslog. (syslog-ng and rsyslog seem to support exactly this!) Okay, let’s just look at this description: WAN aware? What the heck is that supposed to mean? You mean it won’t work well on a LAN? Does that mean it knows the Internets? That’s just a strange description to start with. Oh, and it’s the first property mentioned! The rest of the description sounds like a transport protocol. Interesting. Why not stick with syslog that is well known, has proven to work, and has integration libraries built already. I never understood why vendors implemented their own transport protocols. They are hard (very hard) to implement and even harder for producers and consumers to adopt to. Oh well.
When people talk about UCF, they keep bringing up ArcSight’s CEF. Well, I am greatly responsible for that specification. But guess what? It’s not a transport protocol! It’s a syntax definition. It tells a log producer how to format their log file. Not how to transport it. Because, there is always syslog that a lot of machines have installed already and it’s easy to use. (And in newer versions you get encryption, caching, etc.).
Now, my last point about standards. Why do vendors keep trying to come up with standards by themselves? It just doesn’t make any sense. How is going to adapt it? At ArcSight, about 4 years ago, we came up with CEF because CEE didn’t move fast enough and we wanted something that our partners could easily use. An analyst wrote that ArcSight is planning to take CEF to the IETF. I hope they are not going to do that. I don’t have any control over that anymore, but that would be stupid. We rather push CEE through IETF. If you have a chance, compare the CEE syntax proposal with CEF. Notice something? Yes. It’s very similar. Again, I might have had something to do with that. Anyways. Vendors should not define logging standards!
On a good note: CEE is moving forward and just released the architecture overview for public commentary. Check them out!
June 28, 2010
All the Data That’s Fit to Visualize
Last week I posted the introductionary video for a talk that I gave at Source Boston in 2008. I just found the entire video of that talk. Enjoy:
Talk by Raffael Marty:
With the ever-growing amount of data collected in IT environments, we need new methods and tools to deal with them. Event and Log Analysis is becoming one of the main tools for analysts to investigate and comprehend the state of their networks, hosts, applications, and business processes. Recent developments, such as regulatory compliance and an increased focus on insider threat have increased the demand for analytical tools to help in the process. Visualization is offering a new, more effective, and simpler approach to data analysis. To date, security visualization, has mostly failed to deliver effective tools and methods. This presentation will show what the New York Times has to teach us about effective visualizations. Visualization for the masses and not visualization for the experts. Insider Threat, Governance, Risk, and Compliance (GRC), and Perimeter Threat all require effective visualization methods and they are right in front of us – in the newspaper.
December 1, 2009
Applied Security Visualization Book seen in Singapore
A friend just sent me couple of pictures he took in a bookstore in Singapore.
Have you seen the book Applied Security Visualization on the shelf at your local book store? If so, send me a picture and I will post it…
February 17, 2009
Security Visualization and Log Analysis Workshop – Sign up now!
“Log Analysis and Security Visualization” is a two-day training class held on March 9th and 10th 2009 in Boston during the SOURCE Boston conference that addresses the data management and analysis challenges of today’s IT environments.
Students will leave this class with the knowledge to visualize and manage their own IT data. They will learn the basics of log analysis, learn about common data sources, get an overview of visualization techniques, and learn how to generate visual representations of IT data for a number of different use-cases from DoS and worm detection to compliance reporting. The training is filled with hands-on exercises utilizing DAVIX, the open-source data analysis and visualization platform.
Register today to secure your spot.
January 2, 2009
links for 2009-01-02
November 28, 2008
links for 2008-11-28
September 6, 2008
The Process of Writing the Applied Security Visualization Book
A little bit more than two years ago, I approached Jessica Goldstein from Addison Wesley to write a book about security visualization. We sat down during BlackHat 2006 and discussed my idea. It didn’t take much to convince her that they should get me on board. I went home after the conference and started putting together a table of contents. Here is the very first TOC that I submitted:
- Introduction
- Data Sources
- Visualization
- From Data To Visuals
- Visual Security Analysis
- Situational Awareness
- Perimeter Threat
- Compliance
- Insider Threat
- Data Visualization Tools
If you read the book, you will notice that this is pretty much what I ended up with. More or less. An interesting fact is that at the time of submitting the TOC, I had no idea what to exactly write about in the compliance and insider threat chapters. The even more interesting fact is that a lot of people told me that their favorite chapter is the insider threat chapter.
After submitting the TOC to Jessica, she had me fill out some more marketing questions about the book. Things like target audience, competitive books, etc. After handing that in, it went silent for a bit. Jessica was selling the book internally. And then things started to look not so good. Jessica went on maternity leave. Kristin took over and got the proposal review process lined up. I asked some people in the industry to have a look over my proposal and provide feedback to the publisher. Questions like: “Why is Raffy the right person to write this book?” “Is there a market for this book?” etc. were being asked. I received the six really great reviews (thanks guys!) mid December 2006. On December 19th, I received an email with the contract to write the book. I sent the contract off to a friend of mine who is a lawyer, just because I was a bit worried about intellectual property rights. After a few emails also with Addison, I felt much better. They are not at all interested in any IP. They just want the copyright, which was totally fine with me. Then, finally, on January 17th, I signed and was under contract to write about 300 pages about security visualization.
After a few days, I received an ISBN number for the book and a ton of material about style guides and how to go about writing the book. All very exciting. I decided to not write my book in TeX, unlike my masters thesis. That was definitely a smart decision. It turned out that using Word wasn’t that bad. The template from Addision made it really easy to format the text correctly. I actually ended up using VI to write the original text without any formatting. Once it was all done, I copied the raw text into Word and started formatting. The reason for doing this is that I am so much quicker in VI than I am in Word. (And hitting the ESC key in Word is not something you want to be doing too much.)
One of the next steps was to put together a timeline. Well, it was sort of aggressive. The version of the schedule I could find in my archives shows that I was planning on being done mid September 2007. Well, I missed that by only a year 
I attribute a lot to the fact that I didn’ really know how to write (seriously) and to the chatpers for which I had to do a lot of research.
I definitely enjoyed the process of writing the book. The folks at Addison Wesley were awesome. They kept motivating me along the way and provided great insights into the writing process. What I am still very impressed with is the PR aspects. Early on, they hooked me up to film a video cast about the book. After publishing the book, I get about an email a week for some press opportunity. Keep them coming
Here is a fun fact: In ~/Data/projects/vis_addision, where I have all the material for the book, I accumulated 1.1GB of data. Pretty crazy.
Are you thinking about writing a book? Do it, but make sure you have time! I spent a LOT of time in the local coffee shop (picture on the left). I always had printouts with me to work on corrections. The picture on the right I took at 6.30am in Taipei. Yes, it’s a full-time job! I learned a lot! I made amazing connections. And I had fun! One piece of advice: make sure you have a good publisher!
I haven’t seen the book in my local Barnes and Nobles yet. Well, I checked two weeks ago. But a friend (@jjx) sent me this picture. So, apparently some book stores have it in stock:
April 12, 2008
RSA Security Conference 2008 – What a Week
Last week the RSA security conference was held in San Francisco. It’s hard to put all the impressions I gathered during the week into words. Let me just highlight some things that I thought were interesting:
RSA is a business development conference. It’s been that way for years and this year was definitely not different. At all! Don’t believe me? If even I, who is not in business development can collect this many cards, it has to be a biz dev con.
The Security Blogger meetup was great. Unfortunately I had to bounce very early. Sorry guys! I would have loved to stick around. The caliber of people hanging out in that room was crazy. Everyone that has a name, and more importantly a voice in the security industry, was there. Thanks to Jennifer for organizing it. Love the t-shirt!
.
Talking about everybody being at RSA: I met the CISO of the Vatican
.
.
.
.
We had the first in-person common event expression (CEE) meeting. Some people from XDAS
showed up and we had some fairly good discussions around what to do with both the standards, how they can be aligned and how we can move forward.
.
Walking around on the floor, I found some interesting security visualizations. This one is from DeepSight. Very visually appealing. I haven’t spent much time to understand what’s on the displays, but it looks interesting.
.
.
January 24, 2008
Bridging Security and Visualization
OnSecrity just released another video of the conversation we recorded last year during RSA. I am talking about security visualization in light of the book I am working on. This video cast is the sequel to the first one that I posted a few days ago.
One of the topics I am discussing in the video is the “false dichotomy” between security and visualization. This is a topic that I talked about during a talk at the MIT Lincoln Labs at the beginning of December. The presentation showed how there are really two disciplines that come together in security visualization: Security and Visualization. The problem with this is that visualization people don’t know much about security and the other way around. It’s a very interesting topic to explore and it explains some of the mistakes that are being made with visualization tools and is also reflected in visualization research.
Technorati Tags: visualization, security, videocast