Troubleshooting Web Services with Wireshark

My current project features a mammoth Flash component that makes a lot of SOAP web service calls. We’ve happily used SoapUI for standalone testing of our services, but when it comes to figuring out what’s going on between Flash and the SOAP services, additional tools are required.

Firebug is almost ideal for this situation – unfortunately, the current version will show neither the request (the posted XML request) or the response – both tabs are blank, giving you HTTP headers only.

Packet Sniffers are ideally suited for this type of troubleshooting – there is an inherent truthfulness to be found by looking at the packets going over the wire. After poking around with a few packet sniffers, I ended up with Wireshark.

Wireshark (formerly known as Ethereal) is a free “network protocol analyzer”, available for many platforms, that seems pretty solid and has a lot of compelling features that make it useful for me. It’s not perfect, and there are a few things it could do better, but for a free tool, it’s pretty amazing.

Let’s see how it works – the first big feature we’ll talk about is the “capture filter”. This is a filter expression you can apply at the NIC level to restrict the amount of traffic that gets analyzed (i.e. “host 192.168.1.44″).

This is a huge boon, as it allows you to limit the amount of CPU and memory consumed by the sniffing, thus allowing you to be more casual about leaving the sniffer on for long periods of time.

Once the capture is started, we’ll start to see packets showing up in the table as green rows. To limit our view to just that of the SOAP requests going across the wire, we can enter a protocol name (in our case, “http/xml” “xml”) into the Display Filter. (The display filter also has its own expression syntax, allowing more complicated boolean queries etc.)

(Update 2008-03-21: I’ve been notified that “http/xml” doesn’t work with some versions / installations.  Use “xml” instead). 

With the Capture and Display Filters in place, WireShark removes the biggest pain point in network traffic analysis – sifting through all the garbage. Below is a screenshot (with IPs censored to protect the innocent) that shows a series of xml requests that have been sent by our Flash application:

The question now becomes: how do we look at what’s going on in these packets?

In the case of http/xml, Wireshark makes it very, very easy to see the Request that was made to the server – it appears in the Packet Details pane, and even has nice little collapso/expando controls for the various sections. I prefer to expand all (control-right arrow), and then collapse everything except the data I care about. My only complaint here is that Wireshark inexplicably allows you to copy/paste only one line at a time. Fortunately, there’s a workaround that we’ll get to.

The obvious question is – where’s the SOAP response? The answer is that it’s in the “packet bytes”, and not in just one packet, but spread across many. This makes the “Packet Bytes” display (by default, the third pane) pretty useless for our purposes – in fact, I turn off this panel altogether (Edit > Preferences > User Interface > Layout) .

Instead, the trick is to right click the packet you’re interested, and select “Follow TCP Stream”. This will produce a window similar to the following:

The red text was sent by one party (in our case, the Flash application), and the blue text was sent by the other party in response (in our case, the SOAP server).

Sometimes you’ll have to sift through a bit of the output, as the stream tends to include more than you’d expect, but overall, you should be able to find what you’re looking for.

One side note is that following the TCP conversation changes your display filter behind the scenes – you’ll need to change it back if you want to revert to the previous mode of viewing just the http/xml requests.

All-in-all, I’ve been quite satisfied with the tool – again, it has some small flaws, but I think it’s a very beneficial tool, and hope that others can find it as useful as I have.