On reporting

When all finished with your testing, and have collected all the evidence, it is time for the report.  The report is expressly in existence to make it easier for the development team to fix the bugs. A lot of people don't like reporting.  I am distinctly not one of those people.  I love reporting.  Of course, I wrote a bunch of books, so I have good patterns for writing, and I am sure that makes a difference.  Here is my pattern.

Never start with a blank page.  Always have a template.  If you are starting your own report format, you might be able to get something out of this post.  

Start the report with an executive summary.  Believe it or not, the executives do read these.  It should explain:

  • What a vulnerability assessment is
  • When it happened
  • What exactly was tested
  • A high level overview of the findings

Next you want to get into the nitty gritty.  Start this with a technical narrative.  This can be as complex or as simple as fits your process.  I just list the tools I used and devices - if any - and leave it at that.  Many go whole hog though, and include screen shots of use of the tools from the various tests, what was and wasn't found during each, and like that.  I don't bother with this.  I include the proxy log as part of the evidence of my test.  If anyone wants to see what I did they can look there. I've never had anyone ask for more than that.

This is, however, a good place to talk about report length.  Vulnerability assessments are expensive, often between four and twelve thousand dollars.  Companies large enough to have relationship managers of same kind - program managers or sales staff - feel bad turning in short reports for that much money.  I get that, I really do.  I'd personally rather turn in a short but complete report than deliver a tome that will never get read.  Different strokes for different folks.

Now we get to the nitty gritty -  the findings themselves. After making a basic list of the findings that is easy to read - finding, one sentence description, severity - I get straight into the meat and potatoes.

  • Personally, I like a page break at the beginning of every finding.  This makes it easier for a development lead to print a finding, hand it to a staff developer, and say "Here, fix this" if they happen to not have a bug tracking system. Remember, the whole point of the report is to make it easy to fix the bugs.
  • Title the finding, give it a reference number, and list the severity.  I like to list how easy it is to fix, too.
  • Describe the finding.  "DOM Cross Site Scripting is browser injection.  It could allow an attacker to acquire the session information of the logged in user.  DOM XSS is especially dangerous because there is no server-side check possible, as it entirely happens in the JavaScript in the browser."  Like that. Remember that you know all of the findings and what they mean, but some of your end customers will not. It is worth having a template of common finding descriptions for copy and paste purposes (I have a spreadsheet).  
  • Give the technical details.  I do this in a Request and Response format. For Cross Site Scripting, as an example, I would show the request with the payload painted in red, the response with the unencoded attack painted in red, and then a screenshot of the outcome (If I am in a hurry you often get an alert(1);)  I do NOT use screenshots of code.  Don't let the sales folks talk you into that.  Yes, I know, having a table of figures is cool and all that but you can't copy and paste example code from a screenshot.  Just say no to screenshots, unless it is an alert box or something.
  • Finally, and most importantly, remediation advice.  This is how to fix the problem, Here is where a coding background is helpful.  Sure, you can Google XSS and then type "validate your inputs" in the remediation advice, but that is useless.  The advice is where you really earn your money, so take the time to figure it out.  For instance, in the DOM XSS above - have you ever tried to validate inputs using JavaScript? It's a screaming pain in the ass. Find out what the latest libraries that make it easier are, and recommend them.

Finally, this is not a marketing paper.  You don't need to tell 'em what you're gonna tell 'em, tell 'em, and then tell 'em what you told 'em. The conclusion should just be in invitation to schedule a readout call, where you can walk the development staff through the findings, and that's it. After that, proofread, save as a PDF, and call it a day.

If you liked this, start the beginning of the series with On Tools.

Husband. Father. Pentester. Secure software composer. Brewer. Lockpicker. Ninja. Insurrectionist. Lumberjack. All words that have been used to describe me recently. I help people write more secure software.



profile for Bill Sempf on Stack Exchange, a network of free, community-driven Q&A sites