[Dshield] OK, so what would YOU test?
phil at moyen.org
Fri Apr 18 13:56:57 GMT 2008
I'm being tasked for developing security testing strategies for my company.
I work in the testing department, so I am surrounded by traditional software
testers who build traditional test cases which are done both manually and by
automated tools like Silk and Ruby/Watir. But, my boss thinks we need to get
into the business of more pro-actively testing security. Probably a good
thing. We've got large software and online products, but little
comprehensive security testing that I'm aware of, and I've been asking. We
do have best practices and security development policies in place, but as
far as I can tell no good way to make sure they are being followed.
Everybody I've talked to indicates that in-depth security testing is more
ad-hoc than the traditional form of functional testing, which is driven by
devising use cases and looking for correct functionality. This corresponds
with my own experience. Given the way our software is developed, I believe
it won't be necessary to completely compromise a product -- just showing the
existence of a vulnerability (SQL injection hole or a XSS problem, for
example) should be sufficient to trigger the fix process.
Given that we all have to start someplace, I'm getting ready to propose the
following baby steps:
1) Develop some test strategies that can be added to the existing testing
framework. I suspect these would initially take the form of simple special
2) Get more security testing awareness added to product development.
3) Develop more in-house abilities to "hack" products, so that we can add a
parallel security testing process alongside the existing functional and
performance testing processes. This is where more advanced things like
buffer overflows would be tested for, since that kind of testing goes beyond
our typical "type this" test case. This is also where we could purchase
testing tools like AppScan, though I don't believe that tools can completely
replace human hands-on testing.
Given this, I'd like to ask the community for some assistance:
* Can you point me towards tales of other organizations who added security
testing like this? There are lots of tactical-level resources on how to
test. I'm thinking of books like "How to Break Web Software" as an example
of a tactical resource. I'm looking for a more strategic view.
* On a tactical level, if you were going to develop low level use cases for
step 1, what specifically would you test? I'm thinking along the lines of
"Type a [single quote/double quote/specific hex character] into every field;
make sure you get appropriate error messages back" or "type the following
messages". This would let us get started pushing the security testing idea
into the wider development arena, while potentially gathering some
Thanks in advance.
More information about the list