Although black box security testing is extremely important to
do, researchers at Fortify Software have found it isn't enough to
help developers find and repair code flaws.
In a report released Monday about black box security testing --
sometimes referred to as
penetration testing -- Fortify researchers found that such
tests failed to provide adequate results in three areas:
- Security test coverage -- Black box tests don't tell you
what percentage of the code was hit, said Barmak Meftah, vice
president of products and services at Fortify. "Without that
parameter, the gauge of security isn't clear," he said.
- The inability to pinpoint the location of a vulnerability -- A
black box test can tell you only what page the vulnerability is on,
Meftah said. It can't give such details as where on the page or in
the application the vulnerability is.
- Not all input sources are tested -- Because black box tests
address just the Web interface, they don't give you all the
problems, Meftah said. An application has a lot more sources of
input, he said.
 | | | | | Once the issue is found, we can
give more information about the cause of the problem because it
sits inside the application.
Barmak Meftah
VP of products and servicesFortify
Software |
| | | | | |
| |
|
"While black box security testing is an important tool for
analyzing the security of deployed applications, its scope is
limited by the fact that it resides outside of the application,"
Meftah said.
To remedy that, Fortify has created a product to complement
black box testing and give developers and testers greater details
about test results. Fortify Tracer, whose announcement coincides
with the release of this report, sits inside an application and
provides "more measurable and actionable output," Meftah added.
For example,
Fortify
Tracer injects monitors in all of the attack surfaces and
around all the functions of the application. Then when a black box
test finds issues with an application, Fortify Tracer tells how
much of the code was hit and where specifically the problem is.
"Once the issue is found, we can give more information about the
cause of the problem because it sits inside the application,"
Meftah said.
Fortify Tracer currently works on any J2EE executable
(.war/.ear) files. Dashboards communicated key metrics and allow
users to compare runs, inspect issues and find flaws. In addition,
it generates detailed reports showing vulnerabilities according to
their categories, such as cross-site scripting and SQL
injection.
Meftah said Fortify Tracer will be integrated with Watchfire's
AppScan, but the product will also work with any black box security
tester, he said.
Available immediately, Fortify Tracer costs $24,000 per named
end user.