A study of more than 300 penetration tests conducted by security company Imperva over four years from 2000 found that 93% of insecure systems identified during testing have remained vulnerable to attack.
In its report, Imperva said the results showed that many organisations did not bother with repeat penetration tests after problems had supposedly been fixed.
"The information we collected over the years from customers that do repeat penetration tests indicates that failing to perform a repeat penetration test may lead to a false sense of security," it said.
Worryingly, 33% of users that did retest found previously encountered vulnerabilities. "These figures indicate that programmers either did not understand the problem, did not know how to fix it or on many occasions just tried to hide it," Imperva said.
The company also reported that in 10% of cases the retests uncovered new vulnerabilities which had not been identified in earlier tests.
Impreva said users were not spending enough time on building robust testing procedures. "Most of the applications we tested required many man-years' work to construct," the report said, but it found that often users would only allocate two dedicated staff, working between four and 14 days to test the security of an application.
In 60% of the retests, Imperva found new vulnerabilities that were either introduced when programmers corrected previously identified vulnerabilities, or were introduced during the application's development evolution.
When there was a long period between testing cycles, Imperva noted that vulnerabilities fixed during the earlier tests were reintroduced during various change cycles that the applications went through.
According to Imperva, programmers who had not seen the report from the first penetration test introduced some of the changes. It also found that in some cases changes that reintroduced old vulnerabilities were made by the same programmers who introduced the original problems.