Skip to content

DOMAIN 6 · Security Assessment and Testing

Security Testing: Choose Methods That Answer the Right Question

Compare vulnerability assessment and penetration testing, set rules of engagement, and turn findings into verified improvements.

Security testing produces evidence about weaknesses and control behavior under defined conditions. NIST SP 800-115 groups techniques into reviews, target identification and analysis, and target vulnerability validation. It also explains their benefits and limitations. Although published in 2008, its planning principles remain useful; its historical tool examples should not be treated as a current product recommendation.1

Choose a method after identifying the question. Finding exposed services, checking configuration compliance, and demonstrating an attack path are different objectives and require different evidence.

Compare vulnerability assessment and penetration testing

Section titled “Compare vulnerability assessment and penetration testing”
Approach Useful question Main limitation
Vulnerability assessment Which potential weaknesses exist in the examined scope? A detected condition may need validation and business context
Penetration testing Can selected weaknesses be used to achieve a defined attack objective? Time, scope, and rules constrain which paths can be investigated

Scanning can cover many targets, but neither an automated result nor a severity label proves exploitability in every environment. Penetration testing can validate selected weaknesses and expose combinations of failures, but an unsuccessful attempt does not prove the absence of vulnerabilities.1

Before testing, obtain authorization from the appropriate owners and define the permitted targets, exclusions, times, methods, communications, and stop conditions. Clarify how sensitive findings and collected data will be handled. NIST includes a rules-of-engagement template covering such operational boundaries.1

For outsourced or cloud-hosted systems, determine whether the authorizing party controls every proposed target. Permission to test one application should not be interpreted as permission to test its provider’s unrelated infrastructure. This is a scope decision to settle before execution, not after an unexpected impact.

Documentation and configuration review explain intended and configured behavior. Log review can show observed activity. Technical tests add evidence of actual responses to selected inputs. An external viewpoint and an internal viewpoint expose different assumptions, and overt versus covert testing changes the coordination and detection conditions.1

Record the viewpoint, access provided, relevant versions, and time of testing. A result obtained with an administrator account answers a different question from one obtained as an anonymous user. Preserve enough context for another qualified person to understand and, when appropriate, reproduce the finding.

Worked example: a customer document portal

Section titled “Worked example: a customer document portal”

Imagine a fictional portal where customers download invoices. A scan finds no recognized software vulnerability, but a manual permission test shows that customer A can request customer B’s invoice by changing a document identifier.

The report describes the authorization boundary, affected endpoint, minimal evidence, and business impact without collecting unnecessary customer records. The proposed repair checks the requesting customer’s permission for every document. Verification repeats the original case and tests adjacent document routes and user roles. The lesson is methodological: a clean scan did not answer the resource-authorization question.

Separate observations from assumptions. Explain affected assets, preconditions, evidence, impact, recommended treatment, and limitations. Track an accountable owner and verify that the chosen correction addresses the underlying weakness. NIST treats reporting and mitigation as part of the assessment process rather than ending the work at discovery.1

Avoid presenting raw finding counts as a security score. Counts depend on scope, tools, validation, and reporting rules; changes in any of these can change the number without changing actual risk.

  • “A penetration test is a comprehensive audit.” It tests selected paths within explicit constraints.
  • “A scanner’s finding is automatically a confirmed incident.” A possible weakness and an observed compromise are different claims.
  • “Retesting the exact payload is sufficient.” Consider whether equivalent routes remain and whether the correction introduced another weakness.

Control assessment places technical results within an evidence-based assessment. Access control explains the portal example, and secure development addresses preventing recurrence.

  1. NIST, SP 800-115, Technical Guide to Information Security Testing and Assessment, September 2008; sections 2–8 and Appendix B, Rules of Engagement Template. ↩ ↩2 ↩3 ↩4 ↩5

Independent, unofficial study notes. Not endorsed or reviewed by ISC2. CISSP is a registered trademark of ISC2.

Editorial policy & attribution