There are various cases when during an IT/ security assurance projects there are specific requirements to rely on penetration testing projects/ reports completed by a third party.
However, not always, the IT security auditors have the necessary information to assess the quality of the penetration testing projects (focusing on planning, projects delivery and reporting).
And I address here the IT security assurance projects having in scope IT systems and finalizing with an Independent Auditor Report (IT Audit Opinion) and not audits focused on processes (like ISO audits) or on specific IT controls relevant for specific objectives (like ISAE 3402 IT/ Security assurance projects).
IT/ Security auditing standards
In order to have a consistent and objective assessment of third party reports, an IT security auditor must refer to the auditing standards he is using for performing his engagement.
And if we consider the IT audit standards available on the 
market, we must look at ISACA framework. There are also other standards applicable to various industries or specific objectives, focused on management systems, like ISO standards, but ISACA’s are the only ones of general nature which can be used cross-platforms, industries, etc. And as support for this, I performed some quick Internet searches and found mostly the same views.
For example, in one of its papers, NIST recognizes the entities enumerated below as addressing the IT auditing within their standards, further states that all of them taken essentially the same position concerning audits involving information systems and continues focusing on ISACA’s CobIT.
- The American Institute of Certified Public Accountants (AICPA) in several Statements on Auditing Standards (SASs);
- Institute of Internal Auditors Association (IIA) in its Standards for the Professional Practice of Internal Auditing;
- Information Systems Audit and Control Association (ISACA) in its “ITAF™: A Professional Practices Framework for IS Audit/Assurance”
- S. General Accounting Office (GAO) in its Government Auditing Standards and Title 2, Accounting.
ISACA’s “ITAF™: A Professional Practices Framework for IS Audit/Assurance”
Therefore, I will continue with ISACA standards and identify the requirements for using the work of third parties. ISACA developed a specific standard: IS AUDIT and ASSURANCE STANDARD – 1206 USING THE WORK OF OTHER EXPERTS and a related guideline: IS AUDIT and ASSURANCE GUIDELINE – 2206 USING THE WORK OF OTHER EXPERTS.
The standard defines 7 statements regarding the auditor responsibilities when considering the use of work of other experts:
1206.1 IS audit and assurance professionals shall consider using the work of other experts for the engagement, where appropriate.
1206.2 IS audit and assurance professionals shall assess and approve the adequacy of the other experts’ professional qualifications, competencies, relevant experience, resources, independence and quality-control processes prior to the engagement.
1206.3 IS audit and assurance professionals shall assess, review and evaluate the work of other experts as part of the engagement, and document the conclusion on the extent of use and reliance on their work.
1206.4 IS audit and assurance professionals shall determine whether the work of other experts, who are not part of the engagement team, is adequate and complete to conclude on the current engagement objectives, and clearly document the conclusion.
1206.5 IS audit and assurance professionals shall determine whether the work of other experts will be relied upon and incorporated directly or referred to separately in the report.
1206.6 IS audit and assurance professionals shall apply additional test procedures to gain sufficient and appropriate evidence in circumstances where the work of other experts does not provide sufficient and appropriate evidence.
1206.7 IS audit and assurance professionals shall provide an appropriate audit opinion or conclusion, and include any scope limitation where required evidence is not obtained through additional test procedures.
Also, the standard describes several key aspects regarding what the IS audit and assurance professionals should do:
- Consider using the work of other experts in the engagement when there are constraints (e.g., technical knowledge required by the nature of the tasks to be performed, scarce audit resources, time constraints) that could impair the work to be performed or potential gains in the quality of the engagement.
- Document the impact on achieving the engagement objectives if required experts cannot be obtained and insert specific tasks in the engagement plan to manage risk and evidence requirements.
- Consider independence of other experts when using their work.
- Have access to all work papers, supporting documentation and reports of other experts, where such access does not create legal issues.
- Determine and conclude on the extent of use and reliance on the expert’s work where the expert is not granted access to records due to legal issues.
- Document the use of the other expert’s work in the report.
The Guideline also provides detailed guidance to IS audit and assurance professionals when considering the use of work of other experts, structured in the following chapters:
- Considering the Use of Work of Other Experts
- Assessing the Adequacy of Other Experts
- Planning and Reviewing the Work of Other Experts
- Evaluating the Work of Other Experts Who Are Not Part of the Audit Engagement Team
- Additional Test Procedures
- Audit Opinion or Conclusion
Therefore, is clear that the standard requires that the auditor is the ONE who should:
- consider the work of other experts within the assurance engagement in order to achieve the audit objectives
- document the reason of such decision
- take all necessary safeguards to ensure:
- independence and objectivity of other experts,
- appropriate professional qualifications, competencies, relevant experience, resources and use of quality control processes
- have access to all work papers, supporting documentation and reports
- consider additional test procedures to gain sufficient and appropriate audit evidence in circumstances where the work of other experts does not provide such evidence
- consider whether supplemental testing of the other experts’ work is required
- appropriately document the use of the other expert’s work in the report
- be responsible to formulate an appropriate audit opinion or conclusion (including scope limitation where required)
Conclusion
Penetration testing used for IT/ Security assurance engagements falls usually under the “using the work of other experts for the engagement” scenario – IT auditors rarely have such technical knowledge to complete full penetration testing engagements and penetration testers rarely like to perform IT audit activities.
Therefore, the IT auditors usually involve experienced penetration testers in the assurance engagements – from other teams within the same audit company or from an external company. However, the responsibility to manage the full process in full compliance with the auditing standards the auditor applies remain with the auditor.
If the involved penetration testers are from the same audit company, the process can be managed very easy – the auditor can easily perform the assessment of the auditing standard requirements.
However, if the penetration testers are from an external company, there are significant chances that the auditor does not have the view on the independence, competencies, relevant experience, resources and use of quality control processes, but still have the possibility to assess these in the planning phase and decided accordingly regarding using their work.
The worst scenarios is when a set of specific requirements (i.e. regulatory requirements) forces the auditor to use the work of external experts and provide only limited resources for assessing the auditing standards requirements – for example, only the penetration testing report issued by an external expert.
Moreover, the report may be presented to the auditor by the audited entity and not the expert, therefore the auditor has no connection with the expert and cannot perform any or most of the assessments required by the auditing standards. And the auditor remains between standard and specific regulatory requirements – and yes, the auditor may chose or simply be forced not to accept such engagements, governed by poor regulatory requirements.
Bibliography: ISACA’s ITAF™: A Professional Practices Framework for IS Audit/ Assurance, 3rd Edition
Security Café 