Interview and Testing

Many organizations have well-documented processes, but the actual work performance is wildly different from the documented process. Interviews are often used to identify these differences and are invaluable in helping controls assessors understand and clarify business practices and organizational expectations

The interview subjects for an assessment should be defined as part of the chartering process and explicitly named in the preaudit checklist. While the assessor should not be prohibited from interviewing people who have knowledge and experience with a control, the audited organization must make sure the proper person with knowledge of the control is available and prepared for the interview.

Just as the subjects need to be defined, so do the depth and breadth of the interview. Interviews can range from high-level discussions with groups of individuals together, to probing discussions with single subjects.

Similarly, the number of individuals and their roles in the controls should also be defined to ensure the coverage is appropriate for the assessment's purpose.

Proper documentation of the interviews must also be created and maintained. When the interview is conducted via a questionnaire, capturing the interviewee's responses is relatively easy. In-person interviews (or those done via VolP or video conferencing) provide the opportunity for recordings as the primary documentation of the interview. In most jurisdictions, the recorder or recording should confirm the subject's permission to record has been given. This ensures an accurate review of the interview can be performed.

Other techniques, such as taking notes, are certainly helpful but are subject to interpretation. As these artifacts may contain sensitive information, they should be properly secured and consistent with their classification.

Testing

Testing is the process of comparing the actual behavior of a system, process or activity under defined conditions with its expected behavior. A successful test documents that the expected and actual behavior are consistent. If they are not, the test documentation should reflect the difference in performance.

Testing requires that the expected performance be established before the test is conducted. Unless this measure is defined, it is impossible to determine if the control is effective.

Historically, there has often been a great divide between testing and security assessment, with testing being considered a development phase activity and assessment being the ongoing or recurring evaluation of the system. This perspective seems to have originated around the formal systems security testing and accreditation communities. These are considered formal security accreditation and its approval for operational use was something done only once at the end of development, or it was redone only when major changes to the system dictated its need.

The speed at which the threat landscape continues to change dictates a more flexible, nuanced use of all security assessment methods throughout the operational lifetime of a system. As a result, test techniques normally thought of as "for developer use only" can pay big dividends during ongoing or special-purpose security assessments conducted at any time.