ICFR implementation: testing efficiency of control system

In my previous articles I explained how to formalize business processes, how to identify different types of risks attributable to this business process, which types of controls exist and how to assess the level of risk. The next step is to investigate and conclude if the control system is effective or not.

Goals and approach

Best practice: there are two important questions defining the goals of controls testing procedure:

1. Whether the control procedures are implemented in accordance with approved controls design
2. Whether the control procedures are to be authorized and competent persons.

The testing plan should cover the following key issues:

1. Defining the scope of controls for testing
2. Method of testing
3. Defining population and testing unit
4. Testing period
5. Defining the approach for sample for testing from population
6. Volume of sample
7. Documentation of results including the definition of exceptions (over/understatement of acceptable levels of mistake).

Now we will analyze each of the seven points in more details.

Defining the scope of controls for testing

This step depends on the expected level of controls efficiency. Usually in this area the professional judgement is applied by audit teams (internal audit team or external consultants). Below I provide the example of the judgement applied:

In fact, there could be some other or additional criteria for example:

1. Number of incidents
2. Proportion of manual/automated controls in the process
3. Level of process formalization
4. Level of segregation of duties in the process
5. Others.

You could assess each criteria using 0-10 approach weighing each criteria and then calculate total amount.

Based on the total result there could be the following conclusions on the number of controls to be tested (example):

Method of testing

The following methods of testing exist:

1. Interview. Based on the results of interviews the following conclusions could be reached: level of knowledge and competency of the roles involved, transparency of the control procedure, control frequency, identification of breach of authority limitations within control procedure implementation. This method could be used in combination with review and inspection methods.



2. Review/analysis. This is the most appropriate method to get confirmation of there is no documentation of the control. Example: segregation of duties review.
3. Inspection. This includes physical verification of control existence. Example: check of signature, stamp, reports, some electronic documents. This type of testing is effective in relation to manual controls.
4. Re-performance. This includes re-performance of actual transactions selected for testing. The re-performed results are compared to the actual and analyzed for any difference. This method is used usually when the 1-3 types are not possible to apply.

Population and testing unit

For each control procedure there should be a defined total number of such procedures for the period or the population subject to testing.

Testing unit is the controlling procedure for each transaction.

Example: The accountant checks each received invoice for double-counting. Population for this testing is all invoices received for the period, testing unit – is one controlling procedure.

Sampling for population

There should be clear methodology for sampling. The following criteria should be taken into the account:

• Level of control environment stability
• Understanding of major business process points where mistakes occur
• Volume of population for testing
• The significance of this control for the whole process/sub-process
• Required accuracy of testing results
• Expected level of error.

Therefore, there are two types of sample selection: statistical and based on the judgement.

When using the statistical approach fundamental principles of probability theory are applied.

Below is the example:

! Important: the number of tested units could be increased if the criteria mentioned above are used to be more risky or volatile.

Example: The accountant checks each received invoice for double-counting. The number of invoices for the reporting period – 20 000. Basic testing sample is 25.

The following factors influence the application of increasing coefficients:

1. Complexity of control procedure made: the control is manual and performed by different persons, there is a two-options choice in the control procedure, low qualification of personnel: coefficient 40%
2. Expected level of mistake: negative incidents were registered in accounting systems: coefficient 20%
3. Type and purpose of control: there is no automatic reject on doubling of invoices: coefficient 20%.

When taking into account all above-mentioned factors, the sampling will be 25+ 25*40%+25*20%+25*20% = 45.

Documentation of results

It is very important to document the results of testing.

Defining the level and materiality of mistake/deviation

When defining the level of error, the following factors are taken into the account:

• If the operations (in the scope of testing) are routine, repeatable, formalized, then the acceptable level of error can be defined as:
  • 1-1.5% for controls, which are used to be significant for sub-process;
  • 1.5-2% for controls, which are less significant for sub-process.

If the level of error is higher than these limits, in this case the control can be assessed as ineffective.

• Although if in addition to the control tested there is an effective compensating high-level control within the process then the acceptable mistake level could be increased for 2% for significant controls and up to 3% for less significant.
• If within the testing process the same mistake was identified, which repeats for more than 2-3% units in the tested sample, it is concluded that the control is not effective.