In 2010, the BCSC Examinations team developed a risk assessment model (the model) that will enable the examiners to develop a risk-based approach to conducting examinations on registrants directly regulated by the BCSC.
We designed the model to evaluate the risk of non-compliance with regulatory requirements. The primary purpose of the model is to ensure that the Examinations team focuses its resources on higher risk registrants and their activities. We built the model after reviewing similar risk models in use by other Canadian securities commissions and self-regulatory organizations.
A key element in the model is a two part, 46-question survey, sent to all registrants that the BCSC is the primary regulator. The survey collected information about the registrants' history, operations, business practices, and procedures. We used this information to populate our model.
Each survey question maps to one or more risk types or risk controls in the model. All registrants received an identical survey and we used the same risk model to evaluate them. This enabled the Examinations team to evaluate the risk level of all market participants on the same basis at the same point in time. Thanks to the cooperation of our registrants, we achieved a 100% response rate on this questionnaire.
How the Model Calculates Risk
The model evaluates three broad risk types as well as the controls used by each firm to control those risks. A metrics-based approach measures the risks and risk controls. The model identifies 23 specific areas of risk (within three broad risk types) and five specific risk controls for 28 risk measurements.
The three broad risk types are:
- Inherent Risk - which is due to the intrinsic nature of a registrant's business model, the products and services provided, its' client base, business strategies, and financial solvency. Twelve specific areas are measured.
- External Risk - which takes into consideration the state of the world and national financial environment and the registrant's fit within that environment. Three specific risk areas are measured.
- Internal Risk - which relates to the registrant's ability to operate effectively and efficiently based on its resources and processes. Eight specific risks are measured.
Risk Controls - Offsetting these three risk types are the registrant's risk controls, which enable the firm to identify, assess, and appropriately manage the above-identified risks. Five specific risk controls are measured.
Once we quantified all 28 of the above identified risk areas, we used a formula, described below, to assign a risk score for each registrant:
Risk Score = [(Inherent Risk + External Risk+ Internal Risk) - (0.4 x Risk Controls)]
The 0.4 factor applied against the risk controls reflects the level that controls offset the business risk faced by registrants. Controls cannot eliminate risk; they can only reduce risk.
We calculate a registrant's risk score with a three-step process:
- We assign each of the 28 areas of risk a weight, which is a relative measure of its importance in determining overall risk. The total weights of the 3 broad risk types total 100 and similarly the total weights of the risk controls also total 100. The weighting is the same for all registrants to ensure that we evaluate all registrants on the same basis.
- To complete the risk assessment of a registrant, we evaluate each specific risk and risk control and assign a score. For the specific risks, a score of 0 to 5 is used with 5 representing a high level of risk. For the risk controls, a score of 0 to 4 is used with 4 representing a strong level of control.
- We individually multiplied the ratings by weightings assigned to each specific risk or risk control, and we used the formula referenced above to determine the risk score.
The Examinations team analyzed the responses provided by 76 registrants and calculated risk scores for each firm. These scores are the basis for assigning a risk rating to each firm. The risk ratings used are:
Each registrant received a report disclosing their individual risk rating as well as a graphical depiction of their risk score in relation to all other registrants.
The most significant factor affecting risk ratings is not due to the nature of business activity. The factor that had the greatest impact on risk ratings is the level of risk controls that the firm had in place. Firms with high scores for risk factors that had well defined and comprehensive risk controls had a lower risk rating than firms with lower risk scores but weak or poorly defined controls.
This observation is consistent with our examination findings. Registrants that communicate strong, comprehensive policies and procedures communicated to all staff tend to have fewer and less significant deficiencies than firms that do not.
It is the intention of the Examinations team to bi-annually update the risk scores of all registrants directly regulated by the BCSC. On an ongoing basis, the Examinations team will update the risk scores for each registrant after completing a compliance examination.
This will allow a comparison of the risk score from the examination, to the risk score based on management responses to the risk survey. It will also allow the examination team to track changes in a registrant's risk score over time.
The ability to compare risk scores is important because it provides:
- a tool for Examinations staff to analyze the change in a registrant's risk profile and the specific areas of its operations that have the greatest impact on their overall risk score
- over time, a basis for Examinations staff to communicate changes in the risk rating to registrants at the completion of a compliance examination