Measuring risk factors

One of the fascinating subjects that is testing any broking firm intending to measure competence by way of knowledge assessment is "what is the pass mark that we should set in any given assessment".This week we are going to look at some revolutionary ideas that are being considered by a team of experts who are working on the premise that different classes of insurance business present different levels of risk to the customer and the broking firm, and that the risk presented by lack of knowledge varies between different classes.The research is at a very early stage and I am able to give you the idea only in broad outline, but the objective is that readers will be able to consider the information to date and that it will be helpful in deciding benchmarks for your own assessments. In assessing competence in a particular class of business the following should be considered:

• In a 100-question assessment, there is no pass mark, only a measure of risk if a practitioner does not know something
• True competence can be achieved only if an individual knows everything that needs to be known to perform a particular task
• Some knowledge will be more critical than other knowledge
• The process of maintaining competence is a continual process of filling gaps in required knowledge
• There is a point in any given level of knowledge requirement at which a firm will render an individual liable to direct supervision
• Types of product can be grouped in terms of risk to customer and broking firm by reference to certain key performance indicators, for example - potential size of loss, customer protection, history of avoidance by insurers, history of claims of negligence, history of file reviews. To explain this let us consider a multiple choice question paper that covers all technical aspects of six different types of business. The candidate has to answer multiple-choice questions. Each answer is then scored with a risk factor value, according to how many questions were answered correctly. For each product, the target risk factor is 1, so those factors where the risk factor is closest to 1 would be the benchmark for that industry category. A result less than one should cause concern with regard to the need for additional supervision.So, for example, if an individual is advising on satellite insurance he should have a very high standard of knowledge. If something goes wrong, the costs could be substantial. There is little protection for the customer other than the law and the regulator's requirements. On the other hand, the benchmark of knowledge for a practitioner advising on household insurance is 68 marks. Even if something does go wrong, the protection afforded to a private individual at law and via regulation and the ombudsman is very high and the level of potential financial loss is relatively low by comparison.In the graph below you will see how the risk factors grow above the critical '1' level as more answers are incorrect.Does the theory have value? I reserve judgment on that, as I think that there will be many - perhaps too many - variables and factors that will need to be built into the calculations. But the underlying theory is worth considering. I would summarise as follows:
• For every subject there must be questions that any practitioner must get right. (for example, knowledge of duty of disclosure)
• It is what a practitioner does not know that creates risk, not what they do know
• That as practitioners gain experience and advise on higher risk products, so access to supervision and the ability to refer reduces
• Consumer protection reduces the higher the risk of the product
• That risk can be reduced by increasing benchmarks below which direct supervision is required.
• Irrespective of the benchmark, the objective of a knowledge assessment is to measure gaps in knowledge and the purpose of a robust T&C Scheme is to continually strive to fill those gaps. Break this down further so that the theory can be applied to an individual firm's needs. The benchmark increases in direct proportion to experience, type of product and levels of supervision and monitoring that takes place or is available. While this is a new concept, I have to admit that there is some logic in the argument that someone advising on complex multi-million pound risks should be watched more carefully if they do not know something relating to that job than someone advising on household insurance and getting the same number of answers wrong. Do let us have your comments.
• Robin Wood is managing director of RW Associates. Email: ruy.lopez@brokercompliance.co.uk
• Using this CPD pageFor the vast majority of practitioners and indeed support and supervisory staff in our industry, CPD is about regular learning and study that is planned, recorded, timed and evaluated. If you are a member of a professional body with a CPD requirement then there will be certain rules regarding the quality and nature of study material, and the way in which it is recorded.For staff of GISC members this means recording on your individual training file what the learning was, who provided it and when.It might be structured, such as a course, a learning programme or exam study. But it can be unstructured. This form of study encompasses reading the trade press, technical material or taking part in activities to support your professional body. Some CPD requirements are points related (a little antiquated) and others require a time value to be allocated. For example, it might take one hour to read Insurance Times each week. Most of that could be put as a time value but, in reality, perhaps only an half hour was devoted to learning something. The rule is to be honest with yourself and record the time that is relevant. Always take time to make a note of what you felt you gained from the activity. This is useful information for anyone else considering the same activity.In response to the popularity of our CPD programme each week's CPD page can now be downloaded from our website.To download a PDF of this article as it appears in the magazine click here .