In the September issue of Mortgage Solutions we took a broad look at regulatory examinations and, in particular, those which now face mortgage advisers. It was concluded that if someone is going to derive substantial financial and social benefits by giving mortgage advice, then it should be expected they will have achieved a defined minimum level of competence and go on to develop their skills to even greater levels as time passes.
In return it must be expected that the examining bodies authorised by, in this instance, the MCCB to design, develop and administer the examinations do so in a way which will result in a valid, fair and reliable assessment.
When we say an examination is valid we mean: does it measure what it is supposed to measure? The mortgage examinations are designed to assess whether candidates possess sufficient levels of generic technical knowledge, understanding of the mortgage market and its products and, that they can apply this knowledge and understanding in practical situations.
In achieving these aims, examiners will usually take the following steps:
• Identify the tasks that mortgage advisers perform.
• Establish the knowledge required to perform those tasks at a competent level.
• Establish the principles a person must understand if they are to perform at a competent level.
• Consider the circumstances that are likely to arise in which knowledge and understanding will need to be applied in order to make competent judgements.
• Consider the analytical skills required to perform the role of a mortgage adviser competently.
Only when these elements have been analysed can the examiners make decisions on:
• Whether knowledge, understanding and skills need to be assessed through written answers, or whether some other means of assessment will be appropriate.
• The number of questions needed.
• The time that candidates should be allowed to complete the exam.
For instance, if it is a necessary part of a mortgage adviser’s job to be able to write legibly, with correct punctuation, correct spelling and in a grammatically correct manner, then clearly a written exam is required. However, if these attributes are not needed in order to be a comp- etent mortgage adviser, then those aspects will not dictate the means of assessment.
For the mortgage examinations to be fair, exams must provide ample opportunity for success and if people fail then this must be because there is a meaningful deficit in their knowledge, understanding or application of mortgage matters.
So we might, for instance, want to ask why is the pass mark set where it is? Are the Institute of Financial Services (ifs) or CII really saying that a person who scores 69% in an exam is not competent to give mortgage advice, but someone who scores 70% is? Perhaps it is not quite as simple as this, but it is certainly true to say that a person who scores less than the pass mark will not be able to give unsupervised advice. In establishing pass marks, examiners take into account many factors but in particular they will want to establish the risks inherent if it is anything less than 100%.
So, perhaps if a person earning £25,000 per year approaches a mortgage adviser about borrowing £75,000 over 25 years ‘ on which they will end up paying back something in the region of £225,000, or more than 11 years’ worth of their present net salary ‘ they might also be surprised if they knew that in order to become a mortgage adviser, the score required in the examination could well have been 70% or less.
Increasing fairness of assessment, and reducing the current risks involved in assessing a candidate’s knowledge, has resulted in a move towards objective testing of exams. Assessing candidates through objective testing provides many advantages to the adviser, and also the consumer, including fairness and confidence.
Finally, we turn to the degree of reliability that can be placed on the results of a mortgage examination. In other words, if a person scores 80% in an examination on Tuesday how certain can we be that, given the same circumstances, that person would score 80% if the examination was taken again on Thursday.
Clearly for those candidates with exam result scores that are some way either above or below the pass mark, the degree of reliability is not so important. For those who score at or around the pass mark, however, then the reliability becomes extremely important.
In this respect, it is generally accepted that an objective style examination, with definite correct and incorrect answers, is for the most part more reliable than subjective essay style examinations. This contrast becomes particularly relevant in circumstances involving large volumes of candidates where essay style answers are evaluated by a greater number and, therefore, more diverse group of examiners.
Peter Bennett is managing director of Questionbank Management
• By 31 December 2002 it will be necessary for all those who want to be mortgage advisers to have passed either CeMAP, CeMAP Bridge or MAQ. Those who do not do so will have their earnings capacity severely reduced, if not completely removed, so the ifs, CII and the MCCB are under an obligation to ensure that the examinations are indeed valid, fair and reliable.
For more information about the ifs and the qualifications offered, please call the customer and student services department on 01227 818 609 or email firstname.lastname@example.org quot-ing reference number: SS025R.