Reading Centre

A Reading Centre is staffed in the following areas, a lead clinician, principal investigators, senior graders, graders, graders in training, epidemiologists, statisticians, photographers and other staff as needed. The lead clinician and principal investigators are ophthalmologists.

Grader Training

Graders have no medical background, but are usually educated to degree level. A senior grader trains them until they perform to a certain standard. Graders must sit a grading exam at the end of their training. Overall training is for a period of between four and six months with additional specialised training sessions. For a grader to receive certification they need to grade a certain set of exam photographs, where grading has been agreed by the senior grader and the reading centre advisor. They need to grade this set at least twice with a 14-day gap in between. The grading is then entered onto a database and intra and inter-observer agreement are calculated. If this is acceptable a grader can grade alone with some supervision. If not acceptable, further training is followed by re-examination. Graders are generally certified to grade for a specific disease and not as a grader. Currently the problem with grader training is that no universally accepted training schedule exists. In addition there is the question of; who certifies graders in areas when there is no accepted grading form/severity scale

Logistics

After the protocol has been agreed photographs are taken and sent to The Reading Centre. On their arrival the images are checked and catalogued. They are then kept under lock and key for the entire time that they are held at the Reading Centre. Graders do not have access to the photographs and are given the images on a daily basis by the data handler. Once graded, images are given directly back to the data handler and then once again locked away along with the completed graded forms. As images are always double-graded this process ensures that no grader has access to previous grader results. The data collected is then entered into a secure database which is stored on a separate drive, no-one other than the data co-ordinator and IT staff have access to this drive.

Quality Control

For quality control purposes a subset of photographs from each study is double/triple graded and entered in to the database. Again, 14 days has to elapse before a grader is allowed to regrade images they have previously graded. This ensures that graders will be unable to remember or memorise any of their prior grading results. Subsets of photographs are graded by the same person, entered, analysed and discussed. A separate set of photographs are kept for temporal drift, they are graded every year, entered and never discussed in detail. If results of this set are not up to standard then further tuition is given.

A major problem with quality control is that all reading centres currently do their own. This makes it very difficult to assess studies. There is a push for specific quality control procedures or image 'swapping'.

Intra and inter observer agreement can be described in three ways; exact agreement, kappa values and weighted kappas. When presenting these to the graders, specific explanation is given about how they influence the overall results.

Protocol and Time

When a protocol is developed for grading it needs to be categorically decided and stated what we are to be looking for when we grade. Points to consider are; need to be able to answer the proposed question, needs to be specific enough and needs to be quantifiable and reproducible. Most importantly, a study's needs have to be matched with the centre's capabilities and needs to be completed within the given time-frame. A rough guide to how many images can be graded within a specific time are given below:

Experienced Grader; 10 AMD photos/one hour, 3 Diabetic Retinopathy/one hour, 3 FFA/one hour.

Not Experienced Grader; 3 AMD photos/one hour, 1-2 Diabetic Retinopathy/one hour, 1 FFA/one hour.

For concentration purposes and for maximum accuracy a grader should only be allowed to grade for between 4-6 hours a day.

Problems with Grading for Different Studie

In general there are still problems that continue to occur with grading for different studies. As yet, there is no universally accepted grading form and no universally accepted fundus photography system in existence. It is also a concern that there is no official programme for grader training and quality control. When grading for screening purposes, such as diabetic retinopathy, there is no information if high-risk or end stage disease patients are seen or not. These all need to be kept in mind when analysing grading results.

Summary

To summarise the process of grading an important point is to aim before you grade. Make sure that you have taken all aspects of the study into consideration and clarified anything that is not clear; it is imperative that a clear and concise protocol has been developed and agreed upon. Make sure that you have your logistics right. Detailed and structured grading is essential. Finally, be certain that you have your quality assurance correct.

© Copyright 2008-2017. The Reading Centre, Moorfields Eye Hospital NHS Trust Foundation. All rights reserved.