Introduction to Expert Annotators

Expert Annotators play a crucial role in AI, checking and organising the data that AI models learn from. GenAI annotators help AI understand data better, spot harmful and biased information and compare and choose more effective responses, making them an essential part of AI development. 

The media’s view on the importance of Expert Annotators

The importance of data annotators – as they’re often called – has caught the attention of many media outlets. They are vital because they improve the data that AI systems learn from. Kyle Lo, senior applied research scientist at the Allen Institute for AI (AI2), told TechCrunch that "the performance gains are coming from data, once you have a stable training setup". Media stories emphasise that without high-quality data preparation, which annotators provide, AI systems cannot perform well. This is often summarised as "garbage in, garbage out." So if you provide the wrong data, you’ll get inaccurate information back. 

Additionally, high-quality data is so important that even smaller AI models can sometimes perform better than larger ones if their data is well-prepared. As Lo explains, "It is possible that a small model with carefully designed data outperforms a large model." This highlights the critical role of Expert Annotators in ensuring AI systems are effective.

A world without Expert Annotators

An AI world without Expert Annotators would likely be full of errors and inefficiencies. Media reports suggest that poorly prepared data can lead AI systems to make mistakes, sometimes with serious consequences, especially in sensitive areas like healthcare. For example, if an AI model is only using unreviewed data or data from open sources, it could end up giving harmful advice such as suggesting a drug based on just its description. 

Expert Annotators help prevent such risks by reviewing answers, prompts, and entire dialogues before an AI learns from them, ensuring the AI knows what’s correct and relevant. Through their work, annotators tag datasets (i.e. correct, harmful, irrelevant), enabling AI to learn not only what is right but also what should be avoided. As James Betker, a researcher at OpenAI, pointed out on his blog, “training data -- not a model's design, architecture, or any other characteristic -- is the key to increasingly sophisticated, capable AI systems.”

The future needs Expert Annotators

The need for skilled Expert Annotators is expected to grow as AI technology becomes more complex. Sonam Jindal, who leads the AI, Labour and the Economy programme at the Partnership on AI nonprofit, points out in an interview with Time, "as you have a need to have higher-quality AI models, you also need higher quality data". Media stories project a future where data annotation becomes more specialised. This means annotators will be even more important for ensuring AI can perform safely and accurately in more advanced tasks. Today’s Expert Annotators are already skilled in fact-checking, applying critical thinking, switching between various themes and topics and following instructions meticulously.

Conclusion

Expert Annotators are foundational to creating reliable and efficient AI systems and their role is increasingly recognised as essential by the media. As technology continues to evolve, the skills of these professionals will become even more valuable in building AI that is not only advanced but also trustworthy.