When F1 Is Not Enough: Auditing Codebooks for LLM-Assisted Annotation

arXiv:2611.08846v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for text annotation in social science, but standard performance metrics do not explain why errors occur. Low F1 may reflect limitations of the annotation system, defects in the written codebook, or inconsistency in…

Excerpt provided by arXiv cs.AI via its public feed. Full reporting belongs to the original publisher.