Received: 4 August 2022 / Accepted: 31 August 2022 / Published: 2 September 2022

Accurate reference data to validate burned area (BA) products are crucial to obtaining
reliable accuracy metrics for such products. However, the accuracy of reference data can be affected
by numerous factors; hence, we can expect some degree of deviation with respect to real ground
conditions. Since reference data are usually produced by semi-automatic methods, where human-
based image interpretation is an important part of the process, in this study, we analyze the impact of
the interpreter on the accuracy of the reference data. Here, we compare the accuracy metrics of the
FireCCI51 BA product obtained from reference datasets that were produced by different analysts over
60 sites located in tropical regions of South America. Additionally, fire severity, tree cover percentage,
and canopy height were selected as explanatory sources of discrepancies between interpreters’
reference BA classifications. We found significant differences between the FireCCI51 accuracy metrics
obtained with the different reference datasets. The highest accuracies (highest Dice coefficient)
were obtained with the reference dataset produced by the most experienced interpreter. The results
indicated that fire severity is the main source of discrepancy between interpreters. Disagreement
between interpreters was more likely to occur in areas with low fire severity. We conclude that
the training and experience of the interpreter play a crucial role in guaranteeing the quality of the
reference data.