In the context of graph layout, many algorithms have been designed to remove node overlapping, and many quality criteria and associated met-rics have been proposed to evaluate those algorithms. Unfortunately, a complete comparison of the algorithms based on some metrics that evaluate their quality has never been provided and it is thus difficult for a visualisation designer to select the algorithm that best suits their needs. In this paper, we review 22 metrics available in the literature, classify them according to the quality criteria they try to capture, and select a representative one for each class. Based on the selected metrics,we compare 9 node overlap removal algorithms. Our experiment involves 854 synthetic and real-world graphs. Finally, we propose a JavaScript library containing both the algorithms and the criteria, and we provide a Web platform, AGORA, in which one can upload graphs, apply the algorithms and compare/download the results.