In many situations, the choice of the most appropriate algorithm for clustering can turn into a real dilemma. Numerical criteria have been proposed to evaluate the quality of the results of clustering algorithms. However, so many different criteria have been proposed that the dilemma is even worsen. Most quality indices reveal different aspects of the quality of the results and hide others. The aim of this paper is to help with the understanding of this domain and to facilitate the comparison and the choice of clustering algorithm. Our proposal consists in studying both evaluation criteria and clustering algorithms. We start by discussing a selected set of representative criteria, and further conduct a case study on a large set of real data, measuring not only the quality of different representative clustering algorithms but also the impact of each criterion on the ranking of the algorithms. By providing both analytical and empirical results, we hope to clarify the field and facilitate designers choices.