Clustering is probably one of the most frequently used ap- proaches when facing a scaling problem in large collections of documents. In many situations, however, the choice of the most appropriate algo- rithm for clustering can turn into a real dilemma. Numerical criteria have been proposed to evaluate the quality of the results of clustering algorithms. However, so many different criteria have been proposed that the dilemma even worsens. Most criteria reveal different aspects of the quality of the results and hide others. The aim of this paper is to help with the understanding of clustering and to facilitate the comparison and the choice of clustering algorithm for a given purpose. Our proposal consists in studying both quality evaluation criteria and clustering algo- rithms. We start by discussing a selected set of representative criteria, and further conduct a case study on a large set of real data, measuring not only the quality of different representative clustering algorithms but also the impact of each criterion on the ranking of the algorithms. By providing empirical results on large scale corpus of either documents or lexical networks useful to digital library, we hope to clarify the field and facilitate designers' choices.