FAIRness assessment evaluates the degree to which a digital object is Findable, Accessible, Interoperable, and Reusable. Here, our object of interest are semantic resources (from thesauri, terminologies, vocabularies to ontologies). Indeed, we have not yet seen a clear methodology implemented and tooled to automatically assess the level of FAIRness of semantic resources. The main objective of this work is to provide such methodology and tooling to guide semantic stakeholders for: (i) making their semantic resources FAIR through better use of standardized metadata; (ii) selecting relevant FAIR semantic resources for use. We propose a metadata-based automatic FAIRness assessment methodology for ontologies and semantic resources called Ontology FAIRness Evaluator (O'FAIRe). It is based on the projection of the 15 foundational FAIR principles for ontologies, and it is aligned and nourished with relevant state-of-the-art initiatives for FAIRness assessment. We propose 61 questions among 80% are based on the resource metadata descriptions and we review the standard metadata properties (taken from the MOD 1.4 ontology metadata model) that could be used to implement these metadata descriptions and improve the level of FAIRness of any semantic resource. We also demonstrate the importance of relying on ontology libraries or repositories to harmonize and harness unified metadata and thus allow FAIRness assessment. Moreover, we have implemented O'FAIRe in the AgroPortal semantic resource repository and produced a preliminary FAIRness analysis over 149 semantic resources in the agri-food/environment domain.