AtLeastOneCoverage: Fraction of CQs with at least one required element correctly generated

This metric aims to evaluate the breadth of CQ coverage by computing the fraction of competency questions for which the generation includes at least one required element. With cov_i is the individual CQ coverage rate and N the number of CQs: AtLeastOneCoverage = |{ i : cov_i > 0 }| / N.

Example:

N = 4 CQs

CQ coverage rates cov: [1.0, 0.5, 0.0, 0.7]

CQs with cov > 0: {1, 2, 4} → count = 3

Result:

AtLeastOneCoverage = 3 / 4 = 0.75

What is being measured?

This metric reports whether the model engages with every CQ at all. A CQ left entirely unanswered fails this measure, even when others are fully covered. It is reported under both the term-level CQ2Term coverage and the axiom-level CQ2Onto coverage.

Metric information

RDF Resource Description Framework Flyer Icon
Title
Fraction of CQs with at least one required element correctly generated
Identifier
Keywords
Ontology Engineering, Metric, Competency Questions, Coverage
Contact Point
Publisher
Version
0.1.0
License
Category
Competency Question Coverage
Dimension
Task
Applicable for
Benchmark
CQ2OE Benchmark: Competency Questions for Ontology Engineering. A benchmark for evaluating LLM-assisted ontology generation from competency questions.
Status
Active
Active Proposed Inactive Deprecated
Ontology Engineering Group https://oeg.fi.upm.es