ClosureRescueCoverage: CQ coverage credited via reasoner-inferred subsumption closure

This metric aims to evaluate CQ coverage with reasoning, crediting hierarchy axioms missed by strict matching but confirmed by the generated ontology's HermiT-inferred subsumption closure. Each gold axiom is decomposed into atomic SubClassOf and SubPropertyOf pairs; an axiom is rescued only when every one of its atomic pairs is present in the closure. Non-hierarchical axioms are scored only by the strict layer.

Example:

Gold_Axiom_for_CQ7: EquivalentClasses(SeafoodWine, WhiteWine ⊓ DryWine ⊓ pairsWith.Seafood)

Atomic_Pairs: { SeafoodWine ⊑ WhiteWine, SeafoodWine ⊑ DryWine, SeafoodWine ⊑ pairsWith.Seafood }

Generated_Closure_Contains:

  1. SeafoodWine ⊑ WhiteWine → matched
  2. SeafoodWine ⊑ DryWine → matched
  3. SeafoodWine ⊑ pairsWith.Seafood → unmatched

Result:

2 of 3 atomic pairs match → axiom NOT rescued under the all-or-nothing rule. It contributes 0 to CQ7's TP count under this layer.

What is being measured?

This metric reports how much additional CQ coverage reasoning can credit when the explicit axiom strings do not match. In experiments, closure rescue raises mean coverage by an average of +1.4 points but does not turn partial CQ coverage into full CQ coverage in the evaluated runs.

Metric information

RDF Resource Description Framework Flyer Icon
Title
CQ coverage credited via reasoner-inferred subsumption closure
Identifier
Keywords
Ontology Engineering, Metric, Competency Questions, Reasoning, HermiT, Closure
Contact Point
Publisher
Version
0.1.0
License
Category
Competency Question Coverage
Dimension
Task
Applicable for
Benchmark
CQ2OE Benchmark: Competency Questions for Ontology Engineering. A benchmark for evaluating LLM-assisted ontology generation from competency questions.
Status
Active
Active Proposed Inactive Deprecated
Ontology Engineering Group https://oeg.fi.upm.es