Evidence & Intermediate Representations

The research problem

Europe’s cultural and creative assets are documented in heterogeneous and often incomplete forms: spreadsheets, collection databases, archival descriptions, documents, images, audiovisual files, websites and legacy metadata. Before artificial intelligence can reconstruct richer knowledge from these sources, the available evidence must be acquired, identified, contextualised and transformed into representations in which observations and candidate assertions can be processed without prematurely turning them into authoritative knowledge.

ReactAI therefore treats intermediate representation as a distinct research problem. The objective is not to convert every source immediately into a final ontology or knowledge graph. It is to construct a portable, provenance-aware evidence layer between heterogeneous source material and subsequent AI-assisted knowledge production. This intermediate layer must retain the identity and origin of evidence, permit different computational representations of the same underlying assertions, and expose appropriate parts of the evidence and candidate knowledge to human review.

Betwixt

Betwixt is the principal software development through which this research is currently being operationalised. It provides reviewable relational projections of semantic assertions, allowing structured semantic material to move into ordinary rectangular data operations and human review workflows without treating the resulting tables as canonical knowledge representations.

The basic unit remains an atomic semantic assertion. Betwixt can organise these assertions into long or wide relational projections appropriate to a particular operation. Dual projections additionally represent the relationship between an assertion and evidentiary material while preserving the two as distinct semantic assertions. Evidence, candidate values, controlled ranges, contextual information and presentation metadata can therefore appear together in a review workspace without collapsing their different roles.

Figure 1: Evidence and descriptive context remain available while semantic dimensions form the review workspace.

This separation is important for the ReactAI methodology. The intermediate representation is not the Creative Asset Digital Twin and it is not intended to anticipate its complete semantic model. It provides the evidence and candidate-knowledge substrate on which subsequent AI methods can operate. In the ReactAI pipeline, heterogeneous inputs are transformed into an intermediate evidence representation; AI-assisted knowledge production can then enrich, test and reconstruct candidate knowledge; reviewed knowledge can subsequently be prepared for interoperable Digital Twin representations.

Try Betwixt

The current development of Betwixt in R already produces the wide and dual-wide projections of reviewable data in HTML.

Betwixt Review Workbench: a practical overview of the workbench’s philosophy.
betwixt.dataobservatory.eu: Reference implementation in R (planned: Go, perhaps Python.)

From evidence to stabilised knowledge

Betwixt is the implementation foreground of a broader research programme on semantic knowledge production. The Semantic Knowledge Production working paper distinguishes observations and candidate assertions from stabilised semantic states. The Review Algebra formalises candidate generation, review, finalisation and recursive transition between those states. Betwixt addresses the narrower representational problem between them: how the semantic material required for a particular computational or human operation can be projected into a workable relational form while retaining the identities required for provenance-aware processing.

Figure 2: Candidate assertions are reviewed within an explicit scope. Review returns semantic values and outcomes; a separate finalisation policy determines the next stabilised state, with provenance retained throughout.

This architecture deliberately separates observation, candidate knowledge, review and stabilised knowledge. A filesystem observation, OCR result, extracted identifier or AI-generated proposition can contribute evidence without silently becoming an institutional fact. Human review can be concentrated at the level at which judgement is meaningful, while deterministic and AI-assisted processes operate over explicit intermediate representations.

Relational projections

Betwixt currently develops four related projection forms: long, wide, dual-long and dual-wide. Long and wide forms provide alternative relational organisations of atomic assertions. Dual forms additionally expose an assertion connecting the reviewed subject to relevant evidence. The choice of projection is therefore determined by the operation or review task rather than by an assumption that one rectangular representation is the underlying knowledge model.

The R implementation is being developed alongside the conceptual work. Current functionality includes candidate dataset construction, controlled and open candidate ranges, review metadata, relational review projections and browser-based review workspaces. This allows the theoretical distinction between evidence, candidate assertions and reviewed knowledge to be tested against reproducible workflows rather than developed only as an abstract semantic model.

From intermediate representation to federation

The same projection principle also supports exchange between independently governed semantic systems. The Bounded Federation working paper extends the problem from knowledge ingestion to graph-to-graph mediation: only the entities, predicates and assertions required for a particular purpose need to enter the federation operation. Betwixt provides a relational review surface in which those selected assertions can be reused, redefined, excluded or treated as not applicable without requiring the participating knowledge systems to merge or globally align their ontologies.

Figure 4: Bounded federation as graph-to-graph review. Selected assertions from Graph A enter an atomic tuple review workspace; reviewed outcomes warrant what can enter Graph B. The operation is mediation, not merging.

The resulting research programme is therefore broader than an R package, but deliberately narrower than a universal knowledge architecture. It investigates the intermediate representations required to move reproducibly from heterogeneous evidence to candidate semantic knowledge, from candidate knowledge to accountable human review, and from reviewed assertions towards interoperable knowledge systems while delaying irreversible semantic commitments until they are actually required.