How the European Digital Product Passport automatically identifies unknown JSON trees using Weighted Jaccard & PostgreSQL GIN arrays — and how this exact engine transforms industrial MRO materials sourcing, RFQ quotation, and cross-reference cataloging.
In global supply chains, thousands of manufacturers format data differently. Forcing every supplier or customer to declare explicit schema IDs (e.g. v1.0.4-patch2) results in 85% validation rejections due to minor naming mismatches.
The SIMILARITY_MATCH engine in dpp-validator ingests blind JSON payloads, flattens them into abstract structural trees, and queries PostgreSQL GIN indexes to detect the intended industry template in under 4 milliseconds.
The exact same math powers industrial MRO distributors: mapping chaotic supplier spec sheets (SKF, NSK, Timken) against vague customer RFQ requests, auto-building master item catalogs, and generating instant cross-brand quotations.
dpp-validator (Java Quarkus + PostgreSQL Reactive Client).
When an arbitrary JSON document hits /validate/v1, it contains nested objects and arrays. The extractor performs a recursive depth-first walk, transforming the tree into a flat set of dot-notation paths while normalizing array elements with []:
// Input JSON:
{ "Manufacturer_BrandOwner": { "ContactDetails": { "emailAddress": "compliance@shoe.eu" } } }
// Extracted Paths Set (inputProps):
paths = [
"Manufacturer_BrandOwner",
"Manufacturer_BrandOwner.ContactDetails",
"Manufacturer_BrandOwner.ContactDetails.emailAddress"
]
Every registered Schema in json_schemas stores its required_paths TEXT[] indexed with PostgreSQL GIN (Generalized Inverted Index). The repository executes an ultra-fast CTE query:
-- Executed by dpp-validator via Vert.x Reactive SQL Client
WITH schema_scores AS (
SELECT
sm.id, sm.schema_name, sm.schema_version, sm.required_paths_count,
(
-- Count how many of this Schema's required paths are present in the Input
SELECT COUNT(*)::int
FROM unnest(sm.required_paths) AS rp
WHERE rp = ANY($1) -- $1 = array of input properties
) AS matched_count,
$2::int AS input_count
FROM json_schemas sm
),
base_jaccard AS (
SELECT
id, schema_name, schema_version, matched_count, required_paths_count,
CASE
WHEN required_paths_count = 0 THEN 0.0
-- Penalize extra unrecognized fields with a 0.6 dampener
ELSE matched_count::float / (required_paths_count + 0.6 * (input_count - matched_count))::float
END AS jaccard_score
FROM schema_scores
)
Many industrial schemas declare dynamic attributes using regular expressions (e.g. ^mro_spec_[0-9]+$). The Java layer inspects schema_pattern_properties and rewards candidates that match wildcards:
// Pattern Property refinement in Java
int matchedPatterns = countMatchedPatterns(patterns, parsedInput);
int totalMatched = matchedBasePaths + matchedPatterns;
int totalRequired = requiredBasePaths + patterns.size();
candidate.finalScore = Math.max((double) totalMatched / inputCount,
totalMatched / (totalRequired + 0.25 * (inputCount - totalMatched)));
The engine enforces an uncompromising gate: candidates must achieve finalScore >= 0.3 (30% structural overlap). If no candidate reaches this threshold, it returns MatchType.NONE:
// Threshold decision in PlainJsonValidator.java
return candidates
.filter(c -> c.finalScore >= 0.3)
.max(Comparator.comparingDouble(a -> a.finalScore))
.map(this::asMatchResult)
.orElse(MatchResult.emptyResult());
// If MatchType.NONE:
// -> "No JSON matchResult found matching by similarity the input JSON"
Industrial suppliers (SKF, NSK, Timken, SMC, Festo) code identical components differently. A customer RFQ asks for "Ball bearing 20x47x14 rubber seal high temp". Sales engineers waste 45 minutes manually opening vendor PDF catalogs to find whether SKF 6204-2RSH/C3HT matches NSK 6204 DDU C3E.
By adapting the SIMILARITY_MATCH pipeline, the MRO Sourcing Hub flattens technical specs into canonical engineering vectors. It cross-compares customer requirements against hundreds of supplier databases simultaneously, generating an instant match score and cross-reference quote!
| Architecture Layer | DPP Context (EU Regulation) | Industrial MRO Sourcing Context |
|---|---|---|
| 1. Ingestion Target | JSON / JSON-LD Product Passport from brand carriers | Customer RFQ (Excel, PDF, Email, API) & Supplier Spec Sheets |
| 2. Normalization Engine | JsonPropertyExtractor extracts hierarchical JSON keys |
MRO Thesaurus extracts canonical attributes (Bore, OD, Width, Material, RPM, Seal) |
| 3. Schema / Master Catalog | json_schemas table in PostgreSQL (Footwear, Batteries, Textiles) |
mro_master_items Golden Records table with GIN array indexes |
| 4. Matching Formula | Weighted Jaccard over required_paths |
Hybrid: Structural Jaccard (80%) + Numeric Tolerance Scoring (20%) |
| 5. Actionable Output | VALID / INVALID compliance report for EU Registry |
Top 3 Cross-Brand Replacement options with Price, Stock & Tolerance Delta |
{
"item_type": "deep_groove_ball_bearing",
"dimensions": {
"inner_diameter_mm": 20.0,
"outer_diameter_mm": 47.0,
"width_mm": 14.0
},
"sealing": "contact_rubber_seal_both_sides",
"clearance": "C3",
"application": "high_speed_electric_motor"
}{
"matched": true,
"top_matches": [
{
"brand": "SKF",
"part_number": "6204-2RSH/C3",
"similarity_score": 0.985,
"exact_dimensions": true,
"price_usd": 4.85,
"lead_time": "In Stock (2,400 pcs)"
},
{
"brand": "NSK",
"part_number": "6204-DDUC3",
"similarity_score": 0.978,
"exact_dimensions": true,
"price_usd": 4.10,
"lead_time": "In Stock (1,200 pcs)"
}
]
}Deploy an OCR / NLP parser to convert raw supplier catalogs (PDF, CSV, API) and RFQs into normalized key-value trees. Map synonyms using a shared industrial thesaurus (e.g. DIN 625 = ISO 15 = Radial Ball Bearing).
Index millions of parts in PostgreSQL using TEXT[] arrays and GIN indexes. The ANY() and unnest() SQL operators evaluate 100,000 components in under 8ms on standard hardware.
Hook the similarity score into your CRM / ERP (SAP, Odoo). When an RFQ arrives, the engine auto-drafts the quote with the best-matched item, highlighting margins, stock availability, and equivalent brand alternatives.