Is designing a memory graph around known data structure “overfitting” if I never touch the questions? [D]

building a missing data infrastructure and started benchmarking long multi-session conversations (LoCoMo). I know the data looks like: people, facts, claims, events, timestamps, relations. So I extract those into a graph.
I did not look at the QA pairs while building extractors or retrieval rules. No “if question contains X, fetch fact #173.”
Recall is very high and it keeps working on new conversations in the same format.
Is this classical overfitting, or just schema-aware engineering? What is the cleanest test that would convince you it isn’t leakage.