We proposed Adaptive-masking for Graph Embedding (AGE) to improve structured graph embeddings and enhance LLM performance on GraphQA tasks. The method introduced JEPA, a self-supervised learning architecture which enhanced the graph-structure embedding for downstream reasoning tasks. Our node sampler demonstrated its effectiveness in the ablation study, successfully identified key nodes within given graphs. The quantitative results confirmed AGE's consistent performance gain in GraphRAG tasks while maintaining computational cost. We hope this work contributes to structured knowledge representation for intelligent agents and facilitates cross-modal reasoning through structured perceptual representations.</p>\n","updatedAt":"2026-07-06T05:26:04.561Z","author":{"_id":"6657bfab21253c7068862aec","avatarUrl":"/avatars/8191ea86b5eeddaed91a1b7df5bf715d.svg","fullname":"Nguyen Huu Bao Long","name":"baolongnguyenhuu","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":2,"identifiedLanguage":{"language":"en","probability":0.8963307738304138},"editors":["baolongnguyenhuu"],"editorAvatarUrls":["/avatars/8191ea86b5eeddaed91a1b7df5bf715d.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.00052","authors":[{"_id":"6a45c4d74f1dd35e48fb8ea5","user":{"_id":"6657bfab21253c7068862aec","avatarUrl":"/avatars/8191ea86b5eeddaed91a1b7df5bf715d.svg","isPro":false,"fullname":"Nguyen Huu Bao Long","user":"baolongnguyenhuu","type":"user","name":"baolongnguyenhuu"},"name":"Bao Long Nguyen Huu","status":"claimed_verified","statusLastChangedAt":"2026-07-05T21:13:53.464Z","hidden":false},{"_id":"6a45c4d74f1dd35e48fb8ea6","name":"Atsushi Hashimoto","hidden":false}],"publishedAt":"2026-06-30T00:00:00.000Z","submittedOnDailyAt":"2026-07-06T00:00:00.000Z","title":"AGE: Adaptive-masking for Graph Embedding in Graph Retrieval-Augmented Generation","submittedOnDailyBy":{"_id":"6657bfab21253c7068862aec","avatarUrl":"/avatars/8191ea86b5eeddaed91a1b7df5bf715d.svg","isPro":false,"fullname":"Nguyen Huu Bao Long","user":"baolongnguyenhuu","type":"user","name":"baolongnguyenhuu"},"summary":"GraphRAG is an extension of retrieval-augmented generation (RAG) that supports large language models (LLMs) by referring to graph-structured data as external knowledge. While this technique ideally captures intricate relationships, it often struggles with graph representations for LLMs, particularly for frozen LLMs, due to the misalignment between graph-based and text-based latent features. We tackle this issue by introducing the {\\it Adaptive-masking for Graph Embedding (AGE)}. AGE employs a Transformer in a mask-based self-supervised learning (SSL) approach. We designed the architecture similar to text embedding encoders, addressing the latent feature misalignment. In contrast to natural language texts, graphs are concise representations, and there exist {\\it key nodes} that hold dominant contextual information, which are challenging to predict from their surroundings. Masking such key nodes leads to inefficiency in the SSL process. Therefore, AGE focuses on predicting nodes apart from key nodes, utilizing a learnable node sampler. Our experimental results indicate that AGE significantly improves approaches using non-parametric search component in GraphQA tasks, achieving superior accuracy across four benchmark datasets with distinct characteristics.","upvotes":3,"discussionId":"6a45c4d74f1dd35e48fb8ea7","ai_summary":"GraphRAG extends RAG by incorporating graph-structured data for LLMs, addressing latent feature misalignment through Adaptive-masking for Graph Embedding (AGE) that uses Transformer-based self-supervised learning with learnable node sampling.","ai_keywords":["retrieval-augmented generation","large language models","graph-structured data","latent feature misalignment","Transformer","self-supervised learning","GraphQA","key nodes","node sampler"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","organization":{"_id":"652f8404902fe76a6d297852","name":"omron-sinicx","fullname":"OMRON SINIC X Corp.","avatar":"https://www.gravatar.com/avatar/11268582af8ebd0bc9e45081c12a4f40?d=retro&size=100"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"},{"_id":"6657bfab21253c7068862aec","avatarUrl":"/avatars/8191ea86b5eeddaed91a1b7df5bf715d.svg","isPro":false,"fullname":"Nguyen Huu Bao Long","user":"baolongnguyenhuu","type":"user"},{"_id":"69bb5768a3edbe68544c5e4c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/M52S1Y2qWz8ADpx1Ixe0_.png","isPro":false,"fullname":"Zixuan Xu","user":"chloe-taylor202","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"652f8404902fe76a6d297852","name":"omron-sinicx","fullname":"OMRON SINIC X Corp.","avatar":"https://www.gravatar.com/avatar/11268582af8ebd0bc9e45081c12a4f40?d=retro&size=100"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.00052.md","query":{}}">
AGE: Adaptive-masking for Graph Embedding in Graph Retrieval-Augmented Generation
Abstract
GraphRAG extends RAG by incorporating graph-structured data for LLMs, addressing latent feature misalignment through Adaptive-masking for Graph Embedding (AGE) that uses Transformer-based self-supervised learning with learnable node sampling.
GraphRAG is an extension of retrieval-augmented generation (RAG) that supports large language models (LLMs) by referring to graph-structured data as external knowledge. While this technique ideally captures intricate relationships, it often struggles with graph representations for LLMs, particularly for frozen LLMs, due to the misalignment between graph-based and text-based latent features. We tackle this issue by introducing the {\it Adaptive-masking for Graph Embedding (AGE)}. AGE employs a Transformer in a mask-based self-supervised learning (SSL) approach. We designed the architecture similar to text embedding encoders, addressing the latent feature misalignment. In contrast to natural language texts, graphs are concise representations, and there exist {\it key nodes} that hold dominant contextual information, which are challenging to predict from their surroundings. Masking such key nodes leads to inefficiency in the SSL process. Therefore, AGE focuses on predicting nodes apart from key nodes, utilizing a learnable node sampler. Our experimental results indicate that AGE significantly improves approaches using non-parametric search component in GraphQA tasks, achieving superior accuracy across four benchmark datasets with distinct characteristics.
Community
This comment has been hidden (marked as Resolved) This comment has been hidden We proposed Adaptive-masking for Graph Embedding (AGE) to improve structured graph embeddings and enhance LLM performance on GraphQA tasks. The method introduced JEPA, a self-supervised learning architecture which enhanced the graph-structure embedding for downstream reasoning tasks. Our node sampler demonstrated its effectiveness in the ablation study, successfully identified key nodes within given graphs. The quantitative results confirmed AGE's consistent performance gain in GraphRAG tasks while maintaining computational cost. We hope this work contributes to structured knowledge representation for intelligent agents and facilitates cross-modal reasoning through structured perceptual representations.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.00052 in a model README.md to link it from this page.
Cite arxiv.org/abs/2607.00052 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.00052 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.