[R]GNN Model For Fraud Detection Isn't Performing Well[R]
Mirrored from r/MachineLearning for archival readability. Support the source by reading on the original site.
We're writing a research paper on explainable fraud detection GNN model and in the first step we're creating a basic Graph Neural Network for that. We're using the most famous dataset available on this topic i.e IEEE CIS Fraud Detection Dataset and implemented all necessary feature engineering on that data (although majority of feature engineering is already performed in the dataset). Then we constructed a heterogeneous graph on that dataset. Various transaction features like device, transaction id, amount are embedded as nodes and connected with transaction nodes. But the issue is after training the model isn't performing well. It is producing average AUC of 0.87, PR-AUC of 0.52, recall@5% around 0.57 and precision@5% around 0.37 (We tried GCN, GraphSAGE and GAT, all performs almost same for rest data)
Whereas the SOTA models in this topic produce much better metrics. Can anyone tell where potentially we're doing things wrong?
[link] [comments]
More from r/MachineLearning
-
For the people who got reviews back from neurips, cvpr, eccv, etc and also tested their paper through an agentic reviewer like the stanford one, how different were the reviews? [D]
Aug 14
-
Building text to ASCII diffusion model , need advice and guidance [P]
Aug 14
-
A collision-entropy floor for watermark/retrieval AI-text detection. Looking for a sanity check before I take this further [D]
Aug 14
-
Are supervised and unsupervised learning still relevant today? [D]
Aug 14
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.