SpearXAI-Org: Target-Context-Aware Benchmark and Explainable Hybrid Reference Framework for LLM-Assisted Spear-Phishing Detection

Our paper’s biggest contribution isn’t a smarter phishing-detector. In fact, we openly admit that our hybrid classifier itself doesn’t beat a simpler one in any measurable way. What we actually contributed is:

1. A better test set. We built 8,200 emails split into 5 clear types — normal email, generic scams, human-written targeted scams, AI-written targeted scams, and AI-reworded targeted scams. Most past research only asked “scam or not,” lumping AI-written attacks in with everything else. This lets us (and others) test tools specifically against AI-written phishing.
2. A more honest way to test these tools. We checked for common mistakes other studies make — like accidentally letting the same email (or a reworded copy of it) appear in both the “training” and “testing” piles, which makes a tool look smarter than it really is. We fixed that.
3. A real-world reality check. We showed that even a tool that scores 99% in a clean lab test would still flag mostly innocent emails as “false alarms” once used in a real company inbox, where actual scams are rare. This is a really important, often-ignored point.
4. Proof that “fancier” doesn’t always mean “better.” We tested whether combining two AI methods, or adding extra explanation features, actually improved accuracy — and found it mostly didn’t. That’s a useful, honest finding, since many papers overstate how much these extra bells and whistles help.

Bottom line: Our real contribution is a fair, well-tested “rulebook and test set” for judging future phishing-detection tools — not a new, better detector itself.