I came across an article recently that genuinely surprised me. Not because the topic was new to me — but because someone at the same education level, building the same kind of thing, had the exact same thought I did: what happens when you feed real SIEM alerts to an LLM and ask it to think like a Tier 1 analyst?

Durga Sai Sri Ramireddy — MS Cybersecurity at University of Houston — published a detailed writeup about building an AI-powered SOC triage pipeline using Claude API.

38 real alerts. Kerberoasting, AS-REP Roasting, lateral movement via CrackMapExec. The AI got 37 out of 38 right. But the one it got wrong told the whole story.

The Failure That Matters In test case 003, Durga fed the LLM an alert that looked like Kerberoasting on the surface — real account, real event code (4769), real SPN format. The only difference was the encryption type: 0x12 — AES-256. The strongest, most modern Kerberos encryption. Normal traffic. Not an attack. The AI returned:

Severity: High

Verdict: Likely True Positive

Escalate: True

And in its analyst notes, it wrote:

"RC4 encryption type (0x12) is a key indicator..."

0x12 is AES. Not RC4. RC4 is 0x17 — the actual Kerberoasting signature. The model had the mapping completely backwards. And it stated it with High confidence. No hedging. No uncertainty. Just a wrong answer delivered like a fact.

What Is an LLM Hallucination ?

Simply Put An LLM hallucination is when an AI confidently makes up information that sounds completely real. Not random nonsense. Not obviously wrong. Something that fits the pattern of what a correct answer would look like — but is factually incorrect at a specific technical detail. This is the scariest kind of wrong answer in a security context. Because it doesn't look wrong. It looks like analysis. It reads like an experienced analyst wrote it. And if you trust it without verifying — you escalate a false positive, waste 30 minutes investigating normal Kerberos traffic, and slowly start losing faith in the system. Alert fatigue starts with one too many false escalations. And one hallucinated high-confidence verdict is one too many.

Why This Happens ? - The Real Reason

Language models don't check facts. They complete patterns. When the AI saw the test-003 alert — correct EventCode, real account name, real SPN format — it started building toward a Kerberoasting verdict. The surrounding context was too convincing. By the time it hit the encryption type field, it was already pattern-completing toward a Kerberoasting conclusion. So it rationalized 0x12 as the attack indicator — because that's what fit the narrative it was already constructing. This is the actual failure mode of LLMs in security contexts.

Not hallucinating IP addresses into empty fields — the model handled that fine. The failure happens when partial data fits a plausible pattern and the model fills in the specific technical detail incorrectly. The confidence score doesn't tell you which situation you're in. That's the part that makes it dangerous.

"This failure mode is exactly what I'm designing around in my AEGIS Platform — specifically the AEGIS Alert Triage Engine inside CyberSentinel AI, which I'm actively building right now. A single-layer LLM approach has one point of failure. Whatever the model gets wrong, you get wrong. There's nothing catching it before the analyst sees it. So I'm building three layers. Each one catches what the previous one can't."

"If you're working in detection engineering or SOC operations — I'd genuinely love feedback on this architecture before I finalize the implementation. What would you add to Layer 1?"

Layer 1 — Rule-Based Suppression

Deterministic. No ML. No LLM.

Just Python dictionaries and lookup tables.

python - snippet:

KERBEROS_ENCRYPTION_TYPES = {

"0x17": {

"name": "RC4-HMAC",

"suspicious": True,

"reason": "Kerberoasting indicator — RC4 downgrade attack"

},

"0x12": {

"name": "AES-256",

"suspicious": False,

"reason": "Normal modern Kerberos — not an attack indicator"

},

"0x11": {

"name": "AES-128",

"suspicious": False,

"reason": "Normal modern Kerberos"

},

}

The test-003 alert never reaches the LLM with an ambiguous encryption type. Layer 1 looks up 0x12, finds AES-256, marks it not suspicious, and the alert is classified as a likely false positive before any language model is involved.

No hallucination possible. The classification is deterministic.

Layer 1 also handles known false positive patterns — scheduled tasks that always fire, service accounts doing routine authentication, backup software generating process creation events at 2am. These patterns are whitelisted. The LLM never sees them. Analyst never sees them either.

Layer 2 — Random Forest ML Scoring

For alerts that pass Layer 1, a Random Forest classifier trained on historical alert dispositions gives each one a confidence score from 0 to 100.

The training data comes from real analyst decisions — every time an analyst marks an alert as True Positive, False Positive, or Requires Investigation, that decision gets added to the training set. Over time the model learns what genuine attacks look like in that specific environment.

An AES-encrypted Kerberos ticket from a SQL Server SPN requested from a known server IP at a normal time of day — scores LOW. Because that's exactly what months of normal domain traffic looks like in the training data.

The ML model doesn't reason. It doesn't pattern-complete. It scores based on statistical proximity to known good and known bad. That's a fundamentally different kind of intelligence — and for this specific task, it's more reliable than language model reasoning.

Layer 3 — LLM Reasoning via RAG

Only alerts that Layer 1 and Layer 2 can't confidently classify reach the LLM.

By the time an alert gets here it already has:

A rule-based classification from Layer 1

An ML confidence score from Layer 2

Contextual flags from both layers

The LLM's job is not to make the primary verdict. It's to explain and contextualise — to write the analyst-facing summary, suggest next steps, and flag what additional data would confirm or rule out the threat.

And crucially — it queries CyberSage, my RAG knowledge base over MITRE ATT&CK, NIST CSF, and CIS Controls — before generating any output. So even if the LLM would naturally pattern-complete toward a wrong answer, the RAG retrieval grounds it in verified technical documentation first.

The encryption type mapping is in the knowledge base. 0x17 = RC4 = Kerberoasting indicator. 0x12 = AES-256 = normal. The LLM reads that before it reasons. It doesn't have to remember from training weights. It retrieves from a verified source.

What the Triage Output Looks Like ?

Every alert that goes through the AEGIS Alert Triage Engine produces a structured JSON output:

{

"severity": "High",

"verdict": "True Positive",

"mitre_tactic": "TA0006 - Credential Access",

"mitre_technique": "T1558.003 - Kerberoasting", "false_positive_probability": 8,

"escalate": true,

"escalation_reason": "RC4 encryption on user-named SPN — offline cracking likely",

"recommended_next_steps": [

"Check source host for Rubeus or Invoke-Kerberoast",

"Verify if SPN account has privileged access",

"Reset SPN account password with 25+ char random string"

],

"analyst_notes": "RC4 (0x17) confirmed via rule-based lookup — not LLM inference. ML score: 91/100. High confidence true positive.",

"triage_layer": "ML + LLM",

"layer1_result": "Not suppressed — 0x17 flagged as suspicious", "layer2_score": 91

}

The analyst notes field tells the analyst exactly which layer made the classification and how confident each layer was. Full transparency. No black box.

The Comparison Dashboard

"I'm also designing an AI vs Analyst comparison dashboard into the SOC Forge environment — same concept Durga implemented, but with a few additions I think make it significantly more useful:

— BFSI context — every alert will show the FinSecure business impact and the specific regulatory implication under RBI CSCRF or CERT-In

— Layer breakdown — which of the 3 layers classified the alert, the confidence score from each layer, and why

— ServiceNow integration — the Save button creates the actual incident ticket automatically, not just a local record

— Training data capture — analyst dispositions feed back into the Random Forest model so the system learns from every disagreement

The comparison dashboard isn't just for evaluation. That's actually the part I'm most interested in building — the feedback loop. Every time an analyst disagrees with the AI verdict and explains why, that becomes training data. Over time the ML layer improves. The false positive rate goes down. The analyst queue gets cleaner.

That feedback loop is what turns a triage tool into a learning system. And that's what I want AEGIS to be — not a tool you deploy and forget, but one that gets better the more your team uses it.

I'm building this now. If you've built something similar — I'd genuinely like to know how you handled the training data quality problem. How do you make sure analyst disagreements are meaningful signal and not just noise?"

The comparison isn't just for evaluation. It's how the system gets smarter. Every time an analyst disagrees with the AI verdict and explains why — that's training data. Over time the ML layer improves. The false positive rate goes down. The analyst queue gets cleaner.

That feedback loop is what turns a triage tool into a learning system.

What This Achieves ?

Hartono et al. (2024), published in IJMRAI, demonstrated that AI-augmented UEBA in SIEM systems reduced false positives by 40% and cut incident triage time by 78% compared to rule-based SIEM alone. That's my benchmark target.

The 3-layer architecture is designed to exceed it — because it combines rule-based precision (no hallucination on known patterns), ML pattern recognition (environment-specific learning), and LLM reasoning (contextualisation for genuinely ambiguous alerts).

Each layer has a different failure mode. The combination covers what any single layer misses.

The Lesson

Durga's article ended with this:

"Someone has to know that 0x17 is RC4 and 0x12 is AES and why that distinction determines whether you escalate or close the ticket. That person is the analyst. The AI is the tool."

I agree with that completely. But I'd add one thing:

The architecture should also know. Before the LLM ever sees the alert. Before the analyst ever sees the verdict.

That's what Layer 1 is for.

What I'm Building ?

This triage engine is one component of AEGIS — an open-source AI-augmented security operations platform I'm building for Indian BFSI institutions. The full platform includes a behavioral EDR agent, Splunk-based SIEM, NLP threat intelligence engine, and the LLM analyst copilot that houses this triage engine.

Every architectural decision is backed by peer-reviewed research. Every component is designed so the AI assists the analyst — not replaces them.

The full platform is dropping on GitHub soon.

If you're working in detection engineering, blue team, or SOC operations — I'd love your feedback on the architecture. What would you add to Layer 1? What patterns do you see in your environment that a rule-based suppression layer should know about?

Anvesh Raju Vishwaraju | MS Cybersecurity — UNC Charlotte

Building AEGIS — open-source AI-SOC platform for Indian BFSI

GitHub: github.com/its-me-anvesh-var

LinkedIn: linkedin.com/in/arv007

References

1. Durga Sai Sri Ramireddy (2026). I Built an AI-Powered SOC Triage Pipeline and Caught a Hallucination in the Process. Medium. https://durgaramireddy.medium.com/ai-soc-triage-claude-api-hallucination-e2d2b444ab30

2. Hartono et al. (2024). Enhancing User and Entity Behavior Analytics in SIEM Systems Using AI-Powered Anomaly Detection. IJMRAI. — F1-score 0.90, 40% false positive reduction, 78% triage time reduction.

3. Al-Shehari et al. (2023). Insider Threat Detection Model Using Anomaly-Based Isolation Forest Algorithm. IEEE Access, 11, 118170–118185.

4. (2025). Large Language Models for Security Operations Centers: A Comprehensive Survey. arXiv:2509.10858.

5. Gupta et al. (2024). A Comprehensive Survey of Retrieval-Augmented Generation (RAG). arXiv:2410.12837.

Ask any frontier LLM "what sub-techniques fall under T1078 — Valid Accounts?" and you'll get a confident, detailed answer. You'll also get things that are wrong. Not because the model is unintelligent — because MITRE ATT&CK is a living document updated quarterly, and the model's training data has incomplete, inconsistent coverage of its specific taxonomy.

This is the knowledge grounding problem. And in security operations, hallucinated answers have consequences.

The RAG Solution

Retrieval-Augmented Generation, introduced by Lewis et al. at NeurIPS 2020, addresses this by separating retrieval from generation. Instead of asking the model to recall from parametric memory, you retrieve the relevant document chunk at query time and include it in the context window. The model generates from evidence, not memory.

The architecture has three components: a document store, an embedding model, and an LLM. Documents are chunked and embedded into a vector space. At query time, the question is embedded using the same model, and the nearest chunks by cosine similarity are retrieved. Those chunks form the context for the LLM.

Implementation Decisions

For the embedding model I chose sentence-transformers/all-MiniLM-L6-v2 — 384-dimensional dense vectors, lightweight enough to run on CPU, with strong performance on semantic similarity tasks. The SBERT paper (Reimers & Gurevych, 2019) demonstrates that bi-encoder architectures like this outperform cross-encoders for retrieval at scale due to the ability to pre-compute document embeddings.

For the vector store I chose ChromaDB over FAISS or Pinecone. The key constraint was privacy — a SOC analyst querying sensitive incident data should have that data stay on-premise. ChromaDB persists locally with zero infrastructure dependency.

The chunking strategy used RecursiveCharacterTextSplitter with 800-token chunks and 100-token overlap. The overlap is critical — it prevents information loss at chunk boundaries, where a technique description might span a split point.

Measuring Retrieval Quality

I evaluated precision across 50 manually labelled test queries spanning all three frameworks — MITRE ATT&CK, NIST CSF, and CIS Controls. With top-k=5, precision was 85%. The primary failure mode was cross-framework contamination: a query about MITRE lateral movement techniques occasionally retrieved NIST CSF chunks because of surface-level keyword overlap around terms like "network" and "access."

The fix was metadata source filtering — restricting retrieval to a specific framework when the query intent is framework-specific. This brought precision on MITRE-specific queries to 91%.

The Hallucination Guard

Every response includes a lightweight confidence check. If the generated answer substantially exceeds the length of the retrieved context, it signals the model went beyond what was grounded. The model is also prompted to explicitly acknowledge when context is insufficient. In production, a more rigorous approach would embed both answer and context and measure cosine similarity — but the heuristic reliably catches the most common failure mode.

The Multi-Provider Architecture

The LLM layer uses a fallback chain: Ollama locally first (llama3.2:3b on Apple Silicon Metal GPU), then Groq's free API tier, then Claude. This means the tool works with zero API cost and zero data egress when Ollama is running — the entire pipeline from query to answer stays on-device.

Results

Average end-to-end latency with Ollama on M-series Apple Silicon: 1.6 seconds. Retrieval precision: 85% overall, 91% on framework-specific queries. The hallucination guard correctly flagged 7 of 8 cases where the model drifted beyond retrieved context in testing.

What I'd Do Differently

The current implementation uses pure dense retrieval. The BEIR benchmark (Thakur et al., 2021) demonstrates that hybrid retrieval — combining BM25 sparse retrieval with dense retrieval — consistently outperforms either alone on domain-specific corpora, which security frameworks are. A hybrid approach would likely push precision above 90% without requiring more data.

Code and Knowledge Base

The full implementation is at github.com/its-me-anvesh-var/cybersecurity-rag-assistant. The knowledge base currently covers MITRE ATT&CK, NIST CSF, and CIS Controls — contributions of additional framework documents are welcome.

References

1. Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020.

2. Reimers & Gurevych — Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. EMNLP 2019.

3. Thakur et al. — BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. NeurIPS 2021.

4. MITRE ATT&CK Framework v14 — Enterprise Tactics and Techniques.

5. NIST Cybersecurity Framework v1.1.

In most security operations centres, offensive and defensive knowledge exist in separate silos. Penetration testers understand exploitation chains intimately. SOC analysts understand alert patterns. Rarely does one person hold both perspectives simultaneously — and that gap is exactly where attackers operate.

This post documents the research and engineering decisions behind PentestX — an AI-augmented CLI toolkit that attempts to close that gap by pairing every offensive finding with its defensive counterpart.

The Problem

Consider a standard nmap scan that identifies OpenSSH 6.6.1 on port 22. A penetration tester immediately thinks: CVE lookup, Metasploit auxiliary module, credential brute-force. A SOC analyst monitoring that same host thinks: is this port supposed to be open? These two perspectives need to be unified at the point of discovery — not in a post-engagement report written weeks later.

Existing tooling doesn't solve this. Metasploit is an offensive framework. Splunk is a defensive platform. Nothing sits in the middle and speaks both languages simultaneously.

The Approach

PentestX is structured as a modular Python CLI with four layers — recon, vulnerability assessment, credential analysis, and SOC triage — all feeding into a shared AI engine. The key architectural decision was the multi-provider LLM fallback chain, validated by the RouteLLM routing framework (Shnitzer et al., arXiv 2024): Ollama locally first for privacy and zero cost, then Groq, HuggingFace, and Claude as sequential fallbacks.

For the knowledge retrieval layer, I implemented a RAG pipeline following Lewis et al.'s NeurIPS 2020 architecture — LangChain orchestration over ChromaDB with local sentence-transformer embeddings (all-MiniLM-L6-v2). The knowledge base covers MITRE ATT&CK techniques, high-impact CVEs, and Splunk SPL detection queries. Retrieval latency averages under 2 seconds with zero API cost.

Key Finding — IOC Extraction Precision

The log parser module extracts seven IOC types from raw log files using compiled regex patterns. Validated against a 500-line ground-truth dataset spanning Apache access logs, SSH authentication logs, and Wazuh alert exports:

• Precision: 91%

• Recall: 96%

• Primary false positive source: RFC1918 addresses and UUID collision with MD5 pattern

The RFC1918 issue was addressed with explicit CIDR exclusion — private ranges (10.x, 192.168.x, 172.16-31.x) are filtered at extraction time since they're useless for external threat intel lookups. The UUID/MD5 collision was addressed with context-aware length filtering.

The AI Layer in Practice

The most practically useful module is the exploit suggester — it queries the NVD API for CVEs matching a discovered service, then prompts the AI engine with a structured template that requests: most exploitable CVE, Metasploit module, exploitation prerequisites, detection evasion approach, and the corresponding blue team detection query. In a single output, a security professional sees both sides of the attack.

Limitations and Future Work

The regex-based IOC extractor has known weaknesses against obfuscated indicators — hex-encoded IPs and base64-encoded domains currently evade detection. A transformer-based NER model (e.g., fine-tuned on the CyberNER dataset) would improve precision on obfuscated indicators but adds inference latency. The web scanner's payload set covers OWASP Top 10 categories A03 and A07 but doesn't yet handle second-order injection or DOM-based XSS — both are on the roadmap.

Conclusion

PentestX demonstrates that the detection-exploitation gap is addressable with composable tooling and locally-run AI.

The full toolkit, knowledge base, and setup script are available at

github.com/its-me-anvesh-var/pentestx

The most dangerous attackers already think in both languages simultaneously. Defenders need tools that do the same.

References

1. Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020.

2. Shnitzer et al. — RouteLLM: Learning to Route LLMs with Preference Data. arXiv 2024.

3. Paxson et al. — Mining Threat Intelligence from Billion-scale SSH Brute-Force Attacks. USENIX Security 2020.

4. OWASP Testing Guide v4.2 — Web Application Security Testing.

5. MITRE ATT&CK Framework v14 — Enterprise Tactics and Techniques.