Gemini 4 Argon for Cybersecurity: Stronger at Detection Engineering Than Its Score Suggests

Gemini 4 Argon finds nearly as much as the best models we tested and reaches the vulnerable code every time, but it is noisy and expensive.

Overview

Google announced Gemini 4 Argon on September 30 as its frontier model for real-world coding, enterprise knowledge work and cyber defense, first releasing it to trusted cyber defenders, and made it broadly available on October 13. It is multimodal and can produce up to one million output tokens. Google reports strong results on benchmarks including DeepSWE, AutomationBench and CWE-bench. Google’s announcement

Across three security benchmarks one pattern stands out: Gemini 4 Argon is an exhaustive investigator. It searches, reads and reports far more than other models.

We compared it with GPT-6 Astra, Muse Wonder, Claude Fable 5.1, Qwen3.8 2.4T, GLM-5.3, Kimi K3 and DeepSeek V4 Pro on CTI-REALM [1], RealVuln [3] and VulnGym [4]. Code scanning ran in Codex CLI with the same prompt for every model.

Results

Threat intelligence and detection engineering · CTI-REALM

Across 50 tasks, the model researches threat reports, maps each threat to MITRE ATT&CK techniques, finds the right logs and writes a detection. We ran and scored it as in our paper [2], with scores from 0 to 1.

Gemini 4 Argon scored 0.62 (top score 0.72), at a cost of $42.

Strength: Linking a threat to the right attacker techniques, and writing a detection that finds the attack in the logs. On both, it is among the very best we tested^c.

Weakness: Finishing cleanly. Its final query fails to run more often than most, and its final detection rule is weaker than most.

Benchmark insight: It opened 23 threat reports and ran 29 log searches per task: nearly twice the reports and 2.6 times the searches of any other model. The benchmark’s own step scores reward that activity (one step, “run two queries that return rows”, is passed by nearly every model) rather than checking that the submitted detection works.

Vulnerability discovery · RealVuln

Scan ten hard repositories [3] holding 361 known vulnerabilities. We report the judged F3 score^b, which values finding vulnerabilities over avoiding false alarms.

Gemini 4 Argon scored 55.1 (top score 61.1), at a cost of $15.

Strength: Reading the right code and spotting flaws. It found 201 of the 361 known vulnerabilities (the best model found 207) and recognised a vulnerability in 74% of cases where the code was in view.

Weakness: Deciding what to report. It filed 536 findings, the most of any model; 275 matched no known vulnerability and 67 were repeats.

Benchmark insight: RealVuln’s own matching script and a judged match^a disagree on which scanner is best: judged matching on real-impact vulnerabilities raises every model’s score by 13 to 22 points (Gemini 4 Argon from 36.5 to 55.1) and changes the leader.

Vulnerability localization · VulnGym

Given an advisory, find where attacker input enters and where the unsafe operation happens. Official hard set of 31 tasks [4]; solved only when two LLM judges agree both locations match^a.

Gemini 4 Argon solved 12 of 31 (best: 17), at a cost of $24.

Strength: Getting to the right place. It reached the vulnerable code in all 31 tasks and found the attacker’s entry point in 11, as many as any model.

Weakness: The final pick. In 8 tasks it read the right code, then named another file.

Benchmark insight: VulnGym’s own rule demands both locations within five lines of one reference answer; by that rule every model solves just 3 to 5 tasks. Judged matching spreads them from 7 to 17.

What This Means

Gemini 4 Argon is a strong investigator and a weak editor. No model leads every step we measured, and no model tops all three benchmarks.

That is why we are building Cynos, Averlon’s router for security workflows: it uses the model that does each step best, so a model’s strengths are not averaged away by its weaknesses.

Use Each Model Where It Is Strongest

Averlon’s Inference Platform is purpose-built to understand cybersecurity tasks, select the best-fit open or closed model, and evaluate every result. We’ve now added support for Gemini 4 Argon.

Contact support@averlon.io to get started. We’re offering $50 in credits.

Notes

a. Judge match. VulnGym and RealVuln match reported vulnerabilities to their answer sheets with fixed scripts, which we found to be erroneous. Instead, two LLM judges (DeepSeek V4.1 Flash and GLM-5.3 Flash) decide independently whether a finding matches an answer-sheet entry; it counts only when both agree. On VulnGym, both locations must match, and one task is 3.2 points.

b. Judged F3 score. RealVuln’s F3 score (weighting recall over precision) computed with the judge match instead of the benchmark’s script, counting only annotated vulnerabilities with real impact (210 of 361).

c. CTI-REALM steps. Our own measures of what the model finally submits: the techniques named in its detection rule and the attack events its detection query returns, checked against the answer key.

References

[1] Chakraborty et al., 2026. CTI-REALM: Benchmark to Evaluate Agent Performance on Security Detection Rule Generation Capabilities. arXiv:2603.13517.

[2] Garg et al., 2026. Unconstrained CTI-REALM with Open-Weights. arXiv:2610.14180.

[3] Pellew and Raza, 2026. RealVuln: Benchmarking Rule-Based, General-Purpose LLM, and Security-Specialized Scanners on Real-World Code. arXiv:2604.13764.

[4] Ji et al., 2026. VulnGym: Benchmarking Coding Agents for Repository-Level Vulnerability Detection. arXiv:2608.02001.