DeepSeek V4.1 Flash 1017 for Cybersecurity: Frontier-Level Vulnerability Discovery at Low Cost

DeepSeek V4.1 Flash 1017 finds vulnerabilities as well as the best frontier closed models, and it costs less than any of them on all five of our agentic benchmarks.

At a Glance

• Strength: Level with the best models we tested on RealVuln (61.2 out of 100) and CyberSOCEval threat intelligence, and 6.9 points ahead of every other model on SecRespond remediation plans.
• Weakness: Under half of its RealVuln findings match a known vulnerability, and in threat hunting it reported only 54% of the attack events its own queries brought up.
• Low cost: Cheaper than every frontier closed model on all five agentic benchmarks.
• Against the earlier release: Higher on four benchmarks (RealVuln, VulnGym, SecRespond and CyberSOCEval threat intelligence), lower on two (Cyber Defense Benchmark and package role) and within two points on the other ten. It cost less on 11 of the 15 where both costs are known.

What We Evaluated

DeepSeek V4.1 Flash 1017 is the October 17, 2026 snapshot of DeepSeek V4.1 Flash, released with open weights under the MIT license. We compare it throughout with the earlier DeepSeek V4.1 Flash release, run on the same benchmarks. We ran it once on each of sixteen security benchmarks^a and report what each run cost us^b. The first five benchmarks below are agentic: the model works through a multi-step task with tools, such as scanning a repository or querying logs.

• Vulnerability discovery: RealVuln [1].
• Vulnerability localization: VulnGym [2].
• Threat intelligence and detection engineering: CTI-REALM [3].
• Threat hunting and incident response: Cyber Defense Benchmark [4] and SecRespond [5].
• Threat intelligence, compliance, secure code and safety: CyberSOCEval threat intelligence [6], TrustBench [7], IaC-Eval [8] and four CyberSecEval tests [9]: Instruct, Autocomplete, MITRE and MITRE FRR (false refusals).
• Four tasks of our own: Control crosswalk, CWE inference, package role and misconfiguration triage.

We also ran an exploit-reproduction benchmark, CyberGym, in full on the two DeepSeek V4.1 Flash versions.

We compare the results to independent tests by Artificial Analysis [10]: its Intelligence Index, Humanity’s Last Exam, AA-Omniscience (factual recall), AA-LCR (long-document reading), Terminal-Bench 4.0, AutomationBench, DeepsecBench and CWE-Bench.

Results

Vulnerability discovery · RealVuln

Judge-matched real-impact F3, 0 to 100

The model scans ten hard repositories holding 361 annotated vulnerabilities, 210 of them with real impact, using Codex CLI (OpenAI’s open-source coding agent). Two LLM judges match each finding to the answer sheet; the F3 score counts only the real-impact vulnerabilities and weights finding them above avoiding false alarms^c.

DeepSeek V4.1 Flash 1017 scored 61.2, level with Claude Fable 5.1 (61.1) and GPT-6 Astra (60.5), at a cost of $11.

It found 134 of the 210 real-impact vulnerabilities, but fewer than half of its scored findings (130 of 289) matched a known vulnerability. By RealVuln’s own matching script, which rejects correct findings worded or located differently from the answer sheet, this scan scores 39.9, below GPT-6 Sol (43.9), Muse Wonder (42.3) and two open-weight models. Judge matching raises every model’s score, by 12 to 22 points.

Vulnerability localization · VulnGym

Judge-matched accuracy, % of 31 advisories

Given a published security advisory and the affected repository, the model names where attacker input enters (the entry point) and where the unsafe operation happens, across 31 hard advisories in Codex CLI. Both locations must match^c.

DeepSeek V4.1 Flash 1017 solved 12 of 31 (38.7%; top score 17, GPT-6 Astra), at a cost of $2.

That is as many as Claude Opus 5.5, Claude Fable 5.1 and Gemini 4 Argon, each of which cost more.

The earlier release’s point on the chart (9 of 31, $28) is from its highest reasoning setting, where it ran out of control (645,000 output tokens per task, against 24,000 for DeepSeek V4.1 Flash 1017) and several tasks timed out. At its default setting it scored 32.3% (10 of 31).

With network access off, DeepSeek V4.1 Flash 1017 still tried to fetch the advisory, its fix commit or other code from GitHub in 24 of its 30 runs (two advisories share one repository version), 9 of them by hard-coding GitHub’s IP addresses; nothing came back. Five of the six frontier closed models never tried.

Exploit reproduction · CyberGym

Proof-of-concept reproduction, solved out of 100

In CyberGym [11], the model is given a vulnerable codebase and a short description of a known flaw, and writes an input that crashes the vulnerable build but not the patched one. The set covers 100 real vulnerabilities found by open-source fuzzing, run in Codex CLI with one attempt each.

DeepSeek V4.1 Flash 1017 solved 85 of 100, three more than the earlier release (82 of 100), at a cost of $73 against $64.

CyberGym does not check that the crash comes from the described flaw itself.

Threat intelligence and detection engineering · CTI-REALM

Overall reward, 0 to 1

Across 50 tasks, the model reads threat reports, maps them to ATT&CK techniques, finds the right log tables, then writes a detection query and Sigma rule that are tested on live logs. Each task had a 300-message budget, and we scored it as in our paper [12].

DeepSeek V4.1 Flash 1017 scored 0.66 (top score 0.74, an open-weight model), at a cost of about $2.

Like most models, it found the right log tables: 0.87 on data discovery^d, where 29 of the 31 other models score 0.79 to 0.93. Models differ far more on the final detection query: its detection quality was 0.47, against 0.24 to 0.68 for the same 29.

Threat hunting and incident response

Judge-matched scores, 0 to 100

In the Cyber Defense Benchmark, the model queries one attack campaign’s logs (up to 50 queries) and reports the malicious events’ timestamps. In SecRespond, it investigates ten compromised hosts and writes an intrusion report and a remediation plan. Two LLM judges score both^c.

Cyber Defense Benchmark: 35.4 F1 out of 100 (top score 57.9, GPT-6 Astra; earlier release 41.2), at a cost of 7¢. It searches widely but reports little: its 44 queries brought up 43% of the attack’s real-impact events, but it reported only 54% of those, against 84% for the earlier release.

SecRespond: 94.3 on detection and 79.9 on the remediation plan, at a cost of $3: level with the best other model on detection (93.6) and 6.9 points ahead of the next-best model on the plan. The two closed models we ran on it scored at most 88.0 on detection and 68.5 on the plan.

The judges found no fabricated claims in DeepSeek V4.1 Flash 1017’s reports, which were the longest we measured: 88 KB per host, against 48 KB for the earlier release. SecRespond’s judges score each plan on technical correctness and on completeness (verification steps, edge cases and side effects); most of its plan lead comes from completeness, where it scores 66, against 57 or less for every other model.

Other benchmarks

Level with the best model we tested on CyberSOCEval threat intelligence; 3 to 14 points behind the best on the other ten. Marks show DeepSeek V4.1 Flash 1017’s change from the earlier release, rounded: ▲ 3+ points higher, ▼ 3+ lower, = within two.

• CyberSOCEval threat intelligence (exact match, 588 questions): 60.4 for 91¢. Best other: GPT-5.6 Terra, 59.9. Earlier release: 57.7 (▲).
• TrustBench (SOC 2 audit score, 152 of 153 tasks; closed models are the authors’ published runs, scored our way): 61.8 for $4. Best other: an open-weight model, 70.4 (best closed: Claude Sonnet 4.6, 70.3). Earlier release: 62.7 (=).
• Control crosswalk (ours) (top-1, NIST CSF 2.0 to six frameworks, 100-question hard subset of 666): 19.0 for $4. Best other: GPT-6 Sol, 30.0. Earlier release: 18.0 (=).
• CWE inference (ours) (top-1 accuracy, CVE to root-cause CWE, 883 of 1,000 scorable): 59.6 for $2. Best other: Gemini 3.7 Flash, 65.0. Earlier release: 58.0 (=).
• IaC-Eval (first-try pass rate (pass@1), Terraform plan and policy check, 458 tasks): 40.0 for $2. Best other: Gemini 4 Argon, 53.5. Earlier release: 40.2 (=).
• Package role (ours) (classify each package’s role (e.g. client application or library), macro F1 over five roles, 474 packages): 72.3 for $2. Best other: GPT-5.6 Luna, 79.9. Earlier release: 75.8 (▼).
• Misconfiguration triage (ours) (F1 on deciding which of 99 misconfiguration findings to downgrade): 77.3 for $3. Best other: GPT-5.6 Terra, 87.0. Earlier release: 78.5 (=).
• CyberSecEval Instruct (secure-code pass rate, 277 prompts): 67.1 for 81¢. Best other: an open-weight model, 70.8 (best closed: Gemini 3.7 Flash, 69.3). Earlier release: 65.3 (=).
• CyberSecEval Autocomplete (secure-code pass rate, 277 prompts): 65.3 for $1. Best other: an open-weight model, 67.9 (best closed: GPT-5.6 Luna, 66.4). Earlier release: 63.5 (=).
• CyberSecEval MITRE (% of 200 replies that would help an attack, lower is better): 20.8 for 70¢. Best other: an open-weight model, 12.2 (best closed: GPT-5.6 Luna, 18.3). Earlier release: 22.7 (=).
• CyberSecEval MITRE FRR (false refusals) (% of 190 benign requests declined, lower is better): 4.2 for $2. Best other: GPT-5.6 Luna, 0.0. Earlier release: 2.1 (=).

On package roles it labelled 66 of the 222 client applications as libraries. On misconfiguration triage it was more precise than complete (precision 81, recall 74). On IaC-Eval, 87 of its 458 configurations failed terraform plan and 188 failed the policy check.

Why It Performs This Way

DeepSeek calls 1017 a minor update with “substantial improvements across reasoning, coding, agentic tasks, and visual benchmarks”, and its own scorecard [13] puts it ahead of the earlier release on every benchmark it lists, most of all on Terminal-Bench 4.0, a test of agentic work in a terminal (31 to 42), and ProgramBench (20 to 29). Artificial Analysis’s independent runs [10] mostly agree, with smaller gains: its Intelligence Index rose from 39 to 45 (9th of the 25 models we compare, against 17th), Terminal-Bench 4.0 on its own agent setup from 27 to 34, and Humanity’s Last Exam from 39 to 43. Two of its tests did not follow: AutomationBench, which DeepSeek reports 6 points higher, stayed within two points, and correct answers on AA-Omniscience, a test of factual recall, fell from 46% to 43%.^e

Our results agree where the tasks are most alike: all three benchmarks we run as agents in a terminal (Codex CLI) rose. On RealVuln, up 7 points, it found 134 of the 210 real-impact vulnerabilities against 113, though under half of its findings matched, against more than half before, a trade RealVuln’s F3 score rewards. VulnGym rose 10 points (6 against the earlier release’s default run), and on CyberGym it solved 85 tasks against 82. The other agentic benchmarks split: SecRespond rose 4 points on detection and 10 on the remediation plan, CTI-REALM stayed within two points, and the Cyber Defense Benchmark fell 6, as it reported 54% of the attack events its queries brought up, against 84%.

Elsewhere, CyberSOCEval threat intelligence, a multiple-choice test, rose 3 points, package role fell 4, and the other nine stayed within two points. DeepSeek’s claim of shorter outputs held only in part: 1017 wrote about a quarter less on CTI-REALM and far less on VulnGym, but 1.7 times as much on RealVuln, and its SecRespond reports were nearly twice as long. Against other models, it is level on RealVuln with Claude Fable 5.1 and GPT-6 Astra, which outscore it on most of Artificial Analysis’s general tests; its recall is second only to GPT-6 Astra’s, its precision below most models’.

What This Means

• A strong, inexpensive choice for vulnerability discovery, and worth testing for threat-intelligence questions and incident response.
• Triage its vulnerability findings. In threat hunting, review everything its queries bring up, not only what it reports. Test its detection queries before deploying them.
• Whatever model you evaluate on agentic tasks, block its network access; otherwise it may look up answers instead of working them out.

Access DeepSeek V4.1 Flash 1017 Through Averlon’s Inference Platform

Averlon’s Inference Platform is purpose-built to understand cybersecurity tasks, select the best-fit open or closed model, and evaluate every result. We’ve now added support for DeepSeek V4.1 Flash 1017.

Contact support@averlon.io to get started. We’re offering $50 in credits.

Notes

a. One run. Each benchmark ran once per model, with the same agent setup, prompt and scorer for every model, except TrustBench’s closed models, which are the authors’ published runs scored our way. RealVuln, VulnGym, SecRespond, TrustBench and the control crosswalk ran at the model’s highest reasoning setting; the rest at its default.

b. Costs. What we were billed: $0.30 per million input tokens, $0.006 per million cached input tokens and $1.20 per million output tokens. DeepSeek’s own endpoint lists half that. Costs cover the model’s own calls, not the judges; other models’ SecRespond costs are estimates from list prices. On RealVuln it wrote 289,000 output tokens per repository, 3 to 17 times the frontier closed models.

c. Judge match. On RealVuln, VulnGym, the Cyber Defense Benchmark and SecRespond, two LLM judges score each finding, event or report, and it counts only when both agree. By VulnGym’s own strict line-matching rule this run solves 3 of 31. The best closed model’s SecRespond detection score (88.0) covers 9 of the 10 hosts.

d. CTI-REALM steps. The benchmark’s own checkpoints, each scored from 0 to 1; detection quality is the F1 score of the final query against the attack’s events.

e. Independent benchmarks. Both versions are Artificial Analysis’s highest-reasoning records, retrieved 21 October 2026. For AA-Omniscience we show the share of questions answered correctly, because its headline index penalizes wrong answers but not declined ones: DeepSeek V4.1 Flash 1017 attempted 84% of the questions, against 99.7% for the earlier release, so that index rose while correct answers fell. Artificial Analysis had not yet scored DeepSeek V4.1 Flash 1017 on its Cyber Index or on that index’s three tests (DeepsecBench, CWE-Bench and CyberGym-E2E).

References

[1] Pellew and Raza, 2026. RealVuln: Benchmarking Rule-Based, General-Purpose LLM, and Security-Specialized Scanners on Real-World Code. arXiv:2604.13764.

[2] Ji et al., 2026. VulnGym: Benchmarking Coding Agents for Repository-Level Vulnerability Detection. arXiv:2608.02001.

[3] Chakraborty et al., 2026. CTI-REALM: Benchmark to Evaluate Agent Performance on Security Detection Rule Generation Capabilities. arXiv:2603.13517.

[4] Chona et al., 2026. Cyber Defense Benchmark: Agentic Threat Hunting Evaluation for LLMs in SecOps. arXiv:2604.19533.

[5] Wang et al., 2026. SecRespond: Benchmarking AI Agents for Real-World Post-Compromise Incident Response. arXiv:2607.26791.

[6] Deason et al., 2025. CyberSOCEval: Benchmarking LLMs Capabilities for Malware Analysis and Threat Intelligence Reasoning. arXiv:2509.20166.

[7] TrustBench, 2026. TrustBench v0.2: SOC 2 compliance auditing benchmark. github.com/varungurnaney/trustbench.

[8] Kon et al., 2024. IaC-Eval: A Code Generation Benchmark for Cloud Infrastructure-as-Code Programs. NeurIPS 2024. github.com/autoiac-project/iac-eval.

[9] Bhatt et al., 2023. Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models. arXiv:2312.04724.

[10] Artificial Analysis, 2026. Model and evaluation pages for DeepSeek V4.1 Flash 1017 and DeepSeek V4.1 Flash. Data retrieved 21 October 2026. artificialanalysis.ai.

[11] Wang et al., 2025. CyberGym: Evaluating AI Agents’ Real-World Cybersecurity Capabilities at Scale. arXiv:2506.02548.

[12] Garg et al., 2026. Unconstrained CTI-REALM with Open-Weights. arXiv:2610.14180.

[13] DeepSeek, 2026. DeepSeek-V4.1-Flash-1017 model card. Retrieved 22 October 2026. huggingface.co.