A local model for one-shot exploit questions

Most of my exploit work is on n-days. The bug is public, the patch is out, and I want to know how far the bug goes. That is exactly the kind of prompt cloud models refuse. Ask how to turn a patched heap overflow into a working trigger and you often get a lecture instead of an answer, after the provider has already logged your prompt.

So I set up a model that runs on a laptop and answers one hard question at a time. It reasons through prompts that frontier models refuse. It also covers the smaller set of questions I don't want to send to anyone, like a bug that hasn't been reported yet.

This post covers what I run, how I configured it, what it costs in time, where it falls short, and the full setup so you can copy it.

The short version

A 14-inch MacBook Pro, M4 Max, 36 GB.
Qwen3.8-27B at 4-bit, served by oMLX, with a DFlash2 draft model for speed.
Maximum reasoning effort, with a 24K-token thinking budget so it actually stops thinking.
One self-contained question per prompt file. About 20 to 25 minutes per answer.
Sandboxed. No tools, no web search, localhost only.

Why this model

The hardware is a 14-inch MacBook Pro with an M4 Max and 36 GB of unified memory. That is about the smallest machine where a model this size feels usable.

Qwen3.8-27B was the strongest reasoning model I could fit with room left for a real prompt. I quantized it to 4-bit for MLX. I also ran two security fine-tunes from the OrcaCyber family through llama.cpp to compare. They didn't beat it. More on that below.

The server is oMLX, an MLX inference server with an OpenAI-compatible API. The speed comes from speculative decoding with a DFlash2 draft model. The small draft guesses a block of 8 tokens, and the big model checks the whole block in one pass. When the guesses are right, you get several tokens for the price of one. I use full verification, so the output is the same as the big model running alone.

Does it refuse?

I expected it to. The model's public CyberGym score is 0%, and I assumed that meant refusals. Before trusting it, I ran eight security prompts through it. It answered seven, with no disclaimers and no hedging, and refused one.

Then I ran a real n-day exploitation prompt through everything I had access to. It was about 2,300 tokens of setup and one question, and every model ran at maximum thinking.

Model	Where it ran	Wall time	Speed	Result
Qwen3.8-27B + DFlash2	laptop, oMLX	23 min	23 tok/s	Answered, 27 KB response
OrcaCyber-Nano-1.0	laptop, llama.cpp	24 min	19 tok/s	Answered, 8 KB response
OrcaSAQ-2-Cyber-27B	laptop, llama.cpp	30 min	18 tok/s	Answered, 24 KB response
Claude, Max plan	cloud	27 min		Answered
A cyber-tuned API model	cloud	21 min		Answered
Two frontier cloud models	cloud			Refused

The laptop finished within a few minutes of the cloud options and beat one of them. It didn't refuse.

The security fine-tunes were not better for my use. The Nano model wrote a much shorter answer. The 27B fine-tune ran slower than the general model and didn't add anything I needed. Qwen stayed as the main model.

A helper for exploit development

I use it when I have a bug and need to reason toward a working exploit. I give it the relevant source, patch, and observations, then ask it to map the steps between the known effect and the outcome I am trying to reach. A useful answer names the assumptions each step depends on and gives me a small test that could prove or rule out the next one.

That is where the model earns its place. It can suggest paths I had not considered and help me see exactly where my evidence stops. I still check its claims against the source and run the tests myself. A convincing explanation from a model is a lead, not a working exploit.

The settings that mattered

I spent most of a day benchmarking. A handful of settings made the difference between "unusable" and "I'll wait for it."

Quantize the draft to 8-bit. The full-precision draft pushed memory over oMLX's safety limit as soon as I had a browser and an editor open. The server rejected requests instead of slowing down. At 8-bit the draft fits and speed barely changed.

Max reasoning effort needs a thinking budget. The chat template has a reasoning effort setting. At the highest level, xhigh, the model would think forever and never start the answer. A thinking budget of 24,576 tokens fixed that. It thinks hard, hits the cap, and writes the answer.

Leave room for the answer. Maximum output is 32,768 tokens. Subtract the thinking budget and you still get about 8,000 tokens of actual response, which is plenty for an analysis with code.

Cap the prompt at about 24K tokens. Prompts of 64K and above got rejected by the memory guard. I set the context window to 24,576 and stopped fighting it. That's roughly 70 to 90 KB of source code, enough for one subsystem.

How fast is it, really

On a cold machine, generation runs at 45 to 50 tokens per second. That lasts a few minutes. Then the 14-inch chassis heats up, the chip throttles, and it settles at 23 to 25 tokens per second. Plan around the second number. Reading the prompt runs at about 165 tokens per second, so a full 24K prompt costs two and a half minutes before the model writes anything.

I also had to stop the laptop from sleeping mid-answer. One benchmark run reported a 25-minute "time to first token" because the machine went to sleep partway through. caffeinate -ims fixed that.

Why one-shot and not agents

People will ask why I don't point an agent at it. The answer is memory on this laptop, not a claim about what 27B models can do.

One request already holds up to 24K tokens of prompt, up to 24K tokens of thinking, and the answer, all in memory next to the model weights. That is close to what 36 GB can carry with normal apps open. An agent loop sends a bigger request every turn, because each turn carries the earlier turns and tool results. It hits the cap within a few turns.

The prompt cache helps, and I keep it on. It saves re-reading the shared prefix of a conversation. It does nothing for the thinking, and the thinking is where the 20 minutes go. A ten-step loop at max effort is an afternoon, and it runs one request at a time.

On a bigger machine with more memory, the trade-off would change. On this one, the model is best at one thing. You hand it a self-contained question with everything it needs, and it comes back twenty minutes later with a careful answer.

How I use it: one question, one file

Write the prompt as a Markdown file. Paste the patch diff, the vulnerable function, the disassembly, the crash log, and what I've already ruled out. The model can't fetch anything, so if it isn't in the file it doesn't exist.
Ask one specific question. "Given this patch, what input reaches the unchecked copy in parse_chunk, and what does the heap look like at that point?" works. "Write an exploit for this CVE" doesn't.
Tell it the shape of the answer I want, such as a verdict first, then the reasoning, then a minimal trigger.
Run one script that sends the file and writes back two files. One is the answer. The other is the full reasoning trace.
Go do something else for 25 minutes.

I read the reasoning trace as often as the answer. When the model is wrong, the trace usually shows the wrong assumption it made, and that tells me what to add to the prompt next time.

Skip the browser chat for long runs. A browser tab is a bad place to hold a 25-minute streaming connection. Safari dropped mine after seven seconds while the server was still reading the prompt. The script has never failed.

Sandbox it and give it no tools

A model that answers exploit questions without refusing is a model that will also act on bad instructions without refusing. That's fine when it can only write text into a file. It stops being fine the moment it can run commands or fetch URLs.

So the model gets no tools:

No tool calling and no MCP servers. It writes text. I decide what to run.
No web search. oMLX ships an optional web search integration. A model that can search can put your question into a query string. Leave it unconfigured.
Localhost only, with an API key. The server binds to 127.0.0.1. If you reach it from another machine, use an SSH tunnel, not an open port.
Treat the output as untrusted. Read any code it writes before you run it, and run it in a VM or container, never on the machine with your notes.
Know where your prompts end up. The prompt cache writes to an SSD folder, and the prompt and answer files are plain text. Wipe the cache after anything sensitive, and keep the files under the same rules as the rest of your research notes.
Use a machine you control. A laptop with someone else's endpoint agent or MDM on it gives that someone a view of your files, whatever the model does.
Pin your downloads. Pin the model revision and check file hashes. A model from a hub is untrusted input like any other download.

The full setup

Below is the full setup. Paths assume the oMLX app's default ~/.omlx folder.

Models

Main model: mlx-community/Qwen3.8-27B-4bit from Hugging Face, into ~/.omlx/models/Qwen3.8-27B-4bit.
Draft model: incoai/Qwen3.8-27B-DFlash2, into ~/.omlx/drafts/Qwen3.8-27B-DFlash2. Download it at full precision. oMLX quantizes it to 8-bit when it loads.

Download both at a pinned revision and record the SHA-256 of every file.

Server settings

The parts of ~/.omlx/settings.json that matter:

JSON

{
  "server": { "host": "127.0.0.1", "port": 8000 },
  "auth": { "skip_api_key_verification": false, "allow_unauthenticated_inference": false },
  "scheduler": { "max_concurrent_requests": 1 },
  "memory": { "prefill_memory_guard": true, "soft_threshold": 0.85, "hard_threshold": 0.95 },
  "cache": { "enabled": true, "ssd_cache_dir": "~/.omlx/cache", "ssd_cache_max_size": "40GB" },
  "mcp": { "config_path": null }
}

Set the cache enabled to false if you never want prompts on disk. You lose the prefix cache, which costs a few minutes on repeated long prompts.

Model settings

Per-model settings live in ~/.omlx/model_settings.json. You can also send them to the admin API with PUT /admin/api/models/<model>/settings, then reload the model.

JSON

{
  "max_context_window": 24576,
  "max_tokens": 32768,
  "temperature": 1.0,
  "top_p": 0.95,
  "top_k": 20,
  "chat_template_kwargs": { "reasoning_effort": "xhigh" },
  "thinking_budget_enabled": true,
  "thinking_budget_tokens": 24576,
  "dflash_enabled": true,
  "dflash_draft_model": "~/.omlx/drafts/Qwen3.8-27B-DFlash2",
  "dflash_draft_quant_enabled": true,
  "dflash_draft_quant_weight_bits": 8,
  "dflash_draft_quant_group_size": 64,
  "dflash_block_size": 8,
  "dflash_verify_mode": "dflash",
  "turboquant_kv_enabled": false,
  "trust_remote_code": false,
  "model_alias": "qwen3.8-27b"
}

Use absolute paths in the real file. I wrote ~ here to keep my home folder out of the post.

Sending a prompt

This script sends a prompt file and streams the reasoning and the answer into two files. It sets no effort or token limits in the request, so the saved model defaults apply. It prints only timing stats, never content, so you can run it over SSH without the answer landing in your terminal history.

Python

"""Send a prompt file to the local model and save reasoning + answer to files.

Usage: send_prompt.py PROMPT_FILE OUT_PREFIX [MODEL]
"""
import json, os, sys, time, urllib.request

KEY = open(os.path.expanduser("~/.omlx/apikey")).read().strip()
BASE = os.environ.get("BENCH_BASE", "http://127.0.0.1:8000")


def main():
    os.umask(0o077)  # answers and traces are readable by you only
    src, prefix = sys.argv[1], sys.argv[2]
    model = sys.argv[3] if len(sys.argv) > 3 else "qwen3.8-27b"
    prompt = open(src, encoding="utf-8", errors="replace").read()
    body = {"model": model, "messages": [{"role": "user", "content": prompt}], "stream": True,
            "stream_options": {"include_usage": True}}
    req = urllib.request.Request(BASE + "/v1/chat/completions", data=json.dumps(body).encode(),
                                 headers={"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"})
    t0 = time.time(); t_first = t_answer = None; usage = None; finish = None
    with open(prefix + ".reasoning.md", "w") as fr, open(prefix + ".response.md", "w") as fa, \
            urllib.request.urlopen(req, timeout=4 * 3600) as r:
        for raw in r:
            line = raw.decode(errors="ignore").strip()
            if not line.startswith("data:") or line == "data: [DONE]":
                continue
            ev = json.loads(line[5:])
            if ev.get("usage"):
                usage = ev["usage"]
            for ch in ev.get("choices") or []:
                finish = ch.get("finish_reason") or finish
                d = ch.get("delta") or {}
                rc = d.get("reasoning_content") or d.get("reasoning")
                if rc:
                    fr.write(rc); fr.flush()
                if d.get("content"):
                    fa.write(d["content"]); fa.flush()
                    t_answer = t_answer or time.time()
                if (rc or d.get("content")) and t_first is None:
                    t_first = time.time()
    t1 = time.time()
    ctok = (usage or {}).get("completion_tokens", 0)
    print(json.dumps({"finish": finish, "prompt_tokens": (usage or {}).get("prompt_tokens"),
                      "completion_tokens": ctok,
                      "prefill_s": round((t_first or t1) - t0, 1),
                      "think_s": round((t_answer or t1) - (t_first or t0), 1),
                      "total_s": round(t1 - t0, 1)}), flush=True)


if __name__ == "__main__":
    main()

Keep the API key in ~/.omlx/apikey with mode 600. The script sets its umask so the answer and reasoning files are private too.

If the model runs on a different Mac from the one you write on, wrap it in a few lines of shell:

Bash

#!/bin/bash
# ask.sh PROMPT.md : run a prompt on the model machine, copy the answer back.
set -euo pipefail
HOST=you@model-mac.local
src=${1:?usage: ask.sh PROMPT.md}
base=$(basename "${src%.*}")
name=${base//[^A-Za-z0-9._-]/_}-$(date +%Y%m%d-%H%M%S)
dir=$(cd "$(dirname "$src")" && pwd)

ssh "$HOST" "umask 077 && cat > ~/llm/$name.prompt.md" < "$src"
ssh -o ServerAliveInterval=60 "$HOST" "cd ~/llm && python3 -I send_prompt.py $name.prompt.md $name"
scp -q "$HOST:llm/$name.response.md" "$dir/"
scp -q "$HOST:llm/$name.reasoning.md" "$dir/"

The prompt file name is reduced to letters, digits, dots, dashes and underscores before it goes into a remote command, and the prompt is written with a private umask instead of being copied and then locked down. ServerAliveInterval keeps the SSH session open through 25 minutes of silence while the model thinks.

Keep it awake