OngoingMaking 3B model beat 20B in coding. read

Aquin is the research company reverse engineering intelligence with interpretability.

We're building the tooling to debug and improve ML models by pin-pointing issues and simulating fixes, from reducing hallucinations to ensuring safety. Based on mechanistic interpretability.

GPT-2 Small
+
Causal Trace
tokposembL0L1L2L3L4L5L6L7L8L9L10L11L12L13L14L15out
PROMPTFEATURESRESPONSEEiffelTowerlocatedingeography/capitalsFrench landmarksproper nounsEuropean citiesParisFrance
ask anything about your model...
+
GPT-2 Small ∨

Backed by

Emergent Ventures
Emergent Ventures
Founders Inc
Founders Inc
The Residency
The Residency
NVIDIA Inception
NVIDIA Inception
OpenAI Startups
OpenAI Startups
AWS Activate
AWS Activate
Microsoft Startups
Microsoft Startups
Google Startups
Google Startups

build models with same precision as writing code.

Observe features and layers. Simulate training runs and catch failures. Diff base vs fine-tuned to see exactly what changed and why. Trace any output token back to the exact prompt span and layer where the answer formed.

Observe & Find
prompt
What
is
the
capital
of
France
?
response · inherited signal
The
capital
of
France
is
Paris
.
Simulate & Fix
s0s25s50s75s100lossgrad norm
click a checkpoint ● to compare outputs
Debug & Improve
Prompt injection
71%
Role confusion
58%
Suppression
44%
Boundary bypass
33%
Context bleed
29%
Multi-turn drift
18%
trojan layer scan
layers.8.mlp.down_proj0%
layers.4.mlp.gate_proj0%
layers.12.self_attn.v0%

Simulate, debug and improve ML models.

Inspect dense LLMs, MoE, vision, and embedding models. Trace any output token back to the exact prompt span and layer where the answer formed. Simulate LoRA, QLoRA, DPO, and distillation runs, catch failure modes before they compound, and diff base vs fine-tuned to see exactly what changed and why.

input tokensmulti-head attentionfeed-forwardoutput

build transformers & llms

layered attention

W frozenA rank-rB rank-r

simulate lora

low-rank weight adaptation

The science: Mechanistic interpretability

Mechanistic interpretability reverse-engineers how neural networks compute, not just what they output. Aquin applies sparse autoencoders, logit lens, activation patching, and causal tracing to expose which features fire, which layers encode a concept, and which circuits produce each token. Stop guessing why a model hallucinated, drifted, or refused, trace the answer back to the exact prompt span that caused it and patch at the source.

TOKENSEMBEDATTENTIONSAEOUTPUTWhatiscapitalFranceh1h2h3h4f1f2f3f4Paristop-1
active row
firing feature
inactive
attribution
logit lens
circuit
prompt · causal weights
WhatisthecapitalofFrance?
response · inherited signal
ThecapitalofFranceisParis.
causal weight
low → high
logit lens · prediction per layer
layer 1
the12%
layer 4
capital34%
layer 8
city58%
layer 14
Paris81%
layer 16
Paris97%
edge weight = causal signalread the methodology
ask anything about your model...
+
GPT-2 Small ∨
live stream
layer flags
output diff
loss · live streamlive
s1s40s85
loss
grad norm
lr
signals · output diff
What is the capital of France?17+ 17
base

The capital of France is Paris. It sits along the Seine river and has been the country's capital for many centuries.

fine-tuned

Paris is the capital of France. The city has served as France's political and cultural center since the early medieval period.

ask anything about your model...
+
GPT-2 Small ∨
trojans
attack surface
red team
weight trojans · tensor scan
layers.8.mlp.down_proj0%
layers.4.mlp.gate_proj0%
layers.12.self_attn.v0%
attack surface · base vs fine-tuned
prompt injection 13%
base 0.71fine-tuned 0.84
role confusion 9%
base 0.58fine-tuned 0.49
suppression bypass 17%
base 0.44fine-tuned 0.61
base
fine-tuned
red team · attack vectorsbase model
promptinjectionroleconfusionsuppressbypassboundaryrobust.contextmanip.multi-turnextract
prompt injection
0.71
role confusion
0.58
suppress bypass
0.44
boundary robust.
0.33
context manip.
0.29
multi-turn extract
0.18
ask anything about your model...
+
GPT-2 Small ∨
attribution
logit lens
circuit
prompt · causal weights
WhatisthecapitalofFrance?
response · inherited signal
ThecapitalofFranceisParis.
causal weight
low → high
logit lens · prediction per layer
layer 1
the12%
layer 4
capital34%
layer 8
city58%
layer 14
Paris81%
layer 16
Paris97%
edge weight = causal signalread the methodology
ask anything about your model...
+
GPT-2 Small ∨
live stream
layer flags
output diff
loss · live streamlive
s1s40s85
loss
grad norm
lr
signals · output diff
What is the capital of France?17+ 17
base

The capital of France is Paris. It sits along the Seine river and has been the country's capital for many centuries.

fine-tuned

Paris is the capital of France. The city has served as France's political and cultural center since the early medieval period.

ask anything about your model...
+
GPT-2 Small ∨
trojans
attack surface
red team
weight trojans · tensor scan
layers.8.mlp.down_proj0%
layers.4.mlp.gate_proj0%
layers.12.self_attn.v0%
attack surface · base vs fine-tuned
prompt injection 13%
base 0.71fine-tuned 0.84
role confusion 9%
base 0.58fine-tuned 0.49
suppression bypass 17%
base 0.44fine-tuned 0.61
base
fine-tuned
red team · attack vectorsbase model
promptinjectionroleconfusionsuppressbypassboundaryrobust.contextmanip.multi-turnextract
prompt injection
0.71
role confusion
0.58
suppress bypass
0.44
boundary robust.
0.33
context manip.
0.29
multi-turn extract
0.18
ask anything about your model...
+
GPT-2 Small ∨

Work with us

Interpretability tooling, custom SAE databases, mechanistic audits, circuit reports, and hands-on research, experiments, and studies for teams of all sizes.

Book a call

Not sure if Aquin is right for you?

Join the Aquin Research Community

LLM researchers & ML engineers — open research, fellowships, hackathons, and early beta access.

Join Discord
Substack
© 2026 Aquin. All rights reserved.

Aquin