About
Engineer by day, architecture cartographer by night.
I'm an AI/ML engineer at one of the world's largest logistics companies, where I led a real-time translation tool now used daily by thousands of employees and built document-parsing systems for complex, multilingual reports. Before that I developed production AI agents at a Japanese AI startup, and started out training computer-vision systems to read complex visual data.
Alongside that work I build Amadeus, an agentic research assistant that automates the full research workflow — tracking new publications, verifying technical claims, and extracting data directly from charts and equations in papers. Amadeus is also where this site's research agenda comes from: at production scale, long documents make current models slow, expensive, and unreliable in the middle of the context. Those failures aren't edge cases; they're the two walls every long-document AI system hits first.
So I write about — and now formally research — the architectures trying to tear those walls down: state-space models, linear attention, hybrids, and the recursive-reasoning families. My deep-dives are published in Towards AI, the map lives in the Architecture Atlas, the falsifiable version is the research proposal, and the scholarly version is a survey in preparation.
Running alongside that, and deliberately kept separate from it, is a second direction: BitFuse. Where the survey work asks whether attention can be replaced, this one drops the FFN weight precision to {−1, 0, +1} (MLP-only) and asks what specifically that costs — then whether a different dynamic carrier can win it back. It is now a thirty-six-variant controlled programme on the Qwen3 architecture, trained at the real Qwen3-0.6B scale (~596M): MLP-only 1.58-bit ternary FFN weights, three rival recurrent families (KDA, Mamba-2, Qwen Gated DeltaNet + sparse-attention retrieval), heterogeneous retrieval schedules (a DeepSeek-V4 legacy control and the new DeepSeek-V4.1-Flash family), six distinct residual topologies treated as separate algorithms rather than settings, the Kimi K3, Qwen3.8-Flash-Next and KDA×QSA cross-family lines, and then multi-token prediction and multimodal integration — one change at a time so a result stays attributable. The two lines share no hypotheses and no results, on purpose.
Elsewhere
- Medium — kyouma45 · long-form writing, published in Towards AI
- GitHub — Kyouma45 · code and this site's source