---
格式版本: 2
标题: "The First Token Is a Clue: Verbalizing Multi-Token Concepts from the J-lens"
原文链接: "https://arxiv.org/abs/2608.31084"
发布日期: "2026-08-31"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Mon, 31 Aug 2026 16:56:20 UTC (1,084 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-09-02T03:55:50+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-09-02T03:54:32+08:00"
入库时间: "2026-09-01T19:55:50.350Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=Oracle&searchtype=all"
匹配关键词:
  - "performance"
相关厂家:
  - "Oracle"
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 5
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "内容为LLM可解释性论文，与超节点/AI Rack/机柜级AI基础设施完全无关，仅搜索词含Oracle但正文无关。"
AI质检模型: "zj-deepseek-v4-flash"
AI质检时间: "2026-09-02T03:55:57+08:00"
AI主题相关性: 0
AI来源权威性: 5
AI新颖性: 0
AI技术细节: 0
AI商业部署信号: 0
AI完整性: 0
AI摘要: "该研究提出利用Jacobian Lens（J-lens）的首个token线索，在冻结模型中直接恢复多token概念的向量，无需训练或预计算模板。"
AI摘要模型: "ali-deepseek-v4-flash"
AI摘要时间: "2026-09-01T23:06:14.786Z"
采集批次: "2026年9月1日23点56分52秒"
采集批次ID: "20260901-235652-761"
去重键: "https://arxiv.org/abs/2608.31084"
---

## Computer Science > Computation and Language

## Title:The First Token Is a Clue: Verbalizing Multi-Token Concepts from the J-lens

Authors:[Xijie Gong](https://arxiv.org/search/cs?searchtype=author&query=Gong,+X), [Tonghan Wang](https://arxiv.org/search/cs?searchtype=author&query=Wang,+T)

[View PDF](https://arxiv.org/pdf/2608.31084) [HTML (experimental)](https://arxiv.org/html/2608.31084v1)

> Abstract:The Jacobian Lens (J-lens) is a recent tool for interpreting LLMs. It reads a hidden state as a ranked list of vocabulary tokens, leaving multi-token concepts without a representation of their own. The original J-lens work addresses this limitation with Template Lens, which precomputes vectors for a fixed phrase vocabulary, and Oracle Lens, which fine-tunes components to propose phrases and reconstruct phrase vectors. We ask whether multi-token concepts and their vectors can instead be recovered directly from J-lens and the frozen model. We find that the first token of a multi-token concept is about as readable as a single-token concept. Given the correct first token and source prompt, the frozen model recovers the second token in 88.3% of two-token cases. We show that a vector for the complete concept can be recovered from subsequent hidden states in a single forward pass. We therefore use J-lens to propose first tokens and let the frozen model complete candidate concepts. We then recover a vector for each candidate and score it alongside the complete vocabulary. Across 496 multi-hop clozes on Gemma-3-12B-IT, Llama-3.1-8B, and Qwen3-14B, our method achieves an average $\mathrm{Rank@}10$ of 43.1%, compared with 27.6% for Template Lens. Without the J-lens clue, performance drops to 21.6%, showing that the first-token clue substantially improves readout. Causal concept swaps using the recovered vectors achieve an average $\mathrm{succ}@10$ of 61.4%, compared with 26.2% for Template Lens under the same intervention. These results show that first-token clues can guide multi-token concept recovery, while subsequent hidden states provide vectors for readout and intervention.

| Subjects: | Computation and Language (cs.CL) |
| --- | --- |
| Cite as: | [arXiv:2608.31084](https://arxiv.org/abs/2608.31084) \[cs.CL\] |
|  | (or [arXiv:2608.31084v1](https://arxiv.org/abs/2608.31084v1) \[cs.CL\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2608.31084](https://doi.org/10.48550/arXiv.2608.31084) |

## Submission history

From: Xijie Gong \[[view email](https://arxiv.org/show-email/db5ac186/2608.31084)\]  
**\[v1\]** Mon, 31 Aug 2026 16:56:20 UTC (1,084 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2608.31084) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))