---
格式版本: 2
标题: "Stress-Testing Efficient Responsible-AI Evaluation: When Compute Savings Change Benchmark Conclusions"
原文链接: "https://arxiv.org/abs/2608.31108"
发布日期: "2026-08-31"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Mon, 31 Aug 2026 17:13:15 UTC (806 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-09-02T03:47:18+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-09-02T03:46:29+08:00"
入库时间: "2026-09-01T19:47:18.283Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=GPU&searchtype=all"
匹配关键词:
  - "GPU"
  - "AI"
相关厂家:
  []
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 5
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "该arXiv论文讨论负责任AI评估的效率与基准结论稳定性，仅提及GPU能耗，与超节点/AI Rack、机柜级AI基础设施无关。"
AI质检模型: "zj-deepseek-v4-flash"
AI质检时间: "2026-09-02T03:47:33+08:00"
AI主题相关性: 0
AI来源权威性: 2
AI新颖性: 1
AI技术细节: 0
AI商业部署信号: 0
AI完整性: 2
AI摘要: "研究者压力测试高效负责任AI评测的结论稳健性：在BBQ/BBQ-V上对3个模型比较7种批处理、量化与基准缩减设置，发现大batch准确率距基线0.35个百分点以内且在多数场景节能，INT8能耗为基线1.79–4.26倍。"
AI摘要模型: "ali-deepseek-v4-flash"
AI摘要时间: "2026-09-01T23:06:49.487Z"
采集批次: "2026年9月1日23点56分52秒"
采集批次ID: "20260901-235652-761"
去重键: "https://arxiv.org/abs/2608.31108"
---

## Computer Science > Machine Learning

## Title:Stress-Testing Efficient Responsible-AI Evaluation: When Compute Savings Change Benchmark Conclusions

[View PDF](https://arxiv.org/pdf/2608.31108) [HTML (experimental)](https://arxiv.org/html/2608.31108v1)

> Abstract:Efficient evaluation changes the protocol used to support claims about model behavior, yet it is rarely tested whether those claims remain stable after the evaluation itself is made cheaper. We stress-test conclusion robustness in responsible-AI benchmarking by evaluating three dense and mixture-of-experts models on BBQ and BBQ-V under seven conditions spanning batching, quantization, benchmark reduction, and their combinations. Rather than treating preserved aggregate accuracy as sufficient, we compare accuracy, bias severity and prevalence, reasoning quality, subgroup behavior, subset-membership stability, runtime, and measured GPU energy against a full-benchmark BF16 baseline. Larger batching keeps accuracy within 0.35 percentage points of baseline and produces comparatively small subgroup changes, while reducing energy in five of six model--dataset settings. INT8 largely preserves quality but uses 1.79--4.26 $\times$ baseline energy. INT4 causes larger, model- and context-dependent changes. Reduced benchmarks provide the most consistent savings, but very small subsets are substantially more sensitive to which items are retained. Efficient evaluation should therefore be treated as a measurement intervention whose validity must be checked across the conclusions the benchmark is intended to support. Our project website is [this https URL](https://vectorinstitute.github.io/sustainable-rai-evaluation/) and the code is available at [this https URL](https://github.com/VectorInstitute/sustainable-rai-evaluation).

| Subjects: | Machine Learning (cs.LG) |
| --- | --- |
| Cite as: | [arXiv:2608.31108](https://arxiv.org/abs/2608.31108) \[cs.LG\] |
|  | (or [arXiv:2608.31108v1](https://arxiv.org/abs/2608.31108v1) \[cs.LG\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2608.31108](https://doi.org/10.48550/arXiv.2608.31108) |

## Submission history

From: Aravind Narayanan \[[view email](https://arxiv.org/show-email/91c8bdce/2608.31108)\]  
**\[v1\]** Mon, 31 Aug 2026 17:13:15 UTC (806 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2608.31108) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))