Compare commits
22 Commits
b79ae54b7b
...
69de5faf4b
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
69de5faf4b | ||
|
|
477a64ecb4 | ||
|
|
6405733358 | ||
|
|
2e27677f80 | ||
|
|
2e90139df7 | ||
|
|
77139f5e32 | ||
|
|
0dd60d6021 | ||
|
|
43cdde9bab | ||
|
|
0a824610cf | ||
|
|
7e8e692695 | ||
|
|
3f9a2af279 | ||
|
|
a65c63947b | ||
|
|
8494e51823 | ||
|
|
070e016f81 | ||
|
|
8c8599e77d | ||
|
|
b7e8ab33ec | ||
|
|
db327afd79 | ||
|
|
5eb724ab86 | ||
|
|
cfeb25247a | ||
|
|
26a7d53b1b | ||
|
|
e603738330 | ||
|
|
7f0a76098a |
41
docs/research/mnn-kv-cache-prefix.md
Normal file
41
docs/research/mnn-kv-cache-prefix.md
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
# MNN 前缀 KV Cache 调研(2026-06-10)
|
||||||
|
|
||||||
|
## 结论
|
||||||
|
|
||||||
|
当前打包的 MNN.xcframework 已暴露 prefix cache 能力,技术上可以把每个场景**固定的
|
||||||
|
system prompt + few-shot 模板**的 prefill 结果缓存到磁盘,二次调用跳过这部分 prefill。
|
||||||
|
**建议 W6 polish 阶段、用性能自检卡量化 prefill 占比之后再决定是否接入**;当前瓶颈在
|
||||||
|
decode 而非二次 prefill,优先级低于 C1/C2/Live Activity。
|
||||||
|
|
||||||
|
## 依据(`Frameworks/MNN.xcframework/ios-arm64/MNN.framework/Headers/llm/llm.hpp`)
|
||||||
|
|
||||||
|
| API | 行号 | 含义 |
|
||||||
|
|---|---|---|
|
||||||
|
| `bool setPrefixCacheFile(const std::string& filename, int flag = 0)` | :161 | 指定前缀缓存文件;配套私有成员 `mPrefixCacheMode` / `mPrefixLength` / `mIsPrefixFileExist` / `completePrefixWrite()`(:250-255)印证:命中时 prefill 只算增量部分 |
|
||||||
|
| `bool reuse_kv()` | :171 | 读 config 开关 `reuse_kv`,多轮对话内复用 KV(同一会话增量 prefill) |
|
||||||
|
| `void syncPromptCache(const ChatMessages&)` | :176 | decode 结束后同步缓存文本——注释明确说明 cache 在 generate() 后自更新,此接口供做过后处理(如 deleteThinkPart)的调用方提供更准确版本 |
|
||||||
|
| `void setKVCacheInfo(size_t add, size_t remove, ...)` / `eraseHistory(begin, end)` | :158-160 | 更底层的 KV 区间管理,可做部分历史擦除 |
|
||||||
|
|
||||||
|
## 对本项目的适用性
|
||||||
|
|
||||||
|
- 我们所有调用都是「固定模板前缀 + 可变数据后缀」的单轮 `response()`,与 prefix cache
|
||||||
|
的模型吻合。
|
||||||
|
- 模板体量(估):报告识别 ~900 tok、导出报告 ~700 tok、意图抽取 ~300 tok。
|
||||||
|
按性能自检卡实测的 prefill 速率推算,每次调用预计省 **1~3s**。
|
||||||
|
- 多场景共用一个 cache 文件是否支持多前缀未知;最坏情况只对单一场景(建议选「报告识别」,
|
||||||
|
模板最长、调用最频繁)生效。
|
||||||
|
|
||||||
|
## 风险
|
||||||
|
|
||||||
|
1. `flag` 参数语义在头文件无注释,需读 MNN 源码或实验确认。
|
||||||
|
2. OMNI(多模态)分支下行为未验证——我们的 MNN 模型是 Omni 构建。
|
||||||
|
3. cache 文件与模型权重版本绑定:模型更新/重下载后必须失效,否则可能输出乱码。
|
||||||
|
4. `<img>` 标签在 prompt 前部(`analyzeImages` 把图片标签拼在最前),意味着报告识别场景的
|
||||||
|
"固定前缀" 实际不固定 —— **文本场景(导出/意图抽取)才是干净的 prefix cache 候选**。
|
||||||
|
|
||||||
|
## 建议的接入步骤(W6,如性能自检显示 prefill 占比 >30%)
|
||||||
|
|
||||||
|
1. `MNNLLMBridge` init 后调 `setPrefixCacheFile(<AppSupport>/mnn-prefix.cache)`(仅文本场景)。
|
||||||
|
2. 真机 A/B:同一导出报告各跑 3 次,对比 `LlmContext.prefill_us`。
|
||||||
|
3. 异常处理:加载失败或输出劣化时删除 cache 文件并禁用,回退现状。
|
||||||
|
4. `ModelDownloadService.importModel` / 重下载路径上顺手删除旧 cache 文件。
|
||||||
1626
docs/superpowers/plans/2026-06-10-competition-optimizations.md
Normal file
1626
docs/superpowers/plans/2026-06-10-competition-optimizations.md
Normal file
File diff suppressed because it is too large
Load Diff
930
docs/superpowers/plans/2026-06-10-voice-diary.md
Normal file
930
docs/superpowers/plans/2026-06-10-voice-diary.md
Normal file
@@ -0,0 +1,930 @@
|
|||||||
|
# 语音健康日记 Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** 在「健康记录」(`DiaryQuickSheet`)加语音输入:iOS 端侧流式语音识别实时转写,停止后由本地 LLM(Qwen3.5-2B,经 AIRuntime)整理成健康日记草稿,追加进输入框,可一键回退原话。
|
||||||
|
|
||||||
|
**Architecture:** `DiaryQuickSheet`(mic 按钮 + 状态机)→ `SpeechDictationService`(新,AVAudioEngine + SFSpeechRecognizer 端侧流式转写,不落盘音频)→ `DiaryAssistService.organize(transcript:)`(新方法,经 AIRuntime actor 队列)。Spec:`docs/superpowers/specs/2026-06-10-voice-diary-design.md`。
|
||||||
|
|
||||||
|
**Tech Stack:** SwiftUI、Speech framework(`requiresOnDeviceRecognition = true`)、AVFoundation、Swift Testing(`康康Tests`)。
|
||||||
|
|
||||||
|
**工程约定(执行前必读):**
|
||||||
|
- 工程是 Xcode 16 同步组(`PBXFileSystemSynchronizedRootGroup`):`康康/`、`康康Tests/` 下新建文件**自动入 target,不要改 pbxproj 的文件列表**(权限键除外,见 Task 1)。
|
||||||
|
- CLI 编译/测试必须:`export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer`,且加 `-derivedDataPath ./build/cli-dd`(避免和 Xcode 抢 build.db 锁)。
|
||||||
|
- 工程 `SWIFT_DEFAULT_ACTOR_ISOLATION = MainActor`:类型默认 MainActor;系统回调闭包(audio tap、recognitionTask handler)是 nonisolated,**闭包内只碰局部捕获变量,回主线程用 `Task { @MainActor in }`**。
|
||||||
|
- 用户可见文案用 `String(appLoc: "...")`;字号用 `Font.tjScaled(...)`,禁止裸 `.system(size:)`;颜色只用 `Tj.Palette.*`。**不要手改 `Localizable.xcstrings`**(键缺失时回退键名本身,中文键名即兜底文案)。
|
||||||
|
- `git status` 里已有 `康康/Localizable.xcstrings` 的无关改动——**任何 commit 都不要带上它**(逐文件 `git add`)。
|
||||||
|
- spec 偏差说明(已确认的两处小调整):① CLAUDE.md 提到的 `DebugAIRunner` 已不在工程中,prompt 自检改为 `康康Tests` 单元测试 + 真机手测清单;② mic 按钮放「内容」section 标签行右侧(而非输入框内右下角 overlay),避免与文字重叠,仍属"输入框旁"。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 0: 建独立分支
|
||||||
|
|
||||||
|
**Files:** 无(纯 git)
|
||||||
|
|
||||||
|
- [x] **Step 1: 从当前分支建 `feat/voice-diary`**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /Users/xuhuayong/apps/康康
|
||||||
|
git checkout -b feat/voice-diary
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: `Switched to a new branch 'feat/voice-diary'`(`Localizable.xcstrings` 的本地改动会跟着工作区走,不影响)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: 新增麦克风 + 语音识别权限描述(pbxproj)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `康康.xcodeproj/project.pbxproj:430`、`康康.xcodeproj/project.pbxproj:486`(Debug + Release 两个构建配置)
|
||||||
|
|
||||||
|
pbxproj 的 `INFOPLIST_KEY_*` 按字母序排列:Microphone 插在 `NSHealthUpdateUsageDescription` 之后,SpeechRecognition 插在 `NSPhotoLibraryUsageDescription` 之后。每个锚点行在文件中出现 **2 次**(Debug/Release),用 replace_all 一次改两处。
|
||||||
|
|
||||||
|
- [x] **Step 1: 插入 NSMicrophoneUsageDescription(replace_all)**
|
||||||
|
|
||||||
|
用 Edit 工具,`replace_all: true`:
|
||||||
|
|
||||||
|
old_string(注意行首是 4 个 tab):
|
||||||
|
```
|
||||||
|
INFOPLIST_KEY_NSHealthUpdateUsageDescription = "康康不会写入 Apple 健康数据。此说明用于满足 HealthKit 权限校验,你的健康资料只保留在本机。";
|
||||||
|
```
|
||||||
|
|
||||||
|
new_string:
|
||||||
|
```
|
||||||
|
INFOPLIST_KEY_NSHealthUpdateUsageDescription = "康康不会写入 Apple 健康数据。此说明用于满足 HealthKit 权限校验,你的健康资料只保留在本机。";
|
||||||
|
INFOPLIST_KEY_NSMicrophoneUsageDescription = "康康需要使用麦克风进行语音记录,识别全程在本机完成,声音不会上传。";
|
||||||
|
```
|
||||||
|
|
||||||
|
- [x] **Step 2: 插入 NSSpeechRecognitionUsageDescription(replace_all)**
|
||||||
|
|
||||||
|
old_string:
|
||||||
|
```
|
||||||
|
INFOPLIST_KEY_NSPhotoLibraryUsageDescription = "康康需要读取你已有的体检/化验报告照片用于本地识别,不会上传。";
|
||||||
|
```
|
||||||
|
|
||||||
|
new_string:
|
||||||
|
```
|
||||||
|
INFOPLIST_KEY_NSPhotoLibraryUsageDescription = "康康需要读取你已有的体检/化验报告照片用于本地识别,不会上传。";
|
||||||
|
INFOPLIST_KEY_NSSpeechRecognitionUsageDescription = "语音转文字使用 iOS 端侧识别,内容不会发送给 Apple 或任何服务器。";
|
||||||
|
```
|
||||||
|
|
||||||
|
- [x] **Step 3: 验证两个键各出现 2 次**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
grep -c "NSMicrophoneUsageDescription\|NSSpeechRecognitionUsageDescription" 康康.xcodeproj/project.pbxproj
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: `4`
|
||||||
|
|
||||||
|
- [x] **Step 4: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add 康康.xcodeproj/project.pbxproj
|
||||||
|
git commit -m "feat(语音日记): 新增麦克风与语音识别权限描述(端侧识别文案)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: organize prompt(TDD)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Test: `康康Tests/DiaryOrganizePromptTests.swift`(新建)
|
||||||
|
- Modify: `康康/AI/Prompts/DiaryAssistPrompts.swift`(文件末尾 `}` 前加方法)
|
||||||
|
|
||||||
|
- [x] **Step 1: 写失败测试**
|
||||||
|
|
||||||
|
新建 `康康Tests/DiaryOrganizePromptTests.swift`:
|
||||||
|
|
||||||
|
```swift
|
||||||
|
import Testing
|
||||||
|
@testable import 康康
|
||||||
|
|
||||||
|
struct DiaryOrganizePromptTests {
|
||||||
|
@Test func organizePromptContainsTranscriptAndHardRules() {
|
||||||
|
let prompt = DiaryAssistPrompts.organize(transcript: "今天早上头晕量了血压140 90")
|
||||||
|
|
||||||
|
#expect(prompt.contains("今天早上头晕量了血压140 90"))
|
||||||
|
// 健康数据红线:数值/单位/药名/时间不许改,必须写进 prompt
|
||||||
|
#expect(prompt.contains("数值"))
|
||||||
|
#expect(prompt.contains("药名"))
|
||||||
|
// 自适应样式两条规则都在
|
||||||
|
#expect(prompt.contains("一段通顺的话"))
|
||||||
|
#expect(prompt.contains("分行"))
|
||||||
|
// 项目 prompt 规范:禁思考标签
|
||||||
|
#expect(prompt.contains("/no_think"))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func organizePromptTruncatesLongTranscript() {
|
||||||
|
let long = String(repeating: "头晕", count: 2000) // 4000 字符,超过上限
|
||||||
|
let prompt = DiaryAssistPrompts.organize(transcript: long)
|
||||||
|
|
||||||
|
// 整条 prompt 里口述部分被截断到 organizeTranscriptLimit
|
||||||
|
let expectedTail = String(long.prefix(DiaryAssistPrompts.organizeTranscriptLimit))
|
||||||
|
#expect(prompt.contains(expectedTail))
|
||||||
|
#expect(!prompt.contains(String(long.prefix(DiaryAssistPrompts.organizeTranscriptLimit + 2))))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [x] **Step 2: 跑测试确认编译失败(方法还不存在)**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /Users/xuhuayong/apps/康康
|
||||||
|
export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
|
||||||
|
xcodebuild test -project 康康.xcodeproj -scheme 康康 \
|
||||||
|
-destination 'platform=iOS Simulator,name=iPhone 17' \
|
||||||
|
-only-testing:'康康Tests/DiaryOrganizePromptTests' \
|
||||||
|
-derivedDataPath ./build/cli-dd CODE_SIGNING_ALLOWED=NO 2>&1 | tail -20
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: 编译错误 `type 'DiaryAssistPrompts' has no member 'organize'`(TEST FAILED)。
|
||||||
|
|
||||||
|
- [x] **Step 3: 实现 organize prompt**
|
||||||
|
|
||||||
|
在 `康康/AI/Prompts/DiaryAssistPrompts.swift` 的 enum 末尾(`suggest` 方法后、收尾 `}` 前)加:
|
||||||
|
|
||||||
|
```swift
|
||||||
|
// MARK: - 语音口述 → 日记整理
|
||||||
|
|
||||||
|
/// 口述转写稿截断上限(字符)。2B 模型 context 保护:超长口述只取前面部分。
|
||||||
|
static let organizeTranscriptLimit = 1200
|
||||||
|
|
||||||
|
/// 把语音转写稿整理成健康日记草稿。自适应样式:内容少 → 一段通顺的话;
|
||||||
|
/// 多方面 → 按「方面:内容」分行。
|
||||||
|
/// 红线(spec §2):只重组语言,严禁增删改任何数值、单位、药名、时间——
|
||||||
|
/// 2B 模型把 140/90 改成 130/90 即健康数据事故,所以规则放第一条并配 few-shot 强化。
|
||||||
|
static func organize(transcript: String) -> String {
|
||||||
|
let trimmed = String(transcript.prefix(organizeTranscriptLimit))
|
||||||
|
return """
|
||||||
|
你是健康记录助手。下面是用户口述身体状态的语音转写原话,可能口语化、有重复、缺标点。
|
||||||
|
请把它整理成一条清晰的健康日记。
|
||||||
|
|
||||||
|
硬性规则:
|
||||||
|
- 【绝对不许】增加、删除或改动任何数值、单位、药名、时间——原话说 140/90 就必须写 140/90。
|
||||||
|
- 只重组语言:去掉口头语和重复;用第一人称;不加入原话没有的事实。
|
||||||
|
- 内容只涉及一两个方面 → 整理成一段通顺的话(2-4 句)。
|
||||||
|
- 内容涉及多个方面(症状/用药/饮食/睡眠/运动等) → 按「方面:内容」分行。
|
||||||
|
- 不诊断、不给用药建议、不写「建议就医」。
|
||||||
|
- 只输出整理后的日记正文,不要解释、不要 markdown 围栏、不要 <think> 标签。
|
||||||
|
|
||||||
|
示例 1(口述:那个今天早上起来有点头晕然后我量了下血压140 90比平时高一点没吃早饭就出门了):
|
||||||
|
今天早上起来有点头晕,量了血压 140/90,比平时高一点。没吃早饭就出门了。
|
||||||
|
|
||||||
|
示例 2(口述:今天头晕了一上午下午好点了血压早上量的140 90嗯缬沙坦吃了降脂药忘了吃早饭没吃中午吃的清淡晚上散步了半小时):
|
||||||
|
症状:头晕了一上午,下午好转。
|
||||||
|
血压:早上 140/90。
|
||||||
|
用药:缬沙坦已服,降脂药忘服。
|
||||||
|
饮食:早饭未吃,午餐清淡。
|
||||||
|
运动:晚上散步半小时。
|
||||||
|
|
||||||
|
【口述原话】:
|
||||||
|
\(trimmed)
|
||||||
|
|
||||||
|
Output: /no_think
|
||||||
|
"""
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [x] **Step 4: 跑测试确认通过**
|
||||||
|
|
||||||
|
同 Step 2 命令。Expected: `** TEST SUCCEEDED **`,2 个用例通过。
|
||||||
|
|
||||||
|
- [x] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add 康康Tests/DiaryOrganizePromptTests.swift 康康/AI/Prompts/DiaryAssistPrompts.swift
|
||||||
|
git commit -m "feat(语音日记): organize prompt(自适应样式 + 数值不可改红线)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 3: DiaryAssistService.organize
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `康康/Services/DiaryAssistService.swift:99` 之后(`suggest` 方法后、struct 收尾 `}` 前)
|
||||||
|
|
||||||
|
无新单测(纯转发 AIRuntime,LLM 行为靠真机手测;解析逻辑只有 strip + trim,复用已测过的 `stripThinkBlocks`)。
|
||||||
|
|
||||||
|
- [x] **Step 1: 加 organize 方法**
|
||||||
|
|
||||||
|
在 `suggest` 方法的收尾 `}` 之后、struct 收尾 `}` 之前加:
|
||||||
|
|
||||||
|
```swift
|
||||||
|
/// 把语音转写稿整理成健康日记草稿(spec 2026-06-10-voice-diary)。
|
||||||
|
/// 失败(模型未就绪 / 输出为空)抛错,调用方回退为直接使用原话,不卡死。
|
||||||
|
/// 与 suggest 同样走 AIRuntime actor 队列,自然与追问/拍照串行。
|
||||||
|
func organize(transcript: String) async throws -> (text: String, decodeRate: Double) {
|
||||||
|
do {
|
||||||
|
try await AIRuntime.shared.prepare()
|
||||||
|
} catch {
|
||||||
|
throw AssistError.modelNotReady
|
||||||
|
}
|
||||||
|
|
||||||
|
let prompt = DiaryAssistPrompts.organize(transcript: transcript)
|
||||||
|
var collected = ""
|
||||||
|
var lastRate: Double = 0
|
||||||
|
let stream = await AIRuntime.shared.generate(prompt: prompt, maxTokens: 400)
|
||||||
|
for try await chunk in stream {
|
||||||
|
collected += chunk.text
|
||||||
|
if chunk.decodeRate > 0 { lastRate = chunk.decodeRate }
|
||||||
|
}
|
||||||
|
|
||||||
|
let text = HealthExportService.stripThinkBlocks(collected)
|
||||||
|
.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||||
|
guard !text.isEmpty else { throw AssistError.empty }
|
||||||
|
return (text, lastRate)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [x] **Step 2: 编译验证**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /Users/xuhuayong/apps/康康
|
||||||
|
export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
|
||||||
|
xcodebuild -project 康康.xcodeproj -scheme 康康 \
|
||||||
|
-destination 'platform=iOS Simulator,name=iPhone 17' \
|
||||||
|
-configuration Debug build -derivedDataPath ./build/cli-dd \
|
||||||
|
CODE_SIGNING_ALLOWED=NO 2>&1 | grep -E "\.swift:[0-9]+:[0-9]+: (error|warning):|BUILD (SUCCEEDED|FAILED)"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: `BUILD SUCCEEDED`,无新增 warning。
|
||||||
|
|
||||||
|
- [x] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add 康康/Services/DiaryAssistService.swift
|
||||||
|
git commit -m "feat(语音日记): DiaryAssistService.organize 转写稿整理"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 4: SpeechDictationService(端侧流式转写)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `康康/Services/SpeechDictationService.swift`
|
||||||
|
|
||||||
|
硬件绑定,无单测;模拟器路径(`isAvailable == false`)与真机路径在 Task 7 手测。
|
||||||
|
|
||||||
|
- [x] **Step 1: 新建 SpeechDictationService.swift**
|
||||||
|
|
||||||
|
```swift
|
||||||
|
import Foundation
|
||||||
|
import Speech
|
||||||
|
import AVFoundation
|
||||||
|
|
||||||
|
/// 端侧流式语音转写(spec 2026-06-10-voice-diary)。
|
||||||
|
/// AVAudioEngine 麦克风 buffer → SFSpeechAudioBufferRecognitionRequest,
|
||||||
|
/// `requiresOnDeviceRecognition = true` 硬性端侧,识别内容不出设备;**不落盘任何音频**。
|
||||||
|
///
|
||||||
|
/// 生命周期:start(onPartial:) 开始录音并实时回调 partial;stop() 结束并返回最终稿。
|
||||||
|
/// 调用方:DiaryQuickSheet。工程默认 MainActor 隔离,本类型即 MainActor;
|
||||||
|
/// audio tap 与识别回调在系统线程,闭包内只碰局部捕获对象,回主线程统一走 Task { @MainActor }。
|
||||||
|
final class SpeechDictationService {
|
||||||
|
|
||||||
|
enum DictationError: Error, LocalizedError {
|
||||||
|
case unavailable
|
||||||
|
case audioEngineStartFailed(String)
|
||||||
|
|
||||||
|
var errorDescription: String? {
|
||||||
|
switch self {
|
||||||
|
case .unavailable:
|
||||||
|
return String(appLoc: "本机不支持端侧语音识别")
|
||||||
|
case .audioEngineStartFailed(let m):
|
||||||
|
return String(appLoc: "录音启动失败:\(m)")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 优先系统语言;系统语言不支持端侧时兜底中文(demo 机即使系统是英文也能用)。
|
||||||
|
private static func makeRecognizer() -> SFSpeechRecognizer? {
|
||||||
|
if let r = SFSpeechRecognizer(locale: .current), r.supportsOnDeviceRecognition {
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
if let r = SFSpeechRecognizer(locale: Locale(identifier: "zh-CN")),
|
||||||
|
r.supportsOnDeviceRecognition {
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 本机是否支持端侧识别。false(模拟器/老机型)时 UI 隐藏 mic 入口,静默降级。
|
||||||
|
static var isAvailable: Bool { makeRecognizer() != nil }
|
||||||
|
|
||||||
|
private let audioEngine = AVAudioEngine()
|
||||||
|
private var request: SFSpeechAudioBufferRecognitionRequest?
|
||||||
|
private var task: SFSpeechRecognitionTask?
|
||||||
|
/// 识别回调持续刷新;isFinal 或出错时置 didFinish。stop() 用「final 优先、partial 兜底」。
|
||||||
|
private var latestText = ""
|
||||||
|
private var didFinish = false
|
||||||
|
|
||||||
|
private(set) var isRecording = false
|
||||||
|
|
||||||
|
/// 麦克风 + 语音识别两个权限一起申请。任一被拒返回 false。
|
||||||
|
func requestAuthorization() async -> Bool {
|
||||||
|
let speech = await withCheckedContinuation { (c: CheckedContinuation<SFSpeechRecognizerAuthorizationStatus, Never>) in
|
||||||
|
SFSpeechRecognizer.requestAuthorization { c.resume(returning: $0) }
|
||||||
|
}
|
||||||
|
guard speech == .authorized else { return false }
|
||||||
|
return await AVAudioApplication.requestRecordPermission()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 开始录音 + 流式识别。partial 结果在主线程回调(录音面板实时字幕)。
|
||||||
|
func start(onPartial: @escaping (String) -> Void) throws {
|
||||||
|
guard !isRecording else { return }
|
||||||
|
guard let recognizer = Self.makeRecognizer(), recognizer.isAvailable else {
|
||||||
|
throw DictationError.unavailable
|
||||||
|
}
|
||||||
|
|
||||||
|
let session = AVAudioSession.sharedInstance()
|
||||||
|
do {
|
||||||
|
try session.setCategory(.record, mode: .measurement, options: .duckOthers)
|
||||||
|
try session.setActive(true, options: .notifyOthersOnDeactivation)
|
||||||
|
} catch {
|
||||||
|
throw DictationError.audioEngineStartFailed(error.localizedDescription)
|
||||||
|
}
|
||||||
|
|
||||||
|
let request = SFSpeechAudioBufferRecognitionRequest()
|
||||||
|
request.requiresOnDeviceRecognition = true // 红线:识别不出设备
|
||||||
|
request.shouldReportPartialResults = true
|
||||||
|
request.addsPunctuation = true
|
||||||
|
self.request = request
|
||||||
|
latestText = ""
|
||||||
|
didFinish = false
|
||||||
|
|
||||||
|
let input = audioEngine.inputNode
|
||||||
|
let format = input.outputFormat(forBus: 0)
|
||||||
|
// tap 在音频线程跑:只碰局部捕获的 request,不碰 self
|
||||||
|
input.installTap(onBus: 0, bufferSize: 1024, format: format) { buffer, _ in
|
||||||
|
request.append(buffer)
|
||||||
|
}
|
||||||
|
audioEngine.prepare()
|
||||||
|
do {
|
||||||
|
try audioEngine.start()
|
||||||
|
} catch {
|
||||||
|
input.removeTap(onBus: 0)
|
||||||
|
deactivateSession()
|
||||||
|
throw DictationError.audioEngineStartFailed(error.localizedDescription)
|
||||||
|
}
|
||||||
|
|
||||||
|
task = recognizer.recognitionTask(with: request) { [weak self] result, error in
|
||||||
|
// 系统线程 → 主线程
|
||||||
|
Task { @MainActor in
|
||||||
|
guard let self else { return }
|
||||||
|
if let result {
|
||||||
|
self.latestText = result.bestTranscription.formattedString
|
||||||
|
onPartial(self.latestText)
|
||||||
|
if result.isFinal { self.didFinish = true }
|
||||||
|
}
|
||||||
|
if error != nil { self.didFinish = true }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
isRecording = true
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 停止录音,等待最终识别结果(最多 1.5s,超时用最新 partial),返回最终稿。
|
||||||
|
/// 中途识别出错时已拿到的 partial 一样返回(spec 错误表:照常进整理流程)。
|
||||||
|
func stop() async -> String {
|
||||||
|
guard isRecording else { return "" }
|
||||||
|
isRecording = false
|
||||||
|
|
||||||
|
audioEngine.stop()
|
||||||
|
audioEngine.inputNode.removeTap(onBus: 0)
|
||||||
|
request?.endAudio()
|
||||||
|
|
||||||
|
let deadline = Date().addingTimeInterval(1.5)
|
||||||
|
while !didFinish && Date() < deadline {
|
||||||
|
try? await Task.sleep(nanoseconds: 100_000_000)
|
||||||
|
}
|
||||||
|
task?.cancel()
|
||||||
|
task = nil
|
||||||
|
request = nil
|
||||||
|
deactivateSession()
|
||||||
|
return latestText
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 用户直接关 sheet 时的清理:不关心结果,立即停。
|
||||||
|
func abort() {
|
||||||
|
guard isRecording else { return }
|
||||||
|
isRecording = false
|
||||||
|
audioEngine.stop()
|
||||||
|
audioEngine.inputNode.removeTap(onBus: 0)
|
||||||
|
request?.endAudio()
|
||||||
|
task?.cancel()
|
||||||
|
task = nil
|
||||||
|
request = nil
|
||||||
|
deactivateSession()
|
||||||
|
}
|
||||||
|
|
||||||
|
private func deactivateSession() {
|
||||||
|
try? AVAudioSession.sharedInstance().setActive(false, options: .notifyOthersOnDeactivation)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [x] **Step 2: 编译验证**
|
||||||
|
|
||||||
|
同 Task 3 Step 2 命令。Expected: `BUILD SUCCEEDED`。若出现 actor 隔离 warning(标注 error in Swift 6 language mode 的不阻塞),按提示把回调内对 self 的访问收进 `Task { @MainActor in }`,不许用 `nonisolated(unsafe)` 糊。
|
||||||
|
|
||||||
|
- [x] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add 康康/Services/SpeechDictationService.swift
|
||||||
|
git commit -m "feat(语音日记): SpeechDictationService 端侧流式转写(不落盘音频)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 5: DiaryVoicePanel(录音/整理面板视图)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `康康/Features/Diary/DiaryVoicePanel.swift`
|
||||||
|
|
||||||
|
纯展示组件,状态全部外部传入,DiaryQuickSheet(已 600+ 行)不再膨胀。
|
||||||
|
|
||||||
|
- [x] **Step 1: 新建 DiaryVoicePanel.swift**
|
||||||
|
|
||||||
|
```swift
|
||||||
|
import SwiftUI
|
||||||
|
|
||||||
|
/// 「健康记录」语音输入面板(spec 2026-06-10-voice-diary)。
|
||||||
|
/// 两种模式:recording(实时字幕 + 计时 + 停止)/ organizing(AI 整理中,可取消)。
|
||||||
|
/// 纯展示:状态由 DiaryQuickSheet 持有并传入。
|
||||||
|
struct DiaryVoicePanel: View {
|
||||||
|
enum Mode: Equatable {
|
||||||
|
case recording(elapsedSeconds: Int)
|
||||||
|
case organizing
|
||||||
|
}
|
||||||
|
|
||||||
|
let mode: Mode
|
||||||
|
/// recording 时为实时字幕;organizing 时为已定稿的转写稿(置灰展示)。
|
||||||
|
let transcript: String
|
||||||
|
let onStop: () -> Void
|
||||||
|
let onCancelOrganize: () -> Void
|
||||||
|
|
||||||
|
/// 录音上限 3 分钟(超时由 DiaryQuickSheet 的看门狗触发 onStop)。
|
||||||
|
static let maxRecordingSeconds = 180
|
||||||
|
|
||||||
|
var body: some View {
|
||||||
|
VStack(alignment: .leading, spacing: 10) {
|
||||||
|
header
|
||||||
|
transcriptArea
|
||||||
|
if case .recording = mode {
|
||||||
|
stopButton
|
||||||
|
}
|
||||||
|
}
|
||||||
|
.padding(12)
|
||||||
|
.frame(maxWidth: .infinity, alignment: .leading)
|
||||||
|
.background(
|
||||||
|
RoundedRectangle(cornerRadius: Tj.Radius.sm, style: .continuous)
|
||||||
|
.fill(Tj.Palette.paper)
|
||||||
|
)
|
||||||
|
.overlay(
|
||||||
|
RoundedRectangle(cornerRadius: Tj.Radius.sm, style: .continuous)
|
||||||
|
.strokeBorder(Tj.Palette.lineSoft, lineWidth: 1)
|
||||||
|
)
|
||||||
|
.overlay(alignment: .bottom) {
|
||||||
|
if mode == .organizing {
|
||||||
|
AIFlowBar().padding(.horizontal, 1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
.clipShape(RoundedRectangle(cornerRadius: Tj.Radius.sm, style: .continuous))
|
||||||
|
}
|
||||||
|
|
||||||
|
@ViewBuilder
|
||||||
|
private var header: some View {
|
||||||
|
switch mode {
|
||||||
|
case .recording(let elapsed):
|
||||||
|
HStack(spacing: 8) {
|
||||||
|
Image(systemName: "waveform")
|
||||||
|
.font(.tjScaled(12, weight: .semibold))
|
||||||
|
.foregroundStyle(Tj.Palette.brick)
|
||||||
|
.symbolEffect(.variableColor.iterative, options: .repeating)
|
||||||
|
Text("正在听 · 识别在本机完成")
|
||||||
|
.font(.tjScaled(13, weight: .medium))
|
||||||
|
.foregroundStyle(Tj.Palette.text2)
|
||||||
|
Spacer(minLength: 0)
|
||||||
|
Text(Self.format(elapsed))
|
||||||
|
.font(.tjScaled(12, design: .monospaced))
|
||||||
|
.foregroundStyle(elapsed >= Self.maxRecordingSeconds - 30
|
||||||
|
? Tj.Palette.brick : Tj.Palette.text3)
|
||||||
|
}
|
||||||
|
case .organizing:
|
||||||
|
HStack(spacing: 8) {
|
||||||
|
Image(systemName: "sparkles")
|
||||||
|
.font(.tjScaled(12, weight: .semibold))
|
||||||
|
.foregroundStyle(Tj.Palette.brick)
|
||||||
|
.symbolEffect(.pulse, options: .repeating)
|
||||||
|
Text("AI 整理中 · 本地推理")
|
||||||
|
.font(.tjScaled(13, weight: .medium))
|
||||||
|
.foregroundStyle(Tj.Palette.text2)
|
||||||
|
Spacer(minLength: 0)
|
||||||
|
Button("取消") { onCancelOrganize() }
|
||||||
|
.font(.tjScaled(12, weight: .semibold))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@ViewBuilder
|
||||||
|
private var transcriptArea: some View {
|
||||||
|
ScrollViewReader { proxy in
|
||||||
|
ScrollView(showsIndicators: false) {
|
||||||
|
Text(transcript.isEmpty ? String(appLoc: "开始说话…") : transcript)
|
||||||
|
.font(.tjScaled(14))
|
||||||
|
.foregroundStyle(transcriptColor)
|
||||||
|
.frame(maxWidth: .infinity, alignment: .leading)
|
||||||
|
.fixedSize(horizontal: false, vertical: true)
|
||||||
|
Color.clear.frame(height: 1).id("tail")
|
||||||
|
}
|
||||||
|
.frame(maxHeight: 120)
|
||||||
|
.onChange(of: transcript) { _, _ in
|
||||||
|
proxy.scrollTo("tail", anchor: .bottom)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private var transcriptColor: Color {
|
||||||
|
if transcript.isEmpty { return Tj.Palette.text3 }
|
||||||
|
return mode == .organizing ? Tj.Palette.text3 : Tj.Palette.text
|
||||||
|
}
|
||||||
|
|
||||||
|
private var stopButton: some View {
|
||||||
|
Button(action: onStop) {
|
||||||
|
HStack(spacing: 8) {
|
||||||
|
Image(systemName: "stop.circle.fill")
|
||||||
|
Text("说完了,整理成日记")
|
||||||
|
}
|
||||||
|
.font(.tjScaled(14, weight: .semibold))
|
||||||
|
.foregroundStyle(Tj.Palette.paper)
|
||||||
|
.frame(maxWidth: .infinity)
|
||||||
|
.padding(.vertical, 12)
|
||||||
|
.background(
|
||||||
|
RoundedRectangle(cornerRadius: Tj.Radius.sm, style: .continuous)
|
||||||
|
.fill(Tj.Palette.brick)
|
||||||
|
)
|
||||||
|
.contentShape(Rectangle())
|
||||||
|
}
|
||||||
|
.buttonStyle(.plain)
|
||||||
|
}
|
||||||
|
|
||||||
|
private static func format(_ seconds: Int) -> String {
|
||||||
|
String(format: "%d:%02d", seconds / 60, seconds % 60)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#Preview("录音中") {
|
||||||
|
DiaryVoicePanel(mode: .recording(elapsedSeconds: 23),
|
||||||
|
transcript: "今天早上起来有点头晕,量了血压一百四九十",
|
||||||
|
onStop: {}, onCancelOrganize: {})
|
||||||
|
.padding()
|
||||||
|
}
|
||||||
|
|
||||||
|
#Preview("整理中") {
|
||||||
|
DiaryVoicePanel(mode: .organizing,
|
||||||
|
transcript: "今天早上起来有点头晕,量了血压一百四九十",
|
||||||
|
onStop: {}, onCancelOrganize: {})
|
||||||
|
.padding()
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [x] **Step 2: 编译验证**
|
||||||
|
|
||||||
|
同 Task 3 Step 2 命令。Expected: `BUILD SUCCEEDED`。
|
||||||
|
|
||||||
|
- [x] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add 康康/Features/Diary/DiaryVoicePanel.swift
|
||||||
|
git commit -m "feat(语音日记): DiaryVoicePanel 录音/整理面板"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 6: DiaryQuickSheet 接入(mic 按钮 + 状态机 + 回退 pill)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `康康/Features/Diary/DiaryQuickSheet.swift`
|
||||||
|
|
||||||
|
改 5 处:① 状态 + 录音流程函数;② 「内容」标签行加 mic 按钮;③ 输入框下方挂面板 / 提示条 / 回退 pill;④ `canRequestSuggest` 把 organizing 排除;⑤ onDisappear 清理。
|
||||||
|
|
||||||
|
- [x] **Step 1: 加语音状态(`@FocusState` 行之后、`hasContent` 之前)**
|
||||||
|
|
||||||
|
在 `DiaryQuickSheet.swift:38`(`@FocusState private var contentFocused: Bool`)之后插入:
|
||||||
|
|
||||||
|
```swift
|
||||||
|
|
||||||
|
// MARK: 语音输入状态(spec 2026-06-10-voice-diary)
|
||||||
|
|
||||||
|
enum VoicePhase: Equatable { case idle, recording, organizing }
|
||||||
|
@State private var voicePhase: VoicePhase = .idle
|
||||||
|
@State private var liveTranscript = ""
|
||||||
|
@State private var recordingSeconds = 0
|
||||||
|
/// 最近一次最终转写稿,「改用原话」回退用;再次录音时覆盖。
|
||||||
|
@State private var rawTranscript: String?
|
||||||
|
/// 刚追加进正文的整理稿,用于「改用原话」时在正文中定位替换。
|
||||||
|
/// 用户手动编辑掉该段(正文中找不到了)时 pill 自然消失。
|
||||||
|
@State private var organizedAppended: String?
|
||||||
|
/// 一次性提示条文案(整理失败已填原话 / 没听清等),开始新录音时清掉。
|
||||||
|
@State private var voiceNote: String?
|
||||||
|
@State private var voiceDeniedAlert = false
|
||||||
|
@State private var voiceFlowTask: Task<Void, Never>?
|
||||||
|
@State private var recordingWatchdog: Task<Void, Never>?
|
||||||
|
private let dictation = SpeechDictationService()
|
||||||
|
```
|
||||||
|
|
||||||
|
- [x] **Step 2: 「内容」标签行加 mic 按钮**
|
||||||
|
|
||||||
|
把(`DiaryQuickSheet.swift:79-80` 附近):
|
||||||
|
|
||||||
|
```swift
|
||||||
|
VStack(alignment: .leading, spacing: 8) {
|
||||||
|
sectionLabel(String(appLoc: "内容"))
|
||||||
|
```
|
||||||
|
|
||||||
|
改为:
|
||||||
|
|
||||||
|
```swift
|
||||||
|
VStack(alignment: .leading, spacing: 8) {
|
||||||
|
HStack {
|
||||||
|
sectionLabel(String(appLoc: "内容"))
|
||||||
|
Spacer()
|
||||||
|
if SpeechDictationService.isAvailable, voicePhase == .idle {
|
||||||
|
Button(action: startVoice) {
|
||||||
|
HStack(spacing: 4) {
|
||||||
|
Image(systemName: "mic.fill")
|
||||||
|
.font(.tjScaled(11, weight: .semibold))
|
||||||
|
Text("说一段")
|
||||||
|
.font(.tjScaled(12, weight: .semibold))
|
||||||
|
}
|
||||||
|
.foregroundStyle(isLoading ? Tj.Palette.text3 : Tj.Palette.brick)
|
||||||
|
.padding(.horizontal, 10)
|
||||||
|
.padding(.vertical, 5)
|
||||||
|
.background(Capsule().strokeBorder(
|
||||||
|
isLoading ? Tj.Palette.line : Tj.Palette.brick.opacity(0.5),
|
||||||
|
lineWidth: 1))
|
||||||
|
.contentShape(Capsule())
|
||||||
|
}
|
||||||
|
.buttonStyle(.plain)
|
||||||
|
.disabled(isLoading) // AI 追问生成中不抢 AIRuntime 队列
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
(`TextField` 那段不动,仍在该 VStack 内。)
|
||||||
|
|
||||||
|
- [x] **Step 3: 输入框下方挂面板 / 提示条 / 回退 pill**
|
||||||
|
|
||||||
|
在 TextField 的 `.overlay(...)` 闭合后、该 VStack 的收尾 `}` 之前(即原 `DiaryQuickSheet.swift:95` `)` 与 `:96` `}` 之间)插入:
|
||||||
|
|
||||||
|
```swift
|
||||||
|
|
||||||
|
if voicePhase != .idle {
|
||||||
|
DiaryVoicePanel(
|
||||||
|
mode: voicePhase == .organizing
|
||||||
|
? .organizing
|
||||||
|
: .recording(elapsedSeconds: recordingSeconds),
|
||||||
|
transcript: liveTranscript,
|
||||||
|
onStop: stopVoiceAndOrganize,
|
||||||
|
onCancelOrganize: cancelOrganize
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if let note = voiceNote {
|
||||||
|
HStack(spacing: 6) {
|
||||||
|
Image(systemName: "info.circle")
|
||||||
|
.font(.tjScaled(11))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
Text(note)
|
||||||
|
.font(.tjScaled(11))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
Spacer(minLength: 0)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if let organized = organizedAppended,
|
||||||
|
rawTranscript != nil,
|
||||||
|
content.range(of: organized) != nil {
|
||||||
|
Button(action: revertToRawTranscript) {
|
||||||
|
HStack(spacing: 4) {
|
||||||
|
Image(systemName: "arrow.uturn.backward")
|
||||||
|
.font(.tjScaled(10, weight: .semibold))
|
||||||
|
Text("改用原话")
|
||||||
|
.font(.tjScaled(11, weight: .semibold))
|
||||||
|
}
|
||||||
|
.foregroundStyle(Tj.Palette.ink)
|
||||||
|
.padding(.horizontal, 10)
|
||||||
|
.padding(.vertical, 5)
|
||||||
|
.background(Capsule().strokeBorder(Tj.Palette.line, lineWidth: 1))
|
||||||
|
.contentShape(Capsule())
|
||||||
|
}
|
||||||
|
.buttonStyle(.plain)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [x] **Step 4: organizing 期间禁用「AI 追问」+ 关 sheet 清理 + 权限 alert**
|
||||||
|
|
||||||
|
把 `DiaryQuickSheet.swift:48`:
|
||||||
|
|
||||||
|
```swift
|
||||||
|
private var canRequestSuggest: Bool { hasContent && !isLoading }
|
||||||
|
```
|
||||||
|
|
||||||
|
改为:
|
||||||
|
|
||||||
|
```swift
|
||||||
|
private var canRequestSuggest: Bool { hasContent && !isLoading && voicePhase == .idle }
|
||||||
|
```
|
||||||
|
|
||||||
|
把 `DiaryQuickSheet.swift:146`:
|
||||||
|
|
||||||
|
```swift
|
||||||
|
.onDisappear { suggestTask?.cancel() }
|
||||||
|
```
|
||||||
|
|
||||||
|
改为:
|
||||||
|
|
||||||
|
```swift
|
||||||
|
.onDisappear {
|
||||||
|
suggestTask?.cancel()
|
||||||
|
voiceFlowTask?.cancel()
|
||||||
|
recordingWatchdog?.cancel()
|
||||||
|
dictation.abort()
|
||||||
|
}
|
||||||
|
.alert(String(appLoc: "需要麦克风与语音识别权限"), isPresented: $voiceDeniedAlert) {
|
||||||
|
Button(String(appLoc: "前往设置")) {
|
||||||
|
if let url = URL(string: UIApplication.openSettingsURLString) {
|
||||||
|
UIApplication.shared.open(url)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Button(String(appLoc: "取消"), role: .cancel) {}
|
||||||
|
} message: {
|
||||||
|
Text("语音记录全程在本机完成,声音和文字都不会上传。请在设置中允许麦克风和语音识别。")
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [x] **Step 5: 加流程函数(`// MARK: - Actions` 区,`requestSuggestions` 之前)**
|
||||||
|
|
||||||
|
在 `DiaryQuickSheet.swift` 的 `sectionLabel` 函数后插入:
|
||||||
|
|
||||||
|
```swift
|
||||||
|
|
||||||
|
// MARK: 语音输入流程
|
||||||
|
|
||||||
|
private func startVoice() {
|
||||||
|
contentFocused = false
|
||||||
|
voiceNote = nil
|
||||||
|
voiceFlowTask = Task { @MainActor in
|
||||||
|
guard await dictation.requestAuthorization() else {
|
||||||
|
voiceDeniedAlert = true
|
||||||
|
return
|
||||||
|
}
|
||||||
|
do {
|
||||||
|
liveTranscript = ""
|
||||||
|
recordingSeconds = 0
|
||||||
|
try dictation.start { partial in liveTranscript = partial }
|
||||||
|
withAnimation(.snappy(duration: 0.2)) { voicePhase = .recording }
|
||||||
|
// 计时 + 3 分钟看门狗(到点自动停,行为与点「停止」一致)
|
||||||
|
recordingWatchdog = Task { @MainActor in
|
||||||
|
while !Task.isCancelled {
|
||||||
|
try? await Task.sleep(nanoseconds: 1_000_000_000)
|
||||||
|
guard !Task.isCancelled, voicePhase == .recording else { return }
|
||||||
|
recordingSeconds += 1
|
||||||
|
if recordingSeconds >= DiaryVoicePanel.maxRecordingSeconds {
|
||||||
|
stopVoiceAndOrganize()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
voiceNote = error.localizedDescription
|
||||||
|
voicePhase = .idle
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private func stopVoiceAndOrganize() {
|
||||||
|
guard voicePhase == .recording else { return }
|
||||||
|
recordingWatchdog?.cancel()
|
||||||
|
voiceFlowTask = Task { @MainActor in
|
||||||
|
let transcript = (await dictation.stop())
|
||||||
|
.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||||
|
liveTranscript = transcript
|
||||||
|
guard !transcript.isEmpty else {
|
||||||
|
withAnimation(.snappy(duration: 0.2)) { voicePhase = .idle }
|
||||||
|
voiceNote = String(appLoc: "没听清,再试一次")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
rawTranscript = transcript
|
||||||
|
withAnimation(.snappy(duration: 0.2)) { voicePhase = .organizing }
|
||||||
|
do {
|
||||||
|
let result = try await DiaryAssistService.shared.organize(transcript: transcript)
|
||||||
|
guard !Task.isCancelled else { return }
|
||||||
|
appendToContent(result.text)
|
||||||
|
organizedAppended = result.text
|
||||||
|
lastRate = result.decodeRate
|
||||||
|
} catch is CancellationError {
|
||||||
|
// cancelOrganize 已处理回退,这里只收尾
|
||||||
|
} catch {
|
||||||
|
guard !Task.isCancelled else { return }
|
||||||
|
appendToContent(transcript) // 红线 #5:整理失败回退原话,不卡死
|
||||||
|
organizedAppended = nil
|
||||||
|
voiceNote = String(appLoc: "AI 整理失败,已填入原话")
|
||||||
|
}
|
||||||
|
withAnimation(.snappy(duration: 0.2)) { voicePhase = .idle }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 取消整理:中断 LLM,直接填原话(与失败回退同路径)。
|
||||||
|
private func cancelOrganize() {
|
||||||
|
guard voicePhase == .organizing else { return }
|
||||||
|
voiceFlowTask?.cancel()
|
||||||
|
if let raw = rawTranscript {
|
||||||
|
appendToContent(raw)
|
||||||
|
organizedAppended = nil
|
||||||
|
voiceNote = String(appLoc: "已取消整理,填入原话")
|
||||||
|
}
|
||||||
|
withAnimation(.snappy(duration: 0.2)) { voicePhase = .idle }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 「改用原话」:把刚追加的整理稿替换为原始转写稿(spec §2:LLM 改数兜底)。
|
||||||
|
private func revertToRawTranscript() {
|
||||||
|
guard let raw = rawTranscript,
|
||||||
|
let organized = organizedAppended,
|
||||||
|
let range = content.range(of: organized, options: .backwards) else { return }
|
||||||
|
withAnimation(.snappy(duration: 0.18)) {
|
||||||
|
content = content.replacingCharacters(in: range, with: raw)
|
||||||
|
organizedAppended = nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [x] **Step 6: 编译验证(touch 强制重编拿全量警告)**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /Users/xuhuayong/apps/康康
|
||||||
|
touch 康康/Features/Diary/DiaryQuickSheet.swift
|
||||||
|
export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
|
||||||
|
xcodebuild -project 康康.xcodeproj -scheme 康康 \
|
||||||
|
-destination 'platform=iOS Simulator,name=iPhone 17' \
|
||||||
|
-configuration Debug build -derivedDataPath ./build/cli-dd \
|
||||||
|
CODE_SIGNING_ALLOWED=NO 2>&1 | grep -E "\.swift:[0-9]+:[0-9]+: (error|warning):|BUILD (SUCCEEDED|FAILED)"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: `BUILD SUCCEEDED`,无新增 warning。
|
||||||
|
|
||||||
|
- [x] **Step 7: 跑全量单测(确认没碰坏别的)**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
xcodebuild test -project 康康.xcodeproj -scheme 康康 \
|
||||||
|
-destination 'platform=iOS Simulator,name=iPhone 17' \
|
||||||
|
-derivedDataPath ./build/cli-dd CODE_SIGNING_ALLOWED=NO 2>&1 | tail -5
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: `** TEST SUCCEEDED **`。
|
||||||
|
|
||||||
|
- [x] **Step 8: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add 康康/Features/Diary/DiaryQuickSheet.swift
|
||||||
|
git commit -m "feat(语音日记): DiaryQuickSheet 接入语音输入(录音→整理→回退原话)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 7: 验证与手测清单
|
||||||
|
|
||||||
|
**Files:** 无新增代码
|
||||||
|
|
||||||
|
- [x] **Step 1: 模拟器降级路径验证**
|
||||||
|
|
||||||
|
模拟器跑 App(或 Xcode Preview `DiaryQuickSheet`),打开「+ 新建 → 写日记」:
|
||||||
|
- `SpeechDictationService.isAvailable` 在模拟器多半为 false → 「说一段」按钮应**整体不显示**,其余功能照旧。
|
||||||
|
- 若模拟器恰好支持端侧识别(部分 macOS/Xcode 组合会),按钮出现也算通过——继续验证录音面板出现、无崩溃即可。
|
||||||
|
|
||||||
|
- [ ] **Step 2: 真机手测清单(连 iPhone 跑,逐项确认)**
|
||||||
|
|
||||||
|
1. 首次点「说一段」→ 依次弹语音识别 + 麦克风两个系统权限框,文案是 Task 1 写的端侧说明
|
||||||
|
2. 拒绝权限 → 再点按钮弹「前往设置」alert,能跳系统设置
|
||||||
|
3. 录音中:实时字幕逐字上屏、计时走动、说话时 waveform 动画
|
||||||
|
4. 点「说完了,整理成日记」→ 面板转「AI 整理中」(AIFlowBar 流动)→ 整理稿**追加**进输入框(已有手打内容不被覆盖)
|
||||||
|
5. 口述含数值(如"血压一百四九十")→ 整理稿数值未被改动(说 3 条不同口述各验一次)
|
||||||
|
6. 「改用原话」pill 出现;点击 → 整理稿被替换为原始转写稿;再手动编辑正文该段 → pill 消失
|
||||||
|
7. 飞行模式(模型已下载)→ 全流程照常,验证 100% 本地
|
||||||
|
8. 一个字不说就点停止 → 「没听清,再试一次」,回 idle 不卡死
|
||||||
|
9. 模型未下载(或长按删除模型后)→ 整理失败 → 原话直接入框 + 提示
|
||||||
|
10. 录音中直接下滑关 sheet → 无崩溃,再次打开正常
|
||||||
|
11. 「AI 整理中」点取消 → 原话入框 + 「已取消整理,填入原话」
|
||||||
|
|
||||||
|
- [ ] **Step 3: 把手测结果记进 commit(若有 fix,随 fix 一起提)**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git commit --allow-empty -m "test(语音日记): 真机手测清单通过(见 plan Task 7)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Self-Review 记录
|
||||||
|
|
||||||
|
- **Spec 覆盖**:权限(T1)、organize prompt + 自适应 + 数值红线(T2)、Service(T3)、端侧转写不落盘 + 3 分钟上限 + zh 兜底(T4)、面板 + 实时字幕(T5)、mic 入口 + 状态机 + 追加不覆盖 + 改用原话 + 全部错误回退 + organizing 禁用追问(T6)、手测含飞行模式/空转写/取消(T7)。spec 各节均有对应任务。
|
||||||
|
- **占位符**:无 TBD/TODO;所有代码步骤给了完整代码。
|
||||||
|
- **类型一致性**:`SpeechDictationService.isAvailable/requestAuthorization/start(onPartial:)/stop()/abort()` 在 T4 定义、T6 使用一致;`DiaryVoicePanel.Mode`/`maxRecordingSeconds` T5 定义、T6 使用一致;`organize(transcript:) -> (text:, decodeRate:)` T3 定义、T6 解构一致;`AssistError` 复用现有定义。
|
||||||
121
docs/superpowers/specs/2026-06-10-voice-diary-design.md
Normal file
121
docs/superpowers/specs/2026-06-10-voice-diary-design.md
Normal file
@@ -0,0 +1,121 @@
|
|||||||
|
# 语音健康日记(语音转文字 + AI 整理)设计
|
||||||
|
|
||||||
|
> 2026-06-10 · 在「健康记录」(`DiaryQuickSheet`)里加语音输入:iOS 端侧语音识别实时转写,停止后由本地 LLM 整理成健康日记草稿,可编辑后保存。
|
||||||
|
|
||||||
|
## 背景
|
||||||
|
|
||||||
|
「健康记录」目前只能手打文字(`DiaryQuickSheet` → `DiaryEntry`),已有「AI 医生角度多轮追问」辅助。口述比打字门槛低得多,尤其适合身体不适时记录。
|
||||||
|
|
||||||
|
现有两个本地模型(Qwen3.5-2B 文本、Qwen3-VL 视觉)都没有音频编码器,无法做 ASR;引入 Whisper 类模型要 +0.5~1.5GB 体积和一条新推理链路,不可接受。`SFSpeechRecognizer` 支持强制端侧识别(`requiresOnDeviceRecognition = true`),中文质量够用、零体积,与「100% 本地」卖点完全一致。
|
||||||
|
|
||||||
|
## 决策(已与用户确认)
|
||||||
|
|
||||||
|
| 维度 | 决定 |
|
||||||
|
|---|---|
|
||||||
|
| 交互形态 | 说完 → 自动调 LLM 整理成日记草稿(非纯听写) |
|
||||||
|
| 整理样式 | 自适应:口述短 → 一段通顺的话;口述长且多方面 → 自动分点 |
|
||||||
|
| 入口 | `DiaryQuickSheet` 输入框旁麦克风按钮(不动 RecordSheet 骨架) |
|
||||||
|
| 转写链路 | 流式实时转写(AVAudioEngine buffer → 实时字幕),不落盘音频 |
|
||||||
|
| ASR 引擎 | `SFSpeechRecognizer` 端侧;不引入 Whisper;不做云端回退 |
|
||||||
|
|
||||||
|
## 架构
|
||||||
|
|
||||||
|
```
|
||||||
|
DiaryQuickSheet(mic 按钮 + 录音面板)
|
||||||
|
├─► SpeechDictationService(新)── AVAudioEngine + SFSpeechRecognizer(端侧)
|
||||||
|
└─► DiaryAssistService.organize(transcript:)(新方法)──► AIRuntime ──► MNN/MLX
|
||||||
|
```
|
||||||
|
|
||||||
|
符合模块边界:UI 不直接碰 AIRuntime;语音采集是系统能力,封装成独立 Service。
|
||||||
|
|
||||||
|
## 组件
|
||||||
|
|
||||||
|
### 1. `SpeechDictationService`(新,`Services/`,`@MainActor`)
|
||||||
|
|
||||||
|
封装 AVAudioEngine 麦克风采集 + `SFSpeechAudioBufferRecognitionRequest` 流式识别。
|
||||||
|
|
||||||
|
接口:
|
||||||
|
|
||||||
|
- `static var isAvailable: Bool` — 本机是否支持**端侧**中文识别(`supportsOnDeviceRecognition` + locale 检查;模拟器/老机型为 false)
|
||||||
|
- `func requestAuthorization() async -> Bool` — 麦克风 + 语音识别两个权限一起申请
|
||||||
|
- `func start(onPartial: @escaping (String) -> Void) throws` — 开始录音,partial 结果实时回调(录音面板字幕)
|
||||||
|
- `func stop() async -> String` — 停止并返回最终转写稿
|
||||||
|
|
||||||
|
实现要点:
|
||||||
|
|
||||||
|
- `requiresOnDeviceRecognition = true`(硬性,识别内容不出设备)
|
||||||
|
- `addsPunctuation = true`(自动标点)
|
||||||
|
- locale 跟随系统,不支持端侧时 `isAvailable = false`
|
||||||
|
- **不写任何音频文件**,buffer 即用即弃
|
||||||
|
- 录音上限 3 分钟,到点自动 stop
|
||||||
|
|
||||||
|
### 2. `DiaryAssistService.organize(transcript:)`(新方法)
|
||||||
|
|
||||||
|
```swift
|
||||||
|
func organize(transcript: String) async throws -> (text: String, decodeRate: Double)
|
||||||
|
```
|
||||||
|
|
||||||
|
- prompt 加在 `AI/Prompts/DiaryAssistPrompts.swift`:`organizePrompt(transcript:)`
|
||||||
|
- few-shot 两例:短口述 → 一段第一人称通顺文本;长口述(症状/用药/饮食多方面)→ 分点
|
||||||
|
- **硬性约束写进 prompt:只重组语言,不得增删改任何数值、单位、药名、时间**(健康数据,2B 模型改数即事故)
|
||||||
|
- 转写稿超长先截断(保护 context),非流式,await 完整结果
|
||||||
|
- 走 AIRuntime actor 队列,与「多轮追问」「拍照识别」自然串行
|
||||||
|
|
||||||
|
### 3. `DiaryQuickSheet` UI 改动
|
||||||
|
|
||||||
|
- 内容输入框 trailing 加 mic 按钮(`isAvailable == false` 时整个隐藏)
|
||||||
|
- 录音态:输入框下方展开录音面板 —— 实时字幕区 + 脉冲动画(sparkles/waveform `symbolEffect`)+「停止」按钮
|
||||||
|
- 整理态:面板转「AI 整理中」(复用 `AIFlowBar` + tok/s),可取消
|
||||||
|
- 完成:整理稿**追加**进输入框(沿用 `appendToContent`,不覆盖已写内容);面板收起
|
||||||
|
- 完成后显示一次性「改用原话」pill:点击把刚追加的整理稿替换为原始转写稿(原始稿在本次 sheet 生命周期内持有;再次录音或手动编辑该段后 pill 消失)
|
||||||
|
- 整理稿入框后,既有「AI 多轮追问」功能照常可用,无需特殊处理
|
||||||
|
|
||||||
|
## 状态机
|
||||||
|
|
||||||
|
```
|
||||||
|
idle ──(点 mic,权限 OK)──► recording ──(停止/3min 到点)──► organizing ──► done(回 idle)
|
||||||
|
```
|
||||||
|
|
||||||
|
- 实时字幕只显示在录音面板,**停止前不进输入框**
|
||||||
|
- `organizing` 期间 mic 按钮与「AI 追问」按钮禁用(AIRuntime 串行,避免排队困惑)
|
||||||
|
|
||||||
|
## 错误处理(红线 #5:全部有回退,不卡死)
|
||||||
|
|
||||||
|
| 故障 | 行为 |
|
||||||
|
|---|---|
|
||||||
|
| 权限被拒 | 弹说明 alert + 「前往设置」跳系统设置 |
|
||||||
|
| 本机不支持端侧识别(含模拟器) | mic 按钮隐藏,静默降级为纯手打 |
|
||||||
|
| 识别中途出错 | 已拿到的 partial 文本照常进 organizing |
|
||||||
|
| 转写结果为空 | 提示「没听清,再试一次」,回 idle |
|
||||||
|
| LLM 未就绪 / 整理失败 | **原始转写稿直接追加进输入框** + 提示「AI 整理失败,已填入原话」 |
|
||||||
|
|
||||||
|
不做云端识别回退(红线 #1:不引入云服务)。
|
||||||
|
|
||||||
|
## 权限(project.pbxproj 新增两条 INFOPLIST_KEY)
|
||||||
|
|
||||||
|
- `NSMicrophoneUsageDescription`:康康需要使用麦克风进行语音记录,识别全程在本机完成,声音不会上传。
|
||||||
|
- `NSSpeechRecognitionUsageDescription`:语音转文字使用 iOS 端侧识别,内容不会发送给 Apple 或任何服务器。
|
||||||
|
|
||||||
|
## 测试
|
||||||
|
|
||||||
|
- `organize` prompt:`DebugAIRunner` 加自检入口(短/长两条样例口述,肉眼验自适应样式 + 数值不被改动)
|
||||||
|
- 录音链路:真机手测清单(权限首次申请、录音字幕、3 分钟自动停、整理失败回退、「改用原话」)
|
||||||
|
- 模拟器:验证 `isAvailable == false` 时 mic 按钮隐藏
|
||||||
|
|
||||||
|
## 范围边界(不做)
|
||||||
|
|
||||||
|
- 症状 / AI 问答的语音入口
|
||||||
|
- 音频文件保存或回放
|
||||||
|
- Whisper / 任何新模型
|
||||||
|
- Live Activity 集成(前台短流程,无必要)
|
||||||
|
- 多语言听写优化(locale 跟系统,不支持即降级)
|
||||||
|
|
||||||
|
## 卖点映射(§12)
|
||||||
|
|
||||||
|
1. 降低记录门槛 → 卖点 1(影像档案之外的日常记录闭环)
|
||||||
|
2. 「系统端侧 ASR + 本地 LLM 整理」全链路不出设备 → 卖点 2(100% 本地)
|
||||||
|
3. 日记语料变多 → 卖点 3(本地 RAG 长期记忆)
|
||||||
|
|
||||||
|
## 排期
|
||||||
|
|
||||||
|
清单外新功能(红线 #6),本设计即立项讨论结论。工作量约 1~1.5 天,独立小分支插队,不挤占 C1/VL 主线。
|
||||||
@@ -428,8 +428,10 @@
|
|||||||
INFOPLIST_KEY_NSFaceIDUsageDescription = "用于解锁你的健康档案,数据始终保留在本机。";
|
INFOPLIST_KEY_NSFaceIDUsageDescription = "用于解锁你的健康档案,数据始终保留在本机。";
|
||||||
INFOPLIST_KEY_NSHealthShareUsageDescription = "康康会读取 Apple 健康中的生日、性别、身高和血型,用于本地填充个人资料,不会上传。";
|
INFOPLIST_KEY_NSHealthShareUsageDescription = "康康会读取 Apple 健康中的生日、性别、身高和血型,用于本地填充个人资料,不会上传。";
|
||||||
INFOPLIST_KEY_NSHealthUpdateUsageDescription = "康康不会写入 Apple 健康数据。此说明用于满足 HealthKit 权限校验,你的健康资料只保留在本机。";
|
INFOPLIST_KEY_NSHealthUpdateUsageDescription = "康康不会写入 Apple 健康数据。此说明用于满足 HealthKit 权限校验,你的健康资料只保留在本机。";
|
||||||
|
INFOPLIST_KEY_NSMicrophoneUsageDescription = "康康需要使用麦克风进行语音记录,识别全程在本机完成,声音不会上传。";
|
||||||
INFOPLIST_KEY_NSPhotoLibraryAddUsageDescription = "康康会把识别后的报告原图加密保存到 App 沙盒,不会写入你的相册。";
|
INFOPLIST_KEY_NSPhotoLibraryAddUsageDescription = "康康会把识别后的报告原图加密保存到 App 沙盒,不会写入你的相册。";
|
||||||
INFOPLIST_KEY_NSPhotoLibraryUsageDescription = "康康需要读取你已有的体检/化验报告照片用于本地识别,不会上传。";
|
INFOPLIST_KEY_NSPhotoLibraryUsageDescription = "康康需要读取你已有的体检/化验报告照片用于本地识别,不会上传。";
|
||||||
|
INFOPLIST_KEY_NSSpeechRecognitionUsageDescription = "语音转文字使用 iOS 端侧识别,内容不会发送给 Apple 或任何服务器。";
|
||||||
"INFOPLIST_KEY_UIApplicationSceneManifest_Generation[sdk=iphoneos*]" = YES;
|
"INFOPLIST_KEY_UIApplicationSceneManifest_Generation[sdk=iphoneos*]" = YES;
|
||||||
"INFOPLIST_KEY_UIApplicationSceneManifest_Generation[sdk=iphonesimulator*]" = YES;
|
"INFOPLIST_KEY_UIApplicationSceneManifest_Generation[sdk=iphonesimulator*]" = YES;
|
||||||
"INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents[sdk=iphoneos*]" = YES;
|
"INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents[sdk=iphoneos*]" = YES;
|
||||||
@@ -484,8 +486,10 @@
|
|||||||
INFOPLIST_KEY_NSFaceIDUsageDescription = "用于解锁你的健康档案,数据始终保留在本机。";
|
INFOPLIST_KEY_NSFaceIDUsageDescription = "用于解锁你的健康档案,数据始终保留在本机。";
|
||||||
INFOPLIST_KEY_NSHealthShareUsageDescription = "康康会读取 Apple 健康中的生日、性别、身高和血型,用于本地填充个人资料,不会上传。";
|
INFOPLIST_KEY_NSHealthShareUsageDescription = "康康会读取 Apple 健康中的生日、性别、身高和血型,用于本地填充个人资料,不会上传。";
|
||||||
INFOPLIST_KEY_NSHealthUpdateUsageDescription = "康康不会写入 Apple 健康数据。此说明用于满足 HealthKit 权限校验,你的健康资料只保留在本机。";
|
INFOPLIST_KEY_NSHealthUpdateUsageDescription = "康康不会写入 Apple 健康数据。此说明用于满足 HealthKit 权限校验,你的健康资料只保留在本机。";
|
||||||
|
INFOPLIST_KEY_NSMicrophoneUsageDescription = "康康需要使用麦克风进行语音记录,识别全程在本机完成,声音不会上传。";
|
||||||
INFOPLIST_KEY_NSPhotoLibraryAddUsageDescription = "康康会把识别后的报告原图加密保存到 App 沙盒,不会写入你的相册。";
|
INFOPLIST_KEY_NSPhotoLibraryAddUsageDescription = "康康会把识别后的报告原图加密保存到 App 沙盒,不会写入你的相册。";
|
||||||
INFOPLIST_KEY_NSPhotoLibraryUsageDescription = "康康需要读取你已有的体检/化验报告照片用于本地识别,不会上传。";
|
INFOPLIST_KEY_NSPhotoLibraryUsageDescription = "康康需要读取你已有的体检/化验报告照片用于本地识别,不会上传。";
|
||||||
|
INFOPLIST_KEY_NSSpeechRecognitionUsageDescription = "语音转文字使用 iOS 端侧识别,内容不会发送给 Apple 或任何服务器。";
|
||||||
"INFOPLIST_KEY_UIApplicationSceneManifest_Generation[sdk=iphoneos*]" = YES;
|
"INFOPLIST_KEY_UIApplicationSceneManifest_Generation[sdk=iphoneos*]" = YES;
|
||||||
"INFOPLIST_KEY_UIApplicationSceneManifest_Generation[sdk=iphonesimulator*]" = YES;
|
"INFOPLIST_KEY_UIApplicationSceneManifest_Generation[sdk=iphonesimulator*]" = YES;
|
||||||
"INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents[sdk=iphoneos*]" = YES;
|
"INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents[sdk=iphoneos*]" = YES;
|
||||||
|
|||||||
@@ -15,6 +15,13 @@ enum AIRuntimeError: Error, LocalizedError {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// 推理优先级。interactive = 用户正在屏幕前等(识别/问答/自检);
|
||||||
|
/// background = 预生成(报告摘要等),排队让行、解码中可被协作式抢占。
|
||||||
|
nonisolated enum InferencePriority: Sendable, Equatable {
|
||||||
|
case interactive
|
||||||
|
case background
|
||||||
|
}
|
||||||
|
|
||||||
actor AIRuntime {
|
actor AIRuntime {
|
||||||
static let shared = AIRuntime()
|
static let shared = AIRuntime()
|
||||||
|
|
||||||
@@ -29,6 +36,21 @@ actor AIRuntime {
|
|||||||
private(set) var vlStatus: Status = .notReady
|
private(set) var vlStatus: Status = .notReady
|
||||||
private(set) var lastDecodeRate: Double = 0
|
private(set) var lastDecodeRate: Double = 0
|
||||||
|
|
||||||
|
/// 末次文本生成的性能统计(性能自检页消费;两后端归一)。
|
||||||
|
private(set) var lastGenerateStats: GenerateStats?
|
||||||
|
|
||||||
|
/// 当前实际生效的后端标签(性能自检 / PPT 截图用)。
|
||||||
|
var activeBackendLabel: String {
|
||||||
|
if InferenceEngine.current == .mnn, mnnStatus == .ready {
|
||||||
|
return InferenceEngine.cpuSupportsSME2 ? "MNN · SME2" : "MNN · NEON"
|
||||||
|
}
|
||||||
|
#if targetEnvironment(simulator)
|
||||||
|
return "MLX · CPU(模拟器)"
|
||||||
|
#else
|
||||||
|
return "MLX · GPU"
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
private var llmSession: LLMSession?
|
private var llmSession: LLMSession?
|
||||||
private var vlSession: VLSession?
|
private var vlSession: VLSession?
|
||||||
|
|
||||||
@@ -52,30 +74,56 @@ actor AIRuntime {
|
|||||||
// 这里用 actor 内信号量(count = 1):所有「会占显存的重活」(解码 + 模型加载)
|
// 这里用 actor 内信号量(count = 1):所有「会占显存的重活」(解码 + 模型加载)
|
||||||
// 进入前先 await acquireGate(),结束后 releaseGate()。actor 串行执行保证
|
// 进入前先 await acquireGate(),结束后 releaseGate()。actor 串行执行保证
|
||||||
// gateBusy / gateWaiters 的读写天然无并发。
|
// gateBusy / gateWaiters 的读写天然无并发。
|
||||||
|
private struct GateWaiter {
|
||||||
|
let priority: InferencePriority
|
||||||
|
let cont: CheckedContinuation<Void, Never>
|
||||||
|
}
|
||||||
private var gateBusy = false
|
private var gateBusy = false
|
||||||
private var gateWaiters: [CheckedContinuation<Void, Never>] = []
|
private var gateHolderPriority: InferencePriority = .interactive
|
||||||
|
private var preemptRequested = false
|
||||||
|
private var gateWaiters: [GateWaiter] = []
|
||||||
|
|
||||||
private func acquireGate() async {
|
/// interactive 排到所有 background 等待者之前;同优先级保持 FIFO。纯函数,单测覆盖。
|
||||||
|
nonisolated static func gateInsertionIndex(of priority: InferencePriority,
|
||||||
|
in waiting: [InferencePriority]) -> Int {
|
||||||
|
guard priority == .interactive else { return waiting.count }
|
||||||
|
return waiting.firstIndex(of: .background) ?? waiting.count
|
||||||
|
}
|
||||||
|
|
||||||
|
private func acquireGate(_ priority: InferencePriority = .interactive) async {
|
||||||
if !gateBusy {
|
if !gateBusy {
|
||||||
gateBusy = true
|
gateBusy = true
|
||||||
|
gateHolderPriority = priority
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
// 前台请求撞上后台持有者:请其让位 —— 后台解码循环在下一个 token 抛 CancellationError。
|
||||||
|
if priority == .interactive, gateHolderPriority == .background {
|
||||||
|
preemptRequested = true
|
||||||
|
}
|
||||||
await withCheckedContinuation { (cont: CheckedContinuation<Void, Never>) in
|
await withCheckedContinuation { (cont: CheckedContinuation<Void, Never>) in
|
||||||
gateWaiters.append(cont)
|
let idx = Self.gateInsertionIndex(of: priority, in: gateWaiters.map(\.priority))
|
||||||
|
gateWaiters.insert(GateWaiter(priority: priority, cont: cont), at: idx)
|
||||||
}
|
}
|
||||||
// 被 releaseGate 唤醒时即已持有闸门(gateBusy 保持 true)。
|
// 被 releaseGate 唤醒时即已持有闸门(gateBusy 保持 true)。
|
||||||
}
|
}
|
||||||
|
|
||||||
private func releaseGate() {
|
private func releaseGate() {
|
||||||
|
preemptRequested = false
|
||||||
if gateWaiters.isEmpty {
|
if gateWaiters.isEmpty {
|
||||||
gateBusy = false
|
gateBusy = false
|
||||||
} else {
|
} else {
|
||||||
// 把闸门直接交给队首等待者,gateBusy 维持 true,不留空窗。
|
// 把闸门直接交给队首等待者,gateBusy 维持 true,不留空窗。
|
||||||
let next = gateWaiters.removeFirst()
|
let next = gateWaiters.removeFirst()
|
||||||
next.resume()
|
gateHolderPriority = next.priority
|
||||||
|
next.cont.resume()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// 后台持有者每收到一个 token 查一次:前台在排队就让位。
|
||||||
|
private func shouldPreempt(_ priority: InferencePriority) -> Bool {
|
||||||
|
priority == .background && preemptRequested
|
||||||
|
}
|
||||||
|
|
||||||
private init() {}
|
private init() {}
|
||||||
|
|
||||||
/// App 启动时调用一次:给 MLX 的 GPU 缓冲池设上限,避免 reuse cache 在大模型常驻之上
|
/// App 启动时调用一次:给 MLX 的 GPU 缓冲池设上限,避免 reuse cache 在大模型常驻之上
|
||||||
@@ -180,9 +228,12 @@ actor AIRuntime {
|
|||||||
|
|
||||||
/// 流式生成。调用前应先 await prepare()。
|
/// 流式生成。调用前应先 await prepare()。
|
||||||
/// 注意:返回流是同步创建的,但跨 actor 调用 LLMSession 需要 await。
|
/// 注意:返回流是同步创建的,但跨 actor 调用 LLMSession 需要 await。
|
||||||
func generate(prompt: String, maxTokens: Int = 256) -> AsyncThrowingStream<TokenChunk, Error> {
|
/// priority = .background 时排队让行、解码中可被前台请求按 token 抢占(CancellationError 透传)。
|
||||||
|
func generate(prompt: String,
|
||||||
|
maxTokens: Int = 256,
|
||||||
|
priority: InferencePriority = .interactive) -> AsyncThrowingStream<TokenChunk, Error> {
|
||||||
if InferenceEngine.current == .mnn, mnnStatus == .ready {
|
if InferenceEngine.current == .mnn, mnnStatus == .ready {
|
||||||
return mnnGenerate(prompt: prompt, maxTokens: maxTokens)
|
return mnnGenerate(prompt: prompt, maxTokens: maxTokens, priority: priority)
|
||||||
}
|
}
|
||||||
// 在 actor 隔离上下文中捕获快照,Task 内不再访问 self.status / self.llmSession
|
// 在 actor 隔离上下文中捕获快照,Task 内不再访问 self.status / self.llmSession
|
||||||
let snapshotStatus = status
|
let snapshotStatus = status
|
||||||
@@ -195,7 +246,7 @@ actor AIRuntime {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
// 进闸门:保证本次 LLM 解码与任何 VL 解码 / 模型加载串行,绝不并发占显存。
|
// 进闸门:保证本次 LLM 解码与任何 VL 解码 / 模型加载串行,绝不并发占显存。
|
||||||
await self.acquireGate()
|
await self.acquireGate(priority)
|
||||||
do {
|
do {
|
||||||
// session.generate 跨 actor 边界,需要 await
|
// session.generate 跨 actor 边界,需要 await
|
||||||
let stream = await session.generate(prompt: prompt, maxTokens: maxTokens)
|
let stream = await session.generate(prompt: prompt, maxTokens: maxTokens)
|
||||||
@@ -203,12 +254,18 @@ actor AIRuntime {
|
|||||||
// 消费者(UI)提前关闭/取消时,下面的 checkCancellation 让本 Task 尽快退出,
|
// 消费者(UI)提前关闭/取消时,下面的 checkCancellation 让本 Task 尽快退出,
|
||||||
// 连带丢弃 session 流并触发其 onTermination,停止底层 MLX 解码,不空耗 GPU。
|
// 连带丢弃 session 流并触发其 onTermination,停止底层 MLX 解码,不空耗 GPU。
|
||||||
try Task.checkCancellation()
|
try Task.checkCancellation()
|
||||||
|
// 后台任务让位:前台请求在排队时,下一个 token 处主动退出。
|
||||||
|
if self.shouldPreempt(priority) { throw CancellationError() }
|
||||||
// Task 闭包在 generate() 内启动,继承 AIRuntime 的 actor 隔离;
|
// Task 闭包在 generate() 内启动,继承 AIRuntime 的 actor 隔离;
|
||||||
// 调用同 actor 的 recordRate 不需要 await
|
// 调用同 actor 的 recordRate 不需要 await
|
||||||
self.recordRate(chunk.decodeRate)
|
self.recordRate(chunk.decodeRate)
|
||||||
continuation.yield(chunk)
|
continuation.yield(chunk)
|
||||||
}
|
}
|
||||||
|
self.lastGenerateStats = await session.lastStats
|
||||||
continuation.finish()
|
continuation.finish()
|
||||||
|
} catch is CancellationError {
|
||||||
|
// 取消/抢占以 CancellationError 透传,调用方据此区分「让位」与「真失败」。
|
||||||
|
continuation.finish(throwing: CancellationError())
|
||||||
} catch {
|
} catch {
|
||||||
continuation.finish(throwing: AIRuntimeError.inferenceFailed("\(error)"))
|
continuation.finish(throwing: AIRuntimeError.inferenceFailed("\(error)"))
|
||||||
}
|
}
|
||||||
@@ -222,7 +279,9 @@ actor AIRuntime {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// MNN(CPU/SME2)文本流式生成。结构与 MLX 分支一致:进闸门、串行解码、记录速率。
|
/// MNN(CPU/SME2)文本流式生成。结构与 MLX 分支一致:进闸门、串行解码、记录速率。
|
||||||
private func mnnGenerate(prompt: String, maxTokens: Int) -> AsyncThrowingStream<TokenChunk, Error> {
|
private func mnnGenerate(prompt: String,
|
||||||
|
maxTokens: Int,
|
||||||
|
priority: InferencePriority) -> AsyncThrowingStream<TokenChunk, Error> {
|
||||||
let ready = (mnnStatus == .ready)
|
let ready = (mnnStatus == .ready)
|
||||||
return AsyncThrowingStream { continuation in
|
return AsyncThrowingStream { continuation in
|
||||||
let task = Task {
|
let task = Task {
|
||||||
@@ -230,15 +289,21 @@ actor AIRuntime {
|
|||||||
continuation.finish(throwing: AIRuntimeError.notReady)
|
continuation.finish(throwing: AIRuntimeError.notReady)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
await self.acquireGate()
|
await self.acquireGate(priority)
|
||||||
do {
|
do {
|
||||||
let stream = await self.mnn.generate(prompt: prompt, maxTokens: maxTokens)
|
let stream = await self.mnn.generate(prompt: prompt, maxTokens: maxTokens)
|
||||||
for try await chunk in stream {
|
for try await chunk in stream {
|
||||||
try Task.checkCancellation()
|
try Task.checkCancellation()
|
||||||
|
// 后台任务让位:前台请求在排队时,下一个 token 处主动退出
|
||||||
|
//(流终止触发 MNNBackend.onTermination → bridge.cancel())。
|
||||||
|
if self.shouldPreempt(priority) { throw CancellationError() }
|
||||||
self.recordRate(chunk.decodeRate)
|
self.recordRate(chunk.decodeRate)
|
||||||
continuation.yield(chunk)
|
continuation.yield(chunk)
|
||||||
}
|
}
|
||||||
|
self.lastGenerateStats = await self.mnn.lastStats
|
||||||
continuation.finish()
|
continuation.finish()
|
||||||
|
} catch is CancellationError {
|
||||||
|
continuation.finish(throwing: CancellationError())
|
||||||
} catch {
|
} catch {
|
||||||
continuation.finish(throwing: AIRuntimeError.inferenceFailed("\(error)"))
|
continuation.finish(throwing: AIRuntimeError.inferenceFailed("\(error)"))
|
||||||
}
|
}
|
||||||
|
|||||||
19
康康/AI/GenerateStats.swift
Normal file
19
康康/AI/GenerateStats.swift
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
import Foundation
|
||||||
|
|
||||||
|
/// 单次生成的性能统计,两后端(MNN / MLX)归一。
|
||||||
|
/// MNN 取自 LlmContext(prefill_us / decode_us);MLX 取自 GenerateCompletionInfo。
|
||||||
|
struct GenerateStats: Sendable, Equatable {
|
||||||
|
var promptTokens: Int
|
||||||
|
var genTokens: Int
|
||||||
|
/// prefill(读入 prompt)耗时,秒。
|
||||||
|
var prefillSeconds: Double
|
||||||
|
/// decode(逐 token 生成)耗时,秒。
|
||||||
|
var decodeSeconds: Double
|
||||||
|
|
||||||
|
var prefillTokensPerSecond: Double {
|
||||||
|
prefillSeconds > 0 ? Double(promptTokens) / prefillSeconds : 0
|
||||||
|
}
|
||||||
|
var decodeTokensPerSecond: Double {
|
||||||
|
decodeSeconds > 0 ? Double(genTokens) / decodeSeconds : 0
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -8,6 +8,11 @@ import MLXLMCommon
|
|||||||
actor LLMSession {
|
actor LLMSession {
|
||||||
let container: ModelContainer
|
let container: ModelContainer
|
||||||
|
|
||||||
|
/// 末次生成统计(取自流末尾的 .info 完成事件,性能自检用)。
|
||||||
|
private(set) var lastStats: GenerateStats?
|
||||||
|
|
||||||
|
private func record(_ s: GenerateStats) { lastStats = s }
|
||||||
|
|
||||||
init(container: ModelContainer) {
|
init(container: ModelContainer) {
|
||||||
self.container = container
|
self.container = container
|
||||||
}
|
}
|
||||||
@@ -78,9 +83,14 @@ actor LLMSession {
|
|||||||
let rate = elapsed > 0 ? Double(produced) / elapsed : 0
|
let rate = elapsed > 0 ? Double(produced) / elapsed : 0
|
||||||
continuation.yield(TokenChunk(text: text, decodeRate: rate))
|
continuation.yield(TokenChunk(text: text, decodeRate: rate))
|
||||||
|
|
||||||
case .info:
|
case .info(let info):
|
||||||
// 生成完成统计,是流的最后一个事件
|
// 生成完成统计,是流的最后一个事件
|
||||||
break
|
await self.record(GenerateStats(
|
||||||
|
promptTokens: info.promptTokenCount,
|
||||||
|
genTokens: info.generationTokenCount,
|
||||||
|
prefillSeconds: info.promptTime,
|
||||||
|
decodeSeconds: info.generateTime
|
||||||
|
))
|
||||||
|
|
||||||
case .toolCall:
|
case .toolCall:
|
||||||
// 纯文本生成不会触发,switch 穷举
|
// 纯文本生成不会触发,switch 穷举
|
||||||
|
|||||||
@@ -9,6 +9,11 @@ import Foundation
|
|||||||
actor MNNBackend {
|
actor MNNBackend {
|
||||||
private var bridge: MNNLLMBridge?
|
private var bridge: MNNLLMBridge?
|
||||||
|
|
||||||
|
/// 末次生成统计(供 AIRuntime 在流结束后取走,性能自检用)。
|
||||||
|
private(set) var lastStats: GenerateStats?
|
||||||
|
|
||||||
|
private func record(_ s: GenerateStats) { lastStats = s }
|
||||||
|
|
||||||
var isLoaded: Bool { bridge?.isLoaded ?? false }
|
var isLoaded: Bool { bridge?.isLoaded ?? false }
|
||||||
|
|
||||||
/// 从 MNN 模型目录加载(目录含 MNN llm 的 config.json + llm.mnn + 权重 + tokenizer)。
|
/// 从 MNN 模型目录加载(目录含 MNN llm 的 config.json + llm.mnn + 权重 + tokenizer)。
|
||||||
@@ -35,10 +40,17 @@ actor MNNBackend {
|
|||||||
return AsyncThrowingStream { continuation in
|
return AsyncThrowingStream { continuation in
|
||||||
let meter = MNNRateMeter()
|
let meter = MNNRateMeter()
|
||||||
let task = Task.detached(priority: .userInitiated) {
|
let task = Task.detached(priority: .userInitiated) {
|
||||||
_ = box.value.generateText(prompt, maxTokens: Int32(maxTokens)) { piece in
|
let stats = box.value.generateText(prompt, maxTokens: Int32(maxTokens)) { piece in
|
||||||
let rate = meter.tick()
|
let rate = meter.tick()
|
||||||
continuation.yield(TokenChunk(text: piece, decodeRate: rate))
|
continuation.yield(TokenChunk(text: piece, decodeRate: rate))
|
||||||
}
|
}
|
||||||
|
// ObjC 统计对象先抽成 Sendable 的 GenerateStats 再跨 actor 记录。
|
||||||
|
await self.record(GenerateStats(
|
||||||
|
promptTokens: Int(stats.promptTokens),
|
||||||
|
genTokens: Int(stats.genTokens),
|
||||||
|
prefillSeconds: stats.prefillMs / 1000.0,
|
||||||
|
decodeSeconds: stats.decodeMs / 1000.0
|
||||||
|
))
|
||||||
continuation.finish()
|
continuation.finish()
|
||||||
}
|
}
|
||||||
continuation.onTermination = { _ in
|
continuation.onTermination = { _ in
|
||||||
@@ -58,9 +70,15 @@ actor MNNBackend {
|
|||||||
Task.detached(priority: .userInitiated) {
|
Task.detached(priority: .userInitiated) {
|
||||||
let sink = MNNTextSink()
|
let sink = MNNTextSink()
|
||||||
do {
|
do {
|
||||||
_ = try box.value.analyzeImages(paths, prompt: prompt, maxTokens: Int32(maxTokens)) { piece in
|
let stats = try box.value.analyzeImages(paths, prompt: prompt, maxTokens: Int32(maxTokens)) { piece in
|
||||||
sink.append(piece)
|
sink.append(piece)
|
||||||
}
|
}
|
||||||
|
await self.record(GenerateStats(
|
||||||
|
promptTokens: Int(stats.promptTokens),
|
||||||
|
genTokens: Int(stats.genTokens),
|
||||||
|
prefillSeconds: stats.prefillMs / 1000.0,
|
||||||
|
decodeSeconds: stats.decodeMs / 1000.0
|
||||||
|
))
|
||||||
cont.resume(returning: sink.text)
|
cont.resume(returning: sink.text)
|
||||||
} catch {
|
} catch {
|
||||||
cont.resume(throwing: AIRuntimeError.inferenceFailed(error.localizedDescription))
|
cont.resume(throwing: AIRuntimeError.inferenceFailed(error.localizedDescription))
|
||||||
|
|||||||
@@ -76,4 +76,44 @@ enum DiaryAssistPrompts {
|
|||||||
Output: /no_think
|
Output: /no_think
|
||||||
"""
|
"""
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// MARK: - 语音口述 → 日记整理
|
||||||
|
|
||||||
|
/// 口述转写稿截断上限(字符)。2B 模型 context 保护:超长口述只取前面部分。
|
||||||
|
static let organizeTranscriptLimit = 1200
|
||||||
|
|
||||||
|
/// 把语音转写稿整理成健康日记草稿。自适应样式:内容少 → 一段通顺的话;
|
||||||
|
/// 多方面 → 按「方面:内容」分行。
|
||||||
|
/// 红线(spec 2026-06-10-voice-diary §2):只重组语言,严禁增删改任何数值、单位、药名、时间——
|
||||||
|
/// 2B 模型把 140/90 改成 130/90 即健康数据事故,所以规则放第一条并配 few-shot 强化。
|
||||||
|
static func organize(transcript: String) -> String {
|
||||||
|
let trimmed = String(transcript.prefix(organizeTranscriptLimit))
|
||||||
|
return """
|
||||||
|
你是健康记录助手。下面是用户口述身体状态的语音转写原话,可能口语化、有重复、缺标点。
|
||||||
|
请把它整理成一条清晰的健康日记。
|
||||||
|
|
||||||
|
硬性规则:
|
||||||
|
- 【绝对不许】增加、删除或改动任何数值、单位、药名、时间——原话说 140/90 就必须写 140/90。
|
||||||
|
- 只重组语言:去掉口头语和重复;用第一人称;不加入原话没有的事实。
|
||||||
|
- 内容只涉及一两个方面 → 整理成一段通顺的话(2-4 句)。
|
||||||
|
- 内容涉及多个方面(症状/用药/饮食/睡眠/运动等) → 按「方面:内容」分行。
|
||||||
|
- 不诊断、不给用药建议、不写「建议就医」。
|
||||||
|
- 只输出整理后的日记正文,不要解释、不要 markdown 围栏、不要 <think> 标签。
|
||||||
|
|
||||||
|
示例 1(口述:那个今天早上起来有点头晕然后我量了下血压140 90比平时高一点没吃早饭就出门了):
|
||||||
|
今天早上起来有点头晕,量了血压 140/90,比平时高一点。没吃早饭就出门了。
|
||||||
|
|
||||||
|
示例 2(口述:今天头晕了一上午下午好点了血压早上量的140 90嗯缬沙坦吃了降脂药忘了吃早饭没吃中午吃的清淡晚上散步了半小时):
|
||||||
|
症状:头晕了一上午,下午好转。
|
||||||
|
血压:早上 140/90。
|
||||||
|
用药:缬沙坦已服,降脂药忘服。
|
||||||
|
饮食:早饭未吃,午餐清淡。
|
||||||
|
运动:晚上散步半小时。
|
||||||
|
|
||||||
|
【口述原话】:
|
||||||
|
\(trimmed)
|
||||||
|
|
||||||
|
Output: /no_think
|
||||||
|
"""
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
44
康康/AI/Prompts/InsightPrompts.swift
Normal file
44
康康/AI/Prompts/InsightPrompts.swift
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
import Foundation
|
||||||
|
|
||||||
|
/// 本地解读类 prompt:报告大白话摘要 + 趋势一句话解读。
|
||||||
|
/// 红线:不诊断、不荐药;称呼「你」,不出现「患者」(产品定位:自我健康记录)。
|
||||||
|
nonisolated enum InsightPrompts {
|
||||||
|
|
||||||
|
/// 报告整体大白话摘要(归档后台预生成,写回 Report.summary)。
|
||||||
|
static func reportPlainSummary(title: String, typeLabel: String, indicatorLines: String) -> String {
|
||||||
|
"""
|
||||||
|
你是健康档案助手。下面是一份报告的指标列表,请用大白话给本人(称「你」)写 2~3 句整体解读:
|
||||||
|
- 第 1 句:总体情况(共几项、几项异常)。
|
||||||
|
- 之后:点名最值得留意的异常项,用生活化语言说明偏高/偏低意味着什么方向。
|
||||||
|
- 不诊断疾病、不推荐药物或剂量;异常较多时建议「带上报告咨询医生」。
|
||||||
|
- 只输出正文文字,不要标题、列表、JSON、markdown。
|
||||||
|
|
||||||
|
示例:
|
||||||
|
输入:血常规(化验单),指标:白细胞 5.2 (3.5-9.5) normal;血红蛋白 118 (130-175) low;血小板 210 (125-350) normal
|
||||||
|
输出:这份血常规共 3 项,2 项正常,血红蛋白略低于参考范围。血红蛋白偏低通常与贫血方向有关,平时可以多补充含铁食物;如果还伴随乏力头晕,建议带上报告咨询医生。
|
||||||
|
|
||||||
|
现在的报告:\(title)(\(typeLabel))
|
||||||
|
指标:
|
||||||
|
\(indicatorLines)
|
||||||
|
只输出 2~3 句正文。/no_think
|
||||||
|
"""
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 趋势一句话解读(TrendDetailView,按数据指纹缓存)。
|
||||||
|
static func trendInsight(title: String, unit: String, rangeText: String, dataLines: String) -> String {
|
||||||
|
"""
|
||||||
|
你是健康档案助手。下面是「\(title)」的历史记录(单位 \(unit)\(rangeText)),请用大白话给本人(称「你」)写 1~2 句趋势解读:
|
||||||
|
- 说清整体走向(上升/下降/平稳/波动)和当前值与参考范围的关系。
|
||||||
|
- 不诊断疾病、不推荐药物;持续异常时温和建议「复查或咨询医生」。
|
||||||
|
- 只输出正文文字,不要标题、列表、JSON。
|
||||||
|
|
||||||
|
示例:
|
||||||
|
输入:体重,单位 kg,记录:2026-04-01 72.5 / 2026-04-15 71.8 / 2026-05-01 71.2
|
||||||
|
输出:近一个月你的体重稳步下降了约 1.3kg,节奏平缓,继续保持现在的习惯就好。
|
||||||
|
|
||||||
|
现在的记录:
|
||||||
|
\(dataLines)
|
||||||
|
只输出 1~2 句正文。/no_think
|
||||||
|
"""
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -31,12 +31,38 @@ nonisolated enum VLPrompts {
|
|||||||
|
|
||||||
/// VL 模型不知"今天"是哪天,且 few-shot 示例里写死了日期,
|
/// VL 模型不知"今天"是哪天,且 few-shot 示例里写死了日期,
|
||||||
/// 必须把当天日期显式注入 prompt,模型在无报告日期时才会用对正确的回退值。
|
/// 必须把当天日期显式注入 prompt,模型在无报告日期时才会用对正确的回退值。
|
||||||
static func reportExtraction(today: Date = .now) -> String {
|
/// ocrText 非空时把 Vision OCR 的结果作为参考文本注入 —— Vision 抄数字比
|
||||||
|
/// 2B 多模态读密集小字稳;版面与表格结构仍以图片为准。
|
||||||
|
static func reportExtraction(today: Date = .now, ocrText: String = "") -> String {
|
||||||
let f = DateFormatter()
|
let f = DateFormatter()
|
||||||
f.locale = Locale(identifier: "en_US_POSIX")
|
f.locale = Locale(identifier: "en_US_POSIX")
|
||||||
f.dateFormat = "yyyy-MM-dd"
|
f.dateFormat = "yyyy-MM-dd"
|
||||||
let todayStr = f.string(from: today)
|
let todayStr = f.string(from: today)
|
||||||
return reportExtractionTemplate.replacingOccurrences(of: "{{TODAY}}", with: todayStr)
|
let ocrSection: String
|
||||||
|
if ocrText.isEmpty {
|
||||||
|
ocrSection = ""
|
||||||
|
} else {
|
||||||
|
ocrSection = """
|
||||||
|
|
||||||
|
|
||||||
|
OCR 参考文本(系统对同一报告做文字识别的结果,可能有错字、串行或漏行;版面与表格结构以图片为准,但数值、小数点以 OCR 文字更可靠):
|
||||||
|
\(clipOCR(ocrText))
|
||||||
|
|
||||||
|
"""
|
||||||
|
}
|
||||||
|
return reportExtractionTemplate
|
||||||
|
.replacingOccurrences(of: "{{TODAY}}", with: todayStr)
|
||||||
|
.replacingOccurrences(of: "{{OCR_SECTION}}", with: ocrSection)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// OCR 文本截断:限制进入 prompt 的体量(2B 模型上下文有限)。截到最后一个完整行。
|
||||||
|
static func clipOCR(_ text: String, limit: Int = 1800) -> String {
|
||||||
|
guard text.count > limit else { return text }
|
||||||
|
let clipped = String(text.prefix(limit))
|
||||||
|
if let lastNewline = clipped.lastIndex(of: "\n") {
|
||||||
|
return String(clipped[..<lastNewline]) + "\n(后续内容过长已截断)"
|
||||||
|
}
|
||||||
|
return clipped + "\n(后续内容过长已截断)"
|
||||||
}
|
}
|
||||||
|
|
||||||
private static let reportExtractionTemplate: String = #"""
|
private static let reportExtractionTemplate: String = #"""
|
||||||
@@ -84,7 +110,7 @@ JSON schema(严格):
|
|||||||
输入: 一份春季体检,3 项可读
|
输入: 一份春季体检,3 项可读
|
||||||
输出:
|
输出:
|
||||||
{"title":"春季年度体检","type":"checkup","report_date":"2026-04-12","institution":"协和医院","page_count":1,"summary":"血脂偏高、其他正常","indicators":[{"name":"低密度脂蛋白","value":"3.84","unit":"mmol/L","range":"< 3.40","status":"high","source_page":1,"source_box":[0.12,0.31,0.76,0.07]},{"name":"谷丙转氨酶","value":"32","unit":"U/L","range":"9 - 50","status":"normal","source_page":1,"source_box":[0.12,0.39,0.76,0.07]},{"name":"空腹血糖","value":"5.2","unit":"mmol/L","range":"3.9 - 6.1","status":"normal","source_page":1,"source_box":[0.12,0.47,0.76,0.07]}]}
|
{"title":"春季年度体检","type":"checkup","report_date":"2026-04-12","institution":"协和医院","page_count":1,"summary":"血脂偏高、其他正常","indicators":[{"name":"低密度脂蛋白","value":"3.84","unit":"mmol/L","range":"< 3.40","status":"high","source_page":1,"source_box":[0.12,0.31,0.76,0.07]},{"name":"谷丙转氨酶","value":"32","unit":"U/L","range":"9 - 50","status":"normal","source_page":1,"source_box":[0.12,0.39,0.76,0.07]},{"name":"空腹血糖","value":"5.2","unit":"mmol/L","range":"3.9 - 6.1","status":"normal","source_page":1,"source_box":[0.12,0.47,0.76,0.07]}]}
|
||||||
|
{{OCR_SECTION}}
|
||||||
现在请识别图片并输出 JSON:
|
现在请识别图片并输出 JSON:
|
||||||
"""#
|
"""#
|
||||||
|
|
||||||
|
|||||||
@@ -20,6 +20,8 @@ struct HealthExportSheet: View {
|
|||||||
@State private var completed: Bool = false
|
@State private var completed: Bool = false
|
||||||
@State private var copiedFlash: Bool = false
|
@State private var copiedFlash: Bool = false
|
||||||
@State private var answeringTurnID: UUID?
|
@State private var answeringTurnID: UUID?
|
||||||
|
@State private var retrieval: HealthExportService.RetrievalSummary?
|
||||||
|
@State private var turnRetrievals: [UUID: HealthExportService.RetrievalSummary] = [:]
|
||||||
@FocusState private var questionFocused: Bool
|
@FocusState private var questionFocused: Bool
|
||||||
|
|
||||||
// 快捷问答
|
// 快捷问答
|
||||||
@@ -234,9 +236,14 @@ struct HealthExportSheet: View {
|
|||||||
Text(turn.role.transcriptLabel)
|
Text(turn.role.transcriptLabel)
|
||||||
.font(.tjScaled( 11, weight: .semibold))
|
.font(.tjScaled( 11, weight: .semibold))
|
||||||
.foregroundStyle(isUser ? Tj.Palette.paper.opacity(0.8) : Tj.Palette.text3)
|
.foregroundStyle(isUser ? Tj.Palette.paper.opacity(0.8) : Tj.Palette.text3)
|
||||||
|
if !isUser, let summary = turnRetrievals[turn.id] {
|
||||||
|
RetrievalChipsView(summary: summary)
|
||||||
|
}
|
||||||
if turn.id == answeringTurnID && turn.text.isEmpty {
|
if turn.id == answeringTurnID && turn.text.isEmpty {
|
||||||
VStack(alignment: .leading, spacing: 8) {
|
VStack(alignment: .leading, spacing: 8) {
|
||||||
Text("正在查看本地记录…")
|
Text(turnRetrievals[turn.id] == nil
|
||||||
|
? "正在查看本地记录…"
|
||||||
|
: "正在根据这些记录回答…")
|
||||||
.font(.tjScaled( 13))
|
.font(.tjScaled( 13))
|
||||||
.foregroundStyle(Tj.Palette.text3)
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
AIFlowBar()
|
AIFlowBar()
|
||||||
@@ -298,6 +305,9 @@ struct HealthExportSheet: View {
|
|||||||
arrow
|
arrow
|
||||||
phasePill(.generating)
|
phasePill(.generating)
|
||||||
}
|
}
|
||||||
|
if let retrieval {
|
||||||
|
RetrievalChipsView(summary: retrieval)
|
||||||
|
}
|
||||||
if phase == .generating && rate > 0 {
|
if phase == .generating && rate > 0 {
|
||||||
Text(String(format: String(appLoc: "本地推理 · %.1f tok/s"), rate))
|
Text(String(format: String(appLoc: "本地推理 · %.1f tok/s"), rate))
|
||||||
.font(.tjScaled( 11, design: .monospaced))
|
.font(.tjScaled( 11, design: .monospaced))
|
||||||
@@ -477,9 +487,18 @@ struct HealthExportSheet: View {
|
|||||||
task?.cancel()
|
task?.cancel()
|
||||||
task = Task { @MainActor in
|
task = Task { @MainActor in
|
||||||
do {
|
do {
|
||||||
for try await chunk in stream {
|
for try await event in stream {
|
||||||
appendToTurn(id: assistantTurn.id, text: chunk.text)
|
switch event {
|
||||||
if chunk.decodeRate > 0 { rate = chunk.decodeRate }
|
case .retrieved(let summary):
|
||||||
|
withAnimation(.snappy(duration: 0.25)) {
|
||||||
|
turnRetrievals[assistantTurn.id] = summary
|
||||||
|
}
|
||||||
|
case .token(let chunk):
|
||||||
|
appendToTurn(id: assistantTurn.id, text: chunk.text)
|
||||||
|
if chunk.decodeRate > 0 { rate = chunk.decodeRate }
|
||||||
|
case .phaseChanged, .completed:
|
||||||
|
break
|
||||||
|
}
|
||||||
}
|
}
|
||||||
answeringTurnID = nil
|
answeringTurnID = nil
|
||||||
questionFocused = true
|
questionFocused = true
|
||||||
@@ -512,6 +531,7 @@ struct HealthExportSheet: View {
|
|||||||
rate = 0 // 重新生成时清零,避免旧 tok/s 残留显示
|
rate = 0 // 重新生成时清零,避免旧 tok/s 残留显示
|
||||||
error = nil
|
error = nil
|
||||||
completed = false
|
completed = false
|
||||||
|
retrieval = nil
|
||||||
phase = .retrieving
|
phase = .retrieving
|
||||||
|
|
||||||
let stream = HealthExportService.shared.export(conversation: turns, in: ctx)
|
let stream = HealthExportService.shared.export(conversation: turns, in: ctx)
|
||||||
@@ -522,6 +542,8 @@ struct HealthExportSheet: View {
|
|||||||
switch event {
|
switch event {
|
||||||
case .phaseChanged(let ph):
|
case .phaseChanged(let ph):
|
||||||
phase = ph
|
phase = ph
|
||||||
|
case .retrieved(let summary):
|
||||||
|
withAnimation(.snappy(duration: 0.25)) { retrieval = summary }
|
||||||
case .token(let chunk):
|
case .token(let chunk):
|
||||||
content += chunk.text
|
content += chunk.text
|
||||||
if chunk.decodeRate > 0 { rate = chunk.decodeRate }
|
if chunk.decodeRate > 0 { rate = chunk.decodeRate }
|
||||||
@@ -549,6 +571,7 @@ struct HealthExportSheet: View {
|
|||||||
rate = 0
|
rate = 0
|
||||||
completed = false
|
completed = false
|
||||||
content = ""
|
content = ""
|
||||||
|
retrieval = nil
|
||||||
}
|
}
|
||||||
|
|
||||||
private func reset() {
|
private func reset() {
|
||||||
@@ -560,6 +583,8 @@ struct HealthExportSheet: View {
|
|||||||
error = nil
|
error = nil
|
||||||
completed = false
|
completed = false
|
||||||
answeringTurnID = nil
|
answeringTurnID = nil
|
||||||
|
retrieval = nil
|
||||||
|
turnRetrievals = [:]
|
||||||
questionFocused = true
|
questionFocused = true
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -577,6 +602,44 @@ struct HealthExportSheet: View {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// MARK: - 检索结果 chips(本地 RAG 可视化)
|
||||||
|
|
||||||
|
/// 生成开始前先把「本地 RAG 找到了什么」演出来:N 条记录 + 记录名 chips。
|
||||||
|
/// 结构化检索(不用 embedding)的天然优势 —— 每条命中都可解释、可展示(§12 卖点 3)。
|
||||||
|
private struct RetrievalChipsView: View {
|
||||||
|
let summary: HealthExportService.RetrievalSummary
|
||||||
|
|
||||||
|
var body: some View {
|
||||||
|
VStack(alignment: .leading, spacing: 6) {
|
||||||
|
if summary.totalCount == 0 {
|
||||||
|
Text("本地档案中暂无相关记录,将仅按你的描述整理")
|
||||||
|
.font(.tjScaled( 11))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
} else {
|
||||||
|
Text(String(appLoc: "已在本地档案中找到 \(summary.totalCount) 条相关记录"))
|
||||||
|
.font(.tjScaled( 11, weight: .medium))
|
||||||
|
.foregroundStyle(Tj.Palette.leaf)
|
||||||
|
ScrollView(.horizontal, showsIndicators: false) {
|
||||||
|
HStack(spacing: 6) {
|
||||||
|
ForEach(Array(summary.chips.enumerated()), id: \.offset) { _, chip in
|
||||||
|
Text(chip)
|
||||||
|
.font(.tjScaled( 11))
|
||||||
|
.foregroundStyle(Tj.Palette.text2)
|
||||||
|
.lineLimit(1)
|
||||||
|
.padding(.horizontal, 8)
|
||||||
|
.padding(.vertical, 4)
|
||||||
|
.background(Capsule().fill(Tj.Palette.sand2))
|
||||||
|
.overlay(Capsule().strokeBorder(Tj.Palette.lineSoft, lineWidth: 1))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
.padding(.vertical, 1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
.transition(.opacity.combined(with: .move(edge: .top)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// MARK: - 简易 Markdown 渲染(行级)
|
// MARK: - 简易 Markdown 渲染(行级)
|
||||||
|
|
||||||
/// 极简 Markdown 渲染器,够给医生看的报告就行。
|
/// 极简 Markdown 渲染器,够给医生看的报告就行。
|
||||||
|
|||||||
@@ -311,6 +311,9 @@ struct UnifiedCaptureFlow: View {
|
|||||||
}
|
}
|
||||||
|
|
||||||
try? ctx.save()
|
try? ctx.save()
|
||||||
|
// 后台预生成大白话摘要:用户继续操作,详情页打开时秒开。
|
||||||
|
// 低优先级 —— 任何前台 AI 任务(再次拍照/问答)都会让它在下一个 token 让位。
|
||||||
|
Task { await ReportInsightService.shared.pregenerateIfNeeded(report: report, in: ctx) }
|
||||||
onClose()
|
onClose()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -37,6 +37,27 @@ struct DiaryQuickSheet: View {
|
|||||||
@State private var detent: PresentationDetent = .large
|
@State private var detent: PresentationDetent = .large
|
||||||
@FocusState private var contentFocused: Bool
|
@FocusState private var contentFocused: Bool
|
||||||
|
|
||||||
|
// MARK: 语音输入状态(spec 2026-06-10-voice-diary)
|
||||||
|
|
||||||
|
enum VoicePhase: Equatable { case idle, recording, organizing }
|
||||||
|
@State private var voicePhase: VoicePhase = .idle
|
||||||
|
@State private var liveTranscript = ""
|
||||||
|
@State private var recordingSeconds = 0
|
||||||
|
/// 最近一次最终转写稿,「改用原话」回退用;再次录音时覆盖。
|
||||||
|
@State private var rawTranscript: String?
|
||||||
|
/// 刚追加进正文的整理稿,用于「改用原话」时在正文中定位替换。
|
||||||
|
/// 用户手动编辑掉该段(正文中找不到了)时 pill 自然消失。
|
||||||
|
@State private var organizedAppended: String?
|
||||||
|
/// 一次性提示条文案(整理失败已填原话 / 没听清等),开始新录音时清掉。
|
||||||
|
@State private var voiceNote: String?
|
||||||
|
@State private var voiceDeniedAlert = false
|
||||||
|
@State private var voiceFlowTask: Task<Void, Never>?
|
||||||
|
@State private var recordingWatchdog: Task<Void, Never>?
|
||||||
|
/// 必须 @State:struct View 重建(键盘收起/detent 变化都会触发)时普通 let 会换成
|
||||||
|
/// 全新实例,导致 stop() 落在没在录音的新服务上返回空串(「没听清」假错误),
|
||||||
|
/// 且真正在录音的老实例关不掉、麦克风悬挂。@State 保证视图身份期内实例唯一。
|
||||||
|
@State private var dictation = SpeechDictationService()
|
||||||
|
|
||||||
private var hasContent: Bool {
|
private var hasContent: Bool {
|
||||||
!content.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
|
!content.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
|
||||||
}
|
}
|
||||||
@@ -45,7 +66,7 @@ struct DiaryQuickSheet: View {
|
|||||||
if case .loading = phase { return true }
|
if case .loading = phase { return true }
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
private var canRequestSuggest: Bool { hasContent && !isLoading }
|
private var canRequestSuggest: Bool { hasContent && !isLoading && voicePhase == .idle }
|
||||||
private var canSubmit: Bool { hasContent }
|
private var canSubmit: Bool { hasContent }
|
||||||
|
|
||||||
var body: some View {
|
var body: some View {
|
||||||
@@ -77,7 +98,29 @@ struct DiaryQuickSheet: View {
|
|||||||
ScrollView(showsIndicators: false) {
|
ScrollView(showsIndicators: false) {
|
||||||
VStack(alignment: .leading, spacing: 16) {
|
VStack(alignment: .leading, spacing: 16) {
|
||||||
VStack(alignment: .leading, spacing: 8) {
|
VStack(alignment: .leading, spacing: 8) {
|
||||||
sectionLabel(String(appLoc: "内容"))
|
HStack {
|
||||||
|
sectionLabel(String(appLoc: "内容"))
|
||||||
|
Spacer()
|
||||||
|
if SpeechDictationService.isAvailable, voicePhase == .idle {
|
||||||
|
Button(action: startVoice) {
|
||||||
|
HStack(spacing: 4) {
|
||||||
|
Image(systemName: "mic.fill")
|
||||||
|
.font(.tjScaled(11, weight: .semibold))
|
||||||
|
Text("说一段")
|
||||||
|
.font(.tjScaled(12, weight: .semibold))
|
||||||
|
}
|
||||||
|
.foregroundStyle(isLoading ? Tj.Palette.text3 : Tj.Palette.brick)
|
||||||
|
.padding(.horizontal, 10)
|
||||||
|
.padding(.vertical, 5)
|
||||||
|
.background(Capsule().strokeBorder(
|
||||||
|
isLoading ? Tj.Palette.line : Tj.Palette.brick.opacity(0.5),
|
||||||
|
lineWidth: 1))
|
||||||
|
.contentShape(Capsule())
|
||||||
|
}
|
||||||
|
.buttonStyle(.plain)
|
||||||
|
.disabled(isLoading) // AI 追问生成中不抢 AIRuntime 队列
|
||||||
|
}
|
||||||
|
}
|
||||||
TextField("今天身体怎么样?吃了什么药、有什么感觉?",
|
TextField("今天身体怎么样?吃了什么药、有什么感觉?",
|
||||||
text: $content, axis: .vertical)
|
text: $content, axis: .vertical)
|
||||||
.lineLimit(3...8)
|
.lineLimit(3...8)
|
||||||
@@ -93,6 +136,48 @@ struct DiaryQuickSheet: View {
|
|||||||
RoundedRectangle(cornerRadius: Tj.Radius.sm, style: .continuous)
|
RoundedRectangle(cornerRadius: Tj.Radius.sm, style: .continuous)
|
||||||
.strokeBorder(Tj.Palette.line, lineWidth: 1)
|
.strokeBorder(Tj.Palette.line, lineWidth: 1)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
if voicePhase != .idle {
|
||||||
|
DiaryVoicePanel(
|
||||||
|
mode: voicePhase == .organizing
|
||||||
|
? .organizing
|
||||||
|
: .recording(elapsedSeconds: recordingSeconds),
|
||||||
|
transcript: liveTranscript,
|
||||||
|
onStop: stopVoiceAndOrganize,
|
||||||
|
onCancelOrganize: cancelOrganize
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if let note = voiceNote {
|
||||||
|
HStack(spacing: 6) {
|
||||||
|
Image(systemName: "info.circle")
|
||||||
|
.font(.tjScaled(11))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
Text(note)
|
||||||
|
.font(.tjScaled(11))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
Spacer(minLength: 0)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if let organized = organizedAppended,
|
||||||
|
rawTranscript != nil,
|
||||||
|
content.range(of: organized) != nil {
|
||||||
|
Button(action: revertToRawTranscript) {
|
||||||
|
HStack(spacing: 4) {
|
||||||
|
Image(systemName: "arrow.uturn.backward")
|
||||||
|
.font(.tjScaled(10, weight: .semibold))
|
||||||
|
Text("改用原话")
|
||||||
|
.font(.tjScaled(11, weight: .semibold))
|
||||||
|
}
|
||||||
|
.foregroundStyle(Tj.Palette.ink)
|
||||||
|
.padding(.horizontal, 10)
|
||||||
|
.padding(.vertical, 5)
|
||||||
|
.background(Capsule().strokeBorder(Tj.Palette.line, lineWidth: 1))
|
||||||
|
.contentShape(Capsule())
|
||||||
|
}
|
||||||
|
.buttonStyle(.plain)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
assistSection
|
assistSection
|
||||||
@@ -143,7 +228,22 @@ struct DiaryQuickSheet: View {
|
|||||||
.presentationDragIndicator(.hidden)
|
.presentationDragIndicator(.hidden)
|
||||||
.presentationBackground(Tj.Palette.sand)
|
.presentationBackground(Tj.Palette.sand)
|
||||||
.presentationCornerRadius(Tj.Radius.xl)
|
.presentationCornerRadius(Tj.Radius.xl)
|
||||||
.onDisappear { suggestTask?.cancel() }
|
.onDisappear {
|
||||||
|
suggestTask?.cancel()
|
||||||
|
voiceFlowTask?.cancel()
|
||||||
|
recordingWatchdog?.cancel()
|
||||||
|
dictation.abort()
|
||||||
|
}
|
||||||
|
.alert(String(appLoc: "需要麦克风与语音识别权限"), isPresented: $voiceDeniedAlert) {
|
||||||
|
Button(String(appLoc: "前往设置")) {
|
||||||
|
if let url = URL(string: UIApplication.openSettingsURLString) {
|
||||||
|
UIApplication.shared.open(url)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Button(String(appLoc: "取消"), role: .cancel) {}
|
||||||
|
} message: {
|
||||||
|
Text("语音记录全程在本机完成,声音和文字都不会上传。请在设置中允许麦克风和语音识别。")
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// MARK: - AI 辅助区
|
// MARK: - AI 辅助区
|
||||||
@@ -455,6 +555,100 @@ struct DiaryQuickSheet: View {
|
|||||||
.foregroundStyle(Tj.Palette.text2)
|
.foregroundStyle(Tj.Palette.text2)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// MARK: 语音输入流程
|
||||||
|
|
||||||
|
private func startVoice() {
|
||||||
|
contentFocused = false
|
||||||
|
voiceNote = nil
|
||||||
|
voiceFlowTask = Task { @MainActor in
|
||||||
|
guard await dictation.requestAuthorization() else {
|
||||||
|
voiceDeniedAlert = true
|
||||||
|
return
|
||||||
|
}
|
||||||
|
do {
|
||||||
|
liveTranscript = ""
|
||||||
|
recordingSeconds = 0
|
||||||
|
try dictation.start { partial in liveTranscript = partial }
|
||||||
|
withAnimation(.snappy(duration: 0.2)) { voicePhase = .recording }
|
||||||
|
// 计时 + 3 分钟看门狗(到点自动停,行为与点「停止」一致)
|
||||||
|
recordingWatchdog = Task { @MainActor in
|
||||||
|
while !Task.isCancelled {
|
||||||
|
try? await Task.sleep(nanoseconds: 1_000_000_000)
|
||||||
|
guard !Task.isCancelled, voicePhase == .recording else { return }
|
||||||
|
recordingSeconds += 1
|
||||||
|
if recordingSeconds >= DiaryVoicePanel.maxRecordingSeconds {
|
||||||
|
stopVoiceAndOrganize()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
voiceNote = error.localizedDescription
|
||||||
|
voicePhase = .idle
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private func stopVoiceAndOrganize() {
|
||||||
|
guard voicePhase == .recording else { return }
|
||||||
|
recordingWatchdog?.cancel()
|
||||||
|
voiceFlowTask = Task { @MainActor in
|
||||||
|
// 防御兜底:服务返回空(极端情况下实例丢失/最终结果丢失)时,
|
||||||
|
// 用 @State 里的实时字幕——那就是用户亲眼看到的已识别文字。
|
||||||
|
var transcript = (await dictation.stop())
|
||||||
|
.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||||
|
if transcript.isEmpty {
|
||||||
|
transcript = liveTranscript.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||||
|
}
|
||||||
|
liveTranscript = transcript
|
||||||
|
guard !transcript.isEmpty else {
|
||||||
|
withAnimation(.snappy(duration: 0.2)) { voicePhase = .idle }
|
||||||
|
voiceNote = String(appLoc: "没听清,再试一次")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
rawTranscript = transcript
|
||||||
|
withAnimation(.snappy(duration: 0.2)) { voicePhase = .organizing }
|
||||||
|
do {
|
||||||
|
let result = try await DiaryAssistService.shared.organize(transcript: transcript)
|
||||||
|
guard !Task.isCancelled else { return }
|
||||||
|
appendToContent(result.text)
|
||||||
|
organizedAppended = result.text
|
||||||
|
lastRate = result.decodeRate
|
||||||
|
} catch is CancellationError {
|
||||||
|
// cancelOrganize 已处理回退,这里只收尾
|
||||||
|
} catch {
|
||||||
|
guard !Task.isCancelled else { return }
|
||||||
|
appendToContent(transcript) // 红线 #5:整理失败回退原话,不卡死
|
||||||
|
organizedAppended = nil
|
||||||
|
voiceNote = String(appLoc: "AI 整理失败,已填入原话")
|
||||||
|
}
|
||||||
|
withAnimation(.snappy(duration: 0.2)) { voicePhase = .idle }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 取消整理:中断 LLM,直接填原话(与失败回退同路径)。
|
||||||
|
private func cancelOrganize() {
|
||||||
|
guard voicePhase == .organizing else { return }
|
||||||
|
voiceFlowTask?.cancel()
|
||||||
|
if let raw = rawTranscript {
|
||||||
|
appendToContent(raw)
|
||||||
|
organizedAppended = nil
|
||||||
|
voiceNote = String(appLoc: "已取消整理,填入原话")
|
||||||
|
}
|
||||||
|
withAnimation(.snappy(duration: 0.2)) { voicePhase = .idle }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 「改用原话」:把刚追加的整理稿替换为原始转写稿(spec §2:LLM 改数兜底)。
|
||||||
|
private func revertToRawTranscript() {
|
||||||
|
guard let raw = rawTranscript,
|
||||||
|
let organized = organizedAppended,
|
||||||
|
let range = content.range(of: organized, options: .backwards) else { return }
|
||||||
|
withAnimation(.snappy(duration: 0.18)) {
|
||||||
|
content = content.replacingCharacters(in: range, with: raw)
|
||||||
|
organizedAppended = nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// 触发一轮 AI 辅助。把已覆盖的问诊维度(coveredDims)传给 LLM,
|
/// 触发一轮 AI 辅助。把已覆盖的问诊维度(coveredDims)传给 LLM,
|
||||||
/// 要求本轮避开这些维度,从结构上压住跨轮换皮重复。
|
/// 要求本轮避开这些维度,从结构上压住跨轮换皮重复。
|
||||||
private func requestSuggestions() {
|
private func requestSuggestions() {
|
||||||
|
|||||||
141
康康/Features/Diary/DiaryVoicePanel.swift
Normal file
141
康康/Features/Diary/DiaryVoicePanel.swift
Normal file
@@ -0,0 +1,141 @@
|
|||||||
|
import SwiftUI
|
||||||
|
|
||||||
|
/// 「健康记录」语音输入面板(spec 2026-06-10-voice-diary)。
|
||||||
|
/// 两种模式:recording(实时字幕 + 计时 + 停止)/ organizing(AI 整理中,可取消)。
|
||||||
|
/// 纯展示:状态由 DiaryQuickSheet 持有并传入。
|
||||||
|
struct DiaryVoicePanel: View {
|
||||||
|
enum Mode: Equatable {
|
||||||
|
case recording(elapsedSeconds: Int)
|
||||||
|
case organizing
|
||||||
|
}
|
||||||
|
|
||||||
|
let mode: Mode
|
||||||
|
/// recording 时为实时字幕;organizing 时为已定稿的转写稿(置灰展示)。
|
||||||
|
let transcript: String
|
||||||
|
let onStop: () -> Void
|
||||||
|
let onCancelOrganize: () -> Void
|
||||||
|
|
||||||
|
/// 录音上限 3 分钟(超时由 DiaryQuickSheet 的看门狗触发 onStop)。
|
||||||
|
static let maxRecordingSeconds = 180
|
||||||
|
|
||||||
|
var body: some View {
|
||||||
|
VStack(alignment: .leading, spacing: 10) {
|
||||||
|
header
|
||||||
|
transcriptArea
|
||||||
|
if case .recording = mode {
|
||||||
|
stopButton
|
||||||
|
}
|
||||||
|
}
|
||||||
|
.padding(12)
|
||||||
|
.frame(maxWidth: .infinity, alignment: .leading)
|
||||||
|
.background(
|
||||||
|
RoundedRectangle(cornerRadius: Tj.Radius.sm, style: .continuous)
|
||||||
|
.fill(Tj.Palette.paper)
|
||||||
|
)
|
||||||
|
.overlay(
|
||||||
|
RoundedRectangle(cornerRadius: Tj.Radius.sm, style: .continuous)
|
||||||
|
.strokeBorder(Tj.Palette.lineSoft, lineWidth: 1)
|
||||||
|
)
|
||||||
|
.overlay(alignment: .bottom) {
|
||||||
|
if mode == .organizing {
|
||||||
|
AIFlowBar().padding(.horizontal, 1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
.clipShape(RoundedRectangle(cornerRadius: Tj.Radius.sm, style: .continuous))
|
||||||
|
}
|
||||||
|
|
||||||
|
@ViewBuilder
|
||||||
|
private var header: some View {
|
||||||
|
switch mode {
|
||||||
|
case .recording(let elapsed):
|
||||||
|
HStack(spacing: 8) {
|
||||||
|
Image(systemName: "waveform")
|
||||||
|
.font(.tjScaled(12, weight: .semibold))
|
||||||
|
.foregroundStyle(Tj.Palette.brick)
|
||||||
|
.symbolEffect(.variableColor.iterative, options: .repeating)
|
||||||
|
Text("正在听 · 识别在本机完成")
|
||||||
|
.font(.tjScaled(13, weight: .medium))
|
||||||
|
.foregroundStyle(Tj.Palette.text2)
|
||||||
|
Spacer(minLength: 0)
|
||||||
|
Text(Self.format(elapsed))
|
||||||
|
.font(.tjScaled(12, design: .monospaced))
|
||||||
|
.foregroundStyle(elapsed >= Self.maxRecordingSeconds - 30
|
||||||
|
? Tj.Palette.brick : Tj.Palette.text3)
|
||||||
|
}
|
||||||
|
case .organizing:
|
||||||
|
HStack(spacing: 8) {
|
||||||
|
Image(systemName: "sparkles")
|
||||||
|
.font(.tjScaled(12, weight: .semibold))
|
||||||
|
.foregroundStyle(Tj.Palette.brick)
|
||||||
|
.symbolEffect(.pulse, options: .repeating)
|
||||||
|
Text("AI 整理中 · 本地推理")
|
||||||
|
.font(.tjScaled(13, weight: .medium))
|
||||||
|
.foregroundStyle(Tj.Palette.text2)
|
||||||
|
Spacer(minLength: 0)
|
||||||
|
Button("取消") { onCancelOrganize() }
|
||||||
|
.font(.tjScaled(12, weight: .semibold))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@ViewBuilder
|
||||||
|
private var transcriptArea: some View {
|
||||||
|
ScrollViewReader { proxy in
|
||||||
|
ScrollView(showsIndicators: false) {
|
||||||
|
Text(transcript.isEmpty ? String(appLoc: "开始说话…") : transcript)
|
||||||
|
.font(.tjScaled(14))
|
||||||
|
.foregroundStyle(transcriptColor)
|
||||||
|
.frame(maxWidth: .infinity, alignment: .leading)
|
||||||
|
.fixedSize(horizontal: false, vertical: true)
|
||||||
|
Color.clear.frame(height: 1).id("tail")
|
||||||
|
}
|
||||||
|
.frame(maxHeight: 120)
|
||||||
|
.onChange(of: transcript) { _, _ in
|
||||||
|
proxy.scrollTo("tail", anchor: .bottom)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private var transcriptColor: Color {
|
||||||
|
if transcript.isEmpty { return Tj.Palette.text3 }
|
||||||
|
return mode == .organizing ? Tj.Palette.text3 : Tj.Palette.text
|
||||||
|
}
|
||||||
|
|
||||||
|
private var stopButton: some View {
|
||||||
|
Button(action: onStop) {
|
||||||
|
HStack(spacing: 8) {
|
||||||
|
Image(systemName: "stop.circle.fill")
|
||||||
|
Text("说完了,整理成日记")
|
||||||
|
}
|
||||||
|
.font(.tjScaled(14, weight: .semibold))
|
||||||
|
.foregroundStyle(Tj.Palette.paper)
|
||||||
|
.frame(maxWidth: .infinity)
|
||||||
|
.padding(.vertical, 12)
|
||||||
|
.background(
|
||||||
|
RoundedRectangle(cornerRadius: Tj.Radius.sm, style: .continuous)
|
||||||
|
.fill(Tj.Palette.brick)
|
||||||
|
)
|
||||||
|
.contentShape(Rectangle())
|
||||||
|
}
|
||||||
|
.buttonStyle(.plain)
|
||||||
|
}
|
||||||
|
|
||||||
|
private static func format(_ seconds: Int) -> String {
|
||||||
|
String(format: "%d:%02d", seconds / 60, seconds % 60)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#Preview("录音中") {
|
||||||
|
DiaryVoicePanel(mode: .recording(elapsedSeconds: 23),
|
||||||
|
transcript: "今天早上起来有点头晕,量了血压一百四九十",
|
||||||
|
onStop: {}, onCancelOrganize: {})
|
||||||
|
.padding()
|
||||||
|
}
|
||||||
|
|
||||||
|
#Preview("整理中") {
|
||||||
|
DiaryVoicePanel(mode: .organizing,
|
||||||
|
transcript: "今天早上起来有点头晕,量了血压一百四九十",
|
||||||
|
onStop: {}, onCancelOrganize: {})
|
||||||
|
.padding()
|
||||||
|
}
|
||||||
@@ -28,13 +28,13 @@ struct ModelManagementView: View {
|
|||||||
actionButtons
|
actionButtons
|
||||||
.padding(.top, 4)
|
.padding(.top, 4)
|
||||||
|
|
||||||
if service.states[.mnnLLM]?.phase == .ready {
|
if service.states[.mnnLLM]?.phase == .ready || service.states[.llm]?.phase == .ready {
|
||||||
NavigationLink {
|
NavigationLink {
|
||||||
ModelSelfTestView()
|
ModelSelfTestView()
|
||||||
} label: {
|
} label: {
|
||||||
HStack(spacing: 6) {
|
HStack(spacing: 6) {
|
||||||
Image(systemName: "play.circle")
|
Image(systemName: "gauge.with.needle")
|
||||||
Text("运行推理自检")
|
Text("性能自检")
|
||||||
}
|
}
|
||||||
.frame(maxWidth: .infinity)
|
.frame(maxWidth: .infinity)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,11 +1,13 @@
|
|||||||
import SwiftUI
|
import SwiftUI
|
||||||
|
|
||||||
/// 模型推理自检:加载 LLM 跑一段固定 prompt,流式显示输出 + tok/s。
|
/// 性能自检:跑固定 prompt,展示当前后端(MNN·SME2 / MNN·NEON / MLX·GPU)的
|
||||||
/// 模型就绪后从「我的 · 模型管理」进入,用于现场快速验证本地推理是否正常。
|
/// prefill / decode 实测速度,并按后端存档对比 —— 挑战赛考核点的可见证据(§12 卖点 2/6)。
|
||||||
struct ModelSelfTestView: View {
|
struct ModelSelfTestView: View {
|
||||||
@State private var output = ""
|
@State private var output = ""
|
||||||
@State private var phase: Phase = .idle
|
@State private var phase: Phase = .idle
|
||||||
@State private var rate: Double = 0
|
@State private var rate: Double = 0
|
||||||
|
@State private var lastResult: BenchmarkResult?
|
||||||
|
@State private var history: [String: BenchmarkResult] = [:]
|
||||||
|
|
||||||
private enum Phase: Equatable {
|
private enum Phase: Equatable {
|
||||||
case idle, loading, running, done, failed(String)
|
case idle, loading, running, done, failed(String)
|
||||||
@@ -21,8 +23,6 @@ struct ModelSelfTestView: View {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private let prompt = "用中文一句话介绍肝功能里 ALT 这个指标。"
|
|
||||||
|
|
||||||
private var isBusy: Bool { phase == .loading || phase == .running }
|
private var isBusy: Bool { phase == .loading || phase == .running }
|
||||||
|
|
||||||
private var statusColor: Color {
|
private var statusColor: Color {
|
||||||
@@ -34,79 +34,162 @@ struct ModelSelfTestView: View {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var body: some View {
|
var body: some View {
|
||||||
VStack(alignment: .leading, spacing: 16) {
|
ScrollView {
|
||||||
VStack(alignment: .leading, spacing: 6) {
|
VStack(alignment: .leading, spacing: 16) {
|
||||||
Text("测试 PROMPT")
|
promptCard
|
||||||
.font(.tjScaled( 11, weight: .semibold))
|
|
||||||
.tracking(0.5)
|
|
||||||
.foregroundStyle(Tj.Palette.text3)
|
|
||||||
Text(prompt)
|
|
||||||
.font(.tjScaled( 14))
|
|
||||||
.foregroundStyle(Tj.Palette.text)
|
|
||||||
}
|
|
||||||
.padding(14)
|
|
||||||
.frame(maxWidth: .infinity, alignment: .leading)
|
|
||||||
.tjCard()
|
|
||||||
|
|
||||||
|
HStack {
|
||||||
|
Text(phase.label)
|
||||||
|
.font(.tjScaled( 13, weight: .medium))
|
||||||
|
.foregroundStyle(statusColor)
|
||||||
|
.lineLimit(1)
|
||||||
|
Spacer()
|
||||||
|
if rate > 0 {
|
||||||
|
Text(String(format: "%.1f tok/s", rate))
|
||||||
|
.font(.tjScaled( 12, design: .monospaced))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Button {
|
||||||
|
Task { await run() }
|
||||||
|
} label: {
|
||||||
|
Text(isBusy ? "运行中…" : "运行性能自检").frame(maxWidth: .infinity)
|
||||||
|
}
|
||||||
|
.buttonStyle(TjPrimaryButton())
|
||||||
|
.disabled(isBusy)
|
||||||
|
|
||||||
|
if isBusy { AIFlowBar() }
|
||||||
|
|
||||||
|
if let r = lastResult { statsCard(r) }
|
||||||
|
|
||||||
|
outputCard
|
||||||
|
|
||||||
|
if !history.isEmpty { historyCard }
|
||||||
|
}
|
||||||
|
.padding(16)
|
||||||
|
}
|
||||||
|
.background(Tj.Palette.sand.ignoresSafeArea())
|
||||||
|
.navigationTitle("性能自检")
|
||||||
|
.navigationBarTitleDisplayMode(.inline)
|
||||||
|
.onAppear { history = BenchmarkService.load() }
|
||||||
|
}
|
||||||
|
|
||||||
|
private var promptCard: some View {
|
||||||
|
VStack(alignment: .leading, spacing: 6) {
|
||||||
|
Text("测试 PROMPT")
|
||||||
|
.font(.tjScaled( 11, weight: .semibold))
|
||||||
|
.tracking(0.5)
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
Text(BenchmarkService.fixedPrompt)
|
||||||
|
.font(.tjScaled( 14))
|
||||||
|
.foregroundStyle(Tj.Palette.text)
|
||||||
|
}
|
||||||
|
.padding(14)
|
||||||
|
.frame(maxWidth: .infinity, alignment: .leading)
|
||||||
|
.tjCard()
|
||||||
|
}
|
||||||
|
|
||||||
|
private func statsCard(_ r: BenchmarkResult) -> some View {
|
||||||
|
VStack(alignment: .leading, spacing: 10) {
|
||||||
HStack {
|
HStack {
|
||||||
Text(phase.label)
|
Text("本次结果")
|
||||||
.font(.tjScaled( 13, weight: .medium))
|
.font(.tjScaled( 12, weight: .semibold))
|
||||||
.foregroundStyle(statusColor)
|
.foregroundStyle(Tj.Palette.text2)
|
||||||
.lineLimit(1)
|
|
||||||
Spacer()
|
Spacer()
|
||||||
if rate > 0 {
|
TjBadge(text: r.backendLabel, style: .leaf)
|
||||||
Text(String(format: "%.1f tok/s", rate))
|
}
|
||||||
.font(.tjScaled( 12, design: .monospaced))
|
HStack(spacing: 0) {
|
||||||
.foregroundStyle(Tj.Palette.text3)
|
metric(String(appLoc: "读入"), r.prefillTokensPerSecond > 0
|
||||||
|
? String(format: "%.0f tok/s", r.prefillTokensPerSecond) : "—")
|
||||||
|
metric(String(appLoc: "生成"), String(format: "%.1f tok/s", r.decodeTokensPerSecond))
|
||||||
|
metric(String(appLoc: "总耗时"), String(format: "%.1fs", r.totalSeconds))
|
||||||
|
}
|
||||||
|
Text(String(appLoc: "prompt \(r.promptTokens) tok · 生成 \(r.genTokens) tok · 100% 本地"))
|
||||||
|
.font(.tjScaled( 10, design: .monospaced))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
}
|
||||||
|
.padding(14)
|
||||||
|
.frame(maxWidth: .infinity, alignment: .leading)
|
||||||
|
.tjCard()
|
||||||
|
}
|
||||||
|
|
||||||
|
private func metric(_ label: String, _ value: String) -> some View {
|
||||||
|
VStack(spacing: 3) {
|
||||||
|
Text(value)
|
||||||
|
.font(.tjScaled( 15, weight: .semibold, design: .monospaced))
|
||||||
|
.foregroundStyle(Tj.Palette.text)
|
||||||
|
Text(label)
|
||||||
|
.font(.tjScaled( 10))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
}
|
||||||
|
.frame(maxWidth: .infinity)
|
||||||
|
}
|
||||||
|
|
||||||
|
private var outputCard: some View {
|
||||||
|
ScrollView {
|
||||||
|
Text(output.isEmpty ? "(暂无输出)" : output)
|
||||||
|
.font(.system(.footnote, design: .monospaced))
|
||||||
|
.foregroundStyle(Tj.Palette.text)
|
||||||
|
.frame(maxWidth: .infinity, alignment: .leading)
|
||||||
|
.textSelection(.enabled)
|
||||||
|
.padding(12)
|
||||||
|
}
|
||||||
|
.frame(maxHeight: 220)
|
||||||
|
.background(
|
||||||
|
RoundedRectangle(cornerRadius: Tj.Radius.md, style: .continuous)
|
||||||
|
.fill(Tj.Palette.paper)
|
||||||
|
)
|
||||||
|
.overlay(
|
||||||
|
RoundedRectangle(cornerRadius: Tj.Radius.md, style: .continuous)
|
||||||
|
.strokeBorder(Tj.Palette.lineSoft, lineWidth: 1)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
private var historyCard: some View {
|
||||||
|
VStack(alignment: .leading, spacing: 10) {
|
||||||
|
Text("各引擎实测对比")
|
||||||
|
.font(.tjScaled( 12, weight: .semibold))
|
||||||
|
.foregroundStyle(Tj.Palette.text2)
|
||||||
|
ForEach(history.keys.sorted(), id: \.self) { key in
|
||||||
|
if let r = history[key] {
|
||||||
|
HStack {
|
||||||
|
Text(key)
|
||||||
|
.font(.tjScaled( 12, weight: .medium))
|
||||||
|
.foregroundStyle(Tj.Palette.text)
|
||||||
|
Spacer()
|
||||||
|
Text(String(format: String(appLoc: "生成 %.1f tok/s"), r.decodeTokensPerSecond))
|
||||||
|
.font(.tjScaled( 12, design: .monospaced))
|
||||||
|
.foregroundStyle(Tj.Palette.leaf)
|
||||||
|
Text(r.date.formatted(.dateTime.month().day()))
|
||||||
|
.font(.tjScaled( 10))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
Text("在「我的 · 推理引擎」切换引擎后再跑一次,即可对比 SME2 与 GPU。")
|
||||||
Button {
|
.font(.tjScaled( 10))
|
||||||
Task { await run() }
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
} label: {
|
|
||||||
Text(isBusy ? "运行中…" : "运行推理自检").frame(maxWidth: .infinity)
|
|
||||||
}
|
|
||||||
.buttonStyle(TjPrimaryButton())
|
|
||||||
.disabled(isBusy)
|
|
||||||
|
|
||||||
ScrollView {
|
|
||||||
Text(output.isEmpty ? "(暂无输出)" : output)
|
|
||||||
.font(.system(.footnote, design: .monospaced))
|
|
||||||
.foregroundStyle(Tj.Palette.text)
|
|
||||||
.frame(maxWidth: .infinity, alignment: .leading)
|
|
||||||
.textSelection(.enabled)
|
|
||||||
.padding(12)
|
|
||||||
}
|
|
||||||
.frame(maxHeight: 280)
|
|
||||||
.background(
|
|
||||||
RoundedRectangle(cornerRadius: Tj.Radius.md, style: .continuous)
|
|
||||||
.fill(Tj.Palette.paper)
|
|
||||||
)
|
|
||||||
.overlay(
|
|
||||||
RoundedRectangle(cornerRadius: Tj.Radius.md, style: .continuous)
|
|
||||||
.strokeBorder(Tj.Palette.lineSoft, lineWidth: 1)
|
|
||||||
)
|
|
||||||
|
|
||||||
Spacer()
|
|
||||||
}
|
}
|
||||||
.padding(16)
|
.padding(14)
|
||||||
.background(Tj.Palette.sand.ignoresSafeArea())
|
.frame(maxWidth: .infinity, alignment: .leading)
|
||||||
.navigationTitle("推理自检")
|
.tjCard()
|
||||||
.navigationBarTitleDisplayMode(.inline)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@MainActor
|
@MainActor
|
||||||
private func run() async {
|
private func run() async {
|
||||||
output = ""
|
output = ""
|
||||||
rate = 0
|
rate = 0
|
||||||
|
lastResult = nil
|
||||||
phase = .loading
|
phase = .loading
|
||||||
do {
|
do {
|
||||||
try await AIRuntime.shared.prepare()
|
let result = try await BenchmarkService.shared.run { piece, r in
|
||||||
phase = .running
|
output += piece
|
||||||
for try await chunk in await AIRuntime.shared.generate(prompt: prompt, maxTokens: 200) {
|
if r > 0 { rate = r }
|
||||||
output += chunk.text
|
if phase == .loading { phase = .running }
|
||||||
rate = chunk.decodeRate
|
|
||||||
}
|
}
|
||||||
|
lastResult = result
|
||||||
|
history = BenchmarkService.load()
|
||||||
phase = .done
|
phase = .done
|
||||||
} catch {
|
} catch {
|
||||||
phase = .failed(error.localizedDescription)
|
phase = .failed(error.localizedDescription)
|
||||||
|
|||||||
@@ -257,14 +257,7 @@ struct TimelineEntryDetailView: View {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if let sum = r.summary, !sum.isEmpty {
|
ReportSummaryCard(report: r)
|
||||||
card {
|
|
||||||
Text(String(appLoc: "摘要"))
|
|
||||||
.font(.tjScaled( 12, weight: .semibold)).foregroundStyle(Tj.Palette.text2)
|
|
||||||
Text(sum).font(.tjScaled( 14)).foregroundStyle(Tj.Palette.text)
|
|
||||||
.fixedSize(horizontal: false, vertical: true)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if !r.indicators.isEmpty {
|
if !r.indicators.isEmpty {
|
||||||
card {
|
card {
|
||||||
@@ -559,3 +552,52 @@ private struct EvidenceHighlightOverlay: View {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// MARK: - 报告摘要卡(无摘要时后台预生成兜底)
|
||||||
|
|
||||||
|
/// 有摘要直接显示;无摘要且有指标时触发后台预生成(归档时若被抢占,这里兜底),
|
||||||
|
/// 生成期间显示流光线,完成后 SwiftData 观察自动刷新出文本。
|
||||||
|
private struct ReportSummaryCard: View {
|
||||||
|
@Environment(\.modelContext) private var ctx
|
||||||
|
let report: Report
|
||||||
|
@State private var generating = false
|
||||||
|
|
||||||
|
var body: some View {
|
||||||
|
Group {
|
||||||
|
if let sum = report.summary, !sum.isEmpty {
|
||||||
|
container {
|
||||||
|
Text(String(appLoc: "摘要"))
|
||||||
|
.font(.tjScaled( 12, weight: .semibold)).foregroundStyle(Tj.Palette.text2)
|
||||||
|
Text(sum).font(.tjScaled( 14)).foregroundStyle(Tj.Palette.text)
|
||||||
|
.fixedSize(horizontal: false, vertical: true)
|
||||||
|
}
|
||||||
|
} else if generating {
|
||||||
|
container {
|
||||||
|
Text("本地 AI 正在解读这份报告…")
|
||||||
|
.font(.tjScaled( 12)).foregroundStyle(Tj.Palette.text3)
|
||||||
|
AIFlowBar()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
.task {
|
||||||
|
guard (report.summary ?? "").isEmpty, !report.indicators.isEmpty else { return }
|
||||||
|
generating = true
|
||||||
|
await ReportInsightService.shared.pregenerateIfNeeded(report: report, in: ctx)
|
||||||
|
generating = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private func container<C: View>(@ViewBuilder _ body: () -> C) -> some View {
|
||||||
|
VStack(alignment: .leading, spacing: 10) { body() }
|
||||||
|
.padding(14)
|
||||||
|
.frame(maxWidth: .infinity, alignment: .leading)
|
||||||
|
.background(
|
||||||
|
RoundedRectangle(cornerRadius: Tj.Radius.md, style: .continuous)
|
||||||
|
.fill(Tj.Palette.paper)
|
||||||
|
)
|
||||||
|
.overlay(
|
||||||
|
RoundedRectangle(cornerRadius: Tj.Radius.md, style: .continuous)
|
||||||
|
.strokeBorder(Tj.Palette.lineSoft, lineWidth: 1)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -69,7 +69,7 @@ struct TrendDetailView: View {
|
|||||||
}
|
}
|
||||||
chartCard
|
chartCard
|
||||||
statsCard
|
statsCard
|
||||||
aiPlaceholder
|
TrendInsightCard(bucket: bucket)
|
||||||
pointsList
|
pointsList
|
||||||
}
|
}
|
||||||
.padding(.horizontal, 20)
|
.padding(.horizontal, 20)
|
||||||
@@ -318,27 +318,6 @@ struct TrendDetailView: View {
|
|||||||
return ("\(arrow) \(fmt(abs(d)))\(pctStr)", color)
|
return ("\(arrow) \(fmt(abs(d)))\(pctStr)", color)
|
||||||
}
|
}
|
||||||
|
|
||||||
// MARK: AI 解读占位
|
|
||||||
|
|
||||||
private var aiPlaceholder: some View {
|
|
||||||
HStack(spacing: 8) {
|
|
||||||
Image(systemName: "sparkles")
|
|
||||||
.font(.tjScaled( 12))
|
|
||||||
.foregroundStyle(Tj.Palette.text3)
|
|
||||||
Text("AI 趋势解读即将上线")
|
|
||||||
.font(.tjScaled( 12))
|
|
||||||
.foregroundStyle(Tj.Palette.text3)
|
|
||||||
Spacer()
|
|
||||||
}
|
|
||||||
.padding(.horizontal, 14)
|
|
||||||
.padding(.vertical, 12)
|
|
||||||
.frame(maxWidth: .infinity, alignment: .leading)
|
|
||||||
.background(
|
|
||||||
RoundedRectangle(cornerRadius: Tj.Radius.sm, style: .continuous)
|
|
||||||
.fill(Tj.Palette.sand2.opacity(0.6))
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
// MARK: 数据点列表
|
// MARK: 数据点列表
|
||||||
|
|
||||||
/// 跨线按天合并:每天一行,展示该天各线的值。倒序。
|
/// 跨线按天合并:每天一行,展示该天各线的值。倒序。
|
||||||
@@ -423,6 +402,80 @@ struct TrendDetailView: View {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// MARK: - AI 趋势解读卡
|
||||||
|
|
||||||
|
/// 进入页面先查指纹缓存:命中秒显;未命中本地现算(经 TrendInsightService,§3.1)。
|
||||||
|
private struct TrendInsightCard: View {
|
||||||
|
let bucket: SeriesBucket
|
||||||
|
@State private var text: String?
|
||||||
|
@State private var running = false
|
||||||
|
@State private var failedMessage: String?
|
||||||
|
|
||||||
|
var body: some View {
|
||||||
|
VStack(alignment: .leading, spacing: 8) {
|
||||||
|
HStack(spacing: 6) {
|
||||||
|
Image(systemName: "sparkles")
|
||||||
|
.font(.tjScaled( 12))
|
||||||
|
.foregroundStyle(Tj.Palette.ink)
|
||||||
|
Text("AI 解读")
|
||||||
|
.font(.tjScaled( 12, weight: .semibold))
|
||||||
|
.foregroundStyle(Tj.Palette.text2)
|
||||||
|
Spacer()
|
||||||
|
}
|
||||||
|
if let text {
|
||||||
|
Text(text)
|
||||||
|
.font(.tjScaled( 13))
|
||||||
|
.lineSpacing(3)
|
||||||
|
.foregroundStyle(Tj.Palette.text)
|
||||||
|
.fixedSize(horizontal: false, vertical: true)
|
||||||
|
AIDisclaimerFooter()
|
||||||
|
} else if running {
|
||||||
|
Text("本地 AI 解读中…")
|
||||||
|
.font(.tjScaled( 12))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
AIFlowBar()
|
||||||
|
} else if let failedMessage {
|
||||||
|
HStack {
|
||||||
|
Text(failedMessage)
|
||||||
|
.font(.tjScaled( 12))
|
||||||
|
.foregroundStyle(Tj.Palette.text3)
|
||||||
|
Spacer()
|
||||||
|
Button("重试") { Task { await load(force: true) } }
|
||||||
|
.font(.tjScaled( 12, weight: .medium))
|
||||||
|
.foregroundStyle(Tj.Palette.ink)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
.padding(14)
|
||||||
|
.frame(maxWidth: .infinity, alignment: .leading)
|
||||||
|
.background(
|
||||||
|
RoundedRectangle(cornerRadius: Tj.Radius.md, style: .continuous)
|
||||||
|
.fill(Tj.Palette.paper)
|
||||||
|
)
|
||||||
|
.overlay(
|
||||||
|
RoundedRectangle(cornerRadius: Tj.Radius.md, style: .continuous)
|
||||||
|
.strokeBorder(Tj.Palette.lineSoft, lineWidth: 1)
|
||||||
|
)
|
||||||
|
.task(id: bucket.id) { await load(force: false) }
|
||||||
|
}
|
||||||
|
|
||||||
|
@MainActor
|
||||||
|
private func load(force: Bool) async {
|
||||||
|
if !force, let cached = TrendInsightService.shared.cachedText(for: bucket) {
|
||||||
|
text = cached
|
||||||
|
return
|
||||||
|
}
|
||||||
|
running = true
|
||||||
|
failedMessage = nil
|
||||||
|
do {
|
||||||
|
text = try await TrendInsightService.shared.generate(for: bucket)
|
||||||
|
} catch {
|
||||||
|
failedMessage = String(appLoc: "AI 解读暂不可用(模型未就绪或繁忙)")
|
||||||
|
}
|
||||||
|
running = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
enum TrendRange: String, CaseIterable, Identifiable {
|
enum TrendRange: String, CaseIterable, Identifiable {
|
||||||
case all, year, sixMonths, threeMonths
|
case all, year, sixMonths, threeMonths
|
||||||
var id: String { rawValue }
|
var id: String { rawValue }
|
||||||
|
|||||||
@@ -1155,6 +1155,12 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"AI 整理中 · 本地推理" : {
|
||||||
|
|
||||||
|
},
|
||||||
|
"AI 整理失败,已填入原话" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"AI 模型尚未准备好" : {
|
"AI 模型尚未准备好" : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -1230,6 +1236,9 @@
|
|||||||
},
|
},
|
||||||
"AI 生成中 · 本地推理" : {
|
"AI 生成中 · 本地推理" : {
|
||||||
|
|
||||||
|
},
|
||||||
|
"AI 解读" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"AI 解读基于通用健康知识生成,并不掌握你完整的病史与个体情况,仅供日常记录参考。" : {
|
"AI 解读基于通用健康知识生成,并不掌握你完整的病史与个体情况,仅供日常记录参考。" : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -1252,6 +1261,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"AI 解读暂不可用(模型未就绪或繁忙)" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"AI 识别与解读可能出现错误或遗漏:拍照得到的数值、单位、参考范围请务必与原始报告核对," : {
|
"AI 识别与解读可能出现错误或遗漏:拍照得到的数值、单位、参考范围请务必与原始报告核对," : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -1274,9 +1286,6 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
|
||||||
"AI 趋势解读即将上线" : {
|
|
||||||
|
|
||||||
},
|
},
|
||||||
"AI 辅助 · 医生角度查漏补缺" : {
|
"AI 辅助 · 医生角度查漏补缺" : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -1431,6 +1440,16 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"prompt %lld tok · 生成 %lld tok · 100%% 本地" : {
|
||||||
|
"localizations" : {
|
||||||
|
"zh-Hans" : {
|
||||||
|
"stringUnit" : {
|
||||||
|
"state" : "new",
|
||||||
|
"value" : "prompt %1$lld tok · 生成 %2$lld tok · 100%% 本地"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"Qwen3.5-2B 已就绪" : {
|
"Qwen3.5-2B 已就绪" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
@@ -2923,6 +2942,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"停止生成" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"健康日历" : {
|
"健康日历" : {
|
||||||
|
|
||||||
@@ -3386,6 +3408,9 @@
|
|||||||
},
|
},
|
||||||
"最高" : {
|
"最高" : {
|
||||||
|
|
||||||
|
},
|
||||||
|
"写下要整理什么,或先提问补充情况…" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"冠心病" : {
|
"冠心病" : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -3628,6 +3653,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"前往设置" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"剧烈" : {
|
"剧烈" : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -4114,6 +4142,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"各引擎实测对比" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"同一指标记录满 2 次后,会在这里出现时间序列" : {
|
"同一指标记录满 2 次后,会在这里出现时间序列" : {
|
||||||
|
|
||||||
@@ -4362,6 +4393,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"在「我的 · 推理引擎」切换引擎后再跑一次,即可对比 SME2 与 GPU。" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"在内存较小的旧机型上,App 仍可用于手动记录、归档与查看," : {
|
"在内存较小的旧机型上,App 仍可用于手动记录、归档与查看," : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -5259,6 +5293,12 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"已取消整理,填入原话" : {
|
||||||
|
|
||||||
|
},
|
||||||
|
"已在本地档案中找到 %lld 条相关记录" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"已处理 %.1fs · 比云端快 4.2×" : {
|
"已处理 %.1fs · 比云端快 4.2×" : {
|
||||||
"extractionState" : "stale",
|
"extractionState" : "stale",
|
||||||
@@ -5909,6 +5949,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"开始说话…" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"异常项" : {
|
"异常项" : {
|
||||||
"extractionState" : "stale",
|
"extractionState" : "stale",
|
||||||
@@ -6025,6 +6068,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"录音启动失败:%@" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"影像报告" : {
|
"影像报告" : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -6179,6 +6225,12 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"性能自检" : {
|
||||||
|
|
||||||
|
},
|
||||||
|
"总耗时" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"总胆固醇" : {
|
"总胆固醇" : {
|
||||||
"extractionState" : "stale",
|
"extractionState" : "stale",
|
||||||
@@ -7143,6 +7195,7 @@
|
|||||||
|
|
||||||
},
|
},
|
||||||
"推理自检" : {
|
"推理自检" : {
|
||||||
|
"extractionState" : "stale",
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
"en" : {
|
"en" : {
|
||||||
"stringUnit" : {
|
"stringUnit" : {
|
||||||
@@ -7478,6 +7531,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"改用原话" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"放大后整个 App 的文字立即变大,无需重启。设置会被记住。" : {
|
"放大后整个 App 的文字立即变大,无需重启。设置会被记住。" : {
|
||||||
|
|
||||||
@@ -7835,6 +7891,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"日记 ×%lld" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"日记详情" : {
|
"日记详情" : {
|
||||||
|
|
||||||
@@ -8285,6 +8344,12 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"本地 AI 正在解读这份报告…" : {
|
||||||
|
|
||||||
|
},
|
||||||
|
"本地 AI 解读中…" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"本地 RAG · Qwen3 1.7B · 不上传任何数据" : {
|
"本地 RAG · Qwen3 1.7B · 不上传任何数据" : {
|
||||||
"extractionState" : "stale",
|
"extractionState" : "stale",
|
||||||
@@ -8419,6 +8484,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"本地档案中暂无相关记录,将仅按你的描述整理" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"本地识别中" : {
|
"本地识别中" : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -8513,6 +8581,9 @@
|
|||||||
},
|
},
|
||||||
"本月暂无记录" : {
|
"本月暂无记录" : {
|
||||||
|
|
||||||
|
},
|
||||||
|
"本机不支持端侧语音识别" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"本机保存" : {
|
"本机保存" : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -8654,6 +8725,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"本次结果" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"本设备/模拟器不可用,自动回退 MLX" : {
|
"本设备/模拟器不可用,自动回退 MLX" : {
|
||||||
|
|
||||||
@@ -8997,6 +9071,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"正在听 · 识别在本机完成" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"正在本地识别第 1 / 3 页…" : {
|
"正在本地识别第 1 / 3 页…" : {
|
||||||
"extractionState" : "stale",
|
"extractionState" : "stale",
|
||||||
@@ -9069,6 +9146,9 @@
|
|||||||
},
|
},
|
||||||
"正在查看本地记录…" : {
|
"正在查看本地记录…" : {
|
||||||
|
|
||||||
|
},
|
||||||
|
"正在根据这些记录回答…" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"正常" : {
|
"正常" : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -9249,6 +9329,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"没听清,再试一次" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"没有指标 — 点上方「加一项」补一行,或直接保存只存图片" : {
|
"没有指标 — 点上方「加一项」补一行,或直接保存只存图片" : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -9556,6 +9639,12 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"生成" : {
|
||||||
|
|
||||||
|
},
|
||||||
|
"生成 %.1f tok/s" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"生成失败:%@" : {
|
"生成失败:%@" : {
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
@@ -10351,9 +10440,6 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
|
||||||
"继续提问或补充情况…" : {
|
|
||||||
|
|
||||||
},
|
},
|
||||||
"维生素 D" : {
|
"维生素 D" : {
|
||||||
"extractionState" : "stale",
|
"extractionState" : "stale",
|
||||||
@@ -11232,6 +11318,15 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"语音记录全程在本机完成,声音和文字都不会上传。请在设置中允许麦克风和语音识别。" : {
|
||||||
|
|
||||||
|
},
|
||||||
|
"说一段" : {
|
||||||
|
|
||||||
|
},
|
||||||
|
"说完了,整理成日记" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"说说你想给医生看什么" : {
|
"说说你想给医生看什么" : {
|
||||||
"extractionState" : "stale",
|
"extractionState" : "stale",
|
||||||
@@ -11277,6 +11372,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"读入" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"读取生日、性别、身高和血型,确认后填入个人资料" : {
|
"读取生日、性别、身高和血型,确认后填入个人资料" : {
|
||||||
|
|
||||||
@@ -11740,8 +11838,12 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"运行性能自检" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"运行推理自检" : {
|
"运行推理自检" : {
|
||||||
|
"extractionState" : "stale",
|
||||||
"localizations" : {
|
"localizations" : {
|
||||||
"en" : {
|
"en" : {
|
||||||
"stringUnit" : {
|
"stringUnit" : {
|
||||||
@@ -12341,6 +12443,9 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"需要麦克风与语音识别权限" : {
|
||||||
|
|
||||||
},
|
},
|
||||||
"预计耗时 5–8 秒 · 端侧 SME2 加速" : {
|
"预计耗时 5–8 秒 · 端侧 SME2 加速" : {
|
||||||
"extractionState" : "stale",
|
"extractionState" : "stale",
|
||||||
|
|||||||
67
康康/Services/BenchmarkService.swift
Normal file
67
康康/Services/BenchmarkService.swift
Normal file
@@ -0,0 +1,67 @@
|
|||||||
|
import Foundation
|
||||||
|
|
||||||
|
/// 单次性能自检结果。按后端标签归档,供「MNN·SME2 vs MLX·GPU」对比展示(§12 卖点 2/6)。
|
||||||
|
struct BenchmarkResult: Codable, Equatable {
|
||||||
|
var backendLabel: String
|
||||||
|
var promptTokens: Int
|
||||||
|
var genTokens: Int
|
||||||
|
var prefillTokensPerSecond: Double
|
||||||
|
var decodeTokensPerSecond: Double
|
||||||
|
var totalSeconds: Double
|
||||||
|
var date: Date
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 性能自检服务:跑固定 prompt,取 AIRuntime 的归一统计,按后端标签存 UserDefaults。
|
||||||
|
/// UI(ModelSelfTestView)只经本服务调 AIRuntime(§3.1)。
|
||||||
|
@MainActor
|
||||||
|
struct BenchmarkService {
|
||||||
|
static let shared = BenchmarkService()
|
||||||
|
private init() {}
|
||||||
|
|
||||||
|
nonisolated static let storeKey = "kk.benchmark.results"
|
||||||
|
|
||||||
|
/// 固定测试 prompt:跨设备/引擎可比的前提。
|
||||||
|
static let fixedPrompt = "用中文一句话介绍肝功能里 ALT 这个指标。"
|
||||||
|
|
||||||
|
/// 跑一次自检。onToken 把流式输出交给 UI 展示。
|
||||||
|
func run(onToken: @escaping @MainActor (String, Double) -> Void) async throws -> BenchmarkResult {
|
||||||
|
try await AIRuntime.shared.prepare()
|
||||||
|
let start = Date()
|
||||||
|
let stream = await AIRuntime.shared.generate(prompt: Self.fixedPrompt, maxTokens: 128)
|
||||||
|
for try await chunk in stream {
|
||||||
|
onToken(chunk.text, chunk.decodeRate)
|
||||||
|
}
|
||||||
|
let total = Date().timeIntervalSince(start)
|
||||||
|
let label = await AIRuntime.shared.activeBackendLabel
|
||||||
|
let stats = await AIRuntime.shared.lastGenerateStats
|
||||||
|
let result = BenchmarkResult(
|
||||||
|
backendLabel: label,
|
||||||
|
promptTokens: stats?.promptTokens ?? 0,
|
||||||
|
genTokens: stats?.genTokens ?? 0,
|
||||||
|
prefillTokensPerSecond: stats?.prefillTokensPerSecond ?? 0,
|
||||||
|
decodeTokensPerSecond: stats?.decodeTokensPerSecond ?? 0,
|
||||||
|
totalSeconds: total,
|
||||||
|
date: .now
|
||||||
|
)
|
||||||
|
Self.save(result)
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - 存档(静态纯函数,单测覆盖;nonisolated:纯 UserDefaults 操作,无需主线程)
|
||||||
|
|
||||||
|
nonisolated static func save(_ result: BenchmarkResult, defaults: UserDefaults = .standard) {
|
||||||
|
var all = load(defaults: defaults)
|
||||||
|
all[result.backendLabel] = result
|
||||||
|
if let data = try? JSONEncoder().encode(all) {
|
||||||
|
defaults.set(data, forKey: storeKey)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
nonisolated static func load(defaults: UserDefaults = .standard) -> [String: BenchmarkResult] {
|
||||||
|
guard let data = defaults.data(forKey: storeKey),
|
||||||
|
let all = try? JSONDecoder().decode([String: BenchmarkResult].self, from: data) else {
|
||||||
|
return [:]
|
||||||
|
}
|
||||||
|
return all
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,5 +1,6 @@
|
|||||||
import Foundation
|
import Foundation
|
||||||
import UIKit
|
import UIKit
|
||||||
|
import ImageIO
|
||||||
import SwiftData
|
import SwiftData
|
||||||
|
|
||||||
/// VL 解析结果(已结构化,可直接喂 SwiftData 模型构造)。
|
/// VL 解析结果(已结构化,可直接喂 SwiftData 模型构造)。
|
||||||
@@ -142,11 +143,14 @@ actor CaptureService {
|
|||||||
throw CaptureError.modelNotReady
|
throw CaptureError.modelNotReady
|
||||||
}
|
}
|
||||||
let urls = assets.map { FileVault.shared.rootURL.appendingPathComponent($0.relativePath) }
|
let urls = assets.map { FileVault.shared.rootURL.appendingPathComponent($0.relativePath) }
|
||||||
|
// OCR 参考(Vision 本地,<1s/页):给 2B 多模态当数字「抄写员」,降低小字误读。
|
||||||
|
// 任何失败都静默回退为空串,绝不阻断识别主流程(§3.2)。
|
||||||
|
let ocr = await Self.ocrReference(for: urls)
|
||||||
let raw: String
|
let raw: String
|
||||||
do {
|
do {
|
||||||
raw = try await AIRuntime.shared.analyzeReport(
|
raw = try await AIRuntime.shared.analyzeReport(
|
||||||
imageURLs: urls,
|
imageURLs: urls,
|
||||||
prompt: VLPrompts.reportExtraction()
|
prompt: VLPrompts.reportExtraction(ocrText: ocr)
|
||||||
)
|
)
|
||||||
} catch {
|
} catch {
|
||||||
throw CaptureError.inferenceFailed("\(error)")
|
throw CaptureError.inferenceFailed("\(error)")
|
||||||
@@ -160,6 +164,20 @@ actor CaptureService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// 对 Vault 报告图逐页 OCR 拼参考文本。最多 4 页;失败/空文本返回 ""。
|
||||||
|
/// 用 ImageIO 直取 CGImage(不经 UIImage,避免跨 actor 传非 Sendable 引用)。
|
||||||
|
private static func ocrReference(for urls: [URL]) async -> String {
|
||||||
|
var pages: [String] = []
|
||||||
|
for (idx, url) in urls.prefix(4).enumerated() {
|
||||||
|
guard let src = CGImageSourceCreateWithURL(url as CFURL, nil),
|
||||||
|
let cg = CGImageSourceCreateImageAtIndex(src, 0, nil) else { continue }
|
||||||
|
guard let text = try? await OCRService.recognizeText(in: cg),
|
||||||
|
!text.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty else { continue }
|
||||||
|
pages.append(urls.count > 1 ? "【第 \(idx + 1) 页】\n\(text)" : text)
|
||||||
|
}
|
||||||
|
return pages.joined(separator: "\n")
|
||||||
|
}
|
||||||
|
|
||||||
// MARK: - JSON parse(static + 纯函数 → 方便单测)
|
// MARK: - JSON parse(static + 纯函数 → 方便单测)
|
||||||
|
|
||||||
/// 从 VL 输出里抠出第一段合法 JSON 对象并解析。
|
/// 从 VL 输出里抠出第一段合法 JSON 对象并解析。
|
||||||
|
|||||||
@@ -98,4 +98,29 @@ struct DiaryAssistService {
|
|||||||
guard !questions.isEmpty else { throw AssistError.empty }
|
guard !questions.isEmpty else { throw AssistError.empty }
|
||||||
return (Array(questions.prefix(4)), lastRate)
|
return (Array(questions.prefix(4)), lastRate)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// 把语音转写稿整理成健康日记草稿(spec 2026-06-10-voice-diary)。
|
||||||
|
/// 失败(模型未就绪 / 输出为空)抛错,调用方回退为直接使用原话,不卡死。
|
||||||
|
/// 与 suggest 同样走 AIRuntime actor 队列,自然与追问/拍照串行。
|
||||||
|
func organize(transcript: String) async throws -> (text: String, decodeRate: Double) {
|
||||||
|
do {
|
||||||
|
try await AIRuntime.shared.prepare()
|
||||||
|
} catch {
|
||||||
|
throw AssistError.modelNotReady
|
||||||
|
}
|
||||||
|
|
||||||
|
let prompt = DiaryAssistPrompts.organize(transcript: transcript)
|
||||||
|
var collected = ""
|
||||||
|
var lastRate: Double = 0
|
||||||
|
let stream = await AIRuntime.shared.generate(prompt: prompt, maxTokens: 400)
|
||||||
|
for try await chunk in stream {
|
||||||
|
collected += chunk.text
|
||||||
|
if chunk.decodeRate > 0 { lastRate = chunk.decodeRate }
|
||||||
|
}
|
||||||
|
|
||||||
|
let text = HealthExportService.stripThinkBlocks(collected)
|
||||||
|
.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||||
|
guard !text.isEmpty else { throw AssistError.empty }
|
||||||
|
return (text, lastRate)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -35,8 +35,56 @@ struct HealthExportService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// 检索结果摘要 —— 把「本地 RAG 找到了什么」拿给 UI 演出来(§12 卖点 3)。
|
||||||
|
struct RetrievalSummary: Sendable, Equatable {
|
||||||
|
var chips: [String]
|
||||||
|
var indicatorCount: Int
|
||||||
|
var reportCount: Int
|
||||||
|
var symptomCount: Int
|
||||||
|
var diaryCount: Int
|
||||||
|
|
||||||
|
var totalCount: Int { indicatorCount + reportCount + symptomCount + diaryCount }
|
||||||
|
|
||||||
|
/// 同名指标合并计数(保持检索的新→旧顺序),超出 cap 折叠成 "+N"。纯函数,单测覆盖。
|
||||||
|
static func groupedChips(_ names: [String], cap: Int = 8) -> [String] {
|
||||||
|
var order: [String] = []
|
||||||
|
var counts: [String: Int] = [:]
|
||||||
|
for n in names {
|
||||||
|
if counts[n] == nil { order.append(n) }
|
||||||
|
counts[n, default: 0] += 1
|
||||||
|
}
|
||||||
|
var chips = order.map { name -> String in
|
||||||
|
let c = counts[name] ?? 1
|
||||||
|
return c > 1 ? "\(name) ×\(c)" : name
|
||||||
|
}
|
||||||
|
if chips.count > cap {
|
||||||
|
let overflow = chips.count - cap
|
||||||
|
chips = Array(chips.prefix(cap)) + ["+\(overflow)"]
|
||||||
|
}
|
||||||
|
return chips
|
||||||
|
}
|
||||||
|
|
||||||
|
@MainActor
|
||||||
|
static func from(snapshot: Snapshot) -> RetrievalSummary {
|
||||||
|
var chips = groupedChips(snapshot.indicators.map(\.name), cap: 8)
|
||||||
|
chips += snapshot.reports.prefix(3).map(\.title)
|
||||||
|
chips += snapshot.symptoms.prefix(3).map(\.name)
|
||||||
|
if !snapshot.diaries.isEmpty {
|
||||||
|
chips.append(String(appLoc: "日记 ×\(snapshot.diaries.count)"))
|
||||||
|
}
|
||||||
|
return RetrievalSummary(
|
||||||
|
chips: chips,
|
||||||
|
indicatorCount: snapshot.indicators.count,
|
||||||
|
reportCount: snapshot.reports.count,
|
||||||
|
symptomCount: snapshot.symptoms.count,
|
||||||
|
diaryCount: snapshot.diaries.count
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
enum Event {
|
enum Event {
|
||||||
case phaseChanged(Phase)
|
case phaseChanged(Phase)
|
||||||
|
case retrieved(RetrievalSummary)
|
||||||
case token(TokenChunk)
|
case token(TokenChunk)
|
||||||
case completed(persistentID: PersistentIdentifier)
|
case completed(persistentID: PersistentIdentifier)
|
||||||
// .failed 走 stream throw,不在 Event 里
|
// .failed 走 stream throw,不在 Event 里
|
||||||
@@ -80,6 +128,7 @@ struct HealthExportService {
|
|||||||
// —— Phase 2: 检索 ——
|
// —— Phase 2: 检索 ——
|
||||||
continuation.yield(.phaseChanged(.retrieving))
|
continuation.yield(.phaseChanged(.retrieving))
|
||||||
let snapshot = Self.retrieve(intent: intent, ctx: modelContext)
|
let snapshot = Self.retrieve(intent: intent, ctx: modelContext)
|
||||||
|
continuation.yield(.retrieved(RetrievalSummary.from(snapshot: snapshot)))
|
||||||
try Task.checkCancellation()
|
try Task.checkCancellation()
|
||||||
|
|
||||||
// —— Phase 3: 生成 ——
|
// —— Phase 3: 生成 ——
|
||||||
@@ -178,9 +227,10 @@ struct HealthExportService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// 多轮导出页的单轮问答。只回答,不入库。
|
/// 多轮导出页的单轮问答。只回答,不入库。
|
||||||
|
/// 事件流:先 .retrieved(检索可视化),后 .token 流式正文;不发 .phaseChanged / .completed。
|
||||||
func answer(question: String,
|
func answer(question: String,
|
||||||
conversation: [HealthExportDialogueTurn],
|
conversation: [HealthExportDialogueTurn],
|
||||||
in modelContext: ModelContext) -> AsyncThrowingStream<TokenChunk, Error> {
|
in modelContext: ModelContext) -> AsyncThrowingStream<Event, Error> {
|
||||||
AsyncThrowingStream { continuation in
|
AsyncThrowingStream { continuation in
|
||||||
let task = Task { @MainActor in
|
let task = Task { @MainActor in
|
||||||
do {
|
do {
|
||||||
@@ -191,6 +241,7 @@ struct HealthExportService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
let snapshot = Self.retrieveDialogueSnapshot(ctx: modelContext)
|
let snapshot = Self.retrieveDialogueSnapshot(ctx: modelContext)
|
||||||
|
continuation.yield(.retrieved(RetrievalSummary.from(snapshot: snapshot)))
|
||||||
let dataJSON = Self.serializeData(snapshot: snapshot)
|
let dataJSON = Self.serializeData(snapshot: snapshot)
|
||||||
let transcript = HealthExportDialogueTurn.transcript(from: conversation)
|
let transcript = HealthExportDialogueTurn.transcript(from: conversation)
|
||||||
let prompt = HealthExportPrompts.dialogueAnswer(
|
let prompt = HealthExportPrompts.dialogueAnswer(
|
||||||
@@ -209,7 +260,7 @@ struct HealthExportService {
|
|||||||
if clean.count > displayed.count, clean.hasPrefix(displayed) {
|
if clean.count > displayed.count, clean.hasPrefix(displayed) {
|
||||||
let delta = String(clean.dropFirst(displayed.count))
|
let delta = String(clean.dropFirst(displayed.count))
|
||||||
displayed = clean
|
displayed = clean
|
||||||
continuation.yield(TokenChunk(text: delta, decodeRate: chunk.decodeRate))
|
continuation.yield(.token(TokenChunk(text: delta, decodeRate: chunk.decodeRate)))
|
||||||
} else if clean != displayed {
|
} else if clean != displayed {
|
||||||
displayed = clean
|
displayed = clean
|
||||||
}
|
}
|
||||||
@@ -245,6 +296,7 @@ struct HealthExportService {
|
|||||||
|
|
||||||
continuation.yield(.phaseChanged(.retrieving))
|
continuation.yield(.phaseChanged(.retrieving))
|
||||||
let snapshot = Self.retrieveDialogueSnapshot(ctx: modelContext)
|
let snapshot = Self.retrieveDialogueSnapshot(ctx: modelContext)
|
||||||
|
continuation.yield(.retrieved(RetrievalSummary.from(snapshot: snapshot)))
|
||||||
let dataJSON = Self.serializeData(snapshot: snapshot)
|
let dataJSON = Self.serializeData(snapshot: snapshot)
|
||||||
let transcript = HealthExportDialogueTurn.transcript(from: conversation)
|
let transcript = HealthExportDialogueTurn.transcript(from: conversation)
|
||||||
try Task.checkCancellation()
|
try Task.checkCancellation()
|
||||||
|
|||||||
61
康康/Services/ReportInsightService.swift
Normal file
61
康康/Services/ReportInsightService.swift
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
import Foundation
|
||||||
|
import SwiftData
|
||||||
|
|
||||||
|
/// 报告大白话摘要预生成(§3.1:流程经本服务碰 AIRuntime,UI 不直接调)。
|
||||||
|
/// 时机:归档保存后立即后台跑(用户继续操作时完成);详情页打开时兜底重试。
|
||||||
|
/// 写回策略:只在 summary 为空时生成 —— 绝不覆盖 VL 已给出或用户编辑过的摘要。
|
||||||
|
@MainActor
|
||||||
|
final class ReportInsightService {
|
||||||
|
static let shared = ReportInsightService()
|
||||||
|
private init() {}
|
||||||
|
|
||||||
|
/// 进行中的报告 ID,防止「保存后台任务」与「详情页兜底」重复触发。
|
||||||
|
private var inFlight: Set<String> = []
|
||||||
|
|
||||||
|
func pregenerateIfNeeded(report: Report, in ctx: ModelContext) async {
|
||||||
|
guard (report.summary ?? "").isEmpty, !report.indicators.isEmpty else { return }
|
||||||
|
let key = String(describing: report.persistentModelID)
|
||||||
|
guard !inFlight.contains(key) else { return }
|
||||||
|
inFlight.insert(key)
|
||||||
|
defer { inFlight.remove(key) }
|
||||||
|
|
||||||
|
do {
|
||||||
|
try await AIRuntime.shared.prepare()
|
||||||
|
} catch {
|
||||||
|
return // 模型未就绪:静默放弃,详情页下次打开再试
|
||||||
|
}
|
||||||
|
|
||||||
|
let prompt = InsightPrompts.reportPlainSummary(
|
||||||
|
title: report.title,
|
||||||
|
typeLabel: report.type.label,
|
||||||
|
indicatorLines: Self.indicatorLines(for: report.indicators)
|
||||||
|
)
|
||||||
|
var collected = ""
|
||||||
|
do {
|
||||||
|
let stream = await AIRuntime.shared.generate(
|
||||||
|
prompt: prompt, maxTokens: 200, priority: .background)
|
||||||
|
for try await chunk in stream { collected += chunk.text }
|
||||||
|
} catch {
|
||||||
|
return // 被前台任务抢占(CancellationError)或推理失败:放弃,兜底路径再试
|
||||||
|
}
|
||||||
|
let text = HealthExportService.stripThinkBlocks(collected)
|
||||||
|
.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||||
|
guard !text.isEmpty, (report.summary ?? "").isEmpty else { return }
|
||||||
|
report.summary = text
|
||||||
|
try? ctx.save()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 「名 值 单位(参考 range)status」每指标一行;异常项排前,上限 15 行控 prompt 体积。
|
||||||
|
static func indicatorLines(for indicators: [Indicator]) -> String {
|
||||||
|
let sorted = indicators.sorted {
|
||||||
|
($0.status == .normal ? 1 : 0) < ($1.status == .normal ? 1 : 0)
|
||||||
|
}
|
||||||
|
return sorted.prefix(15).map { i in
|
||||||
|
var line = "\(i.name) \(i.value)"
|
||||||
|
if !i.unit.isEmpty { line += " \(i.unit)" }
|
||||||
|
if !i.range.isEmpty { line += "(参考 \(i.range))" }
|
||||||
|
line += " \(i.status.rawValue)"
|
||||||
|
return line
|
||||||
|
}.joined(separator: "\n")
|
||||||
|
}
|
||||||
|
}
|
||||||
151
康康/Services/SpeechDictationService.swift
Normal file
151
康康/Services/SpeechDictationService.swift
Normal file
@@ -0,0 +1,151 @@
|
|||||||
|
import Foundation
|
||||||
|
import Speech
|
||||||
|
import AVFoundation
|
||||||
|
|
||||||
|
/// 端侧流式语音转写(spec 2026-06-10-voice-diary)。
|
||||||
|
/// AVAudioEngine 麦克风 buffer → SFSpeechAudioBufferRecognitionRequest,
|
||||||
|
/// `requiresOnDeviceRecognition = true` 硬性端侧,识别内容不出设备;**不落盘任何音频**。
|
||||||
|
///
|
||||||
|
/// 生命周期:start(onPartial:) 开始录音并实时回调 partial;stop() 结束并返回最终稿。
|
||||||
|
/// 调用方:DiaryQuickSheet。工程默认 MainActor 隔离,本类型即 MainActor;
|
||||||
|
/// audio tap 与识别回调在系统线程,闭包内只碰局部捕获对象,回主线程统一走 Task { @MainActor }。
|
||||||
|
final class SpeechDictationService {
|
||||||
|
|
||||||
|
enum DictationError: Error, LocalizedError {
|
||||||
|
case unavailable
|
||||||
|
case audioEngineStartFailed(String)
|
||||||
|
|
||||||
|
var errorDescription: String? {
|
||||||
|
switch self {
|
||||||
|
case .unavailable:
|
||||||
|
return String(appLoc: "本机不支持端侧语音识别")
|
||||||
|
case .audioEngineStartFailed(let m):
|
||||||
|
return String(appLoc: "录音启动失败:\(m)")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 优先系统语言;系统语言不支持端侧时兜底中文(demo 机即使系统是英文也能用)。
|
||||||
|
private static func makeRecognizer() -> SFSpeechRecognizer? {
|
||||||
|
if let r = SFSpeechRecognizer(locale: .current), r.supportsOnDeviceRecognition {
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
if let r = SFSpeechRecognizer(locale: Locale(identifier: "zh-CN")),
|
||||||
|
r.supportsOnDeviceRecognition {
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 本机是否支持端侧识别。false(模拟器/老机型)时 UI 隐藏 mic 入口,静默降级。
|
||||||
|
static var isAvailable: Bool { makeRecognizer() != nil }
|
||||||
|
|
||||||
|
private let audioEngine = AVAudioEngine()
|
||||||
|
private var request: SFSpeechAudioBufferRecognitionRequest?
|
||||||
|
private var task: SFSpeechRecognitionTask?
|
||||||
|
/// 识别回调持续刷新;isFinal 或出错时置 didFinish。stop() 用「final 优先、partial 兜底」。
|
||||||
|
private var latestText = ""
|
||||||
|
private var didFinish = false
|
||||||
|
|
||||||
|
private(set) var isRecording = false
|
||||||
|
|
||||||
|
/// 麦克风 + 语音识别两个权限一起申请。任一被拒返回 false。
|
||||||
|
func requestAuthorization() async -> Bool {
|
||||||
|
let speech = await withCheckedContinuation { (c: CheckedContinuation<SFSpeechRecognizerAuthorizationStatus, Never>) in
|
||||||
|
SFSpeechRecognizer.requestAuthorization { c.resume(returning: $0) }
|
||||||
|
}
|
||||||
|
guard speech == .authorized else { return false }
|
||||||
|
return await AVAudioApplication.requestRecordPermission()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 开始录音 + 流式识别。partial 结果在主线程回调(录音面板实时字幕)。
|
||||||
|
func start(onPartial: @escaping (String) -> Void) throws {
|
||||||
|
guard !isRecording else { return }
|
||||||
|
guard let recognizer = Self.makeRecognizer(), recognizer.isAvailable else {
|
||||||
|
throw DictationError.unavailable
|
||||||
|
}
|
||||||
|
|
||||||
|
let session = AVAudioSession.sharedInstance()
|
||||||
|
do {
|
||||||
|
try session.setCategory(.record, mode: .measurement, options: .duckOthers)
|
||||||
|
try session.setActive(true, options: .notifyOthersOnDeactivation)
|
||||||
|
} catch {
|
||||||
|
throw DictationError.audioEngineStartFailed(error.localizedDescription)
|
||||||
|
}
|
||||||
|
|
||||||
|
let request = SFSpeechAudioBufferRecognitionRequest()
|
||||||
|
request.requiresOnDeviceRecognition = true // 红线:识别不出设备
|
||||||
|
request.shouldReportPartialResults = true
|
||||||
|
request.addsPunctuation = true
|
||||||
|
self.request = request
|
||||||
|
latestText = ""
|
||||||
|
didFinish = false
|
||||||
|
|
||||||
|
let input = audioEngine.inputNode
|
||||||
|
let format = input.outputFormat(forBus: 0)
|
||||||
|
// tap 在音频线程跑:只碰局部捕获的 request,不碰 self
|
||||||
|
input.installTap(onBus: 0, bufferSize: 1024, format: format) { buffer, _ in
|
||||||
|
request.append(buffer)
|
||||||
|
}
|
||||||
|
audioEngine.prepare()
|
||||||
|
do {
|
||||||
|
try audioEngine.start()
|
||||||
|
} catch {
|
||||||
|
input.removeTap(onBus: 0)
|
||||||
|
deactivateSession()
|
||||||
|
throw DictationError.audioEngineStartFailed(error.localizedDescription)
|
||||||
|
}
|
||||||
|
|
||||||
|
task = recognizer.recognitionTask(with: request) { [weak self] result, error in
|
||||||
|
// 系统线程 → 主线程
|
||||||
|
Task { @MainActor in
|
||||||
|
guard let self else { return }
|
||||||
|
if let result {
|
||||||
|
self.latestText = result.bestTranscription.formattedString
|
||||||
|
onPartial(self.latestText)
|
||||||
|
if result.isFinal { self.didFinish = true }
|
||||||
|
}
|
||||||
|
if error != nil { self.didFinish = true }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
isRecording = true
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 停止录音,等待最终识别结果(最多 1.5s,超时用最新 partial),返回最终稿。
|
||||||
|
/// 中途识别出错时已拿到的 partial 一样返回(spec 错误表:照常进整理流程)。
|
||||||
|
func stop() async -> String {
|
||||||
|
guard isRecording else { return "" }
|
||||||
|
isRecording = false
|
||||||
|
|
||||||
|
audioEngine.stop()
|
||||||
|
audioEngine.inputNode.removeTap(onBus: 0)
|
||||||
|
request?.endAudio()
|
||||||
|
|
||||||
|
let deadline = Date().addingTimeInterval(1.5)
|
||||||
|
while !didFinish && Date() < deadline {
|
||||||
|
try? await Task.sleep(nanoseconds: 100_000_000)
|
||||||
|
}
|
||||||
|
task?.cancel()
|
||||||
|
task = nil
|
||||||
|
request = nil
|
||||||
|
deactivateSession()
|
||||||
|
return latestText
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 用户直接关 sheet 时的清理:不关心结果,立即停。
|
||||||
|
func abort() {
|
||||||
|
guard isRecording else { return }
|
||||||
|
isRecording = false
|
||||||
|
audioEngine.stop()
|
||||||
|
audioEngine.inputNode.removeTap(onBus: 0)
|
||||||
|
request?.endAudio()
|
||||||
|
task?.cancel()
|
||||||
|
task = nil
|
||||||
|
request = nil
|
||||||
|
deactivateSession()
|
||||||
|
}
|
||||||
|
|
||||||
|
private func deactivateSession() {
|
||||||
|
try? AVAudioSession.sharedInstance().setActive(false, options: .notifyOthersOnDeactivation)
|
||||||
|
}
|
||||||
|
}
|
||||||
94
康康/Services/TrendInsightService.swift
Normal file
94
康康/Services/TrendInsightService.swift
Normal file
@@ -0,0 +1,94 @@
|
|||||||
|
import Foundation
|
||||||
|
|
||||||
|
/// 趋势 AI 一句话解读:小预算(≤140 token)+ 按数据指纹缓存(UserDefaults)。
|
||||||
|
/// 数据没变不重算 —— 进趋势详情页秒开;新增/修改记录改变指纹 → 自动重新生成。
|
||||||
|
@MainActor
|
||||||
|
final class TrendInsightService {
|
||||||
|
static let shared = TrendInsightService()
|
||||||
|
private init() {}
|
||||||
|
|
||||||
|
struct Cached: Codable, Equatable {
|
||||||
|
var fingerprint: String
|
||||||
|
var text: String
|
||||||
|
var generatedAt: Date
|
||||||
|
}
|
||||||
|
|
||||||
|
nonisolated static let storePrefix = "kk.trendInsight."
|
||||||
|
|
||||||
|
/// 数据指纹:每条线的 key + 点数 + 首末时间 + 末值/极值。体量小,直接当指纹字符串。
|
||||||
|
nonisolated static func fingerprint(for bucket: SeriesBucket) -> String {
|
||||||
|
var parts: [String] = [bucket.id]
|
||||||
|
for line in bucket.lines {
|
||||||
|
let pts = line.points
|
||||||
|
let first = pts.first.map { Int($0.date.timeIntervalSince1970) } ?? 0
|
||||||
|
let last = pts.last.map { Int($0.date.timeIntervalSince1970) } ?? 0
|
||||||
|
let lastV = pts.last?.value ?? 0
|
||||||
|
let minV = pts.map(\.value).min() ?? 0
|
||||||
|
let maxV = pts.map(\.value).max() ?? 0
|
||||||
|
parts.append("\(line.seriesKey)#\(pts.count)#\(first)#\(last)#\(lastV)#\(minV)#\(maxV)")
|
||||||
|
}
|
||||||
|
return parts.joined(separator: "|")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 命中缓存(指纹一致)返回文本,否则 nil。
|
||||||
|
func cachedText(for bucket: SeriesBucket) -> String? {
|
||||||
|
guard let data = UserDefaults.standard.data(forKey: Self.storePrefix + bucket.id),
|
||||||
|
let c = try? JSONDecoder().decode(Cached.self, from: data),
|
||||||
|
c.fingerprint == Self.fingerprint(for: bucket) else {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return c.text
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 现算一条解读并写缓存。模型未就绪/输出为空时抛错,UI 显示「暂不可用 + 重试」。
|
||||||
|
func generate(for bucket: SeriesBucket) async throws -> String {
|
||||||
|
try await AIRuntime.shared.prepare()
|
||||||
|
let prompt = InsightPrompts.trendInsight(
|
||||||
|
title: bucket.title,
|
||||||
|
unit: bucket.unit,
|
||||||
|
rangeText: Self.rangeText(for: bucket),
|
||||||
|
dataLines: Self.dataLines(for: bucket)
|
||||||
|
)
|
||||||
|
var collected = ""
|
||||||
|
let stream = await AIRuntime.shared.generate(prompt: prompt, maxTokens: 140)
|
||||||
|
for try await chunk in stream { collected += chunk.text }
|
||||||
|
let text = HealthExportService.stripThinkBlocks(collected)
|
||||||
|
.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||||
|
guard !text.isEmpty else { throw AIRuntimeError.inferenceFailed("空输出") }
|
||||||
|
let cached = Cached(fingerprint: Self.fingerprint(for: bucket), text: text, generatedAt: .now)
|
||||||
|
if let data = try? JSONEncoder().encode(cached) {
|
||||||
|
UserDefaults.standard.set(data, forKey: Self.storePrefix + bucket.id)
|
||||||
|
}
|
||||||
|
return text
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 每条线最近 24 个点拼成 "yyyy-MM-dd 值";多线(血压)各占一行带 label 前缀。
|
||||||
|
/// 用 UTC 时区:展示日期仅供模型理解走向,差几小时无影响,但测试与设备时区无关。
|
||||||
|
nonisolated static func dataLines(for bucket: SeriesBucket) -> String {
|
||||||
|
let df = DateFormatter()
|
||||||
|
df.locale = Locale(identifier: "en_US_POSIX")
|
||||||
|
df.timeZone = TimeZone(identifier: "UTC")
|
||||||
|
df.dateFormat = "yyyy-MM-dd"
|
||||||
|
var lines: [String] = []
|
||||||
|
for line in bucket.lines {
|
||||||
|
let pts = line.points.suffix(24)
|
||||||
|
let prefix = bucket.lines.count > 1 ? "\(line.label ?? line.seriesKey):" : ""
|
||||||
|
let series = pts.map { "\(df.string(from: $0.date)) \(fmt($0.value))" }
|
||||||
|
.joined(separator: " / ")
|
||||||
|
lines.append(prefix + series)
|
||||||
|
}
|
||||||
|
return lines.joined(separator: "\n")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// ",参考 lo-hi" 或空串(无参考范围时整段省略)。
|
||||||
|
nonisolated static func rangeText(for bucket: SeriesBucket) -> String {
|
||||||
|
guard let r = bucket.lines.first?.referenceRange else { return "" }
|
||||||
|
return ",参考 \(fmt(r.lowerBound))-\(fmt(r.upperBound))"
|
||||||
|
}
|
||||||
|
|
||||||
|
private nonisolated static func fmt(_ v: Double) -> String {
|
||||||
|
v.truncatingRemainder(dividingBy: 1) == 0
|
||||||
|
? String(format: "%.0f", v)
|
||||||
|
: String(format: "%.1f", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
45
康康Tests/BenchmarkStoreTests.swift
Normal file
45
康康Tests/BenchmarkStoreTests.swift
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
import Testing
|
||||||
|
import Foundation
|
||||||
|
@testable import 康康
|
||||||
|
|
||||||
|
struct BenchmarkStoreTests {
|
||||||
|
|
||||||
|
/// 每个用例独立 suite,避免 Swift Testing 并行执行时互相清空数据。
|
||||||
|
private func freshDefaults(_ name: String) -> UserDefaults {
|
||||||
|
let suite = "test.kk.benchmark.\(name)"
|
||||||
|
let d = UserDefaults(suiteName: suite)!
|
||||||
|
d.removePersistentDomain(forName: suite)
|
||||||
|
return d
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func savesAndLoadsPerBackend() {
|
||||||
|
let d = freshDefaults("savesAndLoads")
|
||||||
|
let mnn = BenchmarkResult(backendLabel: "MNN · SME2", promptTokens: 30, genTokens: 80,
|
||||||
|
prefillTokensPerSecond: 120, decodeTokensPerSecond: 25,
|
||||||
|
totalSeconds: 4.2, date: .now)
|
||||||
|
let mlx = BenchmarkResult(backendLabel: "MLX · GPU", promptTokens: 30, genTokens: 80,
|
||||||
|
prefillTokensPerSecond: 300, decodeTokensPerSecond: 40,
|
||||||
|
totalSeconds: 2.5, date: .now)
|
||||||
|
BenchmarkService.save(mnn, defaults: d)
|
||||||
|
BenchmarkService.save(mlx, defaults: d)
|
||||||
|
let all = BenchmarkService.load(defaults: d)
|
||||||
|
#expect(all.count == 2)
|
||||||
|
#expect(all["MNN · SME2"]?.decodeTokensPerSecond == 25)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func overwritesSameBackend() {
|
||||||
|
let d = freshDefaults("overwrites")
|
||||||
|
let old = BenchmarkResult(backendLabel: "MLX · GPU", promptTokens: 1, genTokens: 1,
|
||||||
|
prefillTokensPerSecond: 1, decodeTokensPerSecond: 1,
|
||||||
|
totalSeconds: 1, date: .now)
|
||||||
|
var new = old
|
||||||
|
new.decodeTokensPerSecond = 99
|
||||||
|
BenchmarkService.save(old, defaults: d)
|
||||||
|
BenchmarkService.save(new, defaults: d)
|
||||||
|
#expect(BenchmarkService.load(defaults: d)["MLX · GPU"]?.decodeTokensPerSecond == 99)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func loadOnEmptyReturnsEmpty() {
|
||||||
|
#expect(BenchmarkService.load(defaults: freshDefaults("loadEmpty")).isEmpty)
|
||||||
|
}
|
||||||
|
}
|
||||||
28
康康Tests/DiaryOrganizePromptTests.swift
Normal file
28
康康Tests/DiaryOrganizePromptTests.swift
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
import Testing
|
||||||
|
@testable import 康康
|
||||||
|
|
||||||
|
struct DiaryOrganizePromptTests {
|
||||||
|
@Test func organizePromptContainsTranscriptAndHardRules() {
|
||||||
|
let prompt = DiaryAssistPrompts.organize(transcript: "今天早上头晕量了血压140 90")
|
||||||
|
|
||||||
|
#expect(prompt.contains("今天早上头晕量了血压140 90"))
|
||||||
|
// 健康数据红线:数值/单位/药名/时间不许改,必须写进 prompt
|
||||||
|
#expect(prompt.contains("数值"))
|
||||||
|
#expect(prompt.contains("药名"))
|
||||||
|
// 自适应样式两条规则都在
|
||||||
|
#expect(prompt.contains("一段通顺的话"))
|
||||||
|
#expect(prompt.contains("分行"))
|
||||||
|
// 项目 prompt 规范:禁思考标签
|
||||||
|
#expect(prompt.contains("/no_think"))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func organizePromptTruncatesLongTranscript() {
|
||||||
|
let long = String(repeating: "头晕", count: 2000) // 4000 字符,超过上限
|
||||||
|
let prompt = DiaryAssistPrompts.organize(transcript: long)
|
||||||
|
|
||||||
|
// 整条 prompt 里口述部分被截断到 organizeTranscriptLimit
|
||||||
|
let expectedTail = String(long.prefix(DiaryAssistPrompts.organizeTranscriptLimit))
|
||||||
|
#expect(prompt.contains(expectedTail))
|
||||||
|
#expect(!prompt.contains(String(long.prefix(DiaryAssistPrompts.organizeTranscriptLimit + 2))))
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -3,7 +3,7 @@ import Testing
|
|||||||
@testable import 康康
|
@testable import 康康
|
||||||
|
|
||||||
struct HealthExportDialogueTests {
|
struct HealthExportDialogueTests {
|
||||||
@Test func dialogueTranscriptKeepsTurnOrderAndRoles() {
|
@Test func dialogueTranscriptKeepsTurnOrderAndRoles() throws {
|
||||||
let turns: [HealthExportDialogueTurn] = [
|
let turns: [HealthExportDialogueTurn] = [
|
||||||
.user("我最近头晕,帮我看看"),
|
.user("我最近头晕,帮我看看"),
|
||||||
.assistant("我会结合你的指标和日记整理。"),
|
.assistant("我会结合你的指标和日记整理。"),
|
||||||
@@ -12,10 +12,13 @@ struct HealthExportDialogueTests {
|
|||||||
|
|
||||||
let transcript = HealthExportDialogueTurn.transcript(from: turns)
|
let transcript = HealthExportDialogueTurn.transcript(from: turns)
|
||||||
|
|
||||||
#expect(transcript.contains("患者: 我最近头晕,帮我看看"))
|
// 用户角色标签是「我」(产品红线:不出现「患者」,见 ca5a3fa)。
|
||||||
|
#expect(transcript.contains("我: 我最近头晕,帮我看看"))
|
||||||
#expect(transcript.contains("康康: 我会结合你的指标和日记整理。"))
|
#expect(transcript.contains("康康: 我会结合你的指标和日记整理。"))
|
||||||
#expect(transcript.contains("患者: 重点看血压"))
|
#expect(transcript.contains("我: 重点看血压"))
|
||||||
#expect(transcript.range(of: "患者: 我最近头晕")!.lowerBound < transcript.range(of: "患者: 重点看血压")!.lowerBound)
|
let first = try #require(transcript.range(of: "我: 我最近头晕"))
|
||||||
|
let second = try #require(transcript.range(of: "我: 重点看血压"))
|
||||||
|
#expect(first.lowerBound < second.lowerBound)
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test func dialogueTranscriptDropsEmptyTurns() {
|
@Test func dialogueTranscriptDropsEmptyTurns() {
|
||||||
@@ -27,8 +30,8 @@ struct HealthExportDialogueTests {
|
|||||||
|
|
||||||
let transcript = HealthExportDialogueTurn.transcript(from: turns)
|
let transcript = HealthExportDialogueTurn.transcript(from: turns)
|
||||||
|
|
||||||
#expect(!transcript.contains("患者: "))
|
#expect(!transcript.contains("我: "))
|
||||||
#expect(transcript.contains("康康: 请补充想看的问题"))
|
#expect(transcript.contains("康康: 请补充想看的问题"))
|
||||||
#expect(transcript.contains("患者: 最近三个月"))
|
#expect(transcript.contains("我: 最近三个月"))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
28
康康Tests/InferencePriorityTests.swift
Normal file
28
康康Tests/InferencePriorityTests.swift
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
import Testing
|
||||||
|
@testable import 康康
|
||||||
|
|
||||||
|
struct InferencePriorityTests {
|
||||||
|
|
||||||
|
@Test func interactiveJumpsAheadOfBackground() {
|
||||||
|
let idx = AIRuntime.gateInsertionIndex(of: .interactive,
|
||||||
|
in: [.interactive, .background, .background])
|
||||||
|
#expect(idx == 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func interactiveKeepsFIFOAmongInteractive() {
|
||||||
|
let idx = AIRuntime.gateInsertionIndex(of: .interactive,
|
||||||
|
in: [.interactive, .interactive])
|
||||||
|
#expect(idx == 2)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func backgroundAlwaysAppends() {
|
||||||
|
let idx = AIRuntime.gateInsertionIndex(of: .background,
|
||||||
|
in: [.interactive, .background])
|
||||||
|
#expect(idx == 2)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func emptyQueueInsertsAtZero() {
|
||||||
|
#expect(AIRuntime.gateInsertionIndex(of: .interactive, in: []) == 0)
|
||||||
|
#expect(AIRuntime.gateInsertionIndex(of: .background, in: []) == 0)
|
||||||
|
}
|
||||||
|
}
|
||||||
26
康康Tests/InsightPromptsTests.swift
Normal file
26
康康Tests/InsightPromptsTests.swift
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
import Testing
|
||||||
|
@testable import 康康
|
||||||
|
|
||||||
|
struct InsightPromptsTests {
|
||||||
|
|
||||||
|
@Test func reportSummaryPromptCarriesDataAndGuards() {
|
||||||
|
let p = InsightPrompts.reportPlainSummary(
|
||||||
|
title: "春季体检", typeLabel: "体检报告",
|
||||||
|
indicatorLines: "血红蛋白 118 g/L(参考 130-175)low")
|
||||||
|
#expect(p.contains("春季体检"))
|
||||||
|
#expect(p.contains("血红蛋白 118"))
|
||||||
|
#expect(p.contains("/no_think"))
|
||||||
|
#expect(p.contains("不诊断"))
|
||||||
|
#expect(!p.contains("患者"))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func trendPromptCarriesDataAndGuards() {
|
||||||
|
let p = InsightPrompts.trendInsight(
|
||||||
|
title: "空腹血糖", unit: "mmol/L", rangeText: ",参考 3.9-6.1",
|
||||||
|
dataLines: "2026-05-01 5.2 / 2026-06-01 5.8")
|
||||||
|
#expect(p.contains("空腹血糖"))
|
||||||
|
#expect(p.contains("2026-06-01 5.8"))
|
||||||
|
#expect(p.contains("/no_think"))
|
||||||
|
#expect(!p.contains("患者"))
|
||||||
|
}
|
||||||
|
}
|
||||||
22
康康Tests/RetrievalSummaryTests.swift
Normal file
22
康康Tests/RetrievalSummaryTests.swift
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
import Testing
|
||||||
|
@testable import 康康
|
||||||
|
|
||||||
|
struct RetrievalSummaryTests {
|
||||||
|
|
||||||
|
@Test func groupsAndCountsPreservingOrder() {
|
||||||
|
let chips = HealthExportService.RetrievalSummary.groupedChips(
|
||||||
|
["血压", "血糖", "血压", "血压", "体重"], cap: 8)
|
||||||
|
#expect(chips == ["血压 ×3", "血糖", "体重"])
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func capsAndAppendsOverflow() {
|
||||||
|
let names = (1...12).map { "指标\($0)" }
|
||||||
|
let chips = HealthExportService.RetrievalSummary.groupedChips(names, cap: 8)
|
||||||
|
#expect(chips.count == 9)
|
||||||
|
#expect(chips.last == "+4")
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func emptyInputGivesEmptyChips() {
|
||||||
|
#expect(HealthExportService.RetrievalSummary.groupedChips([], cap: 8).isEmpty)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -18,14 +18,17 @@ struct SeriesBucketTests {
|
|||||||
seriesKey: seriesKey)
|
seriesKey: seriesKey)
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test func skipsIndicatorsWithoutSeriesKey() {
|
@Test func indicatorsWithoutSeriesKeyFoldIntoLabBucket() {
|
||||||
|
// 「lab 段」特性:无 seriesKey 的指标按 name+unit 归并,同名 ≥minPoints 次即成趋势。
|
||||||
let now = Date()
|
let now = Date()
|
||||||
let items = [
|
let items = [
|
||||||
makeIndicator(value: "5.0", capturedAt: now, seriesKey: nil),
|
makeIndicator(value: "5.0", capturedAt: now.addingTimeInterval(-86_400), seriesKey: nil),
|
||||||
makeIndicator(value: "5.2", capturedAt: now, seriesKey: nil),
|
makeIndicator(value: "5.2", capturedAt: now, seriesKey: nil),
|
||||||
]
|
]
|
||||||
let buckets = SeriesBucket.build(from: items)
|
let buckets = SeriesBucket.build(from: items)
|
||||||
#expect(buckets.isEmpty)
|
#expect(buckets.count == 1)
|
||||||
|
#expect(buckets.first?.kind == .lab)
|
||||||
|
#expect(buckets.first?.lines.first?.points.count == 2)
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test func filtersOutSeriesWithFewerThanMinPoints() {
|
@Test func filtersOutSeriesWithFewerThanMinPoints() {
|
||||||
|
|||||||
13
康康Tests/SpeechDictationAvailabilityTests.swift
Normal file
13
康康Tests/SpeechDictationAvailabilityTests.swift
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
import Testing
|
||||||
|
@testable import 康康
|
||||||
|
|
||||||
|
struct SpeechDictationAvailabilityTests {
|
||||||
|
/// 模拟器冒烟:isAvailable 的探测路径(SFSpeechRecognizer 构造 + supportsOnDeviceRecognition)
|
||||||
|
/// 不许崩溃、不许卡死;模拟器多为 false(mic 按钮隐藏),个别 Xcode/macOS 组合为 true 也合法,
|
||||||
|
/// 所以只断言「可重复调用且结果稳定」,不断言具体值。
|
||||||
|
@Test func isAvailableProbeIsStableAndDoesNotCrash() {
|
||||||
|
let first = SpeechDictationService.isAvailable
|
||||||
|
let second = SpeechDictationService.isAvailable
|
||||||
|
#expect(first == second)
|
||||||
|
}
|
||||||
|
}
|
||||||
43
康康Tests/TrendInsightCacheTests.swift
Normal file
43
康康Tests/TrendInsightCacheTests.swift
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
import Testing
|
||||||
|
import SwiftUI
|
||||||
|
@testable import 康康
|
||||||
|
|
||||||
|
@MainActor
|
||||||
|
struct TrendInsightCacheTests {
|
||||||
|
|
||||||
|
private func bucket(values: [Double]) -> SeriesBucket {
|
||||||
|
let points = values.enumerated().map { i, v in
|
||||||
|
SeriesBucket.Point(id: "p\(i)",
|
||||||
|
date: Date(timeIntervalSince1970: Double(i) * 86_400),
|
||||||
|
value: v, status: .normal)
|
||||||
|
}
|
||||||
|
let line = SeriesBucket.SeriesLine(id: "glucose.fasting", seriesKey: "glucose.fasting",
|
||||||
|
label: nil, color: .blue, points: points,
|
||||||
|
referenceRange: 3.9...6.1)
|
||||||
|
return SeriesBucket(id: "glucose.fasting", title: "空腹血糖", unit: "mmol/L",
|
||||||
|
lines: [line], latestDate: .now, kind: .monitor)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func fingerprintStableForSameData() {
|
||||||
|
let a = TrendInsightService.fingerprint(for: bucket(values: [5.2, 5.5]))
|
||||||
|
let b = TrendInsightService.fingerprint(for: bucket(values: [5.2, 5.5]))
|
||||||
|
#expect(a == b)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func fingerprintChangesWhenDataChanges() {
|
||||||
|
let a = TrendInsightService.fingerprint(for: bucket(values: [5.2, 5.5]))
|
||||||
|
let b = TrendInsightService.fingerprint(for: bucket(values: [5.2, 5.5, 6.0]))
|
||||||
|
#expect(a != b)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func dataLinesFormatsDateAndValue() {
|
||||||
|
let lines = TrendInsightService.dataLines(for: bucket(values: [5.2, 5.5]))
|
||||||
|
#expect(lines.contains("1970-01-01 5.2"))
|
||||||
|
#expect(lines.contains("1970-01-02 5.5"))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func rangeTextRendersReference() {
|
||||||
|
#expect(TrendInsightService.rangeText(for: bucket(values: [5.2]))
|
||||||
|
== ",参考 3.9-6.1")
|
||||||
|
}
|
||||||
|
}
|
||||||
32
康康Tests/VLPromptsOCRTests.swift
Normal file
32
康康Tests/VLPromptsOCRTests.swift
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
import Testing
|
||||||
|
@testable import 康康
|
||||||
|
|
||||||
|
struct VLPromptsOCRTests {
|
||||||
|
|
||||||
|
@Test func emptyOCRKeepsPromptClean() {
|
||||||
|
let p = VLPrompts.reportExtraction(ocrText: "")
|
||||||
|
#expect(!p.contains("OCR 参考文本"))
|
||||||
|
#expect(!p.contains("{{OCR_SECTION}}"))
|
||||||
|
#expect(p.contains("现在请识别图片并输出 JSON"))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func ocrTextIsInjectedBeforeFinalInstruction() {
|
||||||
|
let p = VLPrompts.reportExtraction(ocrText: "尿酸 486 208-428 μmol/L")
|
||||||
|
#expect(p.contains("OCR 参考文本"))
|
||||||
|
#expect(p.contains("尿酸 486"))
|
||||||
|
let ocrPos = p.range(of: "尿酸 486")!.lowerBound
|
||||||
|
let endPos = p.range(of: "现在请识别图片并输出 JSON")!.lowerBound
|
||||||
|
#expect(ocrPos < endPos)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func clipKeepsShortTextIntact() {
|
||||||
|
#expect(VLPrompts.clipOCR("短文本") == "短文本")
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test func clipCutsAtLineBoundary() {
|
||||||
|
let long = Array(repeating: "指标行 1.23 mmol/L", count: 400).joined(separator: "\n")
|
||||||
|
let clipped = VLPrompts.clipOCR(long, limit: 200)
|
||||||
|
#expect(clipped.count < 260)
|
||||||
|
#expect(clipped.hasSuffix("(后续内容过长已截断)"))
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user