Compare commits

...

9 Commits

Author SHA1 Message Date
link2026
836f3d4234 ```
feat(AI): 统一多模态模型架构,整合文本和视觉推理路径

- 将文本生成和VL(图→文)功能合并到单一的Qwen3.5-4B多模态MNN模型
- 移除独立的Qwen3-VL-4B模型依赖,MLX VL改为使用.llm的多模态模型
- 更新ModelKind枚举,新增userFacing集合用于面向用户展示
- MNN后端现在同时支持文本和视觉任务,模拟器回退到MLX

refactor(models): 模型管理和界面调整以适应新的多模态架构

- 更新模型管理界面,只显示统一的Qwen3.5-4B(MNN)模型给用户
- 修改就绪状态检查逻辑,使用ModelKind.userFacing替代allCases
- 更新模型文件清单,从Qwen3.5-2B升级到Qwen3.5-4B-4bit
- 调整模型管理页面UI,突出MNN+SME2端侧加速功能

feat(camera): 添加拍照识别引擎切换功能

- 实现双路径拍照识别:Apple Vision OCR + 文本模型 和 Qwen3-VL直接识别
- 添加预处理逻辑,优化Qwen3-VL对窄长区域图片的识别效果
- 在模型管理页面添加拍照识别引擎选择组件
- 提供用户界面选项,在两种识别方式间切换

style(ui): 优化输入框样式和颜色主题一致性

- 为指标快速表单添加浅色主题偏好
- 统一所有文本输入框的颜色样式(theme)
- 创建EntryInputField组件,替换原有的单行输入+按钮模式
- 实现聊天框风格的条目输入,支持多行自适应和圆形发送按钮

fix(build): 修正Xcode项目配置中的重复框架搜索路径

- 清理project.pbxproj中重复的FRAMEWORK_SEARCH_PATHS配置
- 重新排列Swift桥接头文件配置确保正确引用
- 修复因路径配置重复导致的编译警告问题

test: 增加区域图片预处理和模型清单测试覆盖

- 添加RegionImageCropper.prepareForQwenVL的单元测试
- 验证宽而矮图片的放大和填充逻辑
- 更新ModelManifestTests中的字节数预期值以匹配新模型
- 修正OCRService中VNRecognizedTextObservation类型的处理
```
2026-06-08 23:25:31 +08:00
link2026
b919404412 fix(MNN): 抑制 MNN 第三方头的 -Wdocumentation 警告
MNN 公共头(Executor/Tensor/Interpreter/ImageProcess.hpp)文档注释不规范,
桥接 #include <MNN/llm/llm.hpp> 时触发 13 条 -Wdocumentation 警告。
用 #pragma clang diagnostic 只在解析 MNN 头时关掉,不影响本项目自身文档警告。

device BUILD SUCCEEDED,警告 0。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 21:29:09 +08:00
link2026
ddfd474bb3 feat(AI): MNN 4B 多模态一肩挑文本+视觉,合并为单模型(MLX 仍兜底)
利用 Qwen3.5-4B-MNN 本身是多模态(含 visual.mnn),让同一个 MNN 模型
同时做文本生成与拍照识别 → MNN 路径只需下 1 个模型(7.4GB→2.64GB)。
MLX(.llm/.vl)保留作兜底,尤其开发机 iPhone 15 Pro(A17 无 SME2)。

- MNN.xcframework 重建为 OMNI(MNN_BUILD_LLM_OMNI=ON,加 OpenCV 图像解码);
  构建脚本同步加 OMNI flag
- MNNLLMBridge.analyzeImages:把图片路径拼成 <img>路径</img> 标签 + response,
  Omni 内部 CV::imread 加载(无需桥接 include OpenCV);与 generateText 共用 runResponse
- MNNBackend.analyze:detached 线程跑 blocking VL 调用,聚合为字符串
- AIRuntime:engine=.mnn 且就绪时,prepareVL→prepareMNN、analyzeReport→mnn.analyze;
  否则回退 MLX VL

device + 模拟器 BUILD SUCCEEDED,0 error,OMNI 框架链接干净。
VL 实际识别质量需真机用化验单 A/B(demo 核心)。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 20:52:58 +08:00
link2026
cbacd9461a feat(AI): MNN 文本模型升到 Qwen3.5-4B(taobao-mnn 预转换)
现场机 iPhone 17(A19/SME2)内存与加速均可承载 4B,质量优于 2B。

- ModelKind.mnnLLM rawValue → "Qwen3.5-4B-MNN",displayName → Qwen3.5-4B (MNN/SME2)
- ModelManifest:7 个运行时文件(llm.mnn.weight ~2.45GB + 拆分的
  visual.mnn.weight 188MB),总计 2,836,770,850 bytes(~2.64GiB)
- ModelManifestTests:文件数 7 / 总字节 / URL 更新到 Qwen3.5-4B-MNN
- CLAUDE.md §2:MNN 主模型记为 Qwen3.5-4B,MLX 兜底仍 2B

模拟器 ModelManifestTests TEST SUCCEEDED。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 20:28:14 +08:00
link2026
39b1521f00 feat(AI): MNN 模型纳入下载体系 ModelKind.mnnLLM(Phase 4)
文本 MNN 模型用 taobao-mnn/Qwen3.5-2B-MNN 官方预转换格式(~1.10GiB),
不再从头转换(避开多模态转文本风险,官方转更可靠)。

- ModelStore.ModelKind 新增 .mnnLLM = "Qwen3.5-2B-MNN"
- ModelManifest:.mnnLLM 文件清单(config.json/llm_config.json/llm.mnn/
  llm.mnn.weight 1.1GB/tokenizer.txt/visual.mnn,HF API 实测字节)
- AIRuntime:mnnModelFolder + 就绪判定改走 ModelStore.isComplete(.mnnLLM)
- ModelManagementView:subtitle 加 .mnnLLM 文案(仅此一处,未动其它 WIP)
- ModelManifestTests:+4 条 mnnLLM 断言(文件数/总字节/必需文件/URL)

模拟器 ModelManifestTests TEST SUCCEEDED。下载经现有链路,需上传到
file.myv0.com/Qwen3.5-2B-MNN/(CDN 清单随附)。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 19:38:16 +08:00
link2026
9da3fbc87e feat(Me): 推理引擎切换页 + SME2 状态 + CLAUDE.md 更新(Phase 5 部分)
- InferenceSettingsView:MNN(CPU/SME2)/ MLX(GPU)单选切换,展示当前设备
  SME2 探测状态(A19 启用 / A17 回退);走设计系统卡片,新文件不动 WIP 的
  ModelManagementView
- MeView:「模型管理」下新增「推理引擎」入口,detail 显示 MNN·SME2 / MNN·CPU / MLX·GPU
- CLAUDE.md §2/§12:AI 运行时改为 MNN(主,SME2)+ MLX(兜底)双后端,
  卖点 #2 明确 MNN+Arm SME2 端侧 CPU 加速为挑战赛考核点

模拟器 BUILD SUCCEEDED,0 error。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 19:23:00 +08:00
link2026
f6c0ba7077 feat(AI): 双后端路由 MNN/MLX,AIRuntime 按引擎分发(Phase 3 核心)
- InferenceEngine:引擎枚举(.mnn 默认 / .mlx 兜底)+ UserDefaults 持久化
  + 可用性/SME2 运行时探测(经 MNNLLMBridge)
- MNNBackend:actor 封装 MNNLLMBridge 文本流式生成,detached 线程跑同步
  response、按 UTF-8 边界 yield TokenChunk,串行化交给 AIRuntime 闸门
- AIRuntime:prepare/generate 按引擎分发;.mnn 且模型就绪→MNN,否则回退 MLX
  (过渡期 App 始终可用);prepareVL/单模型常驻时互卸 MNN↔MLX 释放内存
  公有 API 不变,各 Service 零改动

模拟器 BUILD SUCCEEDED,0 error。引擎切换 UI + SME2 指示留待 Phase 5。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 18:58:27 +08:00
link2026
afc6a79dd7 feat(AI/MNN): 集成 MNN.xcframework + ObjC++ 桥(LLM+SME2,Phase 1-2)
挑战赛考核点要求 Qwen + MNN + SME2 + CPU 端侧推理,MLX(GPU)不满足。
本提交打通原生 MNN 集成的工程层:

- scripts/build-mnn-xcframework.sh:从 alibaba/MNN 源码构建 device+sim arm64
  双切片 xcframework,MNN_BUILD_LLM=ON 导出 llm/llm.hpp,MNN_SME2=ON
  (KleidiAI 运行时自动路由:A19/iPhone17 走 SME2,A17 回退 NEON)
- MNNLLMBridge.{h,mm}:ObjC++ 封装 MNN Llm 的加载/流式生成,streambuf 按
  UTF-8 边界聚合回调,getContext() 取 prefill/decode 算 tok/s;模拟器编为桩
  (走 MLX 兜底),SME2 经 sysctl hw.optional.arm.FEAT_SME2 探测
- pbxproj:链接 MNN.xcframework + bridging header
- 二进制 gitignore,由脚本本地生成防历史膨胀

模拟器 BUILD SUCCEEDED(0 error),xcframework 处理 + 桥编译 + 链接通过。
下一步 Phase 3:MNNBackend + AIRuntime 双后端路由。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 18:31:02 +08:00
link2026
06484d09ff feat(AI): LLM 迁移到 mlx-swift-lm 2.31.3 + Qwen3.5-2B
将 SPM 依赖从 mlx-swift-examples 2.29.1 迁到改名延续仓库 mlx-swift-lm
2.31.3(含 qwen3_5 架构、旧 loadContainer API 兼容),文本 LLM 由
Qwen3-1.7B 换为 Qwen3.5-2B-4bit(走 qwen3_5→Qwen35Model 文本路径)。
连带 mlx-swift 0.29.1→0.31.4,顺修弃用 API:
- MLX.GPU.clearCache() → MLX.Memory.clearCache()
- MLX.GPU.set(cacheLimit:) → MLX.Memory.cacheLimit

更新 ModelManifest(.llm 文件清单+精确字节数,~1.63GiB)、ModelManifestTests、
HealthExport.modelTag 默认值。App BUILD SUCCEEDED + ModelManifestTests 通过。

保留作 MNN 改造的 GPU 兜底基线。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 18:00:28 +08:00
30 changed files with 1230 additions and 90 deletions

3
.gitignore vendored
View File

@@ -2,3 +2,6 @@
/Models/ /Models/
/build/ /build/
.DS_Store .DS_Store
# MNN 预编译二进制:由 scripts/build-mnn-xcframework.sh 本地生成,不入库防历史膨胀
/Frameworks/MNN.xcframework/

View File

@@ -22,9 +22,10 @@
| UI | SwiftUI | iOS 17+,用 `@Observable` / `@Model` | | UI | SwiftUI | iOS 17+,用 `@Observable` / `@Model` |
| 持久化 | SwiftData | 见 §5 数据模型 | | 持久化 | SwiftData | 见 §5 数据模型 |
| 图表 | Swift Charts | iOS 16+ 原生 | | 图表 | Swift Charts | iOS 16+ 原生 |
| **AI 运行时** | **MLX Swift (Apple 官方)** | 不要建议 Core ML / llama.cpp / Ollama | | **AI 运行时(主)** | **MNN (alibaba) + Arm SME2 + CPU** | 挑战赛考核点:Qwen + MNN + SME2 端侧 CPU 推理。device-only(xcframework 见 `scripts/build-mnn-xcframework.sh`),A19/iPhone17 启用 SME2、A17 回退 NEON。经 `MNNLLMBridge`(ObjC++)→ `MNNBackend` |
| LLM | Qwen3-1.7B 4bit (HF: `mlx-community/Qwen3-1.7B-4bit`) | ~1.0GB,负责文本生成、关键词抽取、趋势解读 | | **AI 运行时(兜底)** | **MLX Swift (Apple 官方,Metal GPU)** | 双后端:`InferenceEngine` 切换,模拟器/兜底用 MLX。不要建议 Core ML / llama.cpp / Ollama |
| VL | Qwen2.5-VL-3B-Instruct 4bit (HF: `mlx-community/Qwen2.5-VL-3B-Instruct-4bit`) | ~2.0GB,负责拍照→结构化指标 | | LLM | MNN 主:Qwen3.5-4B(`taobao-mnn/Qwen3.5-4B-MNN`,~2.64GiB);MLX 兜底:Qwen3.5-2B-4bit | 文本生成、关键词抽取、趋势解读 |
| VL | Qwen3-VL-4B-Instruct 4bit (MLX `mlx-community/Qwen3-VL-4B-Instruct-4bit`) | 拍照→结构化指标。MNN VL 需 OMNI 构建,暂走 MLX |
| 文档扫描 | VisionKit `VNDocumentCameraView` | 不要自己写透视校正 | | 文档扫描 | VisionKit `VNDocumentCameraView` | 不要自己写透视校正 |
| Face ID | LocalAuthentication | | | Face ID | LocalAuthentication | |
| Live Activity | ActivityKit + WidgetExtension | demo 杀手锏,真机才能测 | | Live Activity | ActivityKit + WidgetExtension | demo 杀手锏,真机才能测 |
@@ -281,7 +282,7 @@ C2 解读 Tab 底部显示一段 diff 文本,**由 `ReportCompareService` 计算
## 12. 评委 PPT 卖点排序(写代码时记住为什么这么做) ## 12. 评委 PPT 卖点排序(写代码时记住为什么这么做)
1. 影像档案系统(统一 VL 拍照 + 归档) — 核心创意 1. 影像档案系统(统一 VL 拍照 + 归档) — 核心创意
2. 100% 本地 + SME2 加速 — 技术亮点 2. 100% 本地 + **MNN + Arm SME2 端侧 CPU 加速**(挑战赛考核点,MLX/GPU 兜底) — 技术亮点
3. 本地 RAG 长期记忆 — 端侧不可替代性 3. 本地 RAG 长期记忆 — 端侧不可替代性
4. 隐私三件套(系统级加密 + Face ID + 永久删除) — 信任建立 4. 隐私三件套(系统级加密 + Face ID + 永久删除) — 信任建立
5. AI 趋势解读 — 长期价值 5. AI 趋势解读 — 长期价值

View File

@@ -0,0 +1,51 @@
#!/bin/sh
# 构建 MNN.xcframework(device arm64 + simulator arm64),含 LLM 引擎 + SME2。
# 产物输出到 康康/../Frameworks/MNN.xcframework(被 .gitignore,不入库,防历史膨胀)。
#
# 用法:
# MNN_SRC=/path/to/MNN sh scripts/build-mnn-xcframework.sh
# 需求:CMake 3.14+、Xcode、约 10-40 分钟。
#
# 关键 flag:
# MNN_BUILD_LLM=ON —— 编入 llm 引擎(并导出 llm/llm.hpp),自动开 MNN_LOW_MEMORY
# MNN_BUILD_LLM_OMNI=ON —— VL(图→文)所需:多模态 Omni + OpenCV 图像解码。
# 统一模型(Qwen3.5-4B-MNN 一肩挑文本+视觉)必须开。
# MNN_SME2=ON —— CMake 默认 ON,A19/iPhone17 运行时经 KleidiAI 自动启用,A17 回退 NEON
# MNN_METAL=OFF —— 考核走 CPU+SME2,关 Metal 保持精简
set -e
MNN_SRC="${MNN_SRC:-/Users/xuhuayong/apps/MNN-src}"
OUT_DIR="$(cd "$(dirname "$0")/.." && pwd)/Frameworks"
TOOLCHAIN_NEW="${MNN_SRC}/cmake/ios.toolchain.new.cmake"
EXTRA="-DMNN_BUILD_LLM=ON -DMNN_BUILD_LLM_OMNI=ON -DMNN_METAL=OFF -DMNN_ARM82=true -DMNN_SME2=ON"
COMMON="-DCMAKE_BUILD_TYPE=Release -DENABLE_BITCODE=0 -DMNN_AAPL_FMWK=1 -DMNN_SEP_BUILD=0 -DMNN_BUILD_SHARED_LIBS=false -DMNN_USE_THREAD_POOL=OFF"
export DEVELOPER_DIR="/Applications/Xcode.app/Contents/Developer"
cd "$MNN_SRC"
# 新版 ios-cmake toolchain(支持 SIMULATORARM64;MNN 自带的旧版只支持 x86_64 模拟器)
if [ ! -f "$TOOLCHAIN_NEW" ]; then
curl -sL "https://raw.githubusercontent.com/leetal/ios-cmake/master/ios.toolchain.cmake" -o "$TOOLCHAIN_NEW"
fi
# device arm64
rm -rf build-dev-arm64 && mkdir build-dev-arm64 && cd build-dev-arm64
cmake .. $COMMON $EXTRA -DCMAKE_TOOLCHAIN_FILE="$TOOLCHAIN_NEW" -DPLATFORM=OS64 -DDEPLOYMENT_TARGET=17.0
make MNN -j16
cd ..
# simulator arm64
rm -rf build-sim-arm64 && mkdir build-sim-arm64 && cd build-sim-arm64
cmake .. $COMMON $EXTRA -DCMAKE_TOOLCHAIN_FILE="$TOOLCHAIN_NEW" -DPLATFORM=SIMULATORARM64 -DDEPLOYMENT_TARGET=17.0
make MNN -j16
cd ..
# 合成 xcframework
rm -rf "$OUT_DIR/MNN.xcframework"
mkdir -p "$OUT_DIR"
xcrun xcodebuild -create-xcframework \
-framework build-dev-arm64/MNN.framework \
-framework build-sim-arm64/MNN.framework \
-output "$OUT_DIR/MNN.xcframework"
echo "✅ 输出: $OUT_DIR/MNN.xcframework"

View File

@@ -10,6 +10,7 @@
FEED000000000000DEAD0001 /* MLXLLM in Frameworks */ = {isa = PBXBuildFile; productRef = FEED000000000000DEAD0003 /* MLXLLM */; }; FEED000000000000DEAD0001 /* MLXLLM in Frameworks */ = {isa = PBXBuildFile; productRef = FEED000000000000DEAD0003 /* MLXLLM */; };
FEED000000000000DEAD0002 /* MLXLMCommon in Frameworks */ = {isa = PBXBuildFile; productRef = FEED000000000000DEAD0004 /* MLXLMCommon */; }; FEED000000000000DEAD0002 /* MLXLMCommon in Frameworks */ = {isa = PBXBuildFile; productRef = FEED000000000000DEAD0004 /* MLXLMCommon */; };
FEED000000000000DEAD0005 /* MLXVLM in Frameworks */ = {isa = PBXBuildFile; productRef = FEED000000000000DEAD0006 /* MLXVLM */; }; FEED000000000000DEAD0005 /* MLXVLM in Frameworks */ = {isa = PBXBuildFile; productRef = FEED000000000000DEAD0006 /* MLXVLM */; };
FEEDFACE000000000000F002 /* MNN.xcframework in Frameworks */ = {isa = PBXBuildFile; fileRef = FEEDFACE000000000000F001 /* MNN.xcframework */; };
/* End PBXBuildFile section */ /* End PBXBuildFile section */
/* Begin PBXContainerItemProxy section */ /* Begin PBXContainerItemProxy section */
@@ -33,6 +34,7 @@
5E463CF92FC403BB0089145B /* 康康.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = "康康.app"; sourceTree = BUILT_PRODUCTS_DIR; }; 5E463CF92FC403BB0089145B /* 康康.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = "康康.app"; sourceTree = BUILT_PRODUCTS_DIR; };
5E463D082FC403BC0089145B /* 康康Tests.xctest */ = {isa = PBXFileReference; explicitFileType = wrapper.cfbundle; includeInIndex = 0; path = "康康Tests.xctest"; sourceTree = BUILT_PRODUCTS_DIR; }; 5E463D082FC403BC0089145B /* 康康Tests.xctest */ = {isa = PBXFileReference; explicitFileType = wrapper.cfbundle; includeInIndex = 0; path = "康康Tests.xctest"; sourceTree = BUILT_PRODUCTS_DIR; };
5E463D122FC403BC0089145B /* 康康UITests.xctest */ = {isa = PBXFileReference; explicitFileType = wrapper.cfbundle; includeInIndex = 0; path = "康康UITests.xctest"; sourceTree = BUILT_PRODUCTS_DIR; }; 5E463D122FC403BC0089145B /* 康康UITests.xctest */ = {isa = PBXFileReference; explicitFileType = wrapper.cfbundle; includeInIndex = 0; path = "康康UITests.xctest"; sourceTree = BUILT_PRODUCTS_DIR; };
FEEDFACE000000000000F001 /* MNN.xcframework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.xcframework; name = MNN.xcframework; path = Frameworks/MNN.xcframework; sourceTree = "<group>"; };
/* End PBXFileReference section */ /* End PBXFileReference section */
/* Begin PBXFileSystemSynchronizedRootGroup section */ /* Begin PBXFileSystemSynchronizedRootGroup section */
@@ -61,6 +63,7 @@
FEED000000000000DEAD0001 /* MLXLLM in Frameworks */, FEED000000000000DEAD0001 /* MLXLLM in Frameworks */,
FEED000000000000DEAD0002 /* MLXLMCommon in Frameworks */, FEED000000000000DEAD0002 /* MLXLMCommon in Frameworks */,
FEED000000000000DEAD0005 /* MLXVLM in Frameworks */, FEED000000000000DEAD0005 /* MLXVLM in Frameworks */,
FEEDFACE000000000000F002 /* MNN.xcframework in Frameworks */,
); );
runOnlyForDeploymentPostprocessing = 0; runOnlyForDeploymentPostprocessing = 0;
}; };
@@ -88,6 +91,7 @@
5E463D0B2FC403BC0089145B /* 康康Tests */, 5E463D0B2FC403BC0089145B /* 康康Tests */,
5E463D152FC403BC0089145B /* 康康UITests */, 5E463D152FC403BC0089145B /* 康康UITests */,
5E463CFA2FC403BB0089145B /* Products */, 5E463CFA2FC403BB0089145B /* Products */,
FEEDFACE000000000000F001 /* MNN.xcframework */,
); );
sourceTree = "<group>"; sourceTree = "<group>";
}; };
@@ -211,7 +215,7 @@
mainGroup = 5E463CF02FC403BB0089145B; mainGroup = 5E463CF02FC403BB0089145B;
minimizedProjectReferenceProxies = 1; minimizedProjectReferenceProxies = 1;
packageReferences = ( packageReferences = (
5E9A1F872FC43C9A0097DD29 /* XCRemoteSwiftPackageReference "mlx-swift-examples" */, 5E9A1F872FC43C9A0097DD29 /* XCRemoteSwiftPackageReference "mlx-swift-lm" */,
); );
preferredProjectObjectVersion = 77; preferredProjectObjectVersion = 77;
productRefGroup = 5E463CFA2FC403BB0089145B /* Products */; productRefGroup = 5E463CFA2FC403BB0089145B /* Products */;
@@ -416,6 +420,7 @@
ENABLE_HARDENED_RUNTIME = YES; ENABLE_HARDENED_RUNTIME = YES;
ENABLE_PREVIEWS = YES; ENABLE_PREVIEWS = YES;
ENABLE_USER_SELECTED_FILES = readonly; ENABLE_USER_SELECTED_FILES = readonly;
FRAMEWORK_SEARCH_PATHS = "$(PROJECT_DIR)/Frameworks";
GENERATE_INFOPLIST_FILE = YES; GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_KEY_CFBundleDisplayName = "康康"; INFOPLIST_KEY_CFBundleDisplayName = "康康";
INFOPLIST_KEY_ITSAppUsesNonExemptEncryption = NO; INFOPLIST_KEY_ITSAppUsesNonExemptEncryption = NO;
@@ -450,6 +455,7 @@
SWIFT_APPROACHABLE_CONCURRENCY = YES; SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_DEFAULT_ACTOR_ISOLATION = MainActor; SWIFT_DEFAULT_ACTOR_ISOLATION = MainActor;
SWIFT_EMIT_LOC_STRINGS = YES; SWIFT_EMIT_LOC_STRINGS = YES;
SWIFT_OBJC_BRIDGING_HEADER = "康康/康康-Bridging-Header.h";
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES; SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0; SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2"; TARGETED_DEVICE_FAMILY = "1,2";
@@ -470,6 +476,7 @@
ENABLE_HARDENED_RUNTIME = YES; ENABLE_HARDENED_RUNTIME = YES;
ENABLE_PREVIEWS = YES; ENABLE_PREVIEWS = YES;
ENABLE_USER_SELECTED_FILES = readonly; ENABLE_USER_SELECTED_FILES = readonly;
FRAMEWORK_SEARCH_PATHS = "$(PROJECT_DIR)/Frameworks";
GENERATE_INFOPLIST_FILE = YES; GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_KEY_CFBundleDisplayName = "康康"; INFOPLIST_KEY_CFBundleDisplayName = "康康";
INFOPLIST_KEY_ITSAppUsesNonExemptEncryption = NO; INFOPLIST_KEY_ITSAppUsesNonExemptEncryption = NO;
@@ -504,6 +511,7 @@
SWIFT_APPROACHABLE_CONCURRENCY = YES; SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_DEFAULT_ACTOR_ISOLATION = MainActor; SWIFT_DEFAULT_ACTOR_ISOLATION = MainActor;
SWIFT_EMIT_LOC_STRINGS = YES; SWIFT_EMIT_LOC_STRINGS = YES;
SWIFT_OBJC_BRIDGING_HEADER = "康康/康康-Bridging-Header.h";
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES; SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0; SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2"; TARGETED_DEVICE_FAMILY = "1,2";
@@ -659,12 +667,12 @@
/* End XCConfigurationList section */ /* End XCConfigurationList section */
/* Begin XCRemoteSwiftPackageReference section */ /* Begin XCRemoteSwiftPackageReference section */
5E9A1F872FC43C9A0097DD29 /* XCRemoteSwiftPackageReference "mlx-swift-examples" */ = { 5E9A1F872FC43C9A0097DD29 /* XCRemoteSwiftPackageReference "mlx-swift-lm" */ = {
isa = XCRemoteSwiftPackageReference; isa = XCRemoteSwiftPackageReference;
repositoryURL = "https://github.com/ml-explore/mlx-swift-examples"; repositoryURL = "https://github.com/ml-explore/mlx-swift-lm";
requirement = { requirement = {
kind = upToNextMajorVersion; kind = exactVersion;
minimumVersion = 2.29.1; version = 2.31.3;
}; };
}; };
/* End XCRemoteSwiftPackageReference section */ /* End XCRemoteSwiftPackageReference section */
@@ -672,17 +680,17 @@
/* Begin XCSwiftPackageProductDependency section */ /* Begin XCSwiftPackageProductDependency section */
FEED000000000000DEAD0003 /* MLXLLM */ = { FEED000000000000DEAD0003 /* MLXLLM */ = {
isa = XCSwiftPackageProductDependency; isa = XCSwiftPackageProductDependency;
package = 5E9A1F872FC43C9A0097DD29 /* XCRemoteSwiftPackageReference "mlx-swift-examples" */; package = 5E9A1F872FC43C9A0097DD29 /* XCRemoteSwiftPackageReference "mlx-swift-lm" */;
productName = MLXLLM; productName = MLXLLM;
}; };
FEED000000000000DEAD0004 /* MLXLMCommon */ = { FEED000000000000DEAD0004 /* MLXLMCommon */ = {
isa = XCSwiftPackageProductDependency; isa = XCSwiftPackageProductDependency;
package = 5E9A1F872FC43C9A0097DD29 /* XCRemoteSwiftPackageReference "mlx-swift-examples" */; package = 5E9A1F872FC43C9A0097DD29 /* XCRemoteSwiftPackageReference "mlx-swift-lm" */;
productName = MLXLMCommon; productName = MLXLMCommon;
}; };
FEED000000000000DEAD0006 /* MLXVLM */ = { FEED000000000000DEAD0006 /* MLXVLM */ = {
isa = XCSwiftPackageProductDependency; isa = XCSwiftPackageProductDependency;
package = 5E9A1F872FC43C9A0097DD29 /* XCRemoteSwiftPackageReference "mlx-swift-examples" */; package = 5E9A1F872FC43C9A0097DD29 /* XCRemoteSwiftPackageReference "mlx-swift-lm" */;
productName = MLXVLM; productName = MLXVLM;
}; };
/* End XCSwiftPackageProductDependency section */ /* End XCSwiftPackageProductDependency section */

View File

@@ -1,13 +1,13 @@
{ {
"originHash" : "6b8265ebd61c6fdfca835dd1f90f17439ca9abc5c11a8b7b5db8790be0349e4d", "originHash" : "facc0ac7c70363ea20f6cd1235de91dea6b06f0d00190946045a6c8ae753abc2",
"pins" : [ "pins" : [
{ {
"identity" : "gzipswift", "identity" : "eventsource",
"kind" : "remoteSourceControl", "kind" : "remoteSourceControl",
"location" : "https://github.com/1024jp/GzipSwift", "location" : "https://github.com/mattt/EventSource.git",
"state" : { "state" : {
"revision" : "731037f6cc2be2ec01562f6597c1d0aa3fe6fd05", "revision" : "a3a85a85214caf642abaa96ae664e4c772a59f6e",
"version" : "6.0.1" "version" : "1.4.1"
} }
}, },
{ {
@@ -15,17 +15,35 @@
"kind" : "remoteSourceControl", "kind" : "remoteSourceControl",
"location" : "https://github.com/ml-explore/mlx-swift", "location" : "https://github.com/ml-explore/mlx-swift",
"state" : { "state" : {
"revision" : "072b684acaae80b6a463abab3a103732f33774bf", "revision" : "dc43e62d7055353c7f99fa071a4e71d29dfddc44",
"version" : "0.29.1" "version" : "0.31.4"
} }
}, },
{ {
"identity" : "mlx-swift-examples", "identity" : "mlx-swift-lm",
"kind" : "remoteSourceControl", "kind" : "remoteSourceControl",
"location" : "https://github.com/ml-explore/mlx-swift-examples", "location" : "https://github.com/ml-explore/mlx-swift-lm",
"state" : { "state" : {
"revision" : "9bff95ca5f0b9e8c021acc4d71a2bbe4a7441631", "revision" : "25b00d4e22e61ec9c41efda47990cd2084ec87ff",
"version" : "2.29.1" "version" : "2.31.3"
}
},
{
"identity" : "swift-asn1",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-asn1.git",
"state" : {
"revision" : "eb50cbd14606a9161cbc5d452f18797c90ef0bab",
"version" : "1.7.0"
}
},
{
"identity" : "swift-atomics",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-atomics.git",
"state" : {
"revision" : "b601256eab081c0f92f059e12818ac1d4f178ff7",
"version" : "1.3.0"
} }
}, },
{ {
@@ -37,6 +55,24 @@
"version" : "1.5.1" "version" : "1.5.1"
} }
}, },
{
"identity" : "swift-crypto",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-crypto.git",
"state" : {
"revision" : "1b6b2e274e85105bfa155183145a1dcfd63331f1",
"version" : "4.5.0"
}
},
{
"identity" : "swift-huggingface",
"kind" : "remoteSourceControl",
"location" : "https://github.com/huggingface/swift-huggingface.git",
"state" : {
"revision" : "b721959445b617d0bf03910b2b4aced345fd93bf",
"version" : "0.9.0"
}
},
{ {
"identity" : "swift-jinja", "identity" : "swift-jinja",
"kind" : "remoteSourceControl", "kind" : "remoteSourceControl",
@@ -46,6 +82,15 @@
"version" : "2.3.6" "version" : "2.3.6"
} }
}, },
{
"identity" : "swift-nio",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio.git",
"state" : {
"revision" : "57c0a08a331aaea9f5d7a932ad94ef43be942a95",
"version" : "2.100.0"
}
},
{ {
"identity" : "swift-numerics", "identity" : "swift-numerics",
"kind" : "remoteSourceControl", "kind" : "remoteSourceControl",
@@ -55,13 +100,31 @@
"version" : "1.1.1" "version" : "1.1.1"
} }
}, },
{
"identity" : "swift-system",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-system.git",
"state" : {
"revision" : "669763cfd5806a67e21972d7e5e2d6b80b1ea985",
"version" : "1.6.5"
}
},
{ {
"identity" : "swift-transformers", "identity" : "swift-transformers",
"kind" : "remoteSourceControl", "kind" : "remoteSourceControl",
"location" : "https://github.com/huggingface/swift-transformers", "location" : "https://github.com/huggingface/swift-transformers",
"state" : { "state" : {
"revision" : "a2e184dddb4757bc943e77fbe99ac6786c53f0b2", "revision" : "58c4bc11963a140358d791f678a60a2745a23146",
"version" : "1.0.0" "version" : "1.2.1"
}
},
{
"identity" : "yyjson",
"kind" : "remoteSourceControl",
"location" : "https://github.com/ibireme/yyjson.git",
"state" : {
"revision" : "8b4a38dc994a110abaec8a400615567bd996105f",
"version" : "0.12.0"
} }
} }
], ],

View File

@@ -32,6 +32,16 @@ actor AIRuntime {
private var llmSession: LLMSession? private var llmSession: LLMSession?
private var vlSession: VLSession? private var vlSession: VLSession?
// MARK: - MNN (CPU/SME2,)
// .mnn , VL() Qwen3.5-4B MNN ()
// MNN,VL 退 MLX Qwen3-VL-4B
private let mnn = MNNBackend()
private(set) var mnnStatus: Status = .notReady
/// MNN (/ Models/Qwen3.5-4B-MNN)
nonisolated static var mnnModelFolder: URL {
ModelStore.shared.localURL(for: .mnnLLM)
}
// MARK: - (§3.1 OOM ) // MARK: - (§3.1 OOM )
// //
// actor , generate() Task; // actor , generate() Task;
@@ -74,12 +84,22 @@ actor AIRuntime {
nonisolated static func configureMLXMemory() { nonisolated static func configureMLXMemory() {
#if !targetEnvironment(simulator) #if !targetEnvironment(simulator)
// 256MB cache : 3GB MB // 256MB cache : 3GB MB
MLX.GPU.set(cacheLimit: 256 * 1024 * 1024) MLX.Memory.cacheLimit = 256 * 1024 * 1024
#endif #endif
} }
/// , /// ,
/// :.mnn MNN(CPU/SME2);.mlx MLX(GPU)
func prepare() async throws { func prepare() async throws {
// MNN MNN;( MLX, MNN )退 MLX,
// App (Phase 5)
let mnnReady = ModelStore.shared.isComplete(for: .mnnLLM)
if InferenceEngine.current == .mnn, mnnReady {
try await prepareMNN()
return
}
// MLX: MNN ()
await unloadMNN()
// , // ,
// return: ready, generate // return: ready, generate
// `guard status == .ready` () // `guard status == .ready` ()
@@ -119,9 +139,51 @@ actor AIRuntime {
} }
} }
/// MNN : MLX LLM/VL
private func prepareMNN() async throws {
while mnnStatus == .loading {
try await Task.sleep(nanoseconds: 80_000_000)
}
if mnnStatus == .ready { return }
let folder = Self.mnnModelFolder
guard ModelStore.shared.isComplete(for: .mnnLLM) else {
mnnStatus = .error("MNN 模型未就绪")
throw AIRuntimeError.notReady
}
await acquireGate()
defer { releaseGate() }
if mnnStatus == .ready { return }
// : MLX LLM/VL, MNN
unloadLLM()
unloadVL()
mnnStatus = .loading
do {
try await mnn.load(folderURL: folder)
mnnStatus = .ready
} catch {
mnnStatus = .error("\(error)")
throw AIRuntimeError.modelLoadFailed("\(error)")
}
}
/// MNN,
private func unloadMNN() async {
guard mnnStatus != .notReady else { return }
await mnn.unload()
mnnStatus = .notReady
MLX.Memory.clearCache()
}
/// await prepare() /// await prepare()
/// :, actor LLMSession await /// :, actor LLMSession await
func generate(prompt: String, maxTokens: Int = 256) -> AsyncThrowingStream<TokenChunk, Error> { func generate(prompt: String, maxTokens: Int = 256) -> AsyncThrowingStream<TokenChunk, Error> {
if InferenceEngine.current == .mnn, mnnStatus == .ready {
return mnnGenerate(prompt: prompt, maxTokens: maxTokens)
}
// actor ,Task 访 self.status / self.llmSession // actor ,Task 访 self.status / self.llmSession
let snapshotStatus = status let snapshotStatus = status
let snapshotSession = llmSession let snapshotSession = llmSession
@@ -159,6 +221,33 @@ actor AIRuntime {
} }
} }
/// MNN(CPU/SME2) MLX :
private func mnnGenerate(prompt: String, maxTokens: Int) -> AsyncThrowingStream<TokenChunk, Error> {
let ready = (mnnStatus == .ready)
return AsyncThrowingStream { continuation in
let task = Task {
guard ready else {
continuation.finish(throwing: AIRuntimeError.notReady)
return
}
await self.acquireGate()
do {
let stream = await self.mnn.generate(prompt: prompt, maxTokens: maxTokens)
for try await chunk in stream {
try Task.checkCancellation()
self.recordRate(chunk.decodeRate)
continuation.yield(chunk)
}
continuation.finish()
} catch {
continuation.finish(throwing: AIRuntimeError.inferenceFailed("\(error)"))
}
self.releaseGate()
}
continuation.onTermination = { _ in task.cancel() }
}
}
private func recordRate(_ rate: Double) { private func recordRate(_ rate: Double) {
if rate > 0 { lastDecodeRate = rate } if rate > 0 { lastDecodeRate = rate }
} }
@@ -167,13 +256,19 @@ actor AIRuntime {
/// VL , load /// VL , load
func prepareVL() async throws { func prepareVL() async throws {
// MNN :VL MNN (+), prepareMNN
if InferenceEngine.current == .mnn, ModelStore.shared.isComplete(for: .mnnLLM) {
try await prepareMNN()
return
}
while vlStatus == .loading { while vlStatus == .loading {
try await Task.sleep(nanoseconds: 80_000_000) try await Task.sleep(nanoseconds: 80_000_000)
} }
if vlStatus == .ready { return } if vlStatus == .ready { return }
// prepare(): isComplete (), // MLX VL .llm Qwen3.5-4B (VLMModelFactory qwen3_5 ),
guard ModelStore.shared.isComplete(for: .vl) else { // Qwen3-VL-4B isComplete ,
guard ModelStore.shared.isComplete(for: .llm) else {
vlStatus = .error("VL 模型未就绪") vlStatus = .error("VL 模型未就绪")
throw AIRuntimeError.notReady throw AIRuntimeError.notReady
} }
@@ -186,11 +281,12 @@ actor AIRuntime {
// OOM (§3.1): VL(~3GB) LLM(~1GB), jetsam // OOM (§3.1): VL(~3GB) LLM(~1GB), jetsam
unloadLLM() unloadLLM()
await unloadMNN()
vlStatus = .loading vlStatus = .loading
do { do {
let session = try await VLSession.load( let session = try await VLSession.load(
folderURL: ModelStore.shared.localURL(for: .vl) folderURL: ModelStore.shared.localURL(for: .llm)
) )
self.vlSession = session self.vlSession = session
vlStatus = .ready vlStatus = .ready
@@ -208,7 +304,7 @@ actor AIRuntime {
guard llmSession != nil else { return } guard llmSession != nil else { return }
llmSession = nil llmSession = nil
status = .notReady status = .notReady
MLX.GPU.clearCache() MLX.Memory.clearCache()
} }
/// VL, ModelContainer MLX /// VL, ModelContainer MLX
@@ -216,7 +312,7 @@ actor AIRuntime {
guard vlSession != nil else { return } guard vlSession != nil else { return }
vlSession = nil vlSession = nil
vlStatus = .notReady vlStatus = .notReady
MLX.GPU.clearCache() MLX.Memory.clearCache()
} }
/// JSON ( VLPrompts.reportExtraction ) /// JSON ( VLPrompts.reportExtraction )
@@ -225,6 +321,16 @@ actor AIRuntime {
func analyzeReport(imageURLs: [URL], func analyzeReport(imageURLs: [URL],
prompt: String, prompt: String,
maxTokens: Int = 512) async throws -> String { maxTokens: Int = 512) async throws -> String {
// MNN : MNN
if InferenceEngine.current == .mnn, mnnStatus == .ready {
await acquireGate()
defer { releaseGate() }
do {
return try await mnn.analyze(imageURLs: imageURLs, prompt: prompt, maxTokens: maxTokens)
} catch {
throw AIRuntimeError.inferenceFailed("\(error)")
}
}
guard vlStatus == .ready, let session = vlSession else { guard vlStatus == .ready, let session = vlSession else {
throw AIRuntimeError.notReady throw AIRuntimeError.notReady
} }

View File

@@ -0,0 +1,41 @@
import Foundation
///
/// - mnn:Qwen + MNN + SME2(CPU),,
/// - mlx:Qwen + MLX(Metal GPU), /
nonisolated enum InferenceEngine: String, CaseIterable, Sendable {
case mnn
case mlx
var displayName: String {
switch self {
case .mnn: return "MNN · CPU/SME2"
case .mlx: return "MLX · GPU"
}
}
/// /MNN device ,退 MLX
var isAvailable: Bool {
switch self {
case .mlx: return true
case .mnn: return MNNLLMBridge.isAvailable()
}
}
// MARK: - (UserDefaults, actor )
private static let key = "kk.inferenceEngine"
/// /退 .mlx() .mnn
static var current: InferenceEngine {
get {
let raw = UserDefaults.standard.string(forKey: key)
let chosen = raw.flatMap(InferenceEngine.init(rawValue:)) ?? .mnn
return chosen.isAvailable ? chosen : .mlx
}
set { UserDefaults.standard.set(newValue.rawValue, forKey: key) }
}
/// :CPU SME2(A19/iPhone17+) UI
static var cpuSupportsSME2: Bool { MNNLLMBridge.cpuSupportsSME2() }
}

View File

@@ -0,0 +1,55 @@
//
// MNNLLMBridge.h
// 康康
//
// Objective-C 接口,封装 MNN-LLM(Qwen)的加载与流式推理。
// 真实实现在 .mm 中以 ObjC++ 调用 <MNN/llm/llm.hpp>;模拟器下编为可用性返回 NO 的桩
// (MNN.framework 仅 device arm64 切片有真实 CPU/SME2 内核,模拟器走 MLX 兜底)。
//
#import <Foundation/Foundation.h>
NS_ASSUME_NONNULL_BEGIN
/// 末次生成的性能统计(取自 MNN LlmContext)。
@interface MNNGenerateStats : NSObject
@property (nonatomic, readonly) int promptTokens;
@property (nonatomic, readonly) int genTokens;
@property (nonatomic, readonly) double prefillMs;
@property (nonatomic, readonly) double decodeMs;
/// 解码速率 tok/s = genTokens / (decodeMs/1000)。demo 卖点 #6 / Live Activity 用。
@property (nonatomic, readonly) double decodeTokensPerSecond;
@end
@interface MNNLLMBridge : NSObject
/// 本构建是否含真实 MNN 运行时(device=YES,simulator 桩=NO)。
+ (BOOL)isAvailable;
/// CPU 是否支持 SME2(运行时探测);A19/iPhone17 YES,A17/iPhone15Pro NO。仅用于 UI 展示加速状态。
+ (BOOL)cpuSupportsSME2;
/// 用 MNN llm 的 config.json 路径加载模型(目录含 llm.mnn / 权重 / tokenizer)。失败返回 nil。
- (nullable instancetype)initWithConfigPath:(NSString *)configPath;
@property (nonatomic, readonly) BOOL isLoaded;
/// 纯文本流式生成。onToken 每解码出一段文本回调一次(在调用线程,同步阻塞直到生成结束)。
/// 返回末次统计。
- (MNNGenerateStats *)generateText:(NSString *)prompt
maxTokens:(int)maxTokens
onToken:(void (^)(NSString *piece))onToken;
/// 图→文(VL,需 MNN_BUILD_LLM_OMNI 构建)。imagePaths 为本地文件路径。
/// 当前文本构建未含 OMNI 时返回 nil 并置 error。
- (nullable MNNGenerateStats *)analyzeImages:(NSArray<NSString *> *)imagePaths
prompt:(NSString *)prompt
maxTokens:(int)maxTokens
onToken:(void (^)(NSString *piece))onToken
error:(NSError *_Nullable *_Nullable)error;
/// 请求取消当前生成(best-effort:置标志,后续 token 不再回调)。
- (void)cancel;
@end
NS_ASSUME_NONNULL_END

View File

@@ -0,0 +1,194 @@
//
// MNNLLMBridge.mm
// 康康
//
// ObjC++ 实现。device 真机用 <MNN/llm/llm.hpp>;模拟器编为桩(返回不可用,上层回退 MLX)。
//
#import "MNNLLMBridge.h"
#include <sys/sysctl.h>
// MARK: - 性能统计(私有 readwrite 重声明)
@interface MNNGenerateStats ()
@property (nonatomic, readwrite) int promptTokens;
@property (nonatomic, readwrite) int genTokens;
@property (nonatomic, readwrite) double prefillMs;
@property (nonatomic, readwrite) double decodeMs;
@end
@implementation MNNGenerateStats
- (double)decodeTokensPerSecond {
return self.decodeMs > 0 ? (self.genTokens / (self.decodeMs / 1000.0)) : 0;
}
@end
// MARK: - SME2 / 可用性探测(device + simulator 都可编)
static BOOL kk_sysctlFlag(const char *name) {
int64_t v = 0; size_t sz = sizeof(v);
if (sysctlbyname(name, &v, &sz, NULL, 0) != 0) return NO;
return v != 0;
}
#if TARGET_OS_SIMULATOR
// ============ 模拟器桩:无真实 MNN ============
@implementation MNNLLMBridge
+ (BOOL)isAvailable { return NO; }
+ (BOOL)cpuSupportsSME2 { return NO; }
- (nullable instancetype)initWithConfigPath:(NSString *)configPath { return nil; }
- (BOOL)isLoaded { return NO; }
- (MNNGenerateStats *)generateText:(NSString *)prompt maxTokens:(int)maxTokens
onToken:(void (^)(NSString *))onToken { return [MNNGenerateStats new]; }
- (nullable MNNGenerateStats *)analyzeImages:(NSArray<NSString *> *)imagePaths prompt:(NSString *)prompt
maxTokens:(int)maxTokens onToken:(void (^)(NSString *))onToken
error:(NSError **)error {
if (error) *error = [NSError errorWithDomain:@"MNN" code:-1
userInfo:@{NSLocalizedDescriptionKey: @"MNN 在模拟器不可用"}];
return nil;
}
- (void)cancel {}
@end
#else
// ============ 真机:真实 MNN-LLM ============
// MNN 第三方头文件的文档注释不规范,会触发一堆 -Wdocumentation 警告(Executor/
// Tensor/Interpreter/ImageProcess.hpp)。只在解析 MNN 头时关掉该警告,不影响本项目。
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wdocumentation"
#include <MNN/llm/llm.hpp>
#pragma clang diagnostic pop
#include <string>
#include <ostream>
#include <streambuf>
#include <atomic>
using MNN::Transformer::Llm;
namespace {
/// 把 MNN 写入 ostream 的解码文本转成 NSString 回调;按 UTF-8 完整边界聚合,避免截断多字节。
class TokenStreamBuf : public std::streambuf {
public:
TokenStreamBuf(void (^onToken)(NSString *), std::atomic<bool> *cancel)
: _onToken(onToken), _cancel(cancel) {}
void flush() {
if (_pending.empty()) return;
emitPending(); // 末尾尽力 emit(即便非完整 UTF-8 也交出去)
_pending.clear();
}
protected:
std::streamsize xsputn(const char *s, std::streamsize n) override {
append(s, (size_t)n);
return n;
}
int overflow(int c) override {
if (c != EOF) { char ch = (char)c; append(&ch, 1); }
return c;
}
private:
void append(const char *s, size_t n) {
if (_cancel && _cancel->load()) return; // 已取消,吞掉不回调
_pending.append(s, n);
// 仅当整个 pending 是合法 UTF-8 才 emit(token 通常是完整字/词,边界自然对齐)
NSString *str = [[NSString alloc] initWithBytes:_pending.data()
length:_pending.size()
encoding:NSUTF8StringEncoding];
if (str) { if (_onToken) _onToken(str); _pending.clear(); }
}
void emitPending() {
NSString *str = [[NSString alloc] initWithBytes:_pending.data()
length:_pending.size()
encoding:NSUTF8StringEncoding];
if (str && _onToken) _onToken(str);
}
void (^_onToken)(NSString *);
std::atomic<bool> *_cancel;
std::string _pending;
};
} // namespace
@implementation MNNLLMBridge {
Llm *_llm;
std::atomic<bool> _cancel;
BOOL _loaded;
}
+ (BOOL)isAvailable { return YES; }
+ (BOOL)cpuSupportsSME2 {
// Apple 通过 sysctl 暴露 ARM 特性位:FEAT_SME2(A19/iPhone17+)。
return kk_sysctlFlag("hw.optional.arm.FEAT_SME2");
}
- (nullable instancetype)initWithConfigPath:(NSString *)configPath {
self = [super init];
if (!self) return nil;
_cancel = false;
_llm = Llm::createLLM(std::string(configPath.UTF8String));
if (_llm == nullptr) return nil;
_loaded = _llm->load();
if (!_loaded) { Llm::destroy(_llm); _llm = nullptr; return nil; }
return self;
}
- (void)dealloc {
if (_llm) { Llm::destroy(_llm); _llm = nullptr; }
}
- (BOOL)isLoaded { return _loaded; }
- (void)cancel { _cancel = true; }
// 统一生成:full 已是最终 prompt(文本,或含 <img>路径</img> 标签)。
// 多模态模型 createLLM 返回 Omni,response 解析 <img> 标签并对路径 CV::imread(OMNI 框架内)。
- (MNNGenerateStats *)runResponse:(NSString *)full
maxTokens:(int)maxTokens
onToken:(void (^)(NSString *))onToken {
_cancel = false;
TokenStreamBuf buf(onToken, &_cancel);
std::ostream os(&buf);
if (_llm) {
_llm->response(std::string(full.UTF8String), &os, nullptr, maxTokens);
}
buf.flush();
return [self statsFromContext];
}
- (MNNGenerateStats *)generateText:(NSString *)prompt
maxTokens:(int)maxTokens
onToken:(void (^)(NSString *))onToken {
return [self runResponse:prompt maxTokens:maxTokens onToken:onToken];
}
- (nullable MNNGenerateStats *)analyzeImages:(NSArray<NSString *> *)imagePaths
prompt:(NSString *)prompt
maxTokens:(int)maxTokens
onToken:(void (^)(NSString *))onToken
error:(NSError **)error {
// 在 prompt 前拼 <img>本地路径</img>;Omni 解析标签并对路径 imread(需 OMNI 框架)。
NSMutableString *full = [NSMutableString string];
for (NSString *p in imagePaths) {
[full appendFormat:@"<img>%@</img>", p];
}
[full appendString:prompt];
return [self runResponse:full maxTokens:maxTokens onToken:onToken];
}
- (MNNGenerateStats *)statsFromContext {
MNNGenerateStats *s = [MNNGenerateStats new];
if (_llm) {
const MNN::Transformer::LlmContext *ctx = _llm->getContext();
if (ctx) {
s.promptTokens = ctx->prompt_len;
s.genTokens = ctx->gen_seq_len;
s.prefillMs = ctx->prefill_us / 1000.0;
s.decodeMs = ctx->decode_us / 1000.0;
}
}
return s;
}
@end
#endif

View File

@@ -0,0 +1,95 @@
import Foundation
/// MNN(CPU / SME2), `MNNLLMBridge`
/// `LLMSession`/`VLSession` actor ; `AIRuntime`
///
/// () Qwen3.5-4B MNN :`generate` ,
/// `analyze` <img> Omni imread ( OMNI ,xcframework )
/// ,; MNN,VL 退 MLX( `AIRuntime`)
actor MNNBackend {
private var bridge: MNNLLMBridge?
var isLoaded: Bool { bridge?.isLoaded ?? false }
/// MNN ( MNN llm config.json + llm.mnn + + tokenizer)
func load(folderURL: URL) throws {
let configPath = folderURL.appendingPathComponent("config.json").path
guard FileManager.default.fileExists(atPath: configPath) else {
throw AIRuntimeError.notReady
}
guard let b = MNNLLMBridge(configPath: configPath) else {
throw AIRuntimeError.modelLoadFailed("MNN createLLM/load 失败")
}
bridge = b
}
func unload() { bridge = nil }
/// `bridge.generateText` , detached 线,
/// yield `TokenChunk`( tok/s) `bridge.cancel()`
func generate(prompt: String, maxTokens: Int) -> AsyncThrowingStream<TokenChunk, Error> {
guard let bridge else {
return AsyncThrowingStream { $0.finish(throwing: AIRuntimeError.notReady) }
}
let box = MNNUncheckedBox(bridge)
return AsyncThrowingStream { continuation in
let meter = MNNRateMeter()
let task = Task.detached(priority: .userInitiated) {
_ = box.value.generateText(prompt, maxTokens: Int32(maxTokens)) { piece in
let rate = meter.tick()
continuation.yield(TokenChunk(text: piece, decodeRate: rate))
}
continuation.finish()
}
continuation.onTermination = { _ in
box.value.cancel()
task.cancel()
}
}
}
/// (VL)(JSON ) <img> ,
/// MNN Omni imread ( OMNI );blocking detached 线
func analyze(imageURLs: [URL], prompt: String, maxTokens: Int) async throws -> String {
guard let bridge else { throw AIRuntimeError.notReady }
let paths = imageURLs.map(\.path)
let box = MNNUncheckedBox(bridge)
return try await withCheckedThrowingContinuation { cont in
Task.detached(priority: .userInitiated) {
let sink = MNNTextSink()
do {
_ = try box.value.analyzeImages(paths, prompt: prompt, maxTokens: Int32(maxTokens)) { piece in
sink.append(piece)
}
cont.resume(returning: sink.text)
} catch {
cont.resume(throwing: AIRuntimeError.inferenceFailed(error.localizedDescription))
}
}
}
}
}
/// 线,
private nonisolated final class MNNTextSink: @unchecked Sendable {
private(set) var text = ""
func append(_ s: String) { text += s }
}
/// Sendable ObjC detached
/// `AIRuntime` :,访
private nonisolated struct MNNUncheckedBox<T>: @unchecked Sendable {
let value: T
init(_ value: T) { self.value = value }
}
/// :线,
private nonisolated final class MNNRateMeter: @unchecked Sendable {
private let start = Date()
private var produced = 0
func tick() -> Double {
produced += 1
let elapsed = Date().timeIntervalSince(start)
return elapsed > 0 ? Double(produced) / elapsed : 0
}
}

View File

@@ -18,16 +18,21 @@ nonisolated enum ModelManifest {
static func files(for kind: ModelKind) -> [ModelFile] { static func files(for kind: ModelKind) -> [ModelFile] {
switch kind { switch kind {
case .llm: case .llm:
// Qwen3.5-4B-4bit:,MLX (LLMModelFactory qwen3_5 )
// (VLMModelFactory qwen3_5) mlx-community/Qwen3.5-4B-4bit
// blob (HF API,2026-06 )(),
// README.md / .gitattributes
return [ return [
ModelFile(path: "config.json", bytes: 937), ModelFile(path: "config.json", bytes: 3_366),
ModelFile(path: "model.safetensors", bytes: 968_080_210), ModelFile(path: "model.safetensors", bytes: 3_034_300_695),
ModelFile(path: "model.safetensors.index.json", bytes: 49_731), ModelFile(path: "model.safetensors.index.json", bytes: 101_944),
ModelFile(path: "tokenizer.json", bytes: 11_422_654), ModelFile(path: "tokenizer.json", bytes: 19_989_343),
ModelFile(path: "tokenizer_config.json", bytes: 9_706), ModelFile(path: "tokenizer_config.json", bytes: 1_139),
ModelFile(path: "vocab.json", bytes: 2_776_833), ModelFile(path: "vocab.json", bytes: 6_722_759),
ModelFile(path: "merges.txt", bytes: 1_671_853), ModelFile(path: "chat_template.jinja", bytes: 7_756),
ModelFile(path: "special_tokens_map.json", bytes: 613), ModelFile(path: "preprocessor_config.json", bytes: 390),
ModelFile(path: "added_tokens.json", bytes: 707), ModelFile(path: "processor_config.json", bytes: 1_300),
ModelFile(path: "video_preprocessor_config.json", bytes: 385),
] ]
case .vl: case .vl:
// Qwen3-VL-4B-Instruct-4bit: mlx-community blob // Qwen3-VL-4B-Instruct-4bit: mlx-community blob
@@ -52,6 +57,20 @@ nonisolated enum ModelManifest {
ModelFile(path: "preprocessor_config.json", bytes: 782), ModelFile(path: "preprocessor_config.json", bytes: 782),
ModelFile(path: "video_preprocessor_config.json", bytes: 817), ModelFile(path: "video_preprocessor_config.json", bytes: 817),
] ]
case .mnnLLM:
// taobao-mnn/Qwen3.5-4B-MNN MNN (HF API ,2026-06)
// :config.json(MNN llm )+ llm_config.json()+ llm.mnn()
// + llm.mnn.weight( ~2.45GB)+ tokenizer.txt + visual.mnn/visual.mnn.weight(,
// mllm,) README/.gitattributes dump
return [
ModelFile(path: "config.json", bytes: 652),
ModelFile(path: "llm_config.json", bytes: 8_693),
ModelFile(path: "llm.mnn", bytes: 3_651_096),
ModelFile(path: "llm.mnn.weight", bytes: 2_629_387_626),
ModelFile(path: "tokenizer.txt", bytes: 6_465_727),
ModelFile(path: "visual.mnn", bytes: 488_096),
ModelFile(path: "visual.mnn.weight", bytes: 196_768_960),
]
} }
} }

View File

@@ -1,14 +1,20 @@
import Foundation import Foundation
nonisolated enum ModelKind: String, CaseIterable { nonisolated enum ModelKind: String, CaseIterable {
/// HuggingFace mlx-community , Models/ /// Models/ / CDN
case llm = "Qwen3-1.7B-4bit" /// Qwen3.5-4B,:
case vl = "Qwen3-VL-4B-Instruct-4bit" /// - mnnLLM:MNN(CPU/SME2,)+,taobao-mnn ,
/// - llm:MLX(GPU),Qwen3.5-4B-4bit (, qwen3_5)
/// - vl:(MLX VL .llm ), switch,/
case llm = "Qwen3.5-4B-4bit"
case vl = "Qwen3-VL-4B-Instruct-4bit"
case mnnLLM = "Qwen3.5-4B-MNN"
var displayName: String { var displayName: String {
switch self { switch self {
case .llm: return "Qwen3-1.7B" case .llm: return "Qwen3.5-4B (MLX)"
case .vl: return "Qwen3-VL-4B" case .vl: return "Qwen3-VL-4B"
case .mnnLLM: return "Qwen3.5-4B (MNN/SME2)"
} }
} }
@@ -17,6 +23,12 @@ nonisolated enum ModelKind: String, CaseIterable {
/// ///
var sentinelFilename: String { "config.json" } var sentinelFilename: String { "config.json" }
/// : / /
/// Qwen3.5-4B(MNN,+,)
/// MLX .llm/.vl ,(),
/// · ,
static let userFacing: [ModelKind] = [.mnnLLM]
} }
/// `@unchecked Sendable`:rootURL let, filesystem(线), /// `@unchecked Sendable`:rootURL let, filesystem(线),

View File

@@ -3,7 +3,7 @@ import Foundation
/// VL (Qwen3-VL) / prompt /// VL (Qwen3-VL) / prompt
/// : JSON,markdown /// : JSON,markdown
/// CaptureService 退(§3.2 退线) /// CaptureService 退(§3.2 退线)
enum VLPrompts { nonisolated enum VLPrompts {
/// JSON ( prompt ): /// JSON ( prompt ):
/// ``` /// ```

View File

@@ -143,6 +143,7 @@ struct IndicatorQuickSheet: View {
.clipShape(RoundedRectangle(cornerRadius: Tj.Radius.xl, style: .continuous)) .clipShape(RoundedRectangle(cornerRadius: Tj.Radius.xl, style: .continuous))
.ignoresSafeArea(edges: .bottom) .ignoresSafeArea(edges: .bottom)
) )
.preferredColorScheme(.light)
.presentationDetents([.large]) .presentationDetents([.large])
.presentationDragIndicator(.hidden) .presentationDragIndicator(.hidden)
.presentationBackground(Tj.Palette.sand) .presentationBackground(Tj.Palette.sand)
@@ -423,6 +424,8 @@ struct IndicatorQuickSheet: View {
TextField(placeholder, text: value) TextField(placeholder, text: value)
.keyboardType(.decimalPad) .keyboardType(.decimalPad)
.font(.tjScaled( 20, weight: .semibold, design: .monospaced)) .font(.tjScaled( 20, weight: .semibold, design: .monospaced))
.foregroundStyle(Tj.Palette.text)
.tint(Tj.Palette.ink)
.multilineTextAlignment(.center) .multilineTextAlignment(.center)
.padding(.vertical, 10) .padding(.vertical, 10)
.frame(width: 90) .frame(width: 90)
@@ -468,6 +471,8 @@ struct IndicatorQuickSheet: View {
sectionLabel(String(appLoc: "指标名")) sectionLabel(String(appLoc: "指标名"))
TextField("例如:血红蛋白", text: $name) TextField("例如:血红蛋白", text: $name)
.textInputAutocapitalization(.never) .textInputAutocapitalization(.never)
.foregroundStyle(Tj.Palette.text)
.tint(Tj.Palette.ink)
.padding(.horizontal, 14) .padding(.horizontal, 14)
.padding(.vertical, 12) .padding(.vertical, 12)
.background(fieldBg) .background(fieldBg)
@@ -489,6 +494,8 @@ struct IndicatorQuickSheet: View {
TextField(monitorFieldPlaceholder, text: $value) TextField(monitorFieldPlaceholder, text: $value)
.keyboardType(.decimalPad) .keyboardType(.decimalPad)
.font(.tjScaled( 18, weight: .semibold, design: .monospaced)) .font(.tjScaled( 18, weight: .semibold, design: .monospaced))
.foregroundStyle(Tj.Palette.text)
.tint(Tj.Palette.ink)
.padding(.horizontal, 14) .padding(.horizontal, 14)
.padding(.vertical, 12) .padding(.vertical, 12)
.background(fieldBg) .background(fieldBg)
@@ -499,6 +506,8 @@ struct IndicatorQuickSheet: View {
TextField("mmol/L", text: $unit) TextField("mmol/L", text: $unit)
.textInputAutocapitalization(.never) .textInputAutocapitalization(.never)
.autocorrectionDisabled() .autocorrectionDisabled()
.foregroundStyle(Tj.Palette.text)
.tint(Tj.Palette.ink)
.padding(.horizontal, 14) .padding(.horizontal, 14)
.padding(.vertical, 12) .padding(.vertical, 12)
.background(fieldBg) .background(fieldBg)
@@ -522,6 +531,8 @@ struct IndicatorQuickSheet: View {
TextField("例如:< 3.40 或 3.9 - 6.1", text: $range) TextField("例如:< 3.40 或 3.9 - 6.1", text: $range)
.textInputAutocapitalization(.never) .textInputAutocapitalization(.never)
.autocorrectionDisabled() .autocorrectionDisabled()
.foregroundStyle(Tj.Palette.text)
.tint(Tj.Palette.ink)
.padding(.horizontal, 14) .padding(.horizontal, 14)
.padding(.vertical, 12) .padding(.vertical, 12)
.background(fieldBg) .background(fieldBg)
@@ -581,6 +592,8 @@ struct IndicatorQuickSheet: View {
sectionLabel(String(appLoc: "备注(可选)")) sectionLabel(String(appLoc: "备注(可选)"))
TextField("例如:空腹采血", text: $note, axis: .vertical) TextField("例如:空腹采血", text: $note, axis: .vertical)
.lineLimit(1...3) .lineLimit(1...3)
.foregroundStyle(Tj.Palette.text)
.tint(Tj.Palette.ink)
.padding(.horizontal, 14) .padding(.horizontal, 14)
.padding(.vertical, 12) .padding(.vertical, 12)
.background(fieldBg) .background(fieldBg)

View File

@@ -0,0 +1,126 @@
import SwiftUI
/// : MNN(CPU/SME2,) MLX(GPU,), SME2
/// ; AI (prepare/generate)
struct InferenceSettingsView: View {
@AppStorage("kk.inferenceEngine") private var engineRaw = InferenceEngine.mnn.rawValue
private var selected: InferenceEngine {
InferenceEngine(rawValue: engineRaw) ?? .mnn
}
var body: some View {
ScrollView {
VStack(spacing: 12) {
HStack {
Text("推理引擎")
.font(.tjTitle())
.foregroundStyle(Tj.Palette.text)
Spacer()
}
.padding(.top, 4)
.padding(.bottom, 6)
ForEach(InferenceEngine.allCases, id: \.self) { engine in
engineRow(engine)
}
sme2Card
noteCard
}
.padding(.horizontal, 16)
.padding(.vertical, 20)
}
.background(Tj.Palette.sand.ignoresSafeArea())
}
private func engineRow(_ engine: InferenceEngine) -> some View {
let available = engine.isAvailable
let isOn = (selected == engine)
return Button {
guard available else { return }
engineRaw = engine.rawValue
} label: {
HStack(spacing: 12) {
ZStack {
Circle().fill(isOn ? Tj.Palette.amber.opacity(0.25) : Tj.Palette.sand2)
Image(systemName: engine == .mnn ? "cpu.fill" : "bolt.fill")
.font(.tjScaled(18))
.foregroundStyle(isOn ? Tj.Palette.ink : Tj.Palette.text2)
}
.frame(width: 44, height: 44)
VStack(alignment: .leading, spacing: 2) {
Text(engine.displayName)
.font(.tjScaled(15, weight: .semibold))
.foregroundStyle(Tj.Palette.text)
Text(subtitle(engine, available: available))
.font(.tjScaled(12))
.foregroundStyle(Tj.Palette.text3)
.lineLimit(2)
}
Spacer()
if isOn {
Image(systemName: "checkmark.circle.fill")
.font(.tjScaled(18))
.foregroundStyle(Tj.Palette.leaf)
}
}
.padding(14)
.tjCard()
.opacity(available ? 1 : 0.45)
}
.buttonStyle(.plain)
.disabled(!available)
}
private func subtitle(_ engine: InferenceEngine, available: Bool) -> String {
switch engine {
case .mnn:
if !available { return String(appLoc: "本设备/模拟器不可用,自动回退 MLX") }
return InferenceEngine.cpuSupportsSME2
? String(appLoc: "端侧 CPU + SME2 加速 · 挑战赛考核路径")
: String(appLoc: "端侧 CPU(本机无 SME2,NEON 回退)")
case .mlx:
return String(appLoc: "Metal GPU · 兜底 / 对照")
}
}
private var sme2Card: some View {
let sme2 = InferenceEngine.cpuSupportsSME2
return HStack(spacing: 12) {
ZStack {
Circle().fill(sme2 ? Tj.Palette.leafSoft : Tj.Palette.sand2)
Image(systemName: sme2 ? "checkmark.seal.fill" : "minus.circle")
.font(.tjScaled(18))
.foregroundStyle(sme2 ? Tj.Palette.ink : Tj.Palette.text2)
}
.frame(width: 44, height: 44)
VStack(alignment: .leading, spacing: 2) {
Text("Arm SME2")
.font(.tjScaled(15, weight: .medium))
.foregroundStyle(Tj.Palette.text)
Text(sme2 ? String(appLoc: "本设备支持,MNN 已启用 SME2 加速")
: String(appLoc: "本设备不支持(需 A19/iPhone 17+)"))
.font(.tjScaled(12))
.foregroundStyle(Tj.Palette.text3)
}
Spacer()
}
.padding(14)
.tjCard()
}
private var noteCard: some View {
Text("MNN 在端侧 CPU 上以 Arm SME2 指令集加速 Qwen 推理(本地、不上云)。切换后下一次 AI 调用生效。")
.font(.tjScaled(12))
.foregroundStyle(Tj.Palette.text3)
.frame(maxWidth: .infinity, alignment: .leading)
.padding(14)
.tjCard()
}
}
#Preview {
InferenceSettingsView()
}

View File

@@ -37,6 +37,7 @@ struct MeView: View {
profileCard profileCard
customMetricsCard customMetricsCard
modelManagementCard modelManagementCard
inferenceEngineCard
languageCard languageCard
fontScaleCard fontScaleCard
faceIDCard faceIDCard
@@ -151,10 +152,26 @@ struct MeView: View {
private var modelDetail: String { private var modelDetail: String {
let states = downloadService.states let states = downloadService.states
if ModelKind.allCases.allSatisfy({ states[$0]?.phase == .ready }) { return String(appLoc: "已就绪") } if ModelKind.userFacing.allSatisfy({ states[$0]?.phase == .ready }) { return String(appLoc: "已就绪") }
if downloadService.isAnyDownloading { return String(appLoc: "下载中…") } if downloadService.isAnyDownloading { return String(appLoc: "下载中…") }
let readyCount = ModelKind.allCases.filter { states[$0]?.phase == .ready }.count let readyCount = ModelKind.userFacing.filter { states[$0]?.phase == .ready }.count
return readyCount == 0 ? String(appLoc: "未下载") : String(appLoc: "\(readyCount)/\(ModelKind.allCases.count) 就绪") return readyCount == 0 ? String(appLoc: "未下载") : String(appLoc: "\(readyCount)/\(ModelKind.userFacing.count) 就绪")
}
private var inferenceEngineCard: some View {
NavigationLink {
InferenceSettingsView()
} label: {
settingsCard(title: String(appLoc: "推理引擎"), detail: engineDetail, icon: "cpu.fill")
}
.buttonStyle(.plain)
}
private var engineDetail: String {
switch InferenceEngine.current {
case .mnn: return InferenceEngine.cpuSupportsSME2 ? "MNN · SME2" : "MNN · CPU"
case .mlx: return "MLX · GPU"
}
} }
private var languageCard: some View { private var languageCard: some View {

View File

@@ -10,25 +10,29 @@ struct ModelManagementView: View {
@State private var showCellularConfirm = false @State private var showCellularConfirm = false
@State private var showImporter = false @State private var showImporter = false
@State private var importError: String? @State private var importError: String?
@AppStorage(QuickRegionRecognitionEngine.storageKey)
private var quickRegionEngineRaw = QuickRegionRecognitionEngine.defaultValue.rawValue
private let monitor = NWPathMonitor() private let monitor = NWPathMonitor()
private let monitorQueue = DispatchQueue(label: "kk.netmonitor") private let monitorQueue = DispatchQueue(label: "kk.netmonitor")
private var allReady: Bool { private var allReady: Bool {
ModelKind.allCases.allSatisfy { service.states[$0]?.phase == .ready } ModelKind.userFacing.allSatisfy { service.states[$0]?.phase == .ready }
} }
var body: some View { var body: some View {
ScrollView { ScrollView {
VStack(spacing: 14) { VStack(spacing: 14) {
ForEach(ModelKind.allCases, id: \.self) { kind in ForEach(ModelKind.userFacing, id: \.self) { kind in
modelCard(kind) modelCard(kind)
} }
recognitionEngineCard
actionButtons actionButtons
.padding(.top, 4) .padding(.top, 4)
if service.states[.llm]?.phase == .ready { if service.states[.mnnLLM]?.phase == .ready {
NavigationLink { NavigationLink {
ModelSelfTestView() ModelSelfTestView()
} label: { } label: {
@@ -76,6 +80,46 @@ struct ModelManagementView: View {
} }
} }
// MARK: -
private var selectedRecognitionEngine: QuickRegionRecognitionEngine {
QuickRegionRecognitionEngine(storedValue: quickRegionEngineRaw)
}
private var recognitionEngineCard: some View {
VStack(alignment: .leading, spacing: 12) {
HStack(alignment: .top, spacing: 10) {
ZStack {
Circle().fill(Tj.Palette.sand2)
Image(systemName: "camera.metering.center.weighted")
.font(.tjScaled( 18))
.foregroundStyle(Tj.Palette.text2)
}
.frame(width: 38, height: 38)
VStack(alignment: .leading, spacing: 3) {
Text("异常项拍照识别")
.font(.tjScaled( 15, weight: .semibold))
.foregroundStyle(Tj.Palette.text)
Text(selectedRecognitionEngine.detail)
.font(.tjScaled( 12))
.foregroundStyle(Tj.Palette.text3)
}
Spacer()
}
Picker("异常项拍照识别", selection: $quickRegionEngineRaw) {
ForEach(QuickRegionRecognitionEngine.allCases) { engine in
Text(engine.title).tag(engine.rawValue)
}
}
.pickerStyle(.segmented)
}
.padding(14)
.frame(maxWidth: .infinity, alignment: .leading)
.tjCard()
}
// MARK: - // MARK: -
private func modelCard(_ kind: ModelKind) -> some View { private func modelCard(_ kind: ModelKind) -> some View {
@@ -146,7 +190,7 @@ struct ModelManagementView: View {
private var actionButtons: some View { private var actionButtons: some View {
if service.isAnyDownloading { if service.isAnyDownloading {
Button { Button {
for kind in ModelKind.allCases { service.cancel(kind) } for kind in ModelKind.userFacing { service.cancel(kind) }
} label: { } label: {
Text("暂停下载").frame(maxWidth: .infinity) Text("暂停下载").frame(maxWidth: .infinity)
} }
@@ -154,7 +198,7 @@ struct ModelManagementView: View {
} else if allReady { } else if allReady {
HStack(spacing: 6) { HStack(spacing: 6) {
Image(systemName: "checkmark.seal.fill") Image(systemName: "checkmark.seal.fill")
Text("两个模型都已就绪") Text("Qwen3.5-4B 已就绪")
} }
.font(.tjScaled( 13, weight: .semibold)) .font(.tjScaled( 13, weight: .semibold))
.foregroundStyle(Tj.Palette.leaf) .foregroundStyle(Tj.Palette.leaf)
@@ -198,8 +242,8 @@ struct ModelManagementView: View {
defer { if scoped { folder.stopAccessingSecurityScopedResource() } } defer { if scoped { folder.stopAccessingSecurityScopedResource() } }
let name = folder.lastPathComponent let name = folder.lastPathComponent
guard let kind = ModelKind.allCases.first(where: { $0.rawValue == name }) else { guard let kind = ModelKind.userFacing.first(where: { $0.rawValue == name }) else {
let names = ModelKind.allCases.map(\.rawValue).joined(separator: "") let names = ModelKind.userFacing.map(\.rawValue).joined(separator: "")
importError = String(appLoc: "请选择名为 \(names) 的文件夹") importError = String(appLoc: "请选择名为 \(names) 的文件夹")
return return
} }
@@ -213,13 +257,14 @@ struct ModelManagementView: View {
// MARK: - // MARK: -
private var totalAllBytes: Int { private var totalAllBytes: Int {
ModelKind.allCases.reduce(0) { $0 + ModelManifest.totalBytes(for: $1) } ModelKind.userFacing.reduce(0) { $0 + ModelManifest.totalBytes(for: $1) }
} }
private func subtitle(_ kind: ModelKind) -> String { private func subtitle(_ kind: ModelKind) -> String {
switch kind { switch kind {
case .llm: return String(appLoc: "文本解读 · 趋势 / 问答") case .llm: return String(appLoc: "文本解读 · 趋势 / 问答(MLX 兜底)")
case .vl: return String(appLoc: "拍照识别报告 → 结构化指标") case .vl: return String(appLoc: "拍照识别报告 → 结构化指标")
case .mnnLLM: return String(appLoc: "文本解读 + 拍照识别 · MNN + SME2 端侧加速")
} }
} }

View File

@@ -370,17 +370,14 @@ private struct ChronicSection: View {
} }
} }
HStack { EntryInputField(placeholder: String(appLoc: "自定义慢病"), text: $newCustomCondition) {
TextField("自定义慢病", text: $newCustomCondition) let trimmed = newCustomCondition.trimmingCharacters(in: .whitespaces)
Button("") { guard !trimmed.isEmpty,
let trimmed = newCustomCondition.trimmingCharacters(in: .whitespaces) !profile.chronicConditions.contains(trimmed) else { return }
guard !trimmed.isEmpty, profile.chronicConditions.append(trimmed)
!profile.chronicConditions.contains(trimmed) else { return } newCustomCondition = ""
profile.chronicConditions.append(trimmed)
newCustomCondition = ""
}
.disabled(newCustomCondition.trimmingCharacters(in: .whitespaces).isEmpty)
} }
.listRowBackground(Color.clear)
} header: { } header: {
Text("慢病(影响参考范围与 AI 解读)") Text("慢病(影响参考范围与 AI 解读)")
} }
@@ -408,6 +405,48 @@ private struct ChronicSection: View {
} }
} }
// MARK: - ( + + )
/// `TextField + ` :(1~4 ),
/// () / / /
private struct EntryInputField: View {
let placeholder: String
@Binding var text: String
var onSubmit: () -> Void
private var canSubmit: Bool {
!text.trimmingCharacters(in: .whitespaces).isEmpty
}
var body: some View {
HStack(alignment: .bottom, spacing: 8) {
TextField(placeholder, text: $text, axis: .vertical)
.lineLimit(1...4)
.padding(.horizontal, 14)
.padding(.vertical, 10)
.background(
RoundedRectangle(cornerRadius: Tj.Radius.md, style: .continuous)
.fill(Tj.Palette.paper)
)
.overlay(
RoundedRectangle(cornerRadius: Tj.Radius.md, style: .continuous)
.strokeBorder(Tj.Palette.line, lineWidth: 1)
)
Button {
if canSubmit { onSubmit() }
} label: {
Image(systemName: "arrow.up.circle.fill")
.font(.tjScaled(28))
.foregroundStyle(canSubmit ? Tj.Palette.ink : Tj.Palette.text3)
}
.buttonStyle(.plain)
.disabled(!canSubmit)
}
.padding(.vertical, 2)
}
}
// MARK: - / / ( @State,) // MARK: - / / ( @State,)
private struct StringListSection: View { private struct StringListSection: View {
@@ -431,16 +470,13 @@ private struct StringListSection: View {
.buttonStyle(.borderless) .buttonStyle(.borderless)
} }
} }
HStack { EntryInputField(placeholder: placeholder, text: $newInput) {
TextField(placeholder, text: $newInput) let trimmed = newInput.trimmingCharacters(in: .whitespaces)
Button("") { guard !trimmed.isEmpty, !items.contains(trimmed) else { return }
let trimmed = newInput.trimmingCharacters(in: .whitespaces) items.append(trimmed)
guard !trimmed.isEmpty, !items.contains(trimmed) else { return } newInput = ""
items.append(trimmed)
newInput = ""
}
.disabled(newInput.trimmingCharacters(in: .whitespaces).isEmpty)
} }
.listRowBackground(Color.clear)
} }
} }
} }

View File

@@ -15,6 +15,8 @@ struct QuickRegionCaptureFlow: View {
@Environment(\.modelContext) private var ctx @Environment(\.modelContext) private var ctx
let onClose: () -> Void let onClose: () -> Void
@AppStorage(QuickRegionRecognitionEngine.storageKey)
private var recognitionEngineRaw = QuickRegionRecognitionEngine.defaultValue.rawValue
@State private var phase: Phase = .idle @State private var phase: Phase = .idle
enum Phase { enum Phase {
@@ -98,8 +100,20 @@ struct QuickRegionCaptureFlow: View {
// MARK: - ( OCR LLM) // MARK: - ( OCR LLM)
/// /,( RegionAdjustView ) /// /,( RegionAdjustView )
/// :Vision OCR Qwen3-1.7B ( indicator-capture-ocr-llm) /// :
/// - Apple Vision:Vision OCR Qwen3-1.7B
/// - Qwen3-VL: Qwen3-VL
private func recognizeRegion(_ image: UIImage) async -> (items: [QuickRegionItem], warning: String?) { private func recognizeRegion(_ image: UIImage) async -> (items: [QuickRegionItem], warning: String?) {
let engine = QuickRegionRecognitionEngine(storedValue: recognitionEngineRaw)
switch engine {
case .appleVision:
return await recognizeWithAppleVision(image)
case .qwenVL:
return await recognizeWithQwenVL(image)
}
}
private func recognizeWithAppleVision(_ image: UIImage) async -> (items: [QuickRegionItem], warning: String?) {
do { do {
let text = try await OCRService.recognizeText(in: image) let text = try await OCRService.recognizeText(in: image)
if Task.isCancelled { return ([], nil) } // : if Task.isCancelled { return ([], nil) } // :
@@ -125,6 +139,30 @@ struct QuickRegionCaptureFlow: View {
} }
} }
private func recognizeWithQwenVL(_ image: UIImage) async -> (items: [QuickRegionItem], warning: String?) {
let prepared = RegionImageCropper.prepareForQwenVL(image)
guard let data = prepared.jpegData(compressionQuality: 0.95) else {
return ([], String(appLoc: "图片编码失败,手动补充"))
}
#if DEBUG
print("🖼️ [Qwen3-VL region] prepared image=\(Int(prepared.size.width))x\(Int(prepared.size.height)), bytes=\(data.count)")
#endif
do {
let parsed = try await CaptureService.shared.recognizeRegion(imageData: data)
if Task.isCancelled { return ([], nil) }
let items = Self.buildItems(from: parsed)
return (items, items.isEmpty ? String(appLoc: "没读出指标,挪一下框再试") : nil)
} catch CaptureError.modelNotReady {
return ([], String(appLoc: "模型未就绪,请在模型管理下载或切回 Apple Vision"))
} catch let CaptureError.parseFailed(msg) {
return ([], String(appLoc: "解析失败:\(msg)"))
} catch let CaptureError.inferenceFailed(msg) {
return ([], Task.isCancelled ? nil : String(appLoc: "识别失败:\(msg)"))
} catch {
return ([], Task.isCancelled ? nil : String(appLoc: "未知错误:\(error.localizedDescription)"))
}
}
/// LLM ,(high/low) /// LLM ,(high/low)
private static func buildItems(from parsed: [ParsedReport.ParsedIndicator]) -> [QuickRegionItem] { private static func buildItems(from parsed: [ParsedReport.ParsedIndicator]) -> [QuickRegionItem] {
let mapped = parsed.map { let mapped = parsed.map {

View File

@@ -0,0 +1,31 @@
import Foundation
enum QuickRegionRecognitionEngine: String, CaseIterable, Identifiable, Sendable {
case appleVision
case qwenVL
static let storageKey = "quickRegionRecognitionEngine"
static let defaultValue: QuickRegionRecognitionEngine = .appleVision
var id: String { rawValue }
init(storedValue: String) {
self = QuickRegionRecognitionEngine(rawValue: storedValue) ?? Self.defaultValue
}
var title: String {
switch self {
case .appleVision: return String(appLoc: "Apple Vision")
case .qwenVL: return String(appLoc: "大模型直读")
}
}
var detail: String {
switch self {
case .appleVision:
return String(appLoc: "系统 OCR + 文本模型解析")
case .qwenVL:
return String(appLoc: "Qwen3.5-4B 多模态直接看图(MNN/MLX)")
}
}
}

View File

@@ -352,6 +352,49 @@ enum RegionImageCropper {
guard rect.width >= 1, rect.height >= 1, let cropped = cg.cropping(to: rect) else { return up } guard rect.width >= 1, rect.height >= 1, let cropped = cg.cropping(to: rect) else { return up }
return UIImage(cgImage: cropped, scale: up.scale, orientation: .up) return UIImage(cgImage: cropped, scale: up.scale, orientation: .up)
} }
/// Qwen3-VL : VL ,processor
/// Qwen3-VL ,Apple Vision OCR
static func prepareForQwenVL(_ image: UIImage,
minimumShortEdge: CGFloat = 448,
maximumLongEdge: CGFloat = 2400,
padding: CGFloat = 64) -> UIImage {
let up = image.normalizedUp()
guard let cg = up.cgImage else { return up }
let sourceSize = CGSize(width: cg.width, height: cg.height)
guard sourceSize.width > 0, sourceSize.height > 0 else { return up }
let short = min(sourceSize.width, sourceSize.height)
let long = max(sourceSize.width, sourceSize.height)
var scale = max(1, minimumShortEdge / short)
if long * scale > maximumLongEdge {
scale = maximumLongEdge / long
}
let contentSize = CGSize(
width: max(1, (sourceSize.width * scale).rounded()),
height: max(1, (sourceSize.height * scale).rounded())
)
let canvasSize = CGSize(
width: contentSize.width + padding * 2,
height: contentSize.height + padding * 2
)
let format = UIGraphicsImageRendererFormat.default()
format.scale = 1
format.opaque = true
let renderer = UIGraphicsImageRenderer(size: canvasSize, format: format)
return renderer.image { ctx in
UIColor.white.setFill()
ctx.fill(CGRect(origin: .zero, size: canvasSize))
UIImage(cgImage: cg, scale: 1, orientation: .up).draw(
in: CGRect(x: padding, y: padding,
width: contentSize.width, height: contentSize.height)
)
}
}
} }
extension UIImage { extension UIImage {

View File

@@ -1314,9 +1314,15 @@
} }
} }
} }
},
"Apple Vision" : {
}, },
"Apple 健康里没有可导入的生日、性别、身高或血型。" : { "Apple 健康里没有可导入的生日、性别、身高或血型。" : {
},
"Arm SME2" : {
}, },
"B 型" : { "B 型" : {
"localizations" : { "localizations" : {
@@ -1386,12 +1392,18 @@
}, },
"lo" : { "lo" : {
},
"Metal GPU · 兜底 / 对照" : {
}, },
"mmHg" : { "mmHg" : {
}, },
"mmol/L" : { "mmol/L" : {
},
"MNN 在端侧 CPU 上以 Arm SME2 指令集加速 Qwen 推理(本地、不上云)。切换后下一次 AI 调用生效。" : {
}, },
"O 型" : { "O 型" : {
"localizations" : { "localizations" : {
@@ -1414,6 +1426,12 @@
} }
} }
} }
},
"Qwen3.5-4B 多模态直接看图(MNN/MLX)" : {
},
"Qwen3.5-4B 已就绪" : {
}, },
"s" : { "s" : {
@@ -1853,6 +1871,7 @@
} }
}, },
"两个模型都已就绪" : { "两个模型都已就绪" : {
"extractionState" : "stale",
"localizations" : { "localizations" : {
"en" : { "en" : {
"stringUnit" : { "stringUnit" : {
@@ -3635,6 +3654,7 @@
} }
}, },
"加" : { "加" : {
"extractionState" : "stale",
"localizations" : { "localizations" : {
"en" : { "en" : {
"stringUnit" : { "stringUnit" : {
@@ -4322,6 +4342,9 @@
} }
} }
} }
},
"图片编码失败,手动补充" : {
}, },
"在「+ 新建 → 指标记录 → %@」记录一次" : { "在「+ 新建 → 指标记录 → %@」记录一次" : {
"localizations" : { "localizations" : {
@@ -4600,6 +4623,9 @@
}, },
"大" : { "大" : {
},
"大模型直读" : {
}, },
"失眠" : { "失眠" : {
"localizations" : { "localizations" : {
@@ -5939,6 +5965,9 @@
}, },
"异常项快拍需要相机。去「设置 → 康康 → 相机」打开后再回来。" : { "异常项快拍需要相机。去「设置 → 康康 → 相机」打开后再回来。" : {
},
"异常项拍照识别" : {
}, },
"强度" : { "强度" : {
"localizations" : { "localizations" : {
@@ -7111,6 +7140,9 @@
} }
} }
} }
},
"推理引擎" : {
}, },
"推理自检" : { "推理自检" : {
"localizations" : { "localizations" : {
@@ -7618,6 +7650,7 @@
} }
}, },
"文本解读 · 趋势 / 问答" : { "文本解读 · 趋势 / 问答" : {
"extractionState" : "stale",
"localizations" : { "localizations" : {
"en" : { "en" : {
"stringUnit" : { "stringUnit" : {
@@ -7638,6 +7671,12 @@
} }
} }
} }
},
"文本解读 · 趋势 / 问答(MLX 兜底)" : {
},
"文本解读 + 拍照识别 · MNN + SME2 端侧加速" : {
}, },
"新建" : { "新建" : {
"localizations" : { "localizations" : {
@@ -8617,6 +8656,15 @@
} }
} }
} }
},
"本设备/模拟器不可用,自动回退 MLX" : {
},
"本设备不支持(需 A19/iPhone 17+)" : {
},
"本设备支持,MNN 已启用 SME2 加速" : {
}, },
"本设备未设置 Face ID 或密码" : { "本设备未设置 Face ID 或密码" : {
"localizations" : { "localizations" : {
@@ -8860,6 +8908,9 @@
} }
} }
} }
},
"模型未就绪,请在模型管理下载或切回 Apple Vision" : {
}, },
"模型未就绪时 App 仍可使用,AI 功能会提示前往下载。" : { "模型未就绪时 App 仍可使用,AI 功能会提示前往下载。" : {
"localizations" : { "localizations" : {
@@ -9872,6 +9923,12 @@
} }
} }
} }
},
"端侧 CPU + SME2 加速 · 挑战赛考核路径" : {
},
"端侧 CPU(本机无 SME2,NEON 回退)" : {
}, },
"第 %lld 轮 · 基于你刚才更新的文本 · %lld 条" : { "第 %lld 轮 · 基于你刚才更新的文本 · %lld 条" : {
"localizations" : { "localizations" : {
@@ -10014,6 +10071,9 @@
} }
} }
} }
},
"系统 OCR + 文本模型解析" : {
}, },
"系统:iOS 17 或更新版本。" : { "系统:iOS 17 或更新版本。" : {
"localizations" : { "localizations" : {

View File

@@ -28,7 +28,7 @@ final class HealthExport {
var inferredLabelCN: String? var inferredLabelCN: String?
// demo // demo
/// tag, "Qwen3-1.7B-4bit" /// tag, "Qwen3.5-2B-4bit"
var modelTag: String var modelTag: String
/// tok/s, demo #6 Live Activity /// tok/s, demo #6 Live Activity
var decodeRate: Double var decodeRate: Double
@@ -44,7 +44,7 @@ final class HealthExport {
inferredTimeToDate: Date? = nil, inferredTimeToDate: Date? = nil,
inferredIntent: String? = nil, inferredIntent: String? = nil,
inferredLabelCN: String? = nil, inferredLabelCN: String? = nil,
modelTag: String = "Qwen3-1.7B-4bit", modelTag: String = "Qwen3.5-2B-4bit",
decodeRate: Double = 0) { decodeRate: Double = 0) {
self.prompt = prompt self.prompt = prompt
self.content = content self.content = content

View File

@@ -7,7 +7,7 @@ struct HealthExportDialogueTurn: Identifiable, Hashable, Sendable {
var transcriptLabel: String { var transcriptLabel: String {
switch self { switch self {
case .user: return String(appLoc: "患者") case .user: return String(appLoc: "")
case .assistant: return String(appLoc: "康康") case .assistant: return String(appLoc: "康康")
} }
} }

View File

@@ -58,7 +58,7 @@ final class ModelDownloadService {
} }
func downloadAll() { func downloadAll() {
for kind in ModelKind.allCases { download(kind) } for kind in ModelKind.userFacing { download(kind) }
} }
/// .part , /// .part ,

View File

@@ -23,7 +23,7 @@ enum OCRService {
let handler = VNImageRequestHandler(cgImage: cgImage, orientation: .up, options: [:]) let handler = VNImageRequestHandler(cgImage: cgImage, orientation: .up, options: [:])
do { do {
try handler.perform([request]) try handler.perform([request])
let obs = (request.results as? [VNRecognizedTextObservation]) ?? [] let obs = request.results ?? []
cont.resume(returning: assemble(obs)) cont.resume(returning: assemble(obs))
} catch { } catch {
cont.resume(throwing: error) cont.resume(throwing: error)

View File

@@ -0,0 +1,6 @@
//
// 康康-Bridging-Header.h
// 把 Objective-C 接口暴露给 Swift。
//
#import "AI/MNN/MNNLLMBridge.h"

View File

@@ -4,8 +4,8 @@ import Foundation
struct ModelManifestTests { struct ModelManifestTests {
@Test func llmHasNineFunctionalFiles() { @Test func llmHasTenFunctionalFiles() {
#expect(ModelManifest.files(for: .llm).count == 9) #expect(ModelManifest.files(for: .llm).count == 10)
} }
@Test func vlHasFourteenFunctionalFiles() { @Test func vlHasFourteenFunctionalFiles() {
@@ -13,13 +13,35 @@ struct ModelManifestTests {
} }
@Test func llmTotalBytesMatchesManifest() { @Test func llmTotalBytesMatchesManifest() {
#expect(ModelManifest.totalBytes(for: .llm) == 984_013_244) #expect(ModelManifest.totalBytes(for: .llm) == 3_061_129_077)
} }
@Test func vlTotalBytesMatchesManifest() { @Test func vlTotalBytesMatchesManifest() {
#expect(ModelManifest.totalBytes(for: .vl) == 3_109_729_929) #expect(ModelManifest.totalBytes(for: .vl) == 3_109_729_929)
} }
@Test func mnnHasSevenFunctionalFiles() {
#expect(ModelManifest.files(for: .mnnLLM).count == 7)
}
@Test func mnnTotalBytesMatchesManifest() {
#expect(ModelManifest.totalBytes(for: .mnnLLM) == 2_836_770_850)
}
@Test func mnnHasEssentialRuntimeFiles() {
let names = ModelManifest.files(for: .mnnLLM).map(\.path)
#expect(names.contains("config.json"))
#expect(names.contains("llm.mnn"))
#expect(names.contains("llm.mnn.weight"))
#expect(names.contains("tokenizer.txt"))
}
@Test func mnnFileURLUsesRepoPath() {
let file = ModelFile(path: "config.json", bytes: 652)
let url = ModelManifest.fileURL(for: .mnnLLM, file: file)
#expect(url.absoluteString == "https://file.myv0.com/Qwen3.5-4B-MNN/config.json")
}
@Test func excludesReadmeAndGitattributes() { @Test func excludesReadmeAndGitattributes() {
for kind in [ModelKind.llm, .vl] { for kind in [ModelKind.llm, .vl] {
let names = ModelManifest.files(for: kind).map(\.path) let names = ModelManifest.files(for: kind).map(\.path)
@@ -40,8 +62,8 @@ struct ModelManifestTests {
} }
@Test func fileURLIsBaseSlashRepoSlashPath() { @Test func fileURLIsBaseSlashRepoSlashPath() {
let file = ModelFile(path: "config.json", bytes: 937) let file = ModelFile(path: "config.json", bytes: 3_366)
let url = ModelManifest.fileURL(for: .llm, file: file) let url = ModelManifest.fileURL(for: .llm, file: file)
#expect(url.absoluteString == "https://file.myv0.com/Qwen3-1.7B-4bit/config.json") #expect(url.absoluteString == "https://file.myv0.com/Qwen3.5-4B-4bit/config.json")
} }
} }

View File

@@ -0,0 +1,18 @@
import Testing
@testable import
struct QuickRegionRecognitionEngineTests {
@Test func defaultsToAppleVisionOCR() {
#expect(QuickRegionRecognitionEngine.defaultValue == .appleVision)
}
@Test func rawValuesAreStableForAppStorage() {
#expect(QuickRegionRecognitionEngine.appleVision.rawValue == "appleVision")
#expect(QuickRegionRecognitionEngine.qwenVL.rawValue == "qwenVL")
}
@Test func unknownStoredValueFallsBackToDefault() {
#expect(QuickRegionRecognitionEngine(storedValue: "missing") == .appleVision)
}
}

View File

@@ -112,4 +112,41 @@ final class RegionImageCropperTests: XCTestCase {
imageFrame: CGRect(x: 0, y: 0, width: 100, height: 100)), imageFrame: CGRect(x: 0, y: 0, width: 100, height: 100)),
.zero) .zero)
} }
/// Qwen3-VL processor
/// VL ,
func testPrepareForQwenVLUpscalesWideShortCropAndAddsPadding() {
let image = solidImage(size: CGSize(width: 320, height: 80))
let prepared = RegionImageCropper.prepareForQwenVL(image)
XCTAssertGreaterThanOrEqual(prepared.size.height, 448)
XCTAssertGreaterThan(prepared.size.width, 320)
XCTAssertEqual(prepared.size.width / prepared.size.height,
4.0,
accuracy: 0.8,
"预处理应大致保留宽条内容比例,只允许白边造成轻微变化")
}
func testPrepareForQwenVLDoesNotEnlargePastLongEdgeLimit() {
let image = solidImage(size: CGSize(width: 5000, height: 900))
let prepared = RegionImageCropper.prepareForQwenVL(image)
XCTAssertLessThanOrEqual(max(prepared.size.width, prepared.size.height), 2400 + 128)
}
private func solidImage(size: CGSize) -> UIImage {
let format = UIGraphicsImageRendererFormat.default()
format.scale = 1
return UIGraphicsImageRenderer(size: size, format: format).image { ctx in
UIColor.white.setFill()
ctx.fill(CGRect(origin: .zero, size: size))
UIColor.black.setFill()
ctx.fill(CGRect(x: size.width * 0.1,
y: size.height * 0.35,
width: size.width * 0.8,
height: size.height * 0.3))
}
}
} }