docs(v2): add migration and api schema drafts

This commit is contained in:
phamnazage-jpg
2026-05-22 13:43:46 +08:00
parent 485b6e7a6b
commit 9cdb7e1112
3 changed files with 769 additions and 2 deletions

View File

@@ -0,0 +1,403 @@
# V2 API Response Schema 细稿 — Batch Auto-Import
日期2026-05-22
关联文档:
- `docs/2026-05-21-BATCH_AUTO_IMPORT_SPEC.md`
- `docs/2026-05-21-BATCH_AUTO_IMPORT_TDD_PLAN.md`
- `docs/2026-05-22-BATCH_AUTO_IMPORT_V2_ARCHITECTURE.md`
- `docs/openapi.yaml`
## 1. 目标
这份文档把 V2 结果 API 细化到 handler 和前端都可直接对照实现的粒度。
覆盖:
- 路由
- query/filter
- response schema
- error schema
- badge / 文案映射
- 示例 JSON
## 2. 资源模型
V2 API 只暴露 3 层资源:
1. `run`
2. `run items`
3. `run item events`
不直接暴露:
- 宿主数据库结构
- `import_batches` 细节
- legacy repo 的内部实现字段
## 3. Endpoint 列表
### 3.1 创建 run
`POST /api/batch-import/runs`
作用:
- 创建 V2 run
- 立即返回 `run_id`
- 可选等待一个短暂 confirm window但不保证全部 item 已 done
### 3.2 列表 runs
`GET /api/batch-import/runs`
作用:
- 查看最近导入批次
- 支持状态筛选和关键词搜索
### 3.3 run 详情
`GET /api/batch-import/runs/{run_id}`
作用:
- 返回 run summary
- 返回汇总告警摘要
### 3.4 item 列表
`GET /api/batch-import/runs/{run_id}/items`
作用:
- 展示批次内各 URL/provider 的状态
- 支持按 stage/access/warning 过滤
### 3.5 item 详情
`GET /api/batch-import/runs/{run_id}/items/{item_id}`
作用:
- 返回一个 item 的全量投影
- 包括 capability、模型纠错、event trail
## 4. Query 参数约定
### 4.1 `GET /api/batch-import/runs`
支持:
- `state`
- `running`
- `completed`
- `completed_with_warnings`
- `failed`
- `cancelled`
- `access_mode`
- `subscription`
- `self_service`
- `q`
- 搜索 `run_id / provider_id / base_url`
- `limit`
- `cursor`
说明:
- 页面显示可以把 `completed_with_warnings` 渲染成 `warning`
- API 不应引入第二套 `warning` 枚举
### 4.2 `GET /api/batch-import/runs/{run_id}/items`
支持:
- `current_stage`
- `probe`
- `provision`
- `confirm`
- `validate`
- `done`
- `confirmation_status`
- `pending`
- `confirmed`
- `advisory`
- `failed`
- `access_status`
- `unknown`
- `active`
- `degraded`
- `broken`
- `has_warning`
- `true/false`
- `provider_id`
- `q`
- 搜索 `provider_id / base_url / item_id`
- `limit`
- `cursor`
## 5. 通用错误结构
建议统一:
```json
{
"error": {
"code": "invalid_request",
"message": "subscription_users is required when access_mode=subscription"
}
}
```
常用 `code`
- `invalid_request`
- `not_found`
- `conflict`
- `internal_error`
## 6. 响应 Schema
### 6.1 `POST /api/batch-import/runs`
```json
{
"run_id": "run_20260522_0001",
"state": "running",
"result_page": "/batch-import/runs/run_20260522_0001",
"total_items": 2,
"active_items": 0,
"degraded_items": 0,
"broken_items": 0,
"warning_items": 0
}
```
约束:
- 这是 `run created` 语义,不是 final result
- `state` 初始一般是 `running`
### 6.2 `GET /api/batch-import/runs`
```json
{
"runs": [
{
"run_id": "run_20260522_0001",
"state": "completed_with_warnings",
"mode": "partial",
"access_mode": "subscription",
"total_items": 2,
"completed_items": 2,
"active_items": 1,
"degraded_items": 1,
"broken_items": 0,
"warning_items": 1,
"started_at": "2026-05-22T12:20:00+08:00",
"finished_at": "2026-05-22T12:20:07+08:00"
}
],
"next_cursor": null
}
```
### 6.3 `GET /api/batch-import/runs/{run_id}`
```json
{
"run": {
"run_id": "run_20260522_0001",
"state": "completed_with_warnings",
"mode": "partial",
"access_mode": "subscription",
"total_items": 2,
"completed_items": 2,
"active_items": 1,
"degraded_items": 1,
"broken_items": 0,
"warning_items": 1,
"started_at": "2026-05-22T12:20:00+08:00",
"finished_at": "2026-05-22T12:20:07+08:00"
},
"recent_warnings": [
"该批次包含 1 条 advisory item建议检查 capability profile 与 retry 轨迹"
]
}
```
### 6.4 `GET /api/batch-import/runs/{run_id}/items`
```json
{
"items": [
{
"item_id": "item_01",
"base_url": "https://kimi.a7m.com.cn/v1",
"provider_id": "kimi-a7m-7d7ac291",
"requested_models": ["kimi-k2.6"],
"resolved_smoke_model": "kimi-k2.6",
"current_stage": "done",
"confirmation_status": "advisory",
"access_status": "active",
"retry_count": 2,
"last_retry_at": "2026-05-22T12:20:05+08:00",
"advisory_messages": [
"该上游不支持 /v1/responses系统已自动回退到 /v1/chat/completions"
],
"last_error_stage": "confirm",
"last_error": "API returned 403: Forbidden"
}
],
"next_cursor": null
}
```
### 6.5 `GET /api/batch-import/runs/{run_id}/items/{item_id}`
```json
{
"item_id": "item_01",
"base_url": "https://kimi.a7m.com.cn/v1",
"provider_id": "kimi-a7m-7d7ac291",
"requested_models": ["kimi-k2.6"],
"raw_models": ["kimi-k2.6"],
"normalized_models": ["kimi-k2.6"],
"recommended_models": [],
"resolved_smoke_model": "kimi-k2.6",
"current_stage": "done",
"confirmation_status": "advisory",
"access_status": "active",
"retry_count": 2,
"last_retry_at": "2026-05-22T12:20:05+08:00",
"channel_id": 12,
"account_id": 4,
"advisory_messages": [
"该上游不支持 /v1/responses系统已自动回退到 /v1/chat/completions"
],
"last_error_stage": "confirm",
"last_error": "API returned 403: Forbidden",
"capability_profile": {
"transport_profile": {
"supports_openai_models": true,
"supports_openai_chat_completions": true,
"supports_openai_responses": false,
"supports_anthropic_messages": false,
"auth_style": "bearer",
"model_id_style": "canonical",
"known_advisories": [
"responses_unsupported_but_chat_ok",
"initial_probe_race_expected"
]
},
"model_profiles": [
{
"raw_model_id": "kimi-k2.6",
"normalized_model_id": "kimi-k2.6",
"supports_stream": true,
"supports_tools": "unknown",
"supports_reasoning_fields": "unknown",
"smoke_chat_ok": true
}
]
},
"events": [
{
"event_id": "evt_01",
"event_type": "retry_scheduled",
"stage": "confirm",
"attempt": 1,
"message": "initial 503 no available accounts, retry scheduled",
"payload_json": "{\"delay_ms\":500}",
"created_at": "2026-05-22T12:20:04+08:00"
}
]
}
```
## 7. Badge / 文案映射
### 7.1 Run state badge
| API 值 | 页面 badge |
|---|---|
| `running` | 蓝色 `running` |
| `completed` | 绿色 `completed` |
| `completed_with_warnings` | 黄色 `warning` |
| `failed` | 红色 `failed` |
| `cancelled` | 灰色 `cancelled` |
### 7.2 Confirmation badge
| API 值 | 页面 badge |
|---|---|
| `pending` | 蓝色 `pending` |
| `confirmed` | 绿色 `confirmed` |
| `advisory` | 黄色 `advisory` |
| `failed` | 红色 `failed` |
### 7.3 Access badge
| API 值 | 页面 badge |
|---|---|
| `unknown` | 灰色 `unknown` |
| `active` | 绿色 `active` |
| `degraded` | 黄色 `degraded` |
| `broken` | 红色 `broken` |
## 8. 分页与排序建议
### 8.1 Runs 列表
默认排序:
- `started_at DESC`
### 8.2 Items 列表
默认排序:
1. `current_stage != done` 的在前
2. `access_status = broken/degraded` 在前
3. `updated_at DESC`
原因:
- 结果页首先服务“快速定位问题”
## 9. Handler 约束
1. 列表接口只返回 projection不返回原始 JSON 大对象
2. item 详情才返回 `capability_profile``events`
3. 不在 handler 里拼装 legacy 表结果
4. 任何页面要显示的 warning 文案,都从 projection 层统一生成
## 10. 最小实现建议
实现顺序建议:
1. 先做 `POST /api/batch-import/runs`
2. 再做 `GET /api/batch-import/runs`
3. 再做 `GET /api/batch-import/runs/{run_id}`
4. 再做 item 列表
5. 最后做 item 详情和 event trail
理由:
- 列表页和 run 详情先能支撑运营查看批次
- 复杂度最高的是 item detail / event trail
## 11. 验证点
文档层:
- 是否所有状态枚举都与 `SPEC/TDD/Architecture` 一致
- 是否还存在 `warning``completed_with_warnings` 混用
- 是否还存在 `confirmed_active` 等旧枚举残留
实现层:
1. `subscription/self_service` 条件必填校验正确
2. run 列表能稳定分页
3. item 详情能展示 capability + event trail
4. 页面无需读取宿主数据库即可解释 warning/broken

View File

@@ -0,0 +1,361 @@
# V2 数据库 Migration 草案 — Batch Auto-Import
日期2026-05-22
关联文档:
- `docs/2026-05-21-BATCH_AUTO_IMPORT_SPEC.md`
- `docs/2026-05-21-BATCH_AUTO_IMPORT_TDD_PLAN.md`
- `docs/2026-05-22-BATCH_AUTO_IMPORT_V2_ARCHITECTURE.md`
## 1. 目标
这份草案定义 V2 在控制面 SQLite 上需要新增的持久化结构,以及从现有 v1 运行态表平滑接入的方式。
V2 的目标不是替换 v1 的执行链,而是新增一套**面向长任务、异步确认、结果页**的 canonical runtime store。
## 2. 设计原则
1. **不破坏现有 v1 表**
- `import_batches`
- `import_batch_items`
- `managed_resources`
- `probe_results`
- `access_closure_records`
2. **V2 新增自己的 canonical state**
- `import_runs`
- `import_run_items`
- `import_run_item_events`
3. **结果页/API 只读 V2 新表**
4. **允许 legacy link**
- item 可以记录 `legacy_batch_id`
- item 可以记录 `legacy_provider_id`
- 仅用于追溯,不用于投影
5. **按 SQLite 友好方式设计**
- 不使用复杂 JSON 索引
- 关键筛选字段保留标量列
- 复杂结构用 `TEXT` JSON 保存
## 3. 迁移命名建议
建议新增两条 migration
1. `0007_batch_import_runs.sql`
2. `0008_batch_import_run_events.sql`
原因:
- `0007` 先建立 run / item 主体
- `0008` 再建立 event trail 与二级索引
这样 rollback/debug 更简单,也便于先实现状态库,再补页面事件流。
## 4. `0007_batch_import_runs.sql`
### 4.1 `import_runs`
```sql
CREATE TABLE import_runs (
run_id TEXT PRIMARY KEY,
mode TEXT NOT NULL,
access_mode TEXT NOT NULL,
state TEXT NOT NULL,
total_items INTEGER NOT NULL DEFAULT 0,
completed_items INTEGER NOT NULL DEFAULT 0,
active_items INTEGER NOT NULL DEFAULT 0,
degraded_items INTEGER NOT NULL DEFAULT 0,
broken_items INTEGER NOT NULL DEFAULT 0,
warning_items INTEGER NOT NULL DEFAULT 0,
started_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
finished_at TEXT NULL,
CHECK (mode IN ('strict', 'partial')),
CHECK (access_mode IN ('subscription', 'self_service')),
CHECK (state IN ('running', 'completed', 'completed_with_warnings', 'failed', 'cancelled'))
);
```
索引:
```sql
CREATE INDEX idx_import_runs_started_at ON import_runs(started_at DESC);
CREATE INDEX idx_import_runs_state ON import_runs(state);
CREATE INDEX idx_import_runs_access_mode ON import_runs(access_mode);
```
### 4.2 `import_run_items`
```sql
CREATE TABLE import_run_items (
item_id TEXT PRIMARY KEY,
run_id TEXT NOT NULL,
base_url TEXT NOT NULL,
provider_id TEXT NOT NULL,
requested_models_json TEXT NOT NULL DEFAULT '[]',
raw_models_json TEXT NOT NULL DEFAULT '[]',
normalized_models_json TEXT NOT NULL DEFAULT '[]',
recommended_models_json TEXT NOT NULL DEFAULT '[]',
resolved_smoke_model TEXT NULL,
capability_profile_json TEXT NOT NULL DEFAULT '{}',
current_stage TEXT NOT NULL,
confirmation_status TEXT NOT NULL,
access_status TEXT NOT NULL,
channel_id INTEGER NULL,
account_id INTEGER NULL,
retry_count INTEGER NOT NULL DEFAULT 0,
confirmation_attempts INTEGER NOT NULL DEFAULT 0,
last_retry_at TEXT NULL,
next_retry_at TEXT NULL,
lease_owner TEXT NULL,
lease_until TEXT NULL,
advisory_messages_json TEXT NOT NULL DEFAULT '[]',
last_error_stage TEXT NULL,
last_error TEXT NULL,
legacy_batch_id INTEGER NULL,
legacy_provider_id TEXT NULL,
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (run_id) REFERENCES import_runs(run_id) ON DELETE CASCADE,
CHECK (current_stage IN ('probe', 'provision', 'confirm', 'validate', 'done')),
CHECK (confirmation_status IN ('pending', 'confirmed', 'advisory', 'failed')),
CHECK (access_status IN ('unknown', 'active', 'degraded', 'broken'))
);
```
索引:
```sql
CREATE INDEX idx_import_run_items_run_id ON import_run_items(run_id);
CREATE INDEX idx_import_run_items_provider_id ON import_run_items(provider_id);
CREATE INDEX idx_import_run_items_current_stage ON import_run_items(current_stage);
CREATE INDEX idx_import_run_items_confirmation_status ON import_run_items(confirmation_status);
CREATE INDEX idx_import_run_items_access_status ON import_run_items(access_status);
CREATE INDEX idx_import_run_items_next_retry_at ON import_run_items(next_retry_at);
CREATE INDEX idx_import_run_items_lease_until ON import_run_items(lease_until);
```
### 4.3 为什么这些列是标量
下列字段必须保留标量列,不能只藏在 JSON 里:
- `provider_id`
- `current_stage`
- `confirmation_status`
- `access_status`
- `next_retry_at`
- `lease_until`
原因:
- worker 要按这些字段轮询
- 结果页列表要按这些字段筛选
- SQLite 下从 JSON 中筛选成本高、代码复杂度高
## 5. `0008_batch_import_run_events.sql`
### 5.1 `import_run_item_events`
```sql
CREATE TABLE import_run_item_events (
event_id TEXT PRIMARY KEY,
run_id TEXT NOT NULL,
item_id TEXT NOT NULL,
event_type TEXT NOT NULL,
stage TEXT NOT NULL,
attempt INTEGER NOT NULL DEFAULT 0,
message TEXT NOT NULL,
payload_json TEXT NOT NULL DEFAULT '{}',
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (run_id) REFERENCES import_runs(run_id) ON DELETE CASCADE,
FOREIGN KEY (item_id) REFERENCES import_run_items(item_id) ON DELETE CASCADE
);
```
索引:
```sql
CREATE INDEX idx_import_run_item_events_run_id ON import_run_item_events(run_id);
CREATE INDEX idx_import_run_item_events_item_id ON import_run_item_events(item_id);
CREATE INDEX idx_import_run_item_events_created_at ON import_run_item_events(created_at);
CREATE INDEX idx_import_run_item_events_stage ON import_run_item_events(stage);
CREATE INDEX idx_import_run_item_events_type ON import_run_item_events(event_type);
```
### 5.2 事件类型建议
建议第一版固定这些值:
- `stage_transition`
- `probe_result`
- `provision_result`
- `retry_scheduled`
- `retry_started`
- `advisory_added`
- `confirmation_result`
- `validation_result`
不要在第一版做开放式自由文本事件类型,以免前后端难以统一。
## 6. 回填与迁移策略
### 6.1 不做历史全量回填
V2 第一版**不要求**把所有旧 `import_batches` 回填成 `import_runs`
原因:
- 历史数据缺少 `retry trail`
- 缺少 capability profile
- 缺少 confirmation 语义
- 强行回填会制造伪精度
### 6.2 从启用 V2 后开始写新表
策略:
1. v1 老入口继续写旧表
2. v2 新入口只写新表,并在 item 上记录:
- `legacy_batch_id`
- `legacy_provider_id`
3. 页面/API 只展示 V2 产生的新 run
### 6.3 兼容旧逻辑
如果 V2 内部仍调用现有 provision/import 逻辑:
- 允许继续产生 `import_batches`
- 但 V2 handler 必须在自己的 `import_run_items` 中写投影
也就是说:
- 旧表是“执行副产物”
- 新表是“V2 真相源”
## 7. Repository 草案
建议新增 repo
```text
internal/store/sqlite/
import_runs_repo.go
import_run_items_repo.go
import_run_item_events_repo.go
```
### 7.1 `ImportRunsRepo`
核心方法:
- `Create(ctx, run ImportRun) error`
- `Update(ctx, run ImportRun) error`
- `Get(ctx, runID string) (ImportRun, error)`
- `List(ctx, filter ListImportRunsFilter) ([]ImportRun, error)`
### 7.2 `ImportRunItemsRepo`
核心方法:
- `Create(ctx, item ImportRunItem) error`
- `Update(ctx, item ImportRunItem) error`
- `Get(ctx, itemID string) (ImportRunItem, error)`
- `ListByRun(ctx, runID string, filter ListImportRunItemsFilter) ([]ImportRunItem, error)`
- `AcquireConfirmLease(ctx, now time.Time, workerID string, leaseFor time.Duration, limit int) ([]ImportRunItem, error)`
### 7.3 `ImportRunItemEventsRepo`
核心方法:
- `Append(ctx, event ImportRunItemEvent) error`
- `ListByItem(ctx, itemID string) ([]ImportRunItemEvent, error)`
## 8. Worker 查询草案
Confirmation worker 轮询建议:
```sql
SELECT *
FROM import_run_items
WHERE current_stage = 'confirm'
AND confirmation_status = 'pending'
AND (next_retry_at IS NULL OR next_retry_at <= ?)
AND (lease_until IS NULL OR lease_until < ?)
ORDER BY
CASE WHEN next_retry_at IS NULL THEN 0 ELSE 1 END,
next_retry_at ASC,
updated_at ASC
LIMIT ?;
```
租约写入建议使用“条件更新”方式,避免并发重复领取。
## 9. 聚合策略
`import_runs` 的统计字段不建议每次列表查询时现算,建议:
- item 更新后同步回写 run summary
- 或在同一事务里重算该 run 的计数
原因:
- run 列表页是高频读
- SQLite 上临时聚合 JSON 和事件表不划算
建议聚合规则:
- `completed_items``current_stage='done'`
- `active_items``access_status='active'`
- `degraded_items``access_status='degraded'`
- `broken_items``access_status='broken'`
- `warning_items``confirmation_status='advisory' OR access_status='degraded'`
## 10. 非功能约束
1. **幂等**
- `run_id``item_id` 由控制面生成,不能依赖数据库自增主键做外部 API 标识
2. **可恢复**
- 任何阶段切换前后都要先写 item
- 任何 retry 调度都要落 event
3. **可审计**
- 详情页必须能从 event trail 解释 warning/broken
4. **可删**
- 删 run 时item 和 event 级联删除
## 11. 实施顺序建议
1. 先上 `0007_batch_import_runs.sql`
2. 实现 `ImportRunsRepo` / `ImportRunItemsRepo`
3. 让 batch service 在 Stage 0~2 先写新表
4. 再上 `0008_batch_import_run_events.sql`
5. 引入 confirmation worker 和 event trail
6. 最后再接结果页/API
## 12. 验证点
文档级实现前检查:
- 新表是否足以支撑结果页所有字段
- 是否存在必须从 legacy 表实时拼接才能看到的字段
- worker 是否能只靠新表完成 confirm/resume
实现后建议最小验证:
1. create run → items 可写入
2. item 进入 confirm → `next_retry_at/lease` 可更新
3. event trail 可回放
4. 控制面重启后 unfinished item 可重新被 worker 捞起

View File

@@ -148,6 +148,8 @@
**文档**`docs/2026-05-21-BATCH_AUTO_IMPORT_SPEC.md`(需求规格)
**TDD 计划**`docs/2026-05-21-BATCH_AUTO_IMPORT_TDD_PLAN.md`(实现路径,已确认开放问题)
**技术架构**`docs/2026-05-22-BATCH_AUTO_IMPORT_V2_ARCHITECTURE.md`运行态状态库、结果页、API、页面字段布局
**Migration 草案**`docs/2026-05-22-BATCH_AUTO_IMPORT_V2_MIGRATION_DRAFT.md`SQLite 新表、索引、lease/retry 字段、legacy link
**API Schema 细稿**`docs/2026-05-22-BATCH_AUTO_IMPORT_V2_API_SCHEMAS.md`run/item 响应结构、筛选参数、badge 文案、错误语义)
**本轮设计收敛**
- 已把真实验收中的三类高频问题写入 v2 方案:
@@ -169,8 +171,9 @@
- OpenAPI 已补齐 `/api/batch-import/runs*`legacy `/api/import-batches/*` 降级为 v1/legacy
**当前剩余项**
- [ ] 按收口后的 canonical contract 输出数据库 migration 草案
- [ ] 按收口后的 OpenAPI 与 projection 字段开始实现
- [x] 按收口后的 canonical contract 输出数据库 migration 草案
- [x] 补齐 run/item API response schema 细稿
- [ ] 按收口后的 OpenAPI、migration、projection 字段开始实现
- [ ] 进入实现前再做一次实现前审阅,确认没有新增分叉
**实现前 Gate**:文档级 review 问题已收口,当前可以进入“按文档写 migration / 接口 / worker”的实现准备阶段