docs(v2): add migration and api schema drafts
This commit is contained in:
403
docs/2026-05-22-BATCH_AUTO_IMPORT_V2_API_SCHEMAS.md
Normal file
403
docs/2026-05-22-BATCH_AUTO_IMPORT_V2_API_SCHEMAS.md
Normal file
@@ -0,0 +1,403 @@
|
||||
# V2 API Response Schema 细稿 — Batch Auto-Import
|
||||
|
||||
日期:2026-05-22
|
||||
关联文档:
|
||||
|
||||
- `docs/2026-05-21-BATCH_AUTO_IMPORT_SPEC.md`
|
||||
- `docs/2026-05-21-BATCH_AUTO_IMPORT_TDD_PLAN.md`
|
||||
- `docs/2026-05-22-BATCH_AUTO_IMPORT_V2_ARCHITECTURE.md`
|
||||
- `docs/openapi.yaml`
|
||||
|
||||
## 1. 目标
|
||||
|
||||
这份文档把 V2 结果 API 细化到 handler 和前端都可直接对照实现的粒度。
|
||||
|
||||
覆盖:
|
||||
|
||||
- 路由
|
||||
- query/filter
|
||||
- response schema
|
||||
- error schema
|
||||
- badge / 文案映射
|
||||
- 示例 JSON
|
||||
|
||||
## 2. 资源模型
|
||||
|
||||
V2 API 只暴露 3 层资源:
|
||||
|
||||
1. `run`
|
||||
2. `run items`
|
||||
3. `run item events`
|
||||
|
||||
不直接暴露:
|
||||
|
||||
- 宿主数据库结构
|
||||
- `import_batches` 细节
|
||||
- legacy repo 的内部实现字段
|
||||
|
||||
## 3. Endpoint 列表
|
||||
|
||||
### 3.1 创建 run
|
||||
|
||||
`POST /api/batch-import/runs`
|
||||
|
||||
作用:
|
||||
|
||||
- 创建 V2 run
|
||||
- 立即返回 `run_id`
|
||||
- 可选等待一个短暂 confirm window,但不保证全部 item 已 done
|
||||
|
||||
### 3.2 列表 runs
|
||||
|
||||
`GET /api/batch-import/runs`
|
||||
|
||||
作用:
|
||||
|
||||
- 查看最近导入批次
|
||||
- 支持状态筛选和关键词搜索
|
||||
|
||||
### 3.3 run 详情
|
||||
|
||||
`GET /api/batch-import/runs/{run_id}`
|
||||
|
||||
作用:
|
||||
|
||||
- 返回 run summary
|
||||
- 返回汇总告警摘要
|
||||
|
||||
### 3.4 item 列表
|
||||
|
||||
`GET /api/batch-import/runs/{run_id}/items`
|
||||
|
||||
作用:
|
||||
|
||||
- 展示批次内各 URL/provider 的状态
|
||||
- 支持按 stage/access/warning 过滤
|
||||
|
||||
### 3.5 item 详情
|
||||
|
||||
`GET /api/batch-import/runs/{run_id}/items/{item_id}`
|
||||
|
||||
作用:
|
||||
|
||||
- 返回一个 item 的全量投影
|
||||
- 包括 capability、模型纠错、event trail
|
||||
|
||||
## 4. Query 参数约定
|
||||
|
||||
### 4.1 `GET /api/batch-import/runs`
|
||||
|
||||
支持:
|
||||
|
||||
- `state`
|
||||
- `running`
|
||||
- `completed`
|
||||
- `completed_with_warnings`
|
||||
- `failed`
|
||||
- `cancelled`
|
||||
- `access_mode`
|
||||
- `subscription`
|
||||
- `self_service`
|
||||
- `q`
|
||||
- 搜索 `run_id / provider_id / base_url`
|
||||
- `limit`
|
||||
- `cursor`
|
||||
|
||||
说明:
|
||||
|
||||
- 页面显示可以把 `completed_with_warnings` 渲染成 `warning`
|
||||
- API 不应引入第二套 `warning` 枚举
|
||||
|
||||
### 4.2 `GET /api/batch-import/runs/{run_id}/items`
|
||||
|
||||
支持:
|
||||
|
||||
- `current_stage`
|
||||
- `probe`
|
||||
- `provision`
|
||||
- `confirm`
|
||||
- `validate`
|
||||
- `done`
|
||||
- `confirmation_status`
|
||||
- `pending`
|
||||
- `confirmed`
|
||||
- `advisory`
|
||||
- `failed`
|
||||
- `access_status`
|
||||
- `unknown`
|
||||
- `active`
|
||||
- `degraded`
|
||||
- `broken`
|
||||
- `has_warning`
|
||||
- `true/false`
|
||||
- `provider_id`
|
||||
- `q`
|
||||
- 搜索 `provider_id / base_url / item_id`
|
||||
- `limit`
|
||||
- `cursor`
|
||||
|
||||
## 5. 通用错误结构
|
||||
|
||||
建议统一:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"code": "invalid_request",
|
||||
"message": "subscription_users is required when access_mode=subscription"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
常用 `code`:
|
||||
|
||||
- `invalid_request`
|
||||
- `not_found`
|
||||
- `conflict`
|
||||
- `internal_error`
|
||||
|
||||
## 6. 响应 Schema
|
||||
|
||||
### 6.1 `POST /api/batch-import/runs`
|
||||
|
||||
```json
|
||||
{
|
||||
"run_id": "run_20260522_0001",
|
||||
"state": "running",
|
||||
"result_page": "/batch-import/runs/run_20260522_0001",
|
||||
"total_items": 2,
|
||||
"active_items": 0,
|
||||
"degraded_items": 0,
|
||||
"broken_items": 0,
|
||||
"warning_items": 0
|
||||
}
|
||||
```
|
||||
|
||||
约束:
|
||||
|
||||
- 这是 `run created` 语义,不是 final result
|
||||
- `state` 初始一般是 `running`
|
||||
|
||||
### 6.2 `GET /api/batch-import/runs`
|
||||
|
||||
```json
|
||||
{
|
||||
"runs": [
|
||||
{
|
||||
"run_id": "run_20260522_0001",
|
||||
"state": "completed_with_warnings",
|
||||
"mode": "partial",
|
||||
"access_mode": "subscription",
|
||||
"total_items": 2,
|
||||
"completed_items": 2,
|
||||
"active_items": 1,
|
||||
"degraded_items": 1,
|
||||
"broken_items": 0,
|
||||
"warning_items": 1,
|
||||
"started_at": "2026-05-22T12:20:00+08:00",
|
||||
"finished_at": "2026-05-22T12:20:07+08:00"
|
||||
}
|
||||
],
|
||||
"next_cursor": null
|
||||
}
|
||||
```
|
||||
|
||||
### 6.3 `GET /api/batch-import/runs/{run_id}`
|
||||
|
||||
```json
|
||||
{
|
||||
"run": {
|
||||
"run_id": "run_20260522_0001",
|
||||
"state": "completed_with_warnings",
|
||||
"mode": "partial",
|
||||
"access_mode": "subscription",
|
||||
"total_items": 2,
|
||||
"completed_items": 2,
|
||||
"active_items": 1,
|
||||
"degraded_items": 1,
|
||||
"broken_items": 0,
|
||||
"warning_items": 1,
|
||||
"started_at": "2026-05-22T12:20:00+08:00",
|
||||
"finished_at": "2026-05-22T12:20:07+08:00"
|
||||
},
|
||||
"recent_warnings": [
|
||||
"该批次包含 1 条 advisory item,建议检查 capability profile 与 retry 轨迹"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 6.4 `GET /api/batch-import/runs/{run_id}/items`
|
||||
|
||||
```json
|
||||
{
|
||||
"items": [
|
||||
{
|
||||
"item_id": "item_01",
|
||||
"base_url": "https://kimi.a7m.com.cn/v1",
|
||||
"provider_id": "kimi-a7m-7d7ac291",
|
||||
"requested_models": ["kimi-k2.6"],
|
||||
"resolved_smoke_model": "kimi-k2.6",
|
||||
"current_stage": "done",
|
||||
"confirmation_status": "advisory",
|
||||
"access_status": "active",
|
||||
"retry_count": 2,
|
||||
"last_retry_at": "2026-05-22T12:20:05+08:00",
|
||||
"advisory_messages": [
|
||||
"该上游不支持 /v1/responses,系统已自动回退到 /v1/chat/completions"
|
||||
],
|
||||
"last_error_stage": "confirm",
|
||||
"last_error": "API returned 403: Forbidden"
|
||||
}
|
||||
],
|
||||
"next_cursor": null
|
||||
}
|
||||
```
|
||||
|
||||
### 6.5 `GET /api/batch-import/runs/{run_id}/items/{item_id}`
|
||||
|
||||
```json
|
||||
{
|
||||
"item_id": "item_01",
|
||||
"base_url": "https://kimi.a7m.com.cn/v1",
|
||||
"provider_id": "kimi-a7m-7d7ac291",
|
||||
"requested_models": ["kimi-k2.6"],
|
||||
"raw_models": ["kimi-k2.6"],
|
||||
"normalized_models": ["kimi-k2.6"],
|
||||
"recommended_models": [],
|
||||
"resolved_smoke_model": "kimi-k2.6",
|
||||
"current_stage": "done",
|
||||
"confirmation_status": "advisory",
|
||||
"access_status": "active",
|
||||
"retry_count": 2,
|
||||
"last_retry_at": "2026-05-22T12:20:05+08:00",
|
||||
"channel_id": 12,
|
||||
"account_id": 4,
|
||||
"advisory_messages": [
|
||||
"该上游不支持 /v1/responses,系统已自动回退到 /v1/chat/completions"
|
||||
],
|
||||
"last_error_stage": "confirm",
|
||||
"last_error": "API returned 403: Forbidden",
|
||||
"capability_profile": {
|
||||
"transport_profile": {
|
||||
"supports_openai_models": true,
|
||||
"supports_openai_chat_completions": true,
|
||||
"supports_openai_responses": false,
|
||||
"supports_anthropic_messages": false,
|
||||
"auth_style": "bearer",
|
||||
"model_id_style": "canonical",
|
||||
"known_advisories": [
|
||||
"responses_unsupported_but_chat_ok",
|
||||
"initial_probe_race_expected"
|
||||
]
|
||||
},
|
||||
"model_profiles": [
|
||||
{
|
||||
"raw_model_id": "kimi-k2.6",
|
||||
"normalized_model_id": "kimi-k2.6",
|
||||
"supports_stream": true,
|
||||
"supports_tools": "unknown",
|
||||
"supports_reasoning_fields": "unknown",
|
||||
"smoke_chat_ok": true
|
||||
}
|
||||
]
|
||||
},
|
||||
"events": [
|
||||
{
|
||||
"event_id": "evt_01",
|
||||
"event_type": "retry_scheduled",
|
||||
"stage": "confirm",
|
||||
"attempt": 1,
|
||||
"message": "initial 503 no available accounts, retry scheduled",
|
||||
"payload_json": "{\"delay_ms\":500}",
|
||||
"created_at": "2026-05-22T12:20:04+08:00"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## 7. Badge / 文案映射
|
||||
|
||||
### 7.1 Run state badge
|
||||
|
||||
| API 值 | 页面 badge |
|
||||
|---|---|
|
||||
| `running` | 蓝色 `running` |
|
||||
| `completed` | 绿色 `completed` |
|
||||
| `completed_with_warnings` | 黄色 `warning` |
|
||||
| `failed` | 红色 `failed` |
|
||||
| `cancelled` | 灰色 `cancelled` |
|
||||
|
||||
### 7.2 Confirmation badge
|
||||
|
||||
| API 值 | 页面 badge |
|
||||
|---|---|
|
||||
| `pending` | 蓝色 `pending` |
|
||||
| `confirmed` | 绿色 `confirmed` |
|
||||
| `advisory` | 黄色 `advisory` |
|
||||
| `failed` | 红色 `failed` |
|
||||
|
||||
### 7.3 Access badge
|
||||
|
||||
| API 值 | 页面 badge |
|
||||
|---|---|
|
||||
| `unknown` | 灰色 `unknown` |
|
||||
| `active` | 绿色 `active` |
|
||||
| `degraded` | 黄色 `degraded` |
|
||||
| `broken` | 红色 `broken` |
|
||||
|
||||
## 8. 分页与排序建议
|
||||
|
||||
### 8.1 Runs 列表
|
||||
|
||||
默认排序:
|
||||
|
||||
- `started_at DESC`
|
||||
|
||||
### 8.2 Items 列表
|
||||
|
||||
默认排序:
|
||||
|
||||
1. `current_stage != done` 的在前
|
||||
2. `access_status = broken/degraded` 在前
|
||||
3. `updated_at DESC`
|
||||
|
||||
原因:
|
||||
|
||||
- 结果页首先服务“快速定位问题”
|
||||
|
||||
## 9. Handler 约束
|
||||
|
||||
1. 列表接口只返回 projection,不返回原始 JSON 大对象
|
||||
2. item 详情才返回 `capability_profile` 与 `events`
|
||||
3. 不在 handler 里拼装 legacy 表结果
|
||||
4. 任何页面要显示的 warning 文案,都从 projection 层统一生成
|
||||
|
||||
## 10. 最小实现建议
|
||||
|
||||
实现顺序建议:
|
||||
|
||||
1. 先做 `POST /api/batch-import/runs`
|
||||
2. 再做 `GET /api/batch-import/runs`
|
||||
3. 再做 `GET /api/batch-import/runs/{run_id}`
|
||||
4. 再做 item 列表
|
||||
5. 最后做 item 详情和 event trail
|
||||
|
||||
理由:
|
||||
|
||||
- 列表页和 run 详情先能支撑运营查看批次
|
||||
- 复杂度最高的是 item detail / event trail
|
||||
|
||||
## 11. 验证点
|
||||
|
||||
文档层:
|
||||
|
||||
- 是否所有状态枚举都与 `SPEC/TDD/Architecture` 一致
|
||||
- 是否还存在 `warning` 与 `completed_with_warnings` 混用
|
||||
- 是否还存在 `confirmed_active` 等旧枚举残留
|
||||
|
||||
实现层:
|
||||
|
||||
1. `subscription/self_service` 条件必填校验正确
|
||||
2. run 列表能稳定分页
|
||||
3. item 详情能展示 capability + event trail
|
||||
4. 页面无需读取宿主数据库即可解释 warning/broken
|
||||
361
docs/2026-05-22-BATCH_AUTO_IMPORT_V2_MIGRATION_DRAFT.md
Normal file
361
docs/2026-05-22-BATCH_AUTO_IMPORT_V2_MIGRATION_DRAFT.md
Normal file
@@ -0,0 +1,361 @@
|
||||
# V2 数据库 Migration 草案 — Batch Auto-Import
|
||||
|
||||
日期:2026-05-22
|
||||
关联文档:
|
||||
|
||||
- `docs/2026-05-21-BATCH_AUTO_IMPORT_SPEC.md`
|
||||
- `docs/2026-05-21-BATCH_AUTO_IMPORT_TDD_PLAN.md`
|
||||
- `docs/2026-05-22-BATCH_AUTO_IMPORT_V2_ARCHITECTURE.md`
|
||||
|
||||
## 1. 目标
|
||||
|
||||
这份草案定义 V2 在控制面 SQLite 上需要新增的持久化结构,以及从现有 v1 运行态表平滑接入的方式。
|
||||
|
||||
V2 的目标不是替换 v1 的执行链,而是新增一套**面向长任务、异步确认、结果页**的 canonical runtime store。
|
||||
|
||||
## 2. 设计原则
|
||||
|
||||
1. **不破坏现有 v1 表**
|
||||
- `import_batches`
|
||||
- `import_batch_items`
|
||||
- `managed_resources`
|
||||
- `probe_results`
|
||||
- `access_closure_records`
|
||||
|
||||
2. **V2 新增自己的 canonical state**
|
||||
- `import_runs`
|
||||
- `import_run_items`
|
||||
- `import_run_item_events`
|
||||
|
||||
3. **结果页/API 只读 V2 新表**
|
||||
|
||||
4. **允许 legacy link**
|
||||
- item 可以记录 `legacy_batch_id`
|
||||
- item 可以记录 `legacy_provider_id`
|
||||
- 仅用于追溯,不用于投影
|
||||
|
||||
5. **按 SQLite 友好方式设计**
|
||||
- 不使用复杂 JSON 索引
|
||||
- 关键筛选字段保留标量列
|
||||
- 复杂结构用 `TEXT` JSON 保存
|
||||
|
||||
## 3. 迁移命名建议
|
||||
|
||||
建议新增两条 migration:
|
||||
|
||||
1. `0007_batch_import_runs.sql`
|
||||
2. `0008_batch_import_run_events.sql`
|
||||
|
||||
原因:
|
||||
|
||||
- `0007` 先建立 run / item 主体
|
||||
- `0008` 再建立 event trail 与二级索引
|
||||
|
||||
这样 rollback/debug 更简单,也便于先实现状态库,再补页面事件流。
|
||||
|
||||
## 4. `0007_batch_import_runs.sql`
|
||||
|
||||
### 4.1 `import_runs`
|
||||
|
||||
```sql
|
||||
CREATE TABLE import_runs (
|
||||
run_id TEXT PRIMARY KEY,
|
||||
mode TEXT NOT NULL,
|
||||
access_mode TEXT NOT NULL,
|
||||
state TEXT NOT NULL,
|
||||
total_items INTEGER NOT NULL DEFAULT 0,
|
||||
completed_items INTEGER NOT NULL DEFAULT 0,
|
||||
active_items INTEGER NOT NULL DEFAULT 0,
|
||||
degraded_items INTEGER NOT NULL DEFAULT 0,
|
||||
broken_items INTEGER NOT NULL DEFAULT 0,
|
||||
warning_items INTEGER NOT NULL DEFAULT 0,
|
||||
started_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
finished_at TEXT NULL,
|
||||
CHECK (mode IN ('strict', 'partial')),
|
||||
CHECK (access_mode IN ('subscription', 'self_service')),
|
||||
CHECK (state IN ('running', 'completed', 'completed_with_warnings', 'failed', 'cancelled'))
|
||||
);
|
||||
```
|
||||
|
||||
索引:
|
||||
|
||||
```sql
|
||||
CREATE INDEX idx_import_runs_started_at ON import_runs(started_at DESC);
|
||||
CREATE INDEX idx_import_runs_state ON import_runs(state);
|
||||
CREATE INDEX idx_import_runs_access_mode ON import_runs(access_mode);
|
||||
```
|
||||
|
||||
### 4.2 `import_run_items`
|
||||
|
||||
```sql
|
||||
CREATE TABLE import_run_items (
|
||||
item_id TEXT PRIMARY KEY,
|
||||
run_id TEXT NOT NULL,
|
||||
base_url TEXT NOT NULL,
|
||||
provider_id TEXT NOT NULL,
|
||||
requested_models_json TEXT NOT NULL DEFAULT '[]',
|
||||
raw_models_json TEXT NOT NULL DEFAULT '[]',
|
||||
normalized_models_json TEXT NOT NULL DEFAULT '[]',
|
||||
recommended_models_json TEXT NOT NULL DEFAULT '[]',
|
||||
resolved_smoke_model TEXT NULL,
|
||||
capability_profile_json TEXT NOT NULL DEFAULT '{}',
|
||||
|
||||
current_stage TEXT NOT NULL,
|
||||
confirmation_status TEXT NOT NULL,
|
||||
access_status TEXT NOT NULL,
|
||||
|
||||
channel_id INTEGER NULL,
|
||||
account_id INTEGER NULL,
|
||||
|
||||
retry_count INTEGER NOT NULL DEFAULT 0,
|
||||
confirmation_attempts INTEGER NOT NULL DEFAULT 0,
|
||||
last_retry_at TEXT NULL,
|
||||
next_retry_at TEXT NULL,
|
||||
|
||||
lease_owner TEXT NULL,
|
||||
lease_until TEXT NULL,
|
||||
|
||||
advisory_messages_json TEXT NOT NULL DEFAULT '[]',
|
||||
last_error_stage TEXT NULL,
|
||||
last_error TEXT NULL,
|
||||
|
||||
legacy_batch_id INTEGER NULL,
|
||||
legacy_provider_id TEXT NULL,
|
||||
|
||||
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
|
||||
FOREIGN KEY (run_id) REFERENCES import_runs(run_id) ON DELETE CASCADE,
|
||||
|
||||
CHECK (current_stage IN ('probe', 'provision', 'confirm', 'validate', 'done')),
|
||||
CHECK (confirmation_status IN ('pending', 'confirmed', 'advisory', 'failed')),
|
||||
CHECK (access_status IN ('unknown', 'active', 'degraded', 'broken'))
|
||||
);
|
||||
```
|
||||
|
||||
索引:
|
||||
|
||||
```sql
|
||||
CREATE INDEX idx_import_run_items_run_id ON import_run_items(run_id);
|
||||
CREATE INDEX idx_import_run_items_provider_id ON import_run_items(provider_id);
|
||||
CREATE INDEX idx_import_run_items_current_stage ON import_run_items(current_stage);
|
||||
CREATE INDEX idx_import_run_items_confirmation_status ON import_run_items(confirmation_status);
|
||||
CREATE INDEX idx_import_run_items_access_status ON import_run_items(access_status);
|
||||
CREATE INDEX idx_import_run_items_next_retry_at ON import_run_items(next_retry_at);
|
||||
CREATE INDEX idx_import_run_items_lease_until ON import_run_items(lease_until);
|
||||
```
|
||||
|
||||
### 4.3 为什么这些列是标量
|
||||
|
||||
下列字段必须保留标量列,不能只藏在 JSON 里:
|
||||
|
||||
- `provider_id`
|
||||
- `current_stage`
|
||||
- `confirmation_status`
|
||||
- `access_status`
|
||||
- `next_retry_at`
|
||||
- `lease_until`
|
||||
|
||||
原因:
|
||||
|
||||
- worker 要按这些字段轮询
|
||||
- 结果页列表要按这些字段筛选
|
||||
- SQLite 下从 JSON 中筛选成本高、代码复杂度高
|
||||
|
||||
## 5. `0008_batch_import_run_events.sql`
|
||||
|
||||
### 5.1 `import_run_item_events`
|
||||
|
||||
```sql
|
||||
CREATE TABLE import_run_item_events (
|
||||
event_id TEXT PRIMARY KEY,
|
||||
run_id TEXT NOT NULL,
|
||||
item_id TEXT NOT NULL,
|
||||
event_type TEXT NOT NULL,
|
||||
stage TEXT NOT NULL,
|
||||
attempt INTEGER NOT NULL DEFAULT 0,
|
||||
message TEXT NOT NULL,
|
||||
payload_json TEXT NOT NULL DEFAULT '{}',
|
||||
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
|
||||
FOREIGN KEY (run_id) REFERENCES import_runs(run_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY (item_id) REFERENCES import_run_items(item_id) ON DELETE CASCADE
|
||||
);
|
||||
```
|
||||
|
||||
索引:
|
||||
|
||||
```sql
|
||||
CREATE INDEX idx_import_run_item_events_run_id ON import_run_item_events(run_id);
|
||||
CREATE INDEX idx_import_run_item_events_item_id ON import_run_item_events(item_id);
|
||||
CREATE INDEX idx_import_run_item_events_created_at ON import_run_item_events(created_at);
|
||||
CREATE INDEX idx_import_run_item_events_stage ON import_run_item_events(stage);
|
||||
CREATE INDEX idx_import_run_item_events_type ON import_run_item_events(event_type);
|
||||
```
|
||||
|
||||
### 5.2 事件类型建议
|
||||
|
||||
建议第一版固定这些值:
|
||||
|
||||
- `stage_transition`
|
||||
- `probe_result`
|
||||
- `provision_result`
|
||||
- `retry_scheduled`
|
||||
- `retry_started`
|
||||
- `advisory_added`
|
||||
- `confirmation_result`
|
||||
- `validation_result`
|
||||
|
||||
不要在第一版做开放式自由文本事件类型,以免前后端难以统一。
|
||||
|
||||
## 6. 回填与迁移策略
|
||||
|
||||
### 6.1 不做历史全量回填
|
||||
|
||||
V2 第一版**不要求**把所有旧 `import_batches` 回填成 `import_runs`。
|
||||
|
||||
原因:
|
||||
|
||||
- 历史数据缺少 `retry trail`
|
||||
- 缺少 capability profile
|
||||
- 缺少 confirmation 语义
|
||||
- 强行回填会制造伪精度
|
||||
|
||||
### 6.2 从启用 V2 后开始写新表
|
||||
|
||||
策略:
|
||||
|
||||
1. v1 老入口继续写旧表
|
||||
2. v2 新入口只写新表,并在 item 上记录:
|
||||
- `legacy_batch_id`
|
||||
- `legacy_provider_id`
|
||||
3. 页面/API 只展示 V2 产生的新 run
|
||||
|
||||
### 6.3 兼容旧逻辑
|
||||
|
||||
如果 V2 内部仍调用现有 provision/import 逻辑:
|
||||
|
||||
- 允许继续产生 `import_batches`
|
||||
- 但 V2 handler 必须在自己的 `import_run_items` 中写投影
|
||||
|
||||
也就是说:
|
||||
|
||||
- 旧表是“执行副产物”
|
||||
- 新表是“V2 真相源”
|
||||
|
||||
## 7. Repository 草案
|
||||
|
||||
建议新增 repo:
|
||||
|
||||
```text
|
||||
internal/store/sqlite/
|
||||
import_runs_repo.go
|
||||
import_run_items_repo.go
|
||||
import_run_item_events_repo.go
|
||||
```
|
||||
|
||||
### 7.1 `ImportRunsRepo`
|
||||
|
||||
核心方法:
|
||||
|
||||
- `Create(ctx, run ImportRun) error`
|
||||
- `Update(ctx, run ImportRun) error`
|
||||
- `Get(ctx, runID string) (ImportRun, error)`
|
||||
- `List(ctx, filter ListImportRunsFilter) ([]ImportRun, error)`
|
||||
|
||||
### 7.2 `ImportRunItemsRepo`
|
||||
|
||||
核心方法:
|
||||
|
||||
- `Create(ctx, item ImportRunItem) error`
|
||||
- `Update(ctx, item ImportRunItem) error`
|
||||
- `Get(ctx, itemID string) (ImportRunItem, error)`
|
||||
- `ListByRun(ctx, runID string, filter ListImportRunItemsFilter) ([]ImportRunItem, error)`
|
||||
- `AcquireConfirmLease(ctx, now time.Time, workerID string, leaseFor time.Duration, limit int) ([]ImportRunItem, error)`
|
||||
|
||||
### 7.3 `ImportRunItemEventsRepo`
|
||||
|
||||
核心方法:
|
||||
|
||||
- `Append(ctx, event ImportRunItemEvent) error`
|
||||
- `ListByItem(ctx, itemID string) ([]ImportRunItemEvent, error)`
|
||||
|
||||
## 8. Worker 查询草案
|
||||
|
||||
Confirmation worker 轮询建议:
|
||||
|
||||
```sql
|
||||
SELECT *
|
||||
FROM import_run_items
|
||||
WHERE current_stage = 'confirm'
|
||||
AND confirmation_status = 'pending'
|
||||
AND (next_retry_at IS NULL OR next_retry_at <= ?)
|
||||
AND (lease_until IS NULL OR lease_until < ?)
|
||||
ORDER BY
|
||||
CASE WHEN next_retry_at IS NULL THEN 0 ELSE 1 END,
|
||||
next_retry_at ASC,
|
||||
updated_at ASC
|
||||
LIMIT ?;
|
||||
```
|
||||
|
||||
租约写入建议使用“条件更新”方式,避免并发重复领取。
|
||||
|
||||
## 9. 聚合策略
|
||||
|
||||
`import_runs` 的统计字段不建议每次列表查询时现算,建议:
|
||||
|
||||
- item 更新后同步回写 run summary
|
||||
- 或在同一事务里重算该 run 的计数
|
||||
|
||||
原因:
|
||||
|
||||
- run 列表页是高频读
|
||||
- SQLite 上临时聚合 JSON 和事件表不划算
|
||||
|
||||
建议聚合规则:
|
||||
|
||||
- `completed_items`:`current_stage='done'`
|
||||
- `active_items`:`access_status='active'`
|
||||
- `degraded_items`:`access_status='degraded'`
|
||||
- `broken_items`:`access_status='broken'`
|
||||
- `warning_items`:`confirmation_status='advisory' OR access_status='degraded'`
|
||||
|
||||
## 10. 非功能约束
|
||||
|
||||
1. **幂等**
|
||||
- `run_id`、`item_id` 由控制面生成,不能依赖数据库自增主键做外部 API 标识
|
||||
|
||||
2. **可恢复**
|
||||
- 任何阶段切换前后都要先写 item
|
||||
- 任何 retry 调度都要落 event
|
||||
|
||||
3. **可审计**
|
||||
- 详情页必须能从 event trail 解释 warning/broken
|
||||
|
||||
4. **可删**
|
||||
- 删 run 时,item 和 event 级联删除
|
||||
|
||||
## 11. 实施顺序建议
|
||||
|
||||
1. 先上 `0007_batch_import_runs.sql`
|
||||
2. 实现 `ImportRunsRepo` / `ImportRunItemsRepo`
|
||||
3. 让 batch service 在 Stage 0~2 先写新表
|
||||
4. 再上 `0008_batch_import_run_events.sql`
|
||||
5. 引入 confirmation worker 和 event trail
|
||||
6. 最后再接结果页/API
|
||||
|
||||
## 12. 验证点
|
||||
|
||||
文档级实现前检查:
|
||||
|
||||
- 新表是否足以支撑结果页所有字段
|
||||
- 是否存在必须从 legacy 表实时拼接才能看到的字段
|
||||
- worker 是否能只靠新表完成 confirm/resume
|
||||
|
||||
实现后建议最小验证:
|
||||
|
||||
1. create run → items 可写入
|
||||
2. item 进入 confirm → `next_retry_at/lease` 可更新
|
||||
3. event trail 可回放
|
||||
4. 控制面重启后 unfinished item 可重新被 worker 捞起
|
||||
@@ -148,6 +148,8 @@
|
||||
**文档**:`docs/2026-05-21-BATCH_AUTO_IMPORT_SPEC.md`(需求规格)
|
||||
**TDD 计划**:`docs/2026-05-21-BATCH_AUTO_IMPORT_TDD_PLAN.md`(实现路径,已确认开放问题)
|
||||
**技术架构**:`docs/2026-05-22-BATCH_AUTO_IMPORT_V2_ARCHITECTURE.md`(运行态状态库、结果页、API、页面字段布局)
|
||||
**Migration 草案**:`docs/2026-05-22-BATCH_AUTO_IMPORT_V2_MIGRATION_DRAFT.md`(SQLite 新表、索引、lease/retry 字段、legacy link)
|
||||
**API Schema 细稿**:`docs/2026-05-22-BATCH_AUTO_IMPORT_V2_API_SCHEMAS.md`(run/item 响应结构、筛选参数、badge 文案、错误语义)
|
||||
|
||||
**本轮设计收敛**:
|
||||
- 已把真实验收中的三类高频问题写入 v2 方案:
|
||||
@@ -169,8 +171,9 @@
|
||||
- OpenAPI 已补齐 `/api/batch-import/runs*`,legacy `/api/import-batches/*` 降级为 v1/legacy
|
||||
|
||||
**当前剩余项**:
|
||||
- [ ] 按收口后的 canonical contract 输出数据库 migration 草案
|
||||
- [ ] 按收口后的 OpenAPI 与 projection 字段开始实现
|
||||
- [x] 按收口后的 canonical contract 输出数据库 migration 草案
|
||||
- [x] 补齐 run/item API response schema 细稿
|
||||
- [ ] 按收口后的 OpenAPI、migration、projection 字段开始实现
|
||||
- [ ] 进入实现前再做一次实现前审阅,确认没有新增分叉
|
||||
|
||||
**实现前 Gate**:文档级 review 问题已收口,当前可以进入“按文档写 migration / 接口 / worker”的实现准备阶段
|
||||
|
||||
Reference in New Issue
Block a user