docs(ci): define cross-service smoke taxonomy

This commit is contained in:
Your Name
2026-04-21 09:38:37 +08:00
parent b3e34c6e36
commit c59204049a
8 changed files with 180 additions and 19 deletions

View File

@@ -132,3 +132,23 @@ git diff --check
3. 已在 `supply-api/cmd/supply-api/main.go` 增加 `staging/prod + config.dev.yaml` 的 fail-fast 拒绝规则,并补充命令行测试覆盖;同时新增 `supply-api/config/config.staging.example.yaml``supply-api/config/config.prod.example.yaml` 模板骨架。
4. 已在 `scripts/devtest/start_dev_stack.sh` 引入 `LIJIAOQIAO_DEVTEST_SUPPLY_CONFIG` 参数,默认改用 `./config/config.staging.example.yaml`,去除 staging 启动链路里的 `config.dev.yaml` 硬编码。
5. `bash -n scripts/devtest/start_dev_stack.sh``go test ./internal/config ./internal/app``go test ./cmd/supply-api ./internal/app ./internal/config` 均通过,且 `git diff --check` 无格式错误。
## P2-D 测试分类与跨服务 smoke 设计完成
执行命令:
```bash
bash -n scripts/ci/backend-verify.sh
bash -n scripts/ci/repo_integrity_check.sh
bash -n scripts/ci/cross_service_smoke.sh
go test -tags=e2e ./e2e
git diff --check
```
执行结果:
1. 已在 `supply-api/e2e/README.md``supply-api/e2e/e2e_test.go` 明确当前 build-tag 套件属于 `service-http`,边界是“单进程、单服务 HTTP surface”不再冒充真实部署 E2E。
2. 已新增 `tests/smoke/README.md`,把测试分类收敛为 `unit``integration``service-http``cross-service-smoke`,并定义最小链路 `gateway -> token-runtime -> supply-api`
3. 已新增 `scripts/ci/cross_service_smoke.sh` 设计骨架,写明 smoke 输入环境变量、输出工件路径以及 `PASS` / `SKIP_LOCAL_PLACEHOLDER` / `FAIL_REAL_SMOKE` 结果契约;当前脚本仍是 design stub不可作为已完成发布证据。
4. 已在 `scripts/ci/backend-verify.sh` 补入 cross-service smoke 入口设计说明,并把 `./e2e` 的对外命名改为 `service-http`;已在 `scripts/ci/repo_integrity_check.sh` 明确其职责仅限代码完整性 gate不是 release gate。
5. `bash -n scripts/ci/backend-verify.sh``bash -n scripts/ci/repo_integrity_check.sh``bash -n scripts/ci/cross_service_smoke.sh``go test -tags=e2e ./e2e` 均通过,且 `git diff --check` 无格式错误。

View File

@@ -384,21 +384,21 @@
- Create: `tests/smoke/README.md`
- Create: `scripts/ci/cross_service_smoke.sh`
- [ ] `P2-D-01` 盘点当前被叫做 `e2e` 的测试真实边界。
- [x] `P2-D-01` 盘点当前被叫做 `e2e` 的测试真实边界。
完成标准:区分进程内、单服务、跨服务三类。
- [ ] `P2-D-02` 定义新的测试分类名。
- [x] `P2-D-02` 定义新的测试分类名。
完成标准:至少包含 `unit``integration``service-http``cross-service-smoke`
- [ ] `P2-D-03``supply-api/e2e/README.md` 改写术语草稿。
- [x] `P2-D-03``supply-api/e2e/README.md` 改写术语草稿。
完成标准README 不再把进程内测试叫成真实部署 E2E。
- [ ] `P2-D-04` 设计 `cross_service_smoke.sh` 的最小链路。
- [x] `P2-D-04` 设计 `cross_service_smoke.sh` 的最小链路。
完成标准:至少覆盖 `gateway -> token-runtime -> supply-api`
- [ ] `P2-D-05` 为 smoke 脚本设计输入环境变量。
- [x] `P2-D-05` 为 smoke 脚本设计输入环境变量。
完成标准包含地址、token、期望模型或 scope。
- [ ] `P2-D-06` 为 smoke 脚本设计输出报告路径。
- [x] `P2-D-06` 为 smoke 脚本设计输出报告路径。
完成标准:输出可被 manifest 收录。
- [ ] `P2-D-07` 设计 `backend-verify.sh` 中新增 smoke 入口。
- [x] `P2-D-07` 设计 `backend-verify.sh` 中新增 smoke 入口。
完成标准:能区分“本地占位跳过”和“真实 smoke 失败”。
- [ ] `P2-D-08` 设计 `repo_integrity_check.sh` 的职责边界说明。
- [x] `P2-D-08` 设计 `repo_integrity_check.sh` 的职责边界说明。
完成标准:明确它不是 release gate只是代码完整性 gate。
---

View File

@@ -11,6 +11,9 @@ CONTRACT_GATE_DOC="${ROOT_DIR}/tests/contract/gateway_token_runtime_supply_chain
CONTRACT_GATE_CHECKLIST="${ROOT_DIR}/docs/plans/2026-04-21-phase1-contract-gate-checklist.md"
CONTRACT_GATE_LOG="${OUT_DIR}/contract_gate_${TS}.log"
CONTRACT_GATE_REPORT="${OUT_DIR}/contract_gate_${TS}.md"
SMOKE_GATE_DOC="${ROOT_DIR}/tests/smoke/README.md"
SMOKE_GATE_LOG="${OUT_DIR}/cross_service_smoke_${TS}.log"
SMOKE_GATE_REPORT="${OUT_DIR}/cross_service_smoke_${TS}.md"
# shellcheck disable=SC1091
source "${LIB_FILE}"
@@ -57,6 +60,7 @@ run_e2e_skip_gate() {
local title="$2"
local out_file="${OUT_DIR}/${step_id,,}_${TS}.out.log"
# 当前 ./e2e 是 supply-api 单服务进程内 HTTP surface 测试,不是跨服务部署 smoke。
log "[INFO] ${step_id} ${title} start"
set +e
bash -lc "cd \"${ROOT_DIR}/supply-api\" && \"${GO_BIN}\" test -tags=e2e -v ./e2e/..." > "${out_file}" 2>&1
@@ -95,7 +99,7 @@ run_step \
run_e2e_skip_gate \
"STEP-04" \
"supply-api E2E gate must not contain placeholder skip"
"supply-api service-http build-tag suite must not contain placeholder skip"
# Phase 1 contract gate execution slot (design only at this stage):
# - command entry: bash "${ROOT_DIR}/scripts/ci/backend-verify.sh" --phase1-contract-gate
@@ -105,6 +109,17 @@ run_e2e_skip_gate \
# - failure semantics: any scenario mismatch, missing required evidence, or non-zero command exit
# must mark the backend verify result as FAIL.
# Phase 2 cross-service smoke slot (design only at this stage):
# - command entry: bash "${ROOT_DIR}/scripts/ci/cross_service_smoke.sh"
# - taxonomy source: ${SMOKE_GATE_DOC}
# - planned artifacts: ${SMOKE_GATE_LOG} and ${SMOKE_GATE_REPORT}
# - expected chain: gateway -> token-runtime -> supply-api
# - status contract:
# * SKIP_LOCAL_PLACEHOLDER: local/mock/placeholder inputs, not a release pass
# * FAIL_REAL_SMOKE: real staging inputs present but any link in the chain fails
# * PASS: real staging smoke succeeds and report is manifest-collectable
# - backend verify must treat SKIP_LOCAL_PLACEHOLDER as non-pass evidence and FAIL_REAL_SMOKE as hard failure.
HAS_FAIL=0
for row in "${STEP_RESULTS[@]}"; do
status="$(echo "${row}" | awk -F'|' '{print $2}')"

View File

@@ -0,0 +1,60 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "$0")/../.." && pwd)"
OUT_DIR="${ROOT_DIR}/reports/archive/gate_verification"
TS="$(date +%F_%H%M%S)"
LOG_FILE="${OUT_DIR}/cross_service_smoke_${TS}.log"
REPORT_FILE="${OUT_DIR}/cross_service_smoke_${TS}.md"
SMOKE_GATEWAY_BASE_URL="${SMOKE_GATEWAY_BASE_URL:-http://127.0.0.1:18080}"
SMOKE_TOKEN_RUNTIME_BASE_URL="${SMOKE_TOKEN_RUNTIME_BASE_URL:-http://127.0.0.1:18081}"
SMOKE_SUPPLY_API_BASE_URL="${SMOKE_SUPPLY_API_BASE_URL:-http://127.0.0.1:18082}"
SMOKE_BEARER_TOKEN="${SMOKE_BEARER_TOKEN:-placeholder-token}"
SMOKE_EXPECTED_SCOPE="${SMOKE_EXPECTED_SCOPE:-supply:read}"
SMOKE_EXPECTED_MODEL="${SMOKE_EXPECTED_MODEL:-gpt-4o-mini}"
SMOKE_ALLOW_LOCAL_PLACEHOLDER="${SMOKE_ALLOW_LOCAL_PLACEHOLDER:-0}"
mkdir -p "${OUT_DIR}"
: > "${LOG_FILE}"
cat > "${REPORT_FILE}" <<EOF
# Cross-Service Smoke Design Report
- 时间戳:${TS}
- 状态:**DESIGN_ONLY**
- gateway${SMOKE_GATEWAY_BASE_URL}
- token-runtime${SMOKE_TOKEN_RUNTIME_BASE_URL}
- supply-api${SMOKE_SUPPLY_API_BASE_URL}
- expected_scope${SMOKE_EXPECTED_SCOPE}
- expected_model${SMOKE_EXPECTED_MODEL}
- allow_local_placeholder${SMOKE_ALLOW_LOCAL_PLACEHOLDER}
## Planned Chain
1. gateway health
2. token-runtime health
3. supply-api health
4. protected request through gateway with real bearer token
5. verify gateway -> token-runtime -> supply-api chain evidence
## Planned Result Contract
- \`PASS\`: real staging smoke passed
- \`SKIP_LOCAL_PLACEHOLDER\`: local/mock/placeholder inputs only
- \`FAIL_REAL_SMOKE\`: real inputs supplied but chain failed
## Note
This script is a Phase P2-D design stub. It defines input/output contracts and artifact paths,
but it must not be treated as completed release evidence yet.
EOF
{
echo "[INFO] cross-service smoke design stub"
echo "[INFO] report: ${REPORT_FILE}"
echo "[INFO] log: ${LOG_FILE}"
echo "[INFO] status: DESIGN_ONLY"
} | tee -a "${LOG_FILE}"
exit 2

View File

@@ -31,7 +31,7 @@ echo "[repo] supply-api repository integration"
cd "${ROOT_DIR}/supply-api"
bash scripts/run_integration_tests.sh ./internal/repository
)
run_go_suite "${ROOT_DIR}" "${GO_BIN}" "supply-api e2e" "supply-api" test -count=1 -tags=e2e ./e2e
run_go_suite "${ROOT_DIR}" "${GO_BIN}" "supply-api service-http" "supply-api" test -count=1 -tags=e2e ./e2e
# Phase 1 contract gate entry (design slot):
# - execute after service-local suites and repository integration
@@ -41,3 +41,8 @@ run_go_suite "${ROOT_DIR}" "${GO_BIN}" "supply-api e2e" "supply-api" test -count
# reports/archive/gate_verification/contract_gate_<timestamp>.md
# - failure semantics: if the contract gate exits non-zero or any required scenario is missing,
# repo_integrity_check must fail and Phase 1 cannot be marked complete.
# Phase 2 boundary note:
# - repo_integrity_check only proves code completeness, syntax, unit/integration and service-local HTTP coverage.
# - cross_service_smoke belongs to release/path validation and must not be redefined as repository integrity.
# - even after smoke is introduced, this script remains a code integrity gate rather than a release gate.

View File

@@ -1,12 +1,19 @@
# E2E 测试说明
# supply-api service-http 测试说明
`e2e/` 目录只存放带 `//go:build e2e` 的端到端测试源码,不再混放伪装成文档的 Go 文件
`e2e/` 目录保留现有 build tag 路径,但这里的测试分类在 Phase P2-D 之后明确记为 `service-http`,不是“真实部署 E2E”
当前测试分层如下
当前边界
- `e2e_test.go`: 核心 HTTP API、鉴权和审计行为的端到端断言
- `playbook_test.go`: 按业务剧本组织的多步骤流程验证
- `production_flow_test.go`: 面向上线前复核的关键流程和安全边界检查
- 进程内:`newE2ESystem` 直接在单进程内装配 handler、内存审计存储、静态 token backend
- 单服务 HTTP覆盖 `supply-api` 的路由、鉴权、审计和关键业务剧本
- 非跨服务:不会启动 `gateway``platform-token-runtime`、真实数据库或外部网络依赖
新的测试分类名:
- `unit`: 单函数、单组件、无跨进程依赖。
- `integration`: 真实数据库/仓储/适配层集成。
- `service-http`: 单服务 HTTP surface在进程内装配依赖。
- `cross-service-smoke`: `gateway -> token-runtime -> supply-api` 的真实跨服务链路。
运行方式:
@@ -22,6 +29,6 @@ go test -tags=e2e ./e2e -run TestPlaybook_SupplierOnboarding
约束说明:
- E2E 测试应保留在 `*_test.go` 文件内。
- 说明文档只保留 Markdown 内容,不内嵌 Go 源码
- 新增剧本时优先复用 `newE2ESystem`,避免重复搭建测试系统。
- `e2e/` 目录下的 build-tag 测试应保留在 `*_test.go` 文件内,但对外一律称为 `service-http`
- 真实跨服务 smoke 不得继续写进 `supply-api/e2e/`,应放到 `tests/smoke/``scripts/ci/cross_service_smoke.sh`
- 新增剧本时优先复用 `newE2ESystem`,避免重复搭建单服务测试系统。

View File

@@ -24,6 +24,9 @@ import (
"lijiaoqiao/supply-api/internal/pkg/logging"
)
// 当前 build-tag 套件运行在单进程内存依赖上,归类为 service-http。
// 它验证 supply-api 的 HTTP surface不代表 gateway -> token-runtime -> supply-api 的真实跨服务 smoke。
type e2eOptions struct {
withdrawEnabled bool
}

51
tests/smoke/README.md Normal file
View File

@@ -0,0 +1,51 @@
# Cross-Service Smoke Design
## 测试边界
`tests/smoke/` 只描述真实跨服务 smoke 的链路与证据,不承载单服务进程内测试。
分类边界:
1. `unit`
- 单函数或单组件测试。
2. `integration`
- 真实数据库、仓储或适配层集成测试。
3. `service-http`
- 单服务 HTTP surface通常在进程内组装依赖。
4. `cross-service-smoke`
- 至少覆盖 `gateway -> token-runtime -> supply-api` 的真实链路。
## 最小链路
最小 smoke 步骤:
1. 检查 `gateway` 健康状态。
2. 检查 `platform-token-runtime` 健康状态。
3. 检查 `supply-api` 健康状态。
4. 使用真实 smoke token 通过 `gateway` 发起一次受保护请求。
5. 验证该请求需要经过 token-runtime 权限解释,并落到 supply-api 受保护 HTTP surface。
## 输入环境变量
`scripts/ci/cross_service_smoke.sh` 计划使用以下输入:
- `SMOKE_GATEWAY_BASE_URL`
- `SMOKE_TOKEN_RUNTIME_BASE_URL`
- `SMOKE_SUPPLY_API_BASE_URL`
- `SMOKE_BEARER_TOKEN`
- `SMOKE_EXPECTED_SCOPE`
- `SMOKE_EXPECTED_MODEL`
- `SMOKE_ALLOW_LOCAL_PLACEHOLDER`
## 输出工件
输出必须稳定落到:
- `reports/archive/gate_verification/cross_service_smoke_<timestamp>.log`
- `reports/archive/gate_verification/cross_service_smoke_<timestamp>.md`
后续要求:
1. smoke markdown 报告可被 release manifest 收录。
2. `SKIP_LOCAL_PLACEHOLDER` 不得计入 release pass。
3. 只有真实 smoke `PASS` 才能作为发布路径证据。