docs(ci): define cross-service smoke taxonomy
This commit is contained in:
@@ -132,3 +132,23 @@ git diff --check
|
||||
3. 已在 `supply-api/cmd/supply-api/main.go` 增加 `staging/prod + config.dev.yaml` 的 fail-fast 拒绝规则,并补充命令行测试覆盖;同时新增 `supply-api/config/config.staging.example.yaml` 与 `supply-api/config/config.prod.example.yaml` 模板骨架。
|
||||
4. 已在 `scripts/devtest/start_dev_stack.sh` 引入 `LIJIAOQIAO_DEVTEST_SUPPLY_CONFIG` 参数,默认改用 `./config/config.staging.example.yaml`,去除 staging 启动链路里的 `config.dev.yaml` 硬编码。
|
||||
5. `bash -n scripts/devtest/start_dev_stack.sh`、`go test ./internal/config ./internal/app`、`go test ./cmd/supply-api ./internal/app ./internal/config` 均通过,且 `git diff --check` 无格式错误。
|
||||
|
||||
## P2-D 测试分类与跨服务 smoke 设计完成
|
||||
|
||||
执行命令:
|
||||
|
||||
```bash
|
||||
bash -n scripts/ci/backend-verify.sh
|
||||
bash -n scripts/ci/repo_integrity_check.sh
|
||||
bash -n scripts/ci/cross_service_smoke.sh
|
||||
go test -tags=e2e ./e2e
|
||||
git diff --check
|
||||
```
|
||||
|
||||
执行结果:
|
||||
|
||||
1. 已在 `supply-api/e2e/README.md` 与 `supply-api/e2e/e2e_test.go` 明确当前 build-tag 套件属于 `service-http`,边界是“单进程、单服务 HTTP surface”,不再冒充真实部署 E2E。
|
||||
2. 已新增 `tests/smoke/README.md`,把测试分类收敛为 `unit`、`integration`、`service-http`、`cross-service-smoke`,并定义最小链路 `gateway -> token-runtime -> supply-api`。
|
||||
3. 已新增 `scripts/ci/cross_service_smoke.sh` 设计骨架,写明 smoke 输入环境变量、输出工件路径以及 `PASS` / `SKIP_LOCAL_PLACEHOLDER` / `FAIL_REAL_SMOKE` 结果契约;当前脚本仍是 design stub,不可作为已完成发布证据。
|
||||
4. 已在 `scripts/ci/backend-verify.sh` 补入 cross-service smoke 入口设计说明,并把 `./e2e` 的对外命名改为 `service-http`;已在 `scripts/ci/repo_integrity_check.sh` 明确其职责仅限代码完整性 gate,不是 release gate。
|
||||
5. `bash -n scripts/ci/backend-verify.sh`、`bash -n scripts/ci/repo_integrity_check.sh`、`bash -n scripts/ci/cross_service_smoke.sh`、`go test -tags=e2e ./e2e` 均通过,且 `git diff --check` 无格式错误。
|
||||
|
||||
@@ -384,21 +384,21 @@
|
||||
- Create: `tests/smoke/README.md`
|
||||
- Create: `scripts/ci/cross_service_smoke.sh`
|
||||
|
||||
- [ ] `P2-D-01` 盘点当前被叫做 `e2e` 的测试真实边界。
|
||||
- [x] `P2-D-01` 盘点当前被叫做 `e2e` 的测试真实边界。
|
||||
完成标准:区分进程内、单服务、跨服务三类。
|
||||
- [ ] `P2-D-02` 定义新的测试分类名。
|
||||
- [x] `P2-D-02` 定义新的测试分类名。
|
||||
完成标准:至少包含 `unit`、`integration`、`service-http`、`cross-service-smoke`。
|
||||
- [ ] `P2-D-03` 在 `supply-api/e2e/README.md` 改写术语草稿。
|
||||
- [x] `P2-D-03` 在 `supply-api/e2e/README.md` 改写术语草稿。
|
||||
完成标准:README 不再把进程内测试叫成真实部署 E2E。
|
||||
- [ ] `P2-D-04` 设计 `cross_service_smoke.sh` 的最小链路。
|
||||
- [x] `P2-D-04` 设计 `cross_service_smoke.sh` 的最小链路。
|
||||
完成标准:至少覆盖 `gateway -> token-runtime -> supply-api`。
|
||||
- [ ] `P2-D-05` 为 smoke 脚本设计输入环境变量。
|
||||
- [x] `P2-D-05` 为 smoke 脚本设计输入环境变量。
|
||||
完成标准:包含地址、token、期望模型或 scope。
|
||||
- [ ] `P2-D-06` 为 smoke 脚本设计输出报告路径。
|
||||
- [x] `P2-D-06` 为 smoke 脚本设计输出报告路径。
|
||||
完成标准:输出可被 manifest 收录。
|
||||
- [ ] `P2-D-07` 设计 `backend-verify.sh` 中新增 smoke 入口。
|
||||
- [x] `P2-D-07` 设计 `backend-verify.sh` 中新增 smoke 入口。
|
||||
完成标准:能区分“本地占位跳过”和“真实 smoke 失败”。
|
||||
- [ ] `P2-D-08` 设计 `repo_integrity_check.sh` 的职责边界说明。
|
||||
- [x] `P2-D-08` 设计 `repo_integrity_check.sh` 的职责边界说明。
|
||||
完成标准:明确它不是 release gate,只是代码完整性 gate。
|
||||
|
||||
---
|
||||
|
||||
@@ -11,6 +11,9 @@ CONTRACT_GATE_DOC="${ROOT_DIR}/tests/contract/gateway_token_runtime_supply_chain
|
||||
CONTRACT_GATE_CHECKLIST="${ROOT_DIR}/docs/plans/2026-04-21-phase1-contract-gate-checklist.md"
|
||||
CONTRACT_GATE_LOG="${OUT_DIR}/contract_gate_${TS}.log"
|
||||
CONTRACT_GATE_REPORT="${OUT_DIR}/contract_gate_${TS}.md"
|
||||
SMOKE_GATE_DOC="${ROOT_DIR}/tests/smoke/README.md"
|
||||
SMOKE_GATE_LOG="${OUT_DIR}/cross_service_smoke_${TS}.log"
|
||||
SMOKE_GATE_REPORT="${OUT_DIR}/cross_service_smoke_${TS}.md"
|
||||
# shellcheck disable=SC1091
|
||||
source "${LIB_FILE}"
|
||||
|
||||
@@ -57,6 +60,7 @@ run_e2e_skip_gate() {
|
||||
local title="$2"
|
||||
local out_file="${OUT_DIR}/${step_id,,}_${TS}.out.log"
|
||||
|
||||
# 当前 ./e2e 是 supply-api 单服务进程内 HTTP surface 测试,不是跨服务部署 smoke。
|
||||
log "[INFO] ${step_id} ${title} start"
|
||||
set +e
|
||||
bash -lc "cd \"${ROOT_DIR}/supply-api\" && \"${GO_BIN}\" test -tags=e2e -v ./e2e/..." > "${out_file}" 2>&1
|
||||
@@ -95,7 +99,7 @@ run_step \
|
||||
|
||||
run_e2e_skip_gate \
|
||||
"STEP-04" \
|
||||
"supply-api E2E gate must not contain placeholder skip"
|
||||
"supply-api service-http build-tag suite must not contain placeholder skip"
|
||||
|
||||
# Phase 1 contract gate execution slot (design only at this stage):
|
||||
# - command entry: bash "${ROOT_DIR}/scripts/ci/backend-verify.sh" --phase1-contract-gate
|
||||
@@ -105,6 +109,17 @@ run_e2e_skip_gate \
|
||||
# - failure semantics: any scenario mismatch, missing required evidence, or non-zero command exit
|
||||
# must mark the backend verify result as FAIL.
|
||||
|
||||
# Phase 2 cross-service smoke slot (design only at this stage):
|
||||
# - command entry: bash "${ROOT_DIR}/scripts/ci/cross_service_smoke.sh"
|
||||
# - taxonomy source: ${SMOKE_GATE_DOC}
|
||||
# - planned artifacts: ${SMOKE_GATE_LOG} and ${SMOKE_GATE_REPORT}
|
||||
# - expected chain: gateway -> token-runtime -> supply-api
|
||||
# - status contract:
|
||||
# * SKIP_LOCAL_PLACEHOLDER: local/mock/placeholder inputs, not a release pass
|
||||
# * FAIL_REAL_SMOKE: real staging inputs present but any link in the chain fails
|
||||
# * PASS: real staging smoke succeeds and report is manifest-collectable
|
||||
# - backend verify must treat SKIP_LOCAL_PLACEHOLDER as non-pass evidence and FAIL_REAL_SMOKE as hard failure.
|
||||
|
||||
HAS_FAIL=0
|
||||
for row in "${STEP_RESULTS[@]}"; do
|
||||
status="$(echo "${row}" | awk -F'|' '{print $2}')"
|
||||
|
||||
60
scripts/ci/cross_service_smoke.sh
Executable file
60
scripts/ci/cross_service_smoke.sh
Executable file
@@ -0,0 +1,60 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
OUT_DIR="${ROOT_DIR}/reports/archive/gate_verification"
|
||||
TS="$(date +%F_%H%M%S)"
|
||||
LOG_FILE="${OUT_DIR}/cross_service_smoke_${TS}.log"
|
||||
REPORT_FILE="${OUT_DIR}/cross_service_smoke_${TS}.md"
|
||||
|
||||
SMOKE_GATEWAY_BASE_URL="${SMOKE_GATEWAY_BASE_URL:-http://127.0.0.1:18080}"
|
||||
SMOKE_TOKEN_RUNTIME_BASE_URL="${SMOKE_TOKEN_RUNTIME_BASE_URL:-http://127.0.0.1:18081}"
|
||||
SMOKE_SUPPLY_API_BASE_URL="${SMOKE_SUPPLY_API_BASE_URL:-http://127.0.0.1:18082}"
|
||||
SMOKE_BEARER_TOKEN="${SMOKE_BEARER_TOKEN:-placeholder-token}"
|
||||
SMOKE_EXPECTED_SCOPE="${SMOKE_EXPECTED_SCOPE:-supply:read}"
|
||||
SMOKE_EXPECTED_MODEL="${SMOKE_EXPECTED_MODEL:-gpt-4o-mini}"
|
||||
SMOKE_ALLOW_LOCAL_PLACEHOLDER="${SMOKE_ALLOW_LOCAL_PLACEHOLDER:-0}"
|
||||
|
||||
mkdir -p "${OUT_DIR}"
|
||||
: > "${LOG_FILE}"
|
||||
|
||||
cat > "${REPORT_FILE}" <<EOF
|
||||
# Cross-Service Smoke Design Report
|
||||
|
||||
- 时间戳:${TS}
|
||||
- 状态:**DESIGN_ONLY**
|
||||
- gateway:${SMOKE_GATEWAY_BASE_URL}
|
||||
- token-runtime:${SMOKE_TOKEN_RUNTIME_BASE_URL}
|
||||
- supply-api:${SMOKE_SUPPLY_API_BASE_URL}
|
||||
- expected_scope:${SMOKE_EXPECTED_SCOPE}
|
||||
- expected_model:${SMOKE_EXPECTED_MODEL}
|
||||
- allow_local_placeholder:${SMOKE_ALLOW_LOCAL_PLACEHOLDER}
|
||||
|
||||
## Planned Chain
|
||||
|
||||
1. gateway health
|
||||
2. token-runtime health
|
||||
3. supply-api health
|
||||
4. protected request through gateway with real bearer token
|
||||
5. verify gateway -> token-runtime -> supply-api chain evidence
|
||||
|
||||
## Planned Result Contract
|
||||
|
||||
- \`PASS\`: real staging smoke passed
|
||||
- \`SKIP_LOCAL_PLACEHOLDER\`: local/mock/placeholder inputs only
|
||||
- \`FAIL_REAL_SMOKE\`: real inputs supplied but chain failed
|
||||
|
||||
## Note
|
||||
|
||||
This script is a Phase P2-D design stub. It defines input/output contracts and artifact paths,
|
||||
but it must not be treated as completed release evidence yet.
|
||||
EOF
|
||||
|
||||
{
|
||||
echo "[INFO] cross-service smoke design stub"
|
||||
echo "[INFO] report: ${REPORT_FILE}"
|
||||
echo "[INFO] log: ${LOG_FILE}"
|
||||
echo "[INFO] status: DESIGN_ONLY"
|
||||
} | tee -a "${LOG_FILE}"
|
||||
|
||||
exit 2
|
||||
@@ -31,7 +31,7 @@ echo "[repo] supply-api repository integration"
|
||||
cd "${ROOT_DIR}/supply-api"
|
||||
bash scripts/run_integration_tests.sh ./internal/repository
|
||||
)
|
||||
run_go_suite "${ROOT_DIR}" "${GO_BIN}" "supply-api e2e" "supply-api" test -count=1 -tags=e2e ./e2e
|
||||
run_go_suite "${ROOT_DIR}" "${GO_BIN}" "supply-api service-http" "supply-api" test -count=1 -tags=e2e ./e2e
|
||||
|
||||
# Phase 1 contract gate entry (design slot):
|
||||
# - execute after service-local suites and repository integration
|
||||
@@ -41,3 +41,8 @@ run_go_suite "${ROOT_DIR}" "${GO_BIN}" "supply-api e2e" "supply-api" test -count
|
||||
# reports/archive/gate_verification/contract_gate_<timestamp>.md
|
||||
# - failure semantics: if the contract gate exits non-zero or any required scenario is missing,
|
||||
# repo_integrity_check must fail and Phase 1 cannot be marked complete.
|
||||
|
||||
# Phase 2 boundary note:
|
||||
# - repo_integrity_check only proves code completeness, syntax, unit/integration and service-local HTTP coverage.
|
||||
# - cross_service_smoke belongs to release/path validation and must not be redefined as repository integrity.
|
||||
# - even after smoke is introduced, this script remains a code integrity gate rather than a release gate.
|
||||
|
||||
@@ -1,12 +1,19 @@
|
||||
# E2E 测试说明
|
||||
# supply-api service-http 测试说明
|
||||
|
||||
`e2e/` 目录只存放带 `//go:build e2e` 的端到端测试源码,不再混放伪装成文档的 Go 文件。
|
||||
`e2e/` 目录保留现有 build tag 路径,但这里的测试分类在 Phase P2-D 之后明确记为 `service-http`,不是“真实部署 E2E”。
|
||||
|
||||
当前测试分层如下:
|
||||
当前边界:
|
||||
|
||||
- `e2e_test.go`: 核心 HTTP API、鉴权和审计行为的端到端断言。
|
||||
- `playbook_test.go`: 按业务剧本组织的多步骤流程验证。
|
||||
- `production_flow_test.go`: 面向上线前复核的关键流程和安全边界检查。
|
||||
- 进程内:`newE2ESystem` 直接在单进程内装配 handler、内存审计存储、静态 token backend。
|
||||
- 单服务 HTTP:覆盖 `supply-api` 的路由、鉴权、审计和关键业务剧本。
|
||||
- 非跨服务:不会启动 `gateway`、`platform-token-runtime`、真实数据库或外部网络依赖。
|
||||
|
||||
新的测试分类名:
|
||||
|
||||
- `unit`: 单函数、单组件、无跨进程依赖。
|
||||
- `integration`: 真实数据库/仓储/适配层集成。
|
||||
- `service-http`: 单服务 HTTP surface,在进程内装配依赖。
|
||||
- `cross-service-smoke`: `gateway -> token-runtime -> supply-api` 的真实跨服务链路。
|
||||
|
||||
运行方式:
|
||||
|
||||
@@ -22,6 +29,6 @@ go test -tags=e2e ./e2e -run TestPlaybook_SupplierOnboarding
|
||||
|
||||
约束说明:
|
||||
|
||||
- E2E 测试应保留在 `*_test.go` 文件内。
|
||||
- 说明文档只保留 Markdown 内容,不内嵌 Go 源码。
|
||||
- 新增剧本时优先复用 `newE2ESystem`,避免重复搭建测试系统。
|
||||
- `e2e/` 目录下的 build-tag 测试应保留在 `*_test.go` 文件内,但对外一律称为 `service-http`。
|
||||
- 真实跨服务 smoke 不得继续写进 `supply-api/e2e/`,应放到 `tests/smoke/` 与 `scripts/ci/cross_service_smoke.sh`。
|
||||
- 新增剧本时优先复用 `newE2ESystem`,避免重复搭建单服务测试系统。
|
||||
|
||||
@@ -24,6 +24,9 @@ import (
|
||||
"lijiaoqiao/supply-api/internal/pkg/logging"
|
||||
)
|
||||
|
||||
// 当前 build-tag 套件运行在单进程内存依赖上,归类为 service-http。
|
||||
// 它验证 supply-api 的 HTTP surface,不代表 gateway -> token-runtime -> supply-api 的真实跨服务 smoke。
|
||||
|
||||
type e2eOptions struct {
|
||||
withdrawEnabled bool
|
||||
}
|
||||
|
||||
51
tests/smoke/README.md
Normal file
51
tests/smoke/README.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# Cross-Service Smoke Design
|
||||
|
||||
## 测试边界
|
||||
|
||||
`tests/smoke/` 只描述真实跨服务 smoke 的链路与证据,不承载单服务进程内测试。
|
||||
|
||||
分类边界:
|
||||
|
||||
1. `unit`
|
||||
- 单函数或单组件测试。
|
||||
2. `integration`
|
||||
- 真实数据库、仓储或适配层集成测试。
|
||||
3. `service-http`
|
||||
- 单服务 HTTP surface,通常在进程内组装依赖。
|
||||
4. `cross-service-smoke`
|
||||
- 至少覆盖 `gateway -> token-runtime -> supply-api` 的真实链路。
|
||||
|
||||
## 最小链路
|
||||
|
||||
最小 smoke 步骤:
|
||||
|
||||
1. 检查 `gateway` 健康状态。
|
||||
2. 检查 `platform-token-runtime` 健康状态。
|
||||
3. 检查 `supply-api` 健康状态。
|
||||
4. 使用真实 smoke token 通过 `gateway` 发起一次受保护请求。
|
||||
5. 验证该请求需要经过 token-runtime 权限解释,并落到 supply-api 受保护 HTTP surface。
|
||||
|
||||
## 输入环境变量
|
||||
|
||||
`scripts/ci/cross_service_smoke.sh` 计划使用以下输入:
|
||||
|
||||
- `SMOKE_GATEWAY_BASE_URL`
|
||||
- `SMOKE_TOKEN_RUNTIME_BASE_URL`
|
||||
- `SMOKE_SUPPLY_API_BASE_URL`
|
||||
- `SMOKE_BEARER_TOKEN`
|
||||
- `SMOKE_EXPECTED_SCOPE`
|
||||
- `SMOKE_EXPECTED_MODEL`
|
||||
- `SMOKE_ALLOW_LOCAL_PLACEHOLDER`
|
||||
|
||||
## 输出工件
|
||||
|
||||
输出必须稳定落到:
|
||||
|
||||
- `reports/archive/gate_verification/cross_service_smoke_<timestamp>.log`
|
||||
- `reports/archive/gate_verification/cross_service_smoke_<timestamp>.md`
|
||||
|
||||
后续要求:
|
||||
|
||||
1. smoke markdown 报告可被 release manifest 收录。
|
||||
2. `SKIP_LOCAL_PLACEHOLDER` 不得计入 release pass。
|
||||
3. 只有真实 smoke `PASS` 才能作为发布路径证据。
|
||||
Reference in New Issue
Block a user