diff --git a/docs/plans/2026-04-21-project-quality-performance-ops-optimization-execution-log.md b/docs/plans/2026-04-21-project-quality-performance-ops-optimization-execution-log.md index 5dd005c7..4339420a 100644 --- a/docs/plans/2026-04-21-project-quality-performance-ops-optimization-execution-log.md +++ b/docs/plans/2026-04-21-project-quality-performance-ops-optimization-execution-log.md @@ -132,3 +132,23 @@ git diff --check 3. 已在 `supply-api/cmd/supply-api/main.go` 增加 `staging/prod + config.dev.yaml` 的 fail-fast 拒绝规则，并补充命令行测试覆盖；同时新增 `supply-api/config/config.staging.example.yaml` 与 `supply-api/config/config.prod.example.yaml` 模板骨架。 4. 已在 `scripts/devtest/start_dev_stack.sh` 引入 `LIJIAOQIAO_DEVTEST_SUPPLY_CONFIG` 参数，默认改用 `./config/config.staging.example.yaml`，去除 staging 启动链路里的 `config.dev.yaml` 硬编码。 5. `bash -n scripts/devtest/start_dev_stack.sh`、`go test ./internal/config ./internal/app`、`go test ./cmd/supply-api ./internal/app ./internal/config` 均通过，且 `git diff --check` 无格式错误。 + +## P2-D 测试分类与跨服务 smoke 设计完成 + +执行命令： + +```bash +bash -n scripts/ci/backend-verify.sh +bash -n scripts/ci/repo_integrity_check.sh +bash -n scripts/ci/cross_service_smoke.sh +go test -tags=e2e ./e2e +git diff --check +``` + +执行结果： + +1. 已在 `supply-api/e2e/README.md` 与 `supply-api/e2e/e2e_test.go` 明确当前 build-tag 套件属于 `service-http`，边界是“单进程、单服务 HTTP surface”，不再冒充真实部署 E2E。 +2. 已新增 `tests/smoke/README.md`，把测试分类收敛为 `unit`、`integration`、`service-http`、`cross-service-smoke`，并定义最小链路 `gateway -> token-runtime -> supply-api`。 +3. 已新增 `scripts/ci/cross_service_smoke.sh` 设计骨架，写明 smoke 输入环境变量、输出工件路径以及 `PASS` / `SKIP_LOCAL_PLACEHOLDER` / `FAIL_REAL_SMOKE` 结果契约；当前脚本仍是 design stub，不可作为已完成发布证据。 +4. 已在 `scripts/ci/backend-verify.sh` 补入 cross-service smoke 入口设计说明，并把 `./e2e` 的对外命名改为 `service-http`；已在 `scripts/ci/repo_integrity_check.sh` 明确其职责仅限代码完整性 gate，不是 release gate。 +5. `bash -n scripts/ci/backend-verify.sh`、`bash -n scripts/ci/repo_integrity_check.sh`、`bash -n scripts/ci/cross_service_smoke.sh`、`go test -tags=e2e ./e2e` 均通过，且 `git diff --check` 无格式错误。 diff --git a/docs/plans/2026-04-21-project-quality-performance-ops-optimization-plan.md b/docs/plans/2026-04-21-project-quality-performance-ops-optimization-plan.md index d5e7d4d4..54dfdcde 100644 --- a/docs/plans/2026-04-21-project-quality-performance-ops-optimization-plan.md +++ b/docs/plans/2026-04-21-project-quality-performance-ops-optimization-plan.md @@ -384,21 +384,21 @@ - Create: `tests/smoke/README.md` - Create: `scripts/ci/cross_service_smoke.sh` -- [ ] `P2-D-01` 盘点当前被叫做 `e2e` 的测试真实边界。 +- [x] `P2-D-01` 盘点当前被叫做 `e2e` 的测试真实边界。 完成标准：区分进程内、单服务、跨服务三类。 -- [ ] `P2-D-02` 定义新的测试分类名。 +- [x] `P2-D-02` 定义新的测试分类名。 完成标准：至少包含 `unit`、`integration`、`service-http`、`cross-service-smoke`。 -- [ ] `P2-D-03` 在 `supply-api/e2e/README.md` 改写术语草稿。 +- [x] `P2-D-03` 在 `supply-api/e2e/README.md` 改写术语草稿。 完成标准：README 不再把进程内测试叫成真实部署 E2E。 -- [ ] `P2-D-04` 设计 `cross_service_smoke.sh` 的最小链路。 +- [x] `P2-D-04` 设计 `cross_service_smoke.sh` 的最小链路。 完成标准：至少覆盖 `gateway -> token-runtime -> supply-api`。 -- [ ] `P2-D-05` 为 smoke 脚本设计输入环境变量。 +- [x] `P2-D-05` 为 smoke 脚本设计输入环境变量。 完成标准：包含地址、token、期望模型或 scope。 -- [ ] `P2-D-06` 为 smoke 脚本设计输出报告路径。 +- [x] `P2-D-06` 为 smoke 脚本设计输出报告路径。 完成标准：输出可被 manifest 收录。 -- [ ] `P2-D-07` 设计 `backend-verify.sh` 中新增 smoke 入口。 +- [x] `P2-D-07` 设计 `backend-verify.sh` 中新增 smoke 入口。 完成标准：能区分“本地占位跳过”和“真实 smoke 失败”。 -- [ ] `P2-D-08` 设计 `repo_integrity_check.sh` 的职责边界说明。 +- [x] `P2-D-08` 设计 `repo_integrity_check.sh` 的职责边界说明。 完成标准：明确它不是 release gate，只是代码完整性 gate。 --- diff --git a/scripts/ci/backend-verify.sh b/scripts/ci/backend-verify.sh index 2aeb3c2a..41926e5f 100755 --- a/scripts/ci/backend-verify.sh +++ b/scripts/ci/backend-verify.sh @@ -11,6 +11,9 @@ CONTRACT_GATE_DOC="${ROOT_DIR}/tests/contract/gateway_token_runtime_supply_chain CONTRACT_GATE_CHECKLIST="${ROOT_DIR}/docs/plans/2026-04-21-phase1-contract-gate-checklist.md" CONTRACT_GATE_LOG="${OUT_DIR}/contract_gate_${TS}.log" CONTRACT_GATE_REPORT="${OUT_DIR}/contract_gate_${TS}.md" +SMOKE_GATE_DOC="${ROOT_DIR}/tests/smoke/README.md" +SMOKE_GATE_LOG="${OUT_DIR}/cross_service_smoke_${TS}.log" +SMOKE_GATE_REPORT="${OUT_DIR}/cross_service_smoke_${TS}.md" # shellcheck disable=SC1091 source "${LIB_FILE}" @@ -57,6 +60,7 @@ run_e2e_skip_gate() { local title="$2" local out_file="${OUT_DIR}/${step_id,,}_${TS}.out.log" + # 当前 ./e2e 是 supply-api 单服务进程内 HTTP surface 测试，不是跨服务部署 smoke。 log "[INFO] ${step_id} ${title} start" set +e bash -lc "cd \"${ROOT_DIR}/supply-api\" && \"${GO_BIN}\" test -tags=e2e -v ./e2e/..." > "${out_file}" 2>&1 @@ -95,7 +99,7 @@ run_step \ run_e2e_skip_gate \ "STEP-04" \ - "supply-api E2E gate must not contain placeholder skip" + "supply-api service-http build-tag suite must not contain placeholder skip" # Phase 1 contract gate execution slot (design only at this stage): # - command entry: bash "${ROOT_DIR}/scripts/ci/backend-verify.sh" --phase1-contract-gate @@ -105,6 +109,17 @@ run_e2e_skip_gate \ # - failure semantics: any scenario mismatch, missing required evidence, or non-zero command exit # must mark the backend verify result as FAIL. +# Phase 2 cross-service smoke slot (design only at this stage): +# - command entry: bash "${ROOT_DIR}/scripts/ci/cross_service_smoke.sh" +# - taxonomy source: ${SMOKE_GATE_DOC} +# - planned artifacts: ${SMOKE_GATE_LOG} and ${SMOKE_GATE_REPORT} +# - expected chain: gateway -> token-runtime -> supply-api +# - status contract: +# * SKIP_LOCAL_PLACEHOLDER: local/mock/placeholder inputs, not a release pass +# * FAIL_REAL_SMOKE: real staging inputs present but any link in the chain fails +# * PASS: real staging smoke succeeds and report is manifest-collectable +# - backend verify must treat SKIP_LOCAL_PLACEHOLDER as non-pass evidence and FAIL_REAL_SMOKE as hard failure. + HAS_FAIL=0 for row in "${STEP_RESULTS[@]}"; do status="$(echo "${row}" | awk -F'|' '{print $2}')" diff --git a/scripts/ci/cross_service_smoke.sh b/scripts/ci/cross_service_smoke.sh new file mode 100755 index 00000000..59a73a71 --- /dev/null +++ b/scripts/ci/cross_service_smoke.sh @@ -0,0 +1,60 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "$0")/../.." && pwd)" +OUT_DIR="${ROOT_DIR}/reports/archive/gate_verification" +TS="$(date +%F_%H%M%S)" +LOG_FILE="${OUT_DIR}/cross_service_smoke_${TS}.log" +REPORT_FILE="${OUT_DIR}/cross_service_smoke_${TS}.md" + +SMOKE_GATEWAY_BASE_URL="${SMOKE_GATEWAY_BASE_URL:-http://127.0.0.1:18080}" +SMOKE_TOKEN_RUNTIME_BASE_URL="${SMOKE_TOKEN_RUNTIME_BASE_URL:-http://127.0.0.1:18081}" +SMOKE_SUPPLY_API_BASE_URL="${SMOKE_SUPPLY_API_BASE_URL:-http://127.0.0.1:18082}" +SMOKE_BEARER_TOKEN="${SMOKE_BEARER_TOKEN:-placeholder-token}" +SMOKE_EXPECTED_SCOPE="${SMOKE_EXPECTED_SCOPE:-supply:read}" +SMOKE_EXPECTED_MODEL="${SMOKE_EXPECTED_MODEL:-gpt-4o-mini}" +SMOKE_ALLOW_LOCAL_PLACEHOLDER="${SMOKE_ALLOW_LOCAL_PLACEHOLDER:-0}" + +mkdir -p "${OUT_DIR}" +: > "${LOG_FILE}" + +cat > "${REPORT_FILE}" < token-runtime -> supply-api chain evidence + +## Planned Result Contract + +- \`PASS\`: real staging smoke passed +- \`SKIP_LOCAL_PLACEHOLDER\`: local/mock/placeholder inputs only +- \`FAIL_REAL_SMOKE\`: real inputs supplied but chain failed + +## Note + +This script is a Phase P2-D design stub. It defines input/output contracts and artifact paths, +but it must not be treated as completed release evidence yet. +EOF + +{ + echo "[INFO] cross-service smoke design stub" + echo "[INFO] report: ${REPORT_FILE}" + echo "[INFO] log: ${LOG_FILE}" + echo "[INFO] status: DESIGN_ONLY" +} | tee -a "${LOG_FILE}" + +exit 2 diff --git a/scripts/ci/repo_integrity_check.sh b/scripts/ci/repo_integrity_check.sh index 7c9ae609..c9077c9a 100755 --- a/scripts/ci/repo_integrity_check.sh +++ b/scripts/ci/repo_integrity_check.sh @@ -31,7 +31,7 @@ echo "[repo] supply-api repository integration" cd "${ROOT_DIR}/supply-api" bash scripts/run_integration_tests.sh ./internal/repository ) -run_go_suite "${ROOT_DIR}" "${GO_BIN}" "supply-api e2e" "supply-api" test -count=1 -tags=e2e ./e2e +run_go_suite "${ROOT_DIR}" "${GO_BIN}" "supply-api service-http" "supply-api" test -count=1 -tags=e2e ./e2e # Phase 1 contract gate entry (design slot): # - execute after service-local suites and repository integration @@ -41,3 +41,8 @@ run_go_suite "${ROOT_DIR}" "${GO_BIN}" "supply-api e2e" "supply-api" test -count # reports/archive/gate_verification/contract_gate_.md # - failure semantics: if the contract gate exits non-zero or any required scenario is missing, # repo_integrity_check must fail and Phase 1 cannot be marked complete. + +# Phase 2 boundary note: +# - repo_integrity_check only proves code completeness, syntax, unit/integration and service-local HTTP coverage. +# - cross_service_smoke belongs to release/path validation and must not be redefined as repository integrity. +# - even after smoke is introduced, this script remains a code integrity gate rather than a release gate. diff --git a/supply-api/e2e/README.md b/supply-api/e2e/README.md index 9456dc1c..fece9842 100644 --- a/supply-api/e2e/README.md +++ b/supply-api/e2e/README.md @@ -1,12 +1,19 @@ -# E2E 测试说明 +# supply-api service-http 测试说明 -`e2e/` 目录只存放带 `//go:build e2e` 的端到端测试源码，不再混放伪装成文档的 Go 文件。 +`e2e/` 目录保留现有 build tag 路径，但这里的测试分类在 Phase P2-D 之后明确记为 `service-http`，不是“真实部署 E2E”。 -当前测试分层如下： +当前边界： -- `e2e_test.go`: 核心 HTTP API、鉴权和审计行为的端到端断言。 -- `playbook_test.go`: 按业务剧本组织的多步骤流程验证。 -- `production_flow_test.go`: 面向上线前复核的关键流程和安全边界检查。 +- 进程内：`newE2ESystem` 直接在单进程内装配 handler、内存审计存储、静态 token backend。 +- 单服务 HTTP：覆盖 `supply-api` 的路由、鉴权、审计和关键业务剧本。 +- 非跨服务：不会启动 `gateway`、`platform-token-runtime`、真实数据库或外部网络依赖。 + +新的测试分类名： + +- `unit`: 单函数、单组件、无跨进程依赖。 +- `integration`: 真实数据库/仓储/适配层集成。 +- `service-http`: 单服务 HTTP surface，在进程内装配依赖。 +- `cross-service-smoke`: `gateway -> token-runtime -> supply-api` 的真实跨服务链路。 运行方式： @@ -22,6 +29,6 @@ go test -tags=e2e ./e2e -run TestPlaybook_SupplierOnboarding 约束说明： -- E2E 测试应保留在 `*_test.go` 文件内。 -- 说明文档只保留 Markdown 内容，不内嵌 Go 源码。 -- 新增剧本时优先复用 `newE2ESystem`，避免重复搭建测试系统。 +- `e2e/` 目录下的 build-tag 测试应保留在 `*_test.go` 文件内，但对外一律称为 `service-http`。 +- 真实跨服务 smoke 不得继续写进 `supply-api/e2e/`，应放到 `tests/smoke/` 与 `scripts/ci/cross_service_smoke.sh`。 +- 新增剧本时优先复用 `newE2ESystem`，避免重复搭建单服务测试系统。 diff --git a/supply-api/e2e/e2e_test.go b/supply-api/e2e/e2e_test.go index 54b8ef09..aaf65b5a 100644 --- a/supply-api/e2e/e2e_test.go +++ b/supply-api/e2e/e2e_test.go @@ -24,6 +24,9 @@ import ( "lijiaoqiao/supply-api/internal/pkg/logging" ) +// 当前 build-tag 套件运行在单进程内存依赖上，归类为 service-http。 +// 它验证 supply-api 的 HTTP surface，不代表 gateway -> token-runtime -> supply-api 的真实跨服务 smoke。 + type e2eOptions struct { withdrawEnabled bool } diff --git a/tests/smoke/README.md b/tests/smoke/README.md new file mode 100644 index 00000000..d1c08776 --- /dev/null +++ b/tests/smoke/README.md @@ -0,0 +1,51 @@ +# Cross-Service Smoke Design + +## 测试边界 + +`tests/smoke/` 只描述真实跨服务 smoke 的链路与证据，不承载单服务进程内测试。 + +分类边界： + +1. `unit` + - 单函数或单组件测试。 +2. `integration` + - 真实数据库、仓储或适配层集成测试。 +3. `service-http` + - 单服务 HTTP surface，通常在进程内组装依赖。 +4. `cross-service-smoke` + - 至少覆盖 `gateway -> token-runtime -> supply-api` 的真实链路。 + +## 最小链路 + +最小 smoke 步骤： + +1. 检查 `gateway` 健康状态。 +2. 检查 `platform-token-runtime` 健康状态。 +3. 检查 `supply-api` 健康状态。 +4. 使用真实 smoke token 通过 `gateway` 发起一次受保护请求。 +5. 验证该请求需要经过 token-runtime 权限解释，并落到 supply-api 受保护 HTTP surface。 + +## 输入环境变量 + +`scripts/ci/cross_service_smoke.sh` 计划使用以下输入： + +- `SMOKE_GATEWAY_BASE_URL` +- `SMOKE_TOKEN_RUNTIME_BASE_URL` +- `SMOKE_SUPPLY_API_BASE_URL` +- `SMOKE_BEARER_TOKEN` +- `SMOKE_EXPECTED_SCOPE` +- `SMOKE_EXPECTED_MODEL` +- `SMOKE_ALLOW_LOCAL_PLACEHOLDER` + +## 输出工件 + +输出必须稳定落到： + +- `reports/archive/gate_verification/cross_service_smoke_.log` +- `reports/archive/gate_verification/cross_service_smoke_.md` + +后续要求： + +1. smoke markdown 报告可被 release manifest 收录。 +2. `SKIP_LOCAL_PLACEHOLDER` 不得计入 release pass。 +3. 只有真实 smoke `PASS` 才能作为发布路径证据。