diff options
| author | Somhairle H. Marisol <[email protected]> | 2026-09-18 08:27:41 +0800 |
|---|---|---|
| committer | Somhairle H. Marisol <[email protected]> | 2026-09-18 08:27:41 +0800 |
| commit | 088735b948d46896b8af30efcb0a2dc5d362b97f (patch) | |
| tree | 0dcab0d5ebc65309267a42d7cda827e9fdd866e7 /docs/recovery-01-results.md | |
| parent | bf6681eb29ac8b0c80ca17b2b5869f7de3da1198 (diff) | |
| download | strategy-lab-088735b948d46896b8af30efcb0a2dc5d362b97f.tar.gz | |
docs(release): 全周期交接文档入库(含 ui-shadcn 迁移交付说明)
[变更性质]
纯文档提交,无运行时逻辑。
[文档内容]
补齐此前各轮未入库的交接/验收文档:backend-auth/backend-domain/
domain-authorization-user(认证与授权域)、etf-recovery-release-
handoff(ETF 修复 + ops 演练定稿与生产部署命令)、recovery-* 系列、
frontend/parent-ui-findings(UI 迁移上下文)、worker/integration 等,
以及本轮 docs/ui-shadcn-handoff.md(shadcn-svelte 迁移交接,含
Chart.svelte 契约、runes $state 踩坑记录与 375/768/1440 验证证据)。
[更新方案]
按主题分文;每份文档只记录可复现的命令、验证结果与语义边界,
不导出密钥或生产敏感路径。
[影响范围]
文档渠道:后续 leader/client 审阅入口;与代码提交一一对应便于回溯。
Diffstat (limited to 'docs/recovery-01-results.md')
| -rw-r--r-- | docs/recovery-01-results.md | 136 |
1 files changed, 136 insertions, 0 deletions
diff --git a/docs/recovery-01-results.md b/docs/recovery-01-results.md new file mode 100644 index 0000000..0e57ab7 --- /dev/null +++ b/docs/recovery-01-results.md @@ -0,0 +1,136 @@ +# Recovery 01 results — real listed-ETF ingestion and failure semantics + +Task: docs/recovery-task-01.md. Date: 2026-09-17. Scope: this repo. No +deployment, no production image replacement, no commits, no user DB access, +no restarts. The user's failed dataset was not refetched or mutated. + +## Skills actually loaded (native skill tool calls, this session) + +1. systematic-debugging — loaded before any fix attempt. +2. test-driven-development — loaded; RED→GREEN cycles recorded below. +3. writing-plans — loaded; plan written to docs/recovery-01-plan.md. +4. verification-before-completion — loaded before this handoff. +No fallback to manual SKILL.md reads was needed. Discovery succeeded. + +## Root cause (with real upstream evidence) + +- Symptom reproduced live first attempt: `ak.fund_etf_hist_em(symbol="159399", + start 20251231, end 20260917)` → + `ConnectionError: RemoteDisconnected('Remote end closed connection without response')`. +- Direct replay of the exact endpoint akshare calls + (`https://push2his.eastmoney.com/api/qt/stock/kline/get`, secid `0.159399`) + with no UA and with a browser UA both failed identically → the eastmoney + kline host is genuinely refusing/silently dropping connections from this + environment. Not a UA/timeout bug, not a code bug in the worker. +- Verified against installed akshare 1.18.94: there is no tencent listed-ETF + daily adapter (`fund_etf_hist_em`, `fund_etf_hist_min_em`, + `fund_etf_hist_sina` are the only ETF end-of-day ETF adapters). The + pre-existing `source_unavailable` message at worker/data.py:124 is accurate. +- Live-verified alternative provider: `ak.fund_etf_hist_sina(symbol="sz159399")` + → 381 rows (2025-02-27..2026-09-16), columns + `date,open,high,low,close,volume,amount,postVol,postAmt`, unadjusted, no date + parameters (full history; must be sliced locally), volume unit 股 (shares). +- Units cross-verified live on the same session with stock sz000001 2026-09-16: + tencent volume 949,626 (lots, 手) vs sina 94,962,632 (shares, 股) — exact ×100, + identical turnover (1,106,652,900 vs 1,106,652,940). Hence honest units + warnings on sina output rather than silent rescaling. + +## Change (smallest correct; worker + trivial frontend label) + +- `worker/data.py`: + - new `_fetch_sina`: etf-only, daily-only, adjustment none-only (explicit + `unsupported_*` rejections otherwise), sina symbol `sz159399`, full-history + fetch sliced locally to the requested window, canonical identity + `SZ#159399` preserved, verbatim provider numbers, source_warnings with + slicing/units/unadjusted disclosure. + - `SUPPORTED_SOURCES` now `{eastmoney, tencent, sina, auto}`; auto chain for + ETF: eastmoney → sina (honestly labeled `provider_fallback` warning naming + both providers); stock/index auto chains unchanged (eastmoney → tencent). + - explicit `source=tencent` + etf still raises the unchanged, accurate + "tencent source has no listed-ETF daily adapter". +- `worker/main.py`: `ALLOWED_SOURCES` += `sina`. +- `frontend/src/lib/format.ts`: `sina` → `新浪` source label (display only, + no UI redesign). Backend (server/) unchanged: it never passes or validates + a source field (datasets.rs validates frequency/adjustment/asset_type only). + +## TDD evidence + +Tests added in `tests/worker/test_data.py` (synthetic fixtures are clearly +labeled synthetic; live-captured provider shapes are quoted in comments): + +- test_sina_etf_serves_159399_with_honest_provenance +- test_sina_rejects_adjustment_and_range_semantics +- test_sina_rejects_stock_and_index +- test_auto_etf_falls_back_from_eastmoney_to_sina_honestly +- test_auto_etf_fails_when_no_provider_is_viable +- test_tencent_etf_rejection_message_unchanged +- test_sina_registered_in_explicit_source_surface + +RED: 6 of the above failed against the unmodified worker (the tencent +rejection test passed as pre-existing behavior). GREEN after implementation. + +## Verification commands and fresh results + +- `python -m pytest tests/worker -q` → **56 passed** (0.79s), final run. +- `python -m pytest tests -q` → **56 passed** (0.72s), final run. +- `npm run check` (frontend) → **0 errors, 0 warnings**. +- `npm test` (frontend) → **13 files / 45 tests passed**. +- Live exact-request fetch (real upstreams, single bounded attempts, + SIGALRM 180s wall clock): + `python -m worker.main fetch --request artifacts/recovery-01/req159399.json + --output artifacts/recovery-01/etf_out` → **status: ready**, ~0.9s. + +## Exact reported dataset facts (159399, requested 2025-12-31..2026-09-17) + +- provider **sina**, endpoint `fund_etf_hist_sina`, params `{"symbol": "sz159399"}` +- row_count **173**, actual_start **2025-12-31**, actual_end **2026-09-16** +- columns: `date,open,high,low,close,volume,amount,symbol` (amount requested) +- adjustment **none (unadjusted)**; prices CNY decimals; **volume in 股 + (shares)** — 100× the lot convention of eastmoney/tencent (verified ×100 on + the same session, not silently converted) +- 2026-09-17 has **no bar**: today's session end-of-day not yet published; the + warning "actual coverage 2025-12-31..2026-09-16 differs from request; gaps + kept" is on the manifest. All calendar gaps present-truthful (holidays etc.). +- identity preserved all the way through: normalized symbol `159399` CSV and + canonical `SZ#159399` contract in adapter tests. +- fetch → persisted raw JSON (`objects/raw_159399_*.json`) + normalized CSV + (`objects/159399.csv`) + preview contract in `result.json` (columns, + row_count, coverage, last-20 rows), exercised outside production in + `artifacts/recovery-01/etf_out/`. + +## Comparison cases (truthful, live attempts) + +- stock 600000 auto (unchanged path): attempt 1 failed (eastmoney down + + tencent proxy.finance.qq.com connect timeout); attempt 2 → **ready** + provider **tencent**, endpoint stock_zh_a_hist_tx, 173 rows, + 2025-12-31..2026-09-16. +- index 000300 auto (unchanged path): attempts 1–2 failed — attempt 1 tencent + proxy connect timeout; attempt 2 eastmoney RemoteDisconnected + tencent + returned an empty payload to akshare ("Length mismatch"). Environmental, + pre-existing flakiness; not caused by this change and out of this task's + repair scope. +- 159399 with source=tencent: explicitly, correctly rejected + (source_unavailable). 159399 with qfq: with eastmoney down auto now fails + honestly because sina has no adjustment factors (tested). + +## Remaining blockers + +1. eastmoney push2his connectivity from this environment is down (affects + stock/index first choice too); recovery is environmental, workarounds live + in fallback chains. +2. tencent proxy.finance.qq.com flaky/intermittent under this environment; + index fetch unverified live in this session. +3. Full release gate (docs/skill-assisted-development.md) is a separate + acceptance workflow; this task only fixes worker ETF ingestion and its + failure semantics. Parent independent verification still required. + +## Changed files + +- worker/data.py +- worker/main.py +- tests/worker/test_data.py (new tests only) +- frontend/src/lib/format.ts (one-line source label) +- docs/recovery-01-plan.md (new) +- docs/recovery-01-results.md (this file, new) +- artifacts/recovery-01/ (isolated live-fetch artifacts: 3 request files, + etf_out/, stock_out/, index_out/) |
