# Recovery 01 plan: 159399 listed-ETF ingestion and failure semantics

Evidence-based plan. All live evidence below was collected 2026-09-17 with bounded
real calls (single attempts, ≤15s timeouts, no retries). Skill loading is recorded
in the session transcript: systematic-debugging, test-driven-development,
writing-plans were invoked via the native skill tool before this plan; if not
visible, fallback statement: skills were loaded with the skill tool and their
content is authoritative for this task.

## Symptom (reproduced)

User defect: cash-flow ETF 159399, cn, 2025-12-31..2026-09-17, daily, unadjusted
fails with `eastmoney failed (ConnectionError: RemoteDisconnected ...)` then
`tencent fallback failed (tencent source has no listed-ETF daily adapter)`.

## Root cause (Phase 1 evidence)

1. **Eastmoney kline host is genuinely unreachable from this environment.**
   Direct replay of exactly what akshare 1.18.94's `fund_etf_hist_em` sends
   (`https://push2his.eastmoney.com/api/qt/stock/kline/get`, secid 0.159399,
   both with no User-Agent and with a browser UA) →
   `ConnectionError: RemoteDisconnected('Remote end closed connection without response')`
   in both cases. Hypotheses (wrong UA, missing header) ruled out. No code bug in
   our worker path — `_fetch_eastmoney` is correct; the upstream endpoint is
   refusing/dropping our connections.
2. **Tencent genuinely has no listed-ETF daily adapter.** Verified against the
   installed akshare 1.18.94 surface: only
   `fund_etf_hist_em`, `fund_etf_hist_min_em`, `fund_etf_hist_sina` exist.
   The existing error at `worker/data.py:124` is accurate, not a bug.
3. **A genuine alternative provider exists: sina.** Live probe
   `ak.fund_etf_hist_sina(symbol="sz159399")` → DataFrame, 381 rows,
   columns `date,open,high,low,close,volume,amount,postVol,postAmt`,
   prices decimal CNY, **volume unit is 股 (shares)** — cross-verified live on the
   same session: for stock 000001 2026-09-16 tencent reports volume 949,626 (手)
   while sina reports 94,962,632 (股), a consistent ×100, with identical turnover.
   Returns full history (no date-range parameter); the requested window must be
   sliced locally. Unadjusted only (no adjust parameter).
4. **Identity.** `split_identity("159399")` on an cn ETF already resolves to
   SZ#159399 (6-digit code not starting 3/6/9 → SZ). Preserved.
5. **Resulting behavior.** With source=auto, eastmoney fails → tencent ETF
   fallback is a designed explicit rejection → `provider_unavailable` failure.
   The failure message is truthful but the product is unusable for ETFs while
   eastmoney is down even though a genuine provider exists and is live.

## Adjustment behaviors per provider (verified, not assumed)

- eastmoney `fund_etf_hist_em`: adjust none|qfq|hfq — currently unreachable.
- sina `fund_etf_hist_sina`: unadjusted only; no date params; volume in 股.
- tencent: no listed-ETF daily adapter (explicit rejection).

## Chosen repair (smallest correct change, worker only)

Backend inspection: `server/src/datasets.rs` validates frequency/adjustment/
asset_type but does not pass or validate a `source` field; jobs.rs builds the
fetch request without it. So the fix is entirely in the worker:

1. Add `_fetch_sina` (asset_type etf, frequency daily, adjustment none only).
   Local slice to requested dates; keep provider numbers verbatim (volume 股);
   attach `source_warnings` recording units and that eastmoney-style adjustments
   are not available from sina.
2. Register `sina` in `SUPPORTED_SOURCES` and in worker `ALLOWED_SOURCES` so it
   is both explicit and honestly reportable. auto chain for ETF becomes
   eastmoney → sina (same symbol, honestly labeled provider_fallback + units
   warning). qfq/hfq with eastmoney down still fails honestly (sina cannot serve).
3. Explicit-source semantics unchanged: `source=tencent` + etf still raises the
   accurate `source_unavailable` message.

## TDD steps

Each step: failing test → run → minimal code → run.

1. Test: sina adapter normalizes a live-captured-shape sina frame
   (fixture labeled synthetic) for 159399 → SZ#159399, sliced to requested
   range, provider="sina", endpoint="fund_etf_hist_sina", units warning present.
2. Test: sina rejects adjustment qfq/hfq and non-daily frequency with explicit
   `unsupported_*` errors.
3. Test: auto ETF chain — eastmoney monkeypatched to fail → served by sina with
   `provider_fallback` warning naming eastmoney/sina honestly.
4. Test: explicit sina request for stock/index rejected (out of verified scope).
5. Run full worker suite; then live bounded end-to-end run of the exact
   requested dataset via `python -m worker.main fetch` into an isolated output
   dir; report exact rows/coverage/source/units; no production touched.

## Constraints honored

- All network calls bounded (≤15s per call, one call per attempt, 180s wall
  clock via SIGALRM already in main.py). No retries, no auto-refetch of user
  data, no production writes, no deployment, no commits.
- The user's failed dataset is never mutated; coverage/consumer decisions
  reported truthfully (2026-09-17 has no EOD bar yet — intraday at report time).