diff options
Diffstat (limited to 'docs/search-fix.md')
| -rw-r--r-- | docs/search-fix.md | 111 |
1 files changed, 111 insertions, 0 deletions
diff --git a/docs/search-fix.md b/docs/search-fix.md new file mode 100644 index 0000000..7e27304 --- /dev/null +++ b/docs/search-fix.md @@ -0,0 +1,111 @@ +# Production instrument-search fix — handoff notes + +Date: 2026-09-17. Scope: production search stall on `/api/instruments?q=...` +(0 bytes in ~20s). All work confined to this repo; nothing deployed, user DB +untouched, no services restarted. + +## Root causes fixed + +1. **worker/data.py** — the old `_catalog_frame` fetched the ENTIRE A-share + stock catalog (`stock_info_a_code_name`) and the ENTIRE ETF live snapshot + (`fund_etf_spot_em`) via AKShare on every search query. Those calls are + unbounded and stalled inside the worker container until the Rust-side + runner timeout fired, so the browser's fetch starved and kept spinning. + Replaced with **bounded direct provider suggestion search**: + - Eastmoney `https://searchapi.eastmoney.com/api/suggest/get` + (`input=<q>`, `type=14`, bounded `count`), JSON, per-call timeout **4s**. + - Tencent `https://smartbox.gtimg.cn/s3/?q=<q>&t=all`, text hint + `v_hint="sh~600000~name~py~GP-A^..."`, per-call timeout **4s**. + - No synthetic data, no full catalogs anywhere in the search path anymore. +2. **worker (main.py/data.py) envelope** — search now prints one JSON object + envelope `{"source","status","items","error","providers"}` with explicit + `status: ready|failed`. `run_search` exits non-zero on failure. +3. **server/src/worker.rs** — container runtime bound reduced **120s → 12s** + (outer bound incl. container startup; provider calls are ≤2×4s inside). + The stdout is now parsed as the full JSON envelope; the old code extracted + `first[...]..last[...]` which silently turned a failed-status envelope into + an empty success. A `failed` status now surfaces as a real error + (`search_unavailable`, provider error code in the message), and **only + ready responses enter the server-side TTL cache**. +4. **server/src/main.rs** — `instruments()` success source label is now + `provider_suggest`; failures keep returning HTTP 200 with + `status: unavailable: ...` so the existing UI contract is unchanged. +5. **Frontend** (`DatasetWizard.svelte` + new `lib/instrumentSearch.ts`): + - 350 ms debounce on input, prior in-flight queries are **aborted** + (AbortController) and stale responses can never repaint. + - Hard **AbortController timer (12s)** bound; hung requests surface + 搜索超时，请重试 instead of an eternal spinner. + - Non-`ok` envelope status / provider failure shows + 搜索数据源暂时不可用，请稍后重试 with a **重试** button; the spinner is + always cleared (`onFinish` in `finally`). + - Result rows carry per-item **provenance** (东财 / 腾讯). + +## Asset-class rules (validated identity, never guessed from code shape) + +Live-probed provider classifications (2026-09-17, both endpoints): + +| Provider | Field | accepted | +|-----------|------------------|-----------------------------| +| Eastmoney | `Classify` | `AStock`→stock, `Index`→index, `Fund`→etf (cross-confirmed, see below) | +| Eastmoney | `Classify` (excluded) | `Bond`,`HK`,`OTCFUND`, others | +| Eastmoney | `MktNum` | `1`→SH, `0`→SZ (others dropped) | +| Tencent | hint tag | `GP-A`→stock, `ETF`→etf, `ZS`→index; `LOF`/others excluded | +| Tencent | hint market | `sh`/`sz` only (`bj` dropped) | + +- An asset's class comes only from the provider's own classification field — + a numeric code like `000300` is never inferred to be an index by shape + (test: `test_search_numeric_code_not_auto_index`). +- **Verified live**: Eastmoney `Classify=Fund` is ambiguous — ETF `510300` + and LOF `160706` share the same `Classify` and `SecurityType`. Therefore an + Eastmoney *Fund* item is only emitted when Tencent presents the same code + with tag `ETF`. Tencent alone classifies ETF/LOF unambiguously. When + Tencent unavailable, Eastmoney-only fund suggestions are suppressed + (`test_search_etf_ambiguous_fund_excluded_without_tencent_confirmation`, + `test_search_lof_not_present_when_eastmoney_only`). Guaranteed: no LOF + ever surfaces as an ETF. +- Per-item provenance is preserved: identical codes may appear once per + actual source (`source: eastmoney` / `source: tencent`). + +## Live verification (fresh container `strategy-lab-worker:local`, rebuilt bd9e9f06d56b) + +``` +docker run --rm --network=bridge strategy-lab-worker:local \ + python -m worker.main search --query 600000 --limit 5 +→ {"status":"ready","items":[{"symbol":"600000","...":"浦发银行","asset_type":"stock",...}],...} + wall time ≈1.9s (container start included; previously 0 bytes / 20+s) + +q=510300 → etf SH#510300 (eastmoney+tencent) +q=沪深300 → index SH#000300 / SZ#399300 (+ ETFs), providers ok +q=160706 → {"items":[]} (LOF correctly suppressed, providers ok, status ready) +``` + +## Tests actually run and passing + +- `server`: `cargo test` → **54 passed**; `cargo build --release` OK. +- `worker`: `.venv/bin/python -m pytest tests` (repo root) → **49 passed**, + of which new/updated in `tests/worker/test_data.py` (real shapes for + stock/ETF/index, Chinese query `浦发银行`/`沪深300`, unsupported classes + bond/HK/OTC/LOF, provider failure + `requests.Timeout` → explicit failed + status, partial-provider warning) and `tests/worker/test_fetch_cli.py::test_search_cli_envelope_contract`. +- `frontend`: `npm run test` (vitest) → **45 passed** across 13 files, + including new `src/lib/instrumentSearch.test.ts` (success, unavailable + status → Chinese message, provider rejection, hard-timer abort → 超时 + wording with spinner cleared, **stale-prior-query never repaints and never + hangs**, dispose invalidates); `npm run check` → 0 errors/warnings; + `npm run build` → OK. + +## Deployment state + +- Worker image rebuilt exactly as `strategy-lab-worker:local` (context repo + root, `-f worker/Dockerfile`): id `bd9e9f06d56b`. +- Frontend `dist/` rebuilt from the fixed sources. +- Server binary built release but **not deployed / no restarts performed**. +- Rollback note: previous worker image tag remains on the host; server + changes are code-only (no config/DB changes). + +## Known limitation (documented honestly) + +When Tencent is unavailable and only Eastmoney responds, fund-class +suggestions are suppressed (stock/index still return). This trades a +slightly narrower ETF list for a hard guarantee that no LOF is presented +as an ETF without provider confirmation. |