-
[OPIK-6128] [QA] feat: Trace Explore smoke (SDK-create + UI-verify) (#6823)
frostbyte_neo 发布于
2026-05-25 08:10:28 +00:00 -
[OPIK-6128] [QA] feat: Trace Explore smoke (SDK-create + UI-verify)
-
feat(bridge): add POST /traces route invoking @opik.track
-
feat(sdk): add createTrace on PythonSdkClient
-
feat(fixtures): add trace fixture extending failure-artifacts
-
fix(bridge): use opik.flush_tracker (flush_all does not exist)
-
feat(fe): add testId prop to MetricCard for E2E metrics-card selectors
-
feat(backend-client): add getProject, getTrace, getTraceSpans wrappers
-
feat(pom): add LogsPage and TracePanelPage
-
feat(matchers): add six custom span matchers
-
test(trace-explore): add @t1-smoke trace-explore CUJ
-
fix(e2e): harden POMs and wire storageState for staging verification
- LogsPage.waitForReady waits for tr[data-row-id] (real rows), not the skeleton
- LogsPage.countTraces falls back to 'Traces N' text when metrics-card-count testid
is not yet deployed (FE testid was added in this PR; staging needs the FE redeploy) - LogsPage.openTraceById rebuilds the URL from the projectId tracked in goto() so it
doesn't accidentally drop default query params captured by the FE - TracePanelPage.closeButton uses getByRole('button', {name: 'Close'}) because the
TraceDetailsPanel custom header replaces the default Sheet close button (which
was the one with data-testid='side-panel-close') - TracePanelPage.waitForFullyLoaded first waits for the trace= query param to land
- BackendClient.getTraceSpans accepts an optional projectId; falls back to looking
it up via getTraceById(traceId).projectId. getSpansByProject requires it. - playwright.config wires storageState from .auth/.json or OPIK_STORAGE_STATE
- refactor(trace-explore): make tests UI-first and independent
The previous draft used REST to read spans and project metadata, which defeats
the point of a UI smoke test: we want to assert that what the SDK logs actually
renders in the UI. Test 2 was REST-only and didn't verify any UI surface; tests
also coupled to fixture-name conventions that hinted at shared state.Changes:
- TracePanelPage drops readSpans()/readMetadata() (both REST) in favour of
DOM-scoped locators: traceNameInHeader, inputValue, outputValue,
spansCountLabel, inputSection, outputSection. All locators are scoped to
the panel's data-testid='traces' root so they never collide with row
previews in the table behind the side-pane. - LogsPage drops the BackendClient dependency (no REST in POM) and gains
breadcrumbProjectLink for project-metadata verification via UI. - BackendClient drops getTrace/getProject/getTraceSpans wrappers (unused now).
- trace.fixture exposes the seeded input/output strings on TraceRef so tests
can assert exactly what was logged appears in the UI. - The custom Span[] matchers are removed: they operated on REST data and were
the wrong abstraction for a UI-first smoke. Plain Playwright assertions on
panel locators are clearer and stricter. - Tests rewritten:
- Test 1: trace appears in Logs table; panel renders trace name + seeded
input/output + the expected Spans (1) label. - Test 2: while the panel is open, the Logs breadcrumb shows the project
name. Verifies project metadata via UI, not REST. - Test 3: no longer depends on opikTrace; seeds three traces itself with
the testNamespace prefix and asserts UI count + row order.
- Test 1: trace appears in Logs table; panel renders trace name + seeded
All three tests now pass when run in isolation, sequentially with WORKERS=1,
and concurrently with WORKERS=3 (each on its own project).- refactor(trace-explore): collapse to two orthogonal smoke tests
The previous three tests overlapped: Test 1 mixed Logs-table and panel
assertions, Test 2 was a near-empty breadcrumb-only check, and Test 3
duplicated the count assertion from Test 1 to also check row ordering.Replace with two tests where each has one clear concern:
-
Logs view shows seeded traces with correct count and ordering
- Seeds 3 traces, asserts count=3 and row order via DOM (data-row-id).
- No panel interaction.
-
Trace panel renders the seeded trace header, input, output, span count,
and project breadcrumb- Seeds 1 trace via opikTrace fixture, opens panel via URL, asserts
all panel content + breadcrumb on the underlying Logs page. - No table-level count/order assertions.
- Seeds 1 trace via opikTrace fixture, opens panel via URL, asserts
Both tests pass individually, sequentially with WORKERS=1 (35s), and in
parallel with WORKERS=2 (19s). Each test runs in its own project via the
existing fixture cascade.- feat(e2e-auth): mint storage state via login flow at globalSetup for cloud
Make the suite CI-ready by removing dependence on pre-captured .auth files.
What changed:
- env.config: cloud deployments now accept OPIK_TEST_USER_EMAIL +
OPIK_TEST_USER_PASSWORD as an alternative to OPIK_API_KEY. Either path
satisfies the validation. - global-setup: for non-oss deployments without a pre-captured auth state,
POSTs {email, plainTextPassword} to /api/auth/login (auth lives at
the Comet root domain, not under /opik), saves the resulting browser
storage state to .auth/user.json, and propagates the minted API key via
process.env.OPIK_API_KEY. OSS deployments skip this step entirely. Mirrors
the legacy tests_end_to_end/typescript-tests/ pattern. - playwright.config: storageState resolves to .auth/user.json for
cloud/self-hosted and undefined for oss. Removes the deployment-named-file
lookup and OPIK_STORAGE_STATE workaround. - python-sdk-client: forwards process.env.OPIK_API_KEY as X-Opik-Api-Key on
every bridge request (read at request time, not at module load) so the
bridge picks up the minted key even though it was spawned before
globalSetup ran. - bridge routes/projects.py + routes/traces.py: read the X-Opik-Api-Key
header and pass it to opik.Opik(api_key=...). The trace route additionally
set_global_client(client, context_wise=True) so @opik.track's global
client uses the request-scoped credentials. When the header is absent the
routes fall back to OPIK_API_KEY env, preserving local-dev defaults. - typescript-sdk: drops the hard require of env.apiKey for oss deployments.
- .env.local.example + .env.cloud.example: document the env-var contract
for both deployment modes.
Verification:
- OSS (http://localhost:5174, no creds): 2/2 passed in 15.6s
- Cloud (staging, fresh .auth/user.json): 2/2 passed in 36.0s
CI authors set OPIK_DEPLOYMENT=<oss|cloud> + OPIK_BASE_URL (+ creds for
cloud) — no pre-staged .auth files required.- refactor(e2e): address baz-reviewer feedback on PR #6823
- MetricCard: add 'metrics-card--value' testid on the value so
LogsPage.countTraces() can target the value-only element instead of parsing
the whole card text. The card also renders a delta percentage that would
produce a wrong number once a previous-period baseline exists (the count
regression baz flagged). - LogsPage.countTraces: switch primary selector to '-value' and keep the
'Traces N' body-text fallback for staging deploys that don't yet have the
new FE testid. - global-setup auth: tighten the contract. The previous early-return required
BOTH a pre-set OPIK_API_KEY AND an existing .auth/user.json — meaning a
fresh cloud run with only OPIK_API_KEY would fall through to the
email/password requirement and abort with an unclear message. Now:- apiKey + storage state on disk => power-user debug path, skip login.
- email+password => canonical CI path, mint storage state via login.
- apiKey alone with no storage state => clear, actionable error.
env.config message updated to explain both paths.
- bridge: extract opik_factory.make_opik_client(workspace, api_key) helper
so projects.py + traces.py + future routes share the wiring (was duplicated
per route). - bridge: register an app-wide ApiError -> JSONResponse handler in main.py so
each route no longer needs a try/except wrapping ApiError just to re-raise
the same status/body as HTTPException. routes/projects.py and traces.py
drop their nested try blocks accordingly.
Verified both modes still pass:
- OSS (http://localhost:5174): 2/2 in 15.4s
- Cloud (staging, fresh user.json): 2/2 in 26.7s
下载附件
-