fix(integ): widen the truth matchers the runners disagree with
Three literals were host-dependent and CI proved it: the disk example's stat line carries the mtime actions/checkout stamps on data/example.json, its restore root is /tmp on Linux where it was /var/folders on macOS, and the TS redis cache example echoes $REDIS_URL, which is db 1 locally and db 0 on the runners. All three got captured as literals because --emit runs both of its captures on one machine, so it sees run-to-run volatility and is blind to host-to-host volatility. That limitation is now in its docstring with the three classes to widen by hand. The workflow loops now keep going after a failure and fail the step at the end, so one stale truth file reports every other one in the same run instead of hiding the examples listed below it. Both runs this round aborted early and hid nine Python and two TypeScript files.
This commit is contained in:
@@ -97,12 +97,18 @@ jobs:
|
||||
run: uv run pytest --ignore=tests/agents/camel
|
||||
|
||||
# Each truth file owns the command it was captured against, so this
|
||||
# list is paths only and cannot drift from what is asserted.
|
||||
# list is paths only and cannot drift from what is asserted. The
|
||||
# runner keeps going after a failure and fails the step at the end,
|
||||
# so one stale truth file reports every other one in the same run
|
||||
# instead of hiding the examples listed below it.
|
||||
- name: Examples (output checked against integ/ truth files)
|
||||
env:
|
||||
REDIS_URL: redis://localhost:6379/0
|
||||
run: |
|
||||
run() { python3 integ/check_example.py "$1" --variant python; }
|
||||
bad=0
|
||||
run() {
|
||||
python3 integ/check_example.py "$1" --variant python || bad=1
|
||||
}
|
||||
run integ/truth/python/ram.json
|
||||
run integ/truth/python/ram_vfs.json
|
||||
run integ/truth/python/ram_python.json
|
||||
@@ -116,6 +122,7 @@ jobs:
|
||||
run integ/truth/python/redis_cache.json
|
||||
run integ/truth/python/redis_vfs.json
|
||||
run integ/truth/version_branching.json
|
||||
exit $bad
|
||||
|
||||
- name: Sync Python dependencies for camel
|
||||
working-directory: python
|
||||
|
||||
@@ -92,12 +92,18 @@ jobs:
|
||||
run: pnpm test
|
||||
|
||||
# Each truth file owns the command it was captured against, so this
|
||||
# list is paths only and cannot drift from what is asserted.
|
||||
# list is paths only and cannot drift from what is asserted. The
|
||||
# runner keeps going after a failure and fails the step at the end,
|
||||
# so one stale truth file reports every other one in the same run
|
||||
# instead of hiding the examples listed below it.
|
||||
- name: Examples (output checked against integ/ truth files)
|
||||
env:
|
||||
REDIS_URL: redis://localhost:6379/0
|
||||
run: |
|
||||
run() { python3 integ/check_example.py "$1" --variant typescript; }
|
||||
bad=0
|
||||
run() {
|
||||
python3 integ/check_example.py "$1" --variant typescript || bad=1
|
||||
}
|
||||
run integ/truth/typescript/ram.json
|
||||
run integ/truth/typescript/ram_vfs.json
|
||||
run integ/truth/typescript/disk.json
|
||||
@@ -115,6 +121,7 @@ jobs:
|
||||
run integ/truth/typescript/redis_cache.json
|
||||
run integ/truth/typescript/redis_vfs.json
|
||||
run integ/truth/version_branching.json
|
||||
exit $bad
|
||||
|
||||
python-fs:
|
||||
needs: changes
|
||||
|
||||
@@ -37,6 +37,19 @@ Line matchers, in the order of preference an author should reach for:
|
||||
differs between the runs becomes a `re` with digit runs generalized, or
|
||||
a `volatile` when even that does not hold. Tighten a `volatile` by hand
|
||||
into an `re` whenever the line has a stable shape worth pinning.
|
||||
|
||||
Both of those runs happen on ONE machine, so `--emit` sees run-to-run
|
||||
volatility and is blind to host-to-host volatility: a line that is
|
||||
stable on the author's Mac and different on CI's Ubuntu is emitted as a
|
||||
literal. Three classes of line have to be widened by hand after an
|
||||
emit, all of which reached CI as literals before this was written down:
|
||||
|
||||
a temp root mkdtemp is /var/folders/... on macOS, /tmp on Linux
|
||||
a checkout mtime actions/checkout stamps the repo files on clone
|
||||
an env value REDIS_URL is db 1 locally and db 0 on the runners
|
||||
|
||||
The rule of thumb: a literal earns its place when the example itself
|
||||
determines it, and has to become an `re` when the machine does.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
|
||||
@@ -38,7 +38,9 @@
|
||||
" 257 1370 15533 /data/example.json",
|
||||
"",
|
||||
"=== stat /data/example.json ===",
|
||||
"name=example.json size=15533 modified=2026-08-17T03:59:32Z type=json",
|
||||
{
|
||||
"re": "name=example\\.json size=15533 modified=\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}Z type=json"
|
||||
},
|
||||
"",
|
||||
"=== tree /data/ ===",
|
||||
"/data/",
|
||||
@@ -134,7 +136,9 @@
|
||||
"example.jsonl",
|
||||
"example.orc",
|
||||
"example.parque…",
|
||||
" loaded (root=/var/folders/4d/s_lrmhbd7xb9jglnmy13sz_w…) ls: example.feather",
|
||||
{
|
||||
"re": " loaded \\(root=[^)]+…\\) ls: example\\.feather"
|
||||
},
|
||||
"example.h5",
|
||||
"example.json",
|
||||
"example.jsonl",
|
||||
|
||||
@@ -15,7 +15,9 @@
|
||||
"lines": [
|
||||
"=== RedisFileCacheStore: FileCache backed by Redis ===",
|
||||
"",
|
||||
" url: redis://localhost:6379/1",
|
||||
{
|
||||
"re": " url: redis://localhost:6379/\\d+"
|
||||
},
|
||||
" keyPrefix: mirage:example:cache:",
|
||||
" cacheLimit: 67108864 bytes",
|
||||
"",
|
||||
|
||||
Reference in New Issue
Block a user