bytecii
808b792a02
`invalidate_subtree` had exactly one caller, the watcher. No `rm -r`, recursive prefix delete or `rename` reached it in either language, so every listing and body cached *below* a deleted or moved directory kept being served: `ls` printed a deleted directory's contents and `cat` returned a deleted file's bytes, both exit 0, until the index TTL expired. Reproduced against live minio with the bucket verified empty, so the reads were provably cache rather than survival. Each mutation site reported `invalidate_after_unlink`, which evicts the path's own listing and its parent's -- complete for a file, blind for a subtree, because everything beneath was cached under its own key and nothing above it evicts one. `cache/context` gains an `invalidate_subtree` beside the two it already had, and `CacheInvalidator` gains the method. It delegates to the manager rather than walking: ancestors can be assembled from `invalidate_after_write` calls because the caller knows the chain, but only the caches know which keys lie beneath a path. Then one rule in both languages -- an op that destroys or moves a whole subtree evicts that subtree -- so every `rm_r`/`remove_prefix` and both endpoints of every `rename` swap over. It is a strict superset of `invalidate_after_unlink`, so a file path behaves exactly as before. `unlink`, non-recursive `rmdir` and `copy` are deliberately untouched. This came out of the kit plan's "delete-side `invalidate_ancestors` asymmetry sweep", whose premise was wrong: `invalidate_ancestors` exists because one op can materialize several levels at once, which no delete does on a real-directory backend, and the keyed-store delete that does need it already had it. box and gdrive only looked like they were missing it -- their `mkdir -p` invalidates per created level in a bespoke loop. Also corrects two `emulate_truncate` docstrings that still named s3/ssh/ram/redis as the emulating backends; #858 left dropbox as the only one. Pinned by `integ/resources/cache/subtree_evict.json`, 16 cases across s3/s3-prefix/gridfs/gridfs-prefix, captured empirically on both hosts and checked falsifiable by reverting the object_store half. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
138 lines
4.7 KiB
Python
138 lines
4.7 KiB
Python
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
|
|
|
from contextvars import ContextVar
|
|
from typing import Protocol
|
|
|
|
from mirage.types import PathSpec
|
|
|
|
|
|
class CacheInvalidator(Protocol):
|
|
"""What this module needs from a cache manager.
|
|
|
|
``mirage.cache.manager.CacheManager`` satisfies this structurally;
|
|
this module never imports it, keeping the dependency one-way:
|
|
core mutators -> cache.context <- mount (pushes a manager).
|
|
"""
|
|
|
|
async def invalidate_after_write(self, path: PathSpec) -> None:
|
|
...
|
|
|
|
async def invalidate_after_unlink(self, path: PathSpec) -> None:
|
|
...
|
|
|
|
async def invalidate_subtree(self, path: PathSpec) -> None:
|
|
...
|
|
|
|
async def cached_bytes(self, path: PathSpec) -> bytes | None:
|
|
...
|
|
|
|
|
|
_active: ContextVar[CacheInvalidator | None] = ContextVar(
|
|
"_active_cache_manager", default=None)
|
|
|
|
|
|
def push_cache_manager(
|
|
manager: CacheInvalidator | None) -> CacheInvalidator | None:
|
|
"""Set the active cache manager for the current async context.
|
|
|
|
Mirrors ``observe.context.push_mount_prefix``: the mount entry point
|
|
pushes its manager before dispatching a command, core backend
|
|
mutators report through :func:`invalidate_after_write` /
|
|
:func:`invalidate_after_unlink`, and the caller restores the
|
|
previous value afterwards.
|
|
|
|
Args:
|
|
manager (CacheInvalidator | None): Manager to activate, or None
|
|
to clear.
|
|
|
|
Returns:
|
|
CacheInvalidator | None: The previously active manager, so
|
|
callers can restore it.
|
|
"""
|
|
prev = _active.get()
|
|
_active.set(manager)
|
|
return prev
|
|
|
|
|
|
def active_cache_manager() -> CacheInvalidator | None:
|
|
"""Return the active cache manager for the current async context."""
|
|
return _active.get()
|
|
|
|
|
|
async def invalidate_after_write(path: PathSpec) -> None:
|
|
"""Report a backend write so caches are invalidated at the mutation
|
|
site. No-op if no cache manager is active.
|
|
|
|
Args:
|
|
path (PathSpec): Resource-relative path that was written.
|
|
"""
|
|
manager = _active.get()
|
|
if manager is not None:
|
|
await manager.invalidate_after_write(path)
|
|
|
|
|
|
async def invalidate_after_unlink(path: PathSpec) -> None:
|
|
"""Report a backend deletion so caches are invalidated at the
|
|
mutation site. No-op if no cache manager is active.
|
|
|
|
Args:
|
|
path (PathSpec): Resource-relative path that was removed.
|
|
"""
|
|
manager = _active.get()
|
|
if manager is not None:
|
|
await manager.invalidate_after_unlink(path)
|
|
|
|
|
|
async def invalidate_subtree(path: PathSpec) -> None:
|
|
"""Report a backend deletion that took a whole subtree with it.
|
|
|
|
``invalidate_after_unlink`` evicts the path's own listing and its
|
|
parent's, which is the whole story for a file. A recursive delete
|
|
or a directory rename also strands every listing and every cached
|
|
body *below* the path, and those were cached under their own keys,
|
|
so nothing above them evicts one: ``ls`` kept printing a deleted
|
|
directory's contents and ``cat`` kept serving a deleted file's
|
|
bytes until the index TTL expired.
|
|
|
|
Unlike :func:`invalidate_ancestors`, this cannot be assembled from
|
|
``invalidate_after_write`` calls, because the set of keys beneath
|
|
the path is only known to the caches themselves.
|
|
|
|
Args:
|
|
path (PathSpec): Root of the subtree that is gone.
|
|
"""
|
|
manager = _active.get()
|
|
if manager is not None:
|
|
await manager.invalidate_subtree(path)
|
|
|
|
|
|
async def invalidate_ancestors(path: PathSpec) -> None:
|
|
"""Evict every ancestor directory listing of ``path``.
|
|
|
|
A single ``invalidate_after_write`` only refreshes the immediate
|
|
parent listing. When an op materializes several missing levels at
|
|
once (``mkdir -p a/b/c``, a bucket write that creates parents), the
|
|
higher ancestors' cached listings stay stale and hide the new
|
|
entries until the index TTL expires. Walking the chain refreshes
|
|
each one.
|
|
|
|
Args:
|
|
path (PathSpec): Mount-relative path that was mutated.
|
|
"""
|
|
parent = path.mount_path.rsplit("/", 1)[0]
|
|
while parent:
|
|
await invalidate_after_write(PathSpec.from_str_path(parent))
|
|
parent = parent.rsplit("/", 1)[0]
|