-
nixl_storage: naive support for files + dynamic (#2936)
frostbyte_neo 发布于
2026-05-18 16:22:23 +00:00 - nixl_storage: naive support for files + dynamic
static is not actually very usable with files: we bump into the OS
limits on the open files very quickly, limiting the size of the cache.
With tons of VRAM, having G3 of comparable size is not helping much, we
really need it to be much bigger.This commit adds support for files in dynamic mode of nixl storage.
There are some naive things done:- No support for sharing the cache storage. We assume the worker has
exclusive access to the files and it is always safe to overwrite
existing files. Note that with TP!=1 we will have several workers
with the same target directory, that's why it's important to have a
worker id as part of the key, so they don't affect each other. - No eviction support. As before, dynamic mode has no evicting support.
This kinda made sense when it was only working with OBJ storages, but
now that is something to be aware of. It's not hard to add eviction
though. - Flat directory with all the cache files. There are going to be a lot
of them, especially provided we don't have eviction. Most filesystems
don't optimize for the case of a directory with millions of files,
and we constantly open/close files there, that might be a source
of additional latency.- There is an existing PR to support sharding the directory, we can
merge it as a band aid. - As a more advanced solution, we can do multi-layer subdirectory
structure, like gds backend does.
- There is an existing PR to support sharding the directory, we can
- We switched directly from having all files open at once, to
open/close every single file on access. We might explore having an
LRU cache of open files instead.
Signed-off-by: Ilya Yanok iyanok@nvidia.com
- Fix formatting
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Fix PR Comments
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Extract _build_descs
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Fix test
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Refactor to increase readability
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Fix another leak issue
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Fix release order
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Treat file not found as a miss
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Add docstring about file create mode 0o644
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Unlink files on failure
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Add missing docs
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Fix pre-commit issues
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Fix mypy error
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Fix test
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Make tests pass
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
- Fix formatting
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
Signed-off-by: Ilya Yanok iyanok@nvidia.com
Signed-off-by: Guy Ealey Morag gealeymorag@nvidia.com
Co-authored-by: Guy Ealey Morag gealeymorag@nvidia.com下载附件