FileArtifactService chose the next version by listing the versions directory
and then created that directory as a separate step. Two saves racing on the
same artifact read the same version list, and the loser's mkdir raised
FileExistsError. An interrupted save was worse: the version directory existed
but held no payload, and because version discovery counts any integer-named
directory, that empty directory became the latest version and load_artifact
returned None instead of the previous good content.
A save now reserves a version by creating a hidden `.{version}.pending`
directory, which mkdir makes atomic and mutually exclusive, writes the payload
and metadata inside it, and publishes the result with a single os.replace onto
`versions/{version}`. Version discovery ignores the staging directories, so a
version becomes visible only once it is complete, and a save that raises
removes its own staging directory. A save whose version list went stale
re-checks before publishing and takes the next free version instead, so
os.replace always lands on a name that does not exist. That last part matters
on Windows, where os.replace maps to MoveFileEx and reports an error when the
destination names an existing directory.
The guarantee is scoped to process-level faults. An exception or a signal never
publishes a partial version, but nothing is fsynced, so a power loss can still
make the rename durable ahead of the file contents and expose an empty
`versions/{version}`. Full durability would need explicit fsyncs and is not
addressed here, and version directories that earlier releases already left
empty are not repaired.
Behavior change: published version numbers are no longer guaranteed to be
contiguous. A reservation abandoned by a signal or a host failure leaves its
`.{version}.pending` directory behind; that directory is never read and never
published, but it holds its version number permanently. There is deliberately
no in-process sweep, because nothing distinguishes an abandoned reservation
from a slow concurrent write, so reclaiming those directories is left to the
operator. The class-level layout comment records the staging directory and the
non-contiguity.
The public artifact-service API and the completed on-disk version layout are
unchanged.
Co-authored-by: George Weale <gweale@google.com>
PiperOrigin-RevId: 967413806
George Weale
94475c9a76