- `FileTree.lookup` resolved each path component by linearly scanning the
node's `children` array. This changes the node's child storage to an
`OrderedDictionary<String, Ptr<FileTreeNode>>` (from swift-collections,
which is already a package dependency) keyed by name, so `lookup`
resolves each component in O(1) while iteration keeps the existing
insertion order.
- Little or no difference in unpack time for images with ~10k files, significant
improvement for images with ~100k files or more.
When unpacking an OCI/tar layer, create() already creates missing parent
directories recursively, so regular files and symlinks with absent
parent entries unpack correctly. link() did not, so a hardlink whose
parent directory had no explicit archive entry failed with "<path> not
found" (e.g. images produced by Bazel rules_img). Mirror create()'s
implicit parent creation in link() so such layers unpack, matching
Docker/containerd.
Adds a direct link() unit test and an end-to-end unpack regression test
covering a hardlink, regular file, and symlink with no explicit parent.
Fixes https://github.com/apple/container/issues/1797
This issue doesn't affect any of our existing products. This is a
preemptive fix for downstream consumers of EXT4.format where, in some
platforms, leading `//` in the path could get resolved into a FileTree
that looks like this
```sh
/
└── /
└── usr
```
- Closes#671.
- Adds optional journal parameter to `EXT4.Formatter.init()`, with nil
default specifying the current no-journal filesystem configuration.
Otherwise the parameter value contains the journal size and mode.
- `minDiskSize` parameter for formatter init specifies the minimum
usable capacity of the resulting filesystem. The resulting disk image
grows past this value to accommodate the filesystem on-disk structures,
including the journal if specified.
Restores a configurable block size in the EXT4 formatter.
https://github.com/apple/containerization/pull/662 resolved a conflict
between the hardcoded log block size and a user-provided block size, but
removed the later. This PR restores the configurable block size without
creating the conflict.
- Fixes a hardlink count decrement threshold
- Fixes unlinking not freeing the first inode
Resolves the failing added tests:
```
✘ Test hardlinkLinksCount() recorded an issue at TestEXT4Format+Link.swift:43:9: Expectation failed: try EXT4.EXT4Reader(blockDevice: afterUnlink).stat("/original").inode.linksCount == 1
✘ Test hardlinkLinksCount() failed after 0.016 seconds with 1 issue.
```
```
✘ Test unlinkFirstInodeFreesInode() recorded an issue at TestEXT4Format+Link.swift:58:9: Expectation failed: try EXT4.EXT4Reader(blockDevice: path).superBlock.freeInodesCount == EXT4.EXT4Reader(blockDevice: emptyPath).superBlock.freeInodesCount
✘ Test unlinkFirstInodeFreesInode() failed after 0.014 seconds with 1 issue.
```
Fixes the `xattr` read loop bounds. Resolves the failing added tests:
> ✘ Test lastXattrNotDroppedAtBufferBoundary() recorded an issue at
TestEXT4ExtendedAttributes.swift:72:13: Expectation failed: (attrs.count
→ 0) == 1
> ✘ Test lastXattrNotDroppedAtBufferBoundary() failed after 0.001
seconds with 1 issue.
> Swift/Array.swift:430: Fatal error: Array index is out of range
Removes the incorrect check for visited inodes. Fixes the added failing
tests:
> ✘ Test sameAbsoluteSymlinkFollowedTwice() recorded an issue at
TestEXT4Reader+IO.swift:502:6: Caught error: symlink loop while
resolving: target/../symlink/file.txt
> ✘ Test sameAbsoluteSymlinkFollowedTwice() failed after 0.009 seconds
with 1 issue.
> ✘ Test sameRelativeSymlinkFollowedTwice() recorded an issue at
TestEXT4Reader+IO.swift:516:6: Caught error: symlink loop while
resolving: ../target/../symlink/file.txt
> ✘ Test sameRelativeSymlinkFollowedTwice() failed after 0.010 seconds
with 1 issue.
Fixes the construction of the path and removed unnecessary code. Fixes
the added failing test:
> ✘ Test fileTreeNodePathWithAbsoluteRoot() recorded an issue at
TestEXT4Reader+IO.swift:600:9: Expectation failed: (dirPtr.pointee.path
→ /) == (FilePath("/dir") → /dir)
> ✘ Test fileTreeNodePathWithAbsoluteRoot() recorded an issue at
TestEXT4Reader+IO.swift:601:9: Expectation failed: (filePtr.pointee.path
→ /) == (FilePath("/dir/file") → /dir/file)
> ✘ Test fileTreeNodePathWithAbsoluteRoot() failed after 0.001 seconds
with 2 issues.
Fixes a force-unwrap crash on the last leaf of an extent tree:
> ContainerizationEXT4/EXT4+Formatter.swift:1148: Fatal error:
Unexpectedly found nil while unwrapping an Optional value
Adds a guard against an empty range in the EXT4 formatter to prevent a
possible crash:
> Swift/arm64e-apple-macos.swiftinterface:6314: Fatal error: Range
requires lowerBound <= upperBound
Removes not used `FilePath.init?(Data)` which had a buffer overread. It
used `String(cString:)` which reads until `\0`, but `Data` is not
null-terminated.
I've added an optional progress handler for rootfs unpacking so that
consumers can show progress. The Ubuntu image takes about 8 seconds to
unpack on my machine, and I'm developing an app where it would be useful
to show progress in the UI.
Total size is determined in an optional first pass that scans archive
headers. Bytes written are then reported during unpacking. The optional
first pass adds 15 (Alpine) to 115 (Ubuntu) ms to unpacking duration on
my machine, depending on image size.
- Fixes EXT4 timestamp encoding for pre-1970 dates
- Fixes EXT4 timestamp decoding for pre-1970 dates
- Fixes the creation date
- Fixes a `UInt32` overflow
Resolves the following failures in the added tests:
```
Swift/arm64e-apple-macos.swiftinterface:38198: Fatal error: Double value cannot be converted to UInt64 because the result would be less than UInt64.min
error: Process '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/libexec/swift/pm/swiftpm-testing-helper --test-bundle-path /Users/Dmitry/Apple/containerization/.build/arm64-apple-macosx/debug/containerizationPackageTests.xctest/Contents/MacOS/containerizationPackageTests --filter encodeNegativeTimestamp /Users/Dmitry/Apple/containerization/.build/arm64-apple-macosx/debug/containerizationPackageTests.xctest/Contents/MacOS/containerizationPackageTests --testing-library swift-testing' exited with unexpected signal code 5
```
```
✘ Test decodeNegativeTimestamp() recorded an issue at TestEXT4Format+Create.swift:100:6: Caught error: not a valid EXT4 superblock
✘ Test decodeNegativeTimestamp() failed after 0.003 seconds with 1 issue.
✘ Suite NegativeTimestampRoundtripTests failed after 0.004 seconds with 1 issue.
✘ Test run with 1 test in 1 suite failed after 0.004 seconds with 1 issue.
```
## Summary
The `EXT4.Inode` struct is public but its properties were internal,
preventing external consumers from reading inode metadata. This change
makes the properties public to enable use cases like reading file
metadata (mode, uid, gid, timestamps) from ext4 filesystems.
Motivating use case: https://github.com/socktainer/socktainer/pull/169🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Optimize unpack a little by trying to reduce allocations in the hot
path. Today for every file we read the entire file into memory and then
pass the data blob to the ext4 writer to eventually be written to the
sparse file. Before being written to the sparse file the data is copied
*again* to a temp buffer before finally hitting write(2) in FileHandle.
This change moves things around such that we can pass an optional buffer
to the ext4 create() (so we can reuse a buffer for file writes), as well
as stops reading entire files into memory by passing the archive entry
itself (wrapped in a ReaderStream object albeit) down to the writer.
Testing with unpacking every platform for
`docker.io/jenkins/jenkins:lts` on an M1 Max:
Old Avg (5 runs): 7.43s
New Avg (5 runs): 5.31s
Continue the allocations journey for anything that is in the codepaths
for pulling images. This time there's a couple spots in archive and ext4
we can get rid of some copies.
- FileTimestamps constructor that actually let you provide values was
not public, so it wasn't possible to pass in values other than nil. This
change makes the other constructor and the underlying fields public.
- Rename EXT4+Format to EXT4+Formatter
- Rename EXT4+Export to EXT4Reader+Export
- Make the superblock publicly accessible in the reader like the docs
for the product states.
- Corrected the typo in the method name (`Extened` → `Extended`)
- Updated all references to reflect the corrected spelling
- Introduced a `@available(*, deprecated)` alias for the original method
to maintain backward compatibility, as it was part of the public API
Kameron Kilchrist