How is read-write layer mounting done on the host with computestorage?

[TBBle]

I was trying to implement a version of cmd/wclayer using the computestorage APIs, and I'm missing some important step for mount.

The flow with the RS1 APIs is:

• ActivateLayer(rwLayerPath)
• PrepareLayer(rwLayerPath, parents)
• volumePath := GetLayerMountPath(rwLayerPath)
• SetVolumeMountPoint(targetPath, volumePath)

With computestorage, it seems we need:

• (I'm guessing this!) vhd.AttachVhd(rwLayerPath+"\sandbox.vhdx")
• AttachLayerStorageFilter(rwLayerPath, parentLayerData)
• volumePath := GetLayerVhdMountPath(someVHDHandle)
• SetVolumeMountPoint(targetPath, volumePath)

The equivalence guess is based on the in-source description of ActivateLayer as mounting filesystems, and PrepareLayer as enabling the filesystem filter.

But I haven't been able to get this to actually work yet.

The current source code I'm trying to use, in case my flow is right but the flags are wrong

// containerSandboxVHDXName is "sandbox.vhdx".
// I have pushed this to https://github.com/TBBle/hcsshim/commit/90725df9c13487cd976604220c5d36742097fc75 if you want to see `getLayerData` for example.

var mountCommand = cli.Command{
	Name:      "mount",
	Usage:     "activates a scratch, optionally mounted to provided target",
	ArgsUsage: "<scratch path> [target path]",
	Before:    appargs.Validate(appargs.NonEmptyString, appargs.Optional(appargs.String)),
	Flags: []cli.Flag{
		cli.StringSliceFlag{
			Name:  "layer, l",
			Usage: "paths to the parent layers for this layer",
		},
	},
	Action: func(cliContext *cli.Context) (err error) {
		path, err := filepath.Abs(cliContext.Args().Get(0))
		if err != nil {
			return err
		}

		sandboxVHDXPath := filepath.Join(path, containerSandboxVHDXName)

		if _, err := os.Stat(sandboxVHDXPath); err != nil {
			return fmt.Errorf("failed to Stat %q: %w", sandboxVHDXPath, err)
		}

		if err := vhd.AttachVhd(sandboxVHDXPath); err != nil {
			return fmt.Errorf("failed to Attach %q: %w", sandboxVHDXPath, err)
		}

		defer func() {
			if err != nil {
				_ = vhd.DetachVhd(sandboxVHDXPath)
			}
		}()

		ctx := context.Background()

		layerData, err := getLayerData(ctx, cliContext.StringSlice("layer"), true)
		if err != nil {
			return err
		}

		err = computestorage.AttachLayerStorageFilter(ctx, path, *layerData)
		if err != nil {
			return err
		}
		defer func() {
			if err != nil {
				_ = computestorage.DetachLayerStorageFilter(ctx, path)
			}
		}()

		// Copied out of vhd.AttachVhd
		vhdHandle, err := vhd.OpenVirtualDisk(
			sandboxVHDXPath,
			vhd.VirtualDiskAccessNone,
			vhd.OpenVirtualDiskFlagCachedIO|vhd.OpenVirtualDiskFlagIgnoreRelativeParentLocator,
		)
		defer windows.CloseHandle(windows.Handle(vhdHandle))

		mountPath, err := computestorage.GetLayerVhdMountPath(ctx, windows.Handle(vhdHandle))
		if err != nil {
			return err
		}

		_, err = fmt.Println(mountPath)
		return err
	},
}

When I try the above source code, the first call for a given newly-created read-write layer fails with:

failed to get vhd mount path: The specified network resource or device is no longer available.

and then recalling it on the same layer path gives:

failed to attach layer storage filter: The parameter is incorrect.

The latter failure indicates that I managed to get some state created, that the deferred DetachLayerStorageFilter + vhd.DetachVhd isn't cleaning up.

Without the computestorage.GetLayerVhdMountPath call, the code passes, but I don't see a new volume in mountvol the way I do with the existing wclayer code, suggesting that something is not right in the VHD-management step. In that case, repeating the call for the same path gives the same "The parameter is incorrect" error, so the state I'm creating that is not being cleaned up must come from computestorage.AttachLayerStorageFilter.

I can't find any existing use-cases for AttachLayerStorageFilter to model off, so I'm just throwing code at the wall to see what sticks, and my current guess is that vhd.AttachVhd is either the wrong thing to use, or I need to call the lower-level APIs in vhd with different flags to get the VHD attached correctly.