Solution or workaround for nested zip/jar files?

[kriegaex]

This test being disabled implies, that you have long been aware of the problem I just had, namely to create a nested zip fille system:

memoryfilesystem/src/test/java/com/github/marschall/memoryfilesystem/ZipFileSystemInteroperabilityTest.java

Lines 45 to 70 in 4cc26b1

	@Test
	@Disabled("broken")
	void createNestedZips() throws IOException {
	FileSystem memoryFileSystem = this.extension.getFileSystem();
	Map<String, String> env = Collections.singletonMap("create", "false");
	Path outerZip = memoryFileSystem.getPath("/file.zip");
	try (OutputStream stream = new JarOutputStream(Files.newOutputStream(outerZip, CREATE_NEW, WRITE))) {
	// nothing, just create an empty jar
	}
	URI uri = URI.create(FS_URI + outerZip.toUri());
	try (FileSystem zipfs = FileSystems.newFileSystem(uri, env)) {
	Path innerZip = zipfs.getPath("hello.zip");
	try (OutputStream stream = new JarOutputStream(Files.newOutputStream(innerZip, CREATE_NEW, WRITE))) {
	// nothing, just create an empty jar
	}
	Map<String, String> env2 = Collections.singletonMap("create", "false");
	// locate file system by using the syntax
	// defined in java.net.JarURLConnection
	URI uri2 = URI.create(FS_URI + innerZip.toUri());
	try (FileSystem zipfs2 = FileSystems.newFileSystem(uri2, env2)) {
	try (BufferedWriter writer = Files.newBufferedWriter(zipfs2.getPath("hello.txt"), US_ASCII, CREATE_NEW, WRITE)) {
	writer.write("world");
	}
	}
	}
	}

To be fair, the issue occurs identically on JimFS.

Is there any workaround you can recommend to create and read from a nested zip file inside another one on MemoryFS, other than to do it outside of the actual outer zip and copy the file into the zip during creation and caching it outside while reading or modifying it? My use case is testing something related to Spring Boot executable JARs, which contain dependency JARs. What if I want to read some of the inner JARs and verify or update their contents?

java.lang.IllegalArgumentException: URI: jar:memory:__g_1:///home/me/projects/my-project/target/my-project-1.0.jar does not contain path info ex. jar:file:/c:/foo.zip!/BAR

  at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:139)
  at java.base/java.nio.file.Path.of(Path.java:209)
  at java.base/java.nio.file.Paths.get(Paths.java:98)
  at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.uriToPath(ZipFileSystemProvider.java:76)
  at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:98)
  at java.base/java.nio.file.FileSystems.newFileSystem(FileSystems.java:339)
  ...

[kriegaex]

Hm, when trying from scratch in isolation, it actually works:

import com.github.marschall.memoryfilesystem.MemoryFileSystemBuilder
import com.google.common.jimfs.Configuration
import com.google.common.jimfs.Jimfs
import spock.lang.Specification
import spock.lang.Unroll

import java.nio.file.FileSystems
import java.nio.file.Files

class NestedZipTest extends Specification {
  @Unroll('#scenario')
  def 'create nested zip file'() {
    given: 'a text file on the default FS'
    def sourceFS = FileSystems.default
    def rootTxtPath = sourceFS.getPath('root.txt')
    Files.write(rootTxtPath, 'Hello root!'.bytes)

    when: 'creating a zip FS on the target FS, adding two text files'
    def outerZipPath = targetFS.getPath('outer.zip')
    if (Files.exists(outerZipPath))
      Files.delete(outerZipPath)
    def outerZipFS = FileSystems.newFileSystem(outerZipPath, [create: 'true'])
    Files.write(outerZipFS.getPath('outer.txt'), 'Hello outer!'.bytes)
    Files.copy(rootTxtPath, outerZipFS.getPath('from-root.txt'))

    and: 'creating a zip FS inside the outer zip file, adding two text files'
    def innerZipPath = outerZipFS.getPath('inner.zip')
    def innerZipFS = FileSystems.newFileSystem(innerZipPath, [create: 'true'])
    Files.write(innerZipFS.getPath('inner.txt'), 'Hello inner!'.bytes)
    Files.copy(rootTxtPath, innerZipFS.getPath('from-root.txt'))

    and: 'creating a zip FS inside the inner zip file, adding two text files'
    def inner2ZipPath = innerZipFS.getPath('inner2.zip')
    def inner2ZipFS = FileSystems.newFileSystem(inner2ZipPath, [create: 'true'])
    Files.write(inner2ZipFS.getPath('inner2.txt'), 'Hello inner2!'.bytes)
    Files.copy(rootTxtPath, inner2ZipFS.getPath('from-root.txt'))

    then: 'no errors occur'
    noExceptionThrown()

    cleanup:
    inner2ZipFS?.close()
    innerZipFS?.close()
    outerZipFS?.close()

    where:
    scenario           | targetFS
    'on disk'          | FileSystems.default
    'JimFS'            | Jimfs.newFileSystem(Configuration.unix().toBuilder().setWorkingDirectory('/').build())
    'MemoryFileSystem' | MemoryFileSystemBuilder.newEmpty().build()
  }
}

Try it in the Groovy Web Console. There, I minimally modified the code to @Grab the two in-memory FS dependencies and to say def sourceFS = MemoryFileSystemBuilder.newEmpty().build(), because the GWC does not allow to write to the default file system. Of course, when running the test, the first iteration trying to do that fails, but running the test locally works. In the GWC, with the modified sourceFS the two in-memory scenarios are working.

It seems as if the problem sits in front of the computer. I am going to try and adjust my actual application code to resemble my test above. Hopefully, I can get it working with this workaround. I will keep you posted.

[kriegaex]

OK, I found the root cause. While trying to debug another problem, I had changed some code to use an URI instead of a regular path. Later, I forgot about it, and it caused problems down the line.

Sorry for the noise, I am closing the issue. You might, however, want to look into the ignored tests in this class (and maybe others, if any), trying to fix them the same way I did, i.e. avoiding URI paths and simply creating new nested file systems from regular paths. Then at least something good would come from this issue, and I could return something to the project that I am using. Feel free to refactor your tests according to my example. The Spock test should be easy to translate to JUnit.