Download Tiles & Store Them For Offline Use

[JaffaKetchup]

Hello,
I have been trying to use flutter_map to create an app where a specific 'Downloads Only' mode can be enabled. The aim is to have tiles loaded whilst not in this mode downloaded to the users device, like caching (except I do not want these tiles to be deleted by the system). Then, once the user is offline or in 'Downloads Only' mode, I'd like to use the images stored from before.

I also have setup multiple map tile servers and have a global variable holding the app's document directory 'globals.saveDir'. I have started doing this using the network_to_file_image package like so:

class AdvancedFileCacheProvider extends TileProvider {
  @override
  ImageProvider getImage(Coords<num> coords, TileLayerOptions options) {
    return NetworkToFileImage(
      url: getTileUrl(coords, options),
      file: File(
        p.join(
            globals.saveDir.path,
            getTileUrl(coords, options)
                .replaceAll('https://', '')
                .replaceAll('http://', '')),
      ),
      debug: true,
    );
  }
}
...
TileLayerOptions(
          urlTemplate: Uri.decodeQueryComponent(urlForMap),
          subdomains: ['a', 'b', 'c'],
          tileProvider: AdvancedFileCacheProvider(),
),
...

This setup prints:

I/flutter (14390): Fetching image from: http://mt.google.com/vt/lyrs=m&x=16324&y=10894&z=15
I/flutter (14390): Fetching image from: http://mt.google.com/vt/lyrs=m&x=16325&y=10894&z=15
I/flutter (14390): Saving image to file: /data/user/0/[appname]/app_flutter/mt.google.com/vt/lyrs=m&x=16324&y=10894&z=15
I/flutter (14390): Saving image to file: /data/user/0/[appname]/app_flutter/mt.google.com/vt/lyrs=m&x=16325&y=10894&z=15
etc...

, which means the downloading of images are working as intended.

However, once I go offline (there is no special mode yet, but I think the network_to_file_image package is supposed to work like this...), the images still try to be fetched from the internet, resulting in a SocketException error.

Has anyone else implemented a similar logic? I understand that this package will cache these tiles, but anything in the cache is vulnerable to the system clearing it, and in my circumstance, this would be quite a large problem.

Thank you very much!
Luka S.

[ibrierley]

Have you first checked that repeated calls to NetworkToFileImage don't try to download from the internet again even when still online (i.e your debug shows saving it, but not repeated attempts to retrieve from file or download..)?

[ibrierley]

I note it says

Meanwhile, network_to_file_image will simply download the image and leave it there. Think WhatsApp or Telegram: someone sends you an image, it's downloaded and kept there
in your Gallery/files forever, or until someone deletes it manually.

Which implies it won't get cleared from cache...

[JaffaKetchup]

Hello @ibrierley.

Thanks for the help, however this issue is the retrieval of the images after they have been downloaded, not the actual downloading. The library says that the downloaded version of a file will always be used if available, so I suppose I should raise the issue there instead. The reason I rose it here is because I think it might be something to do with this library, except I don't quite know what.

I'll raise the issue on that library, but I will not close this issue in the hope that someone has done the same thing I am trying to do.

Many thanks.

[ibrierley]

Yep, I get it's the retrieval issue bit. It's just I hadn't seen any debugging to see what the problem is. I.e I think debug=true that you have, should show it attempting to grab from file rather than from http. However the original issue doesn't make it clear if there is any debug for that, is it looking at the correct filename that exists, or not even trying or what ?

[JaffaKetchup]

Debug=true just enables those messages. Of there are no lines saying retrieval from file, there is no retrieval from file. I have no way of seeing if it tries and just can't find the file name, but I assume it would say that, as well as an exception in the console.

[JaffaKetchup]

I guess what I'm really asking is... Is there a way to download tiles similar to caching, but have those tiles permanantly. It doesn't have to use the forementioned library.

[ibrierley]

Yes, there is a way, but equally this should do it from what I can see.... Looking at their source code, it should output...

if (debug) print("Reading image file: ${file?.path}");

If that is not output, my guess would be it can't see the file. So it's worth checking if that file does exist, what globals.saveDir outputs, and if your path ends up matching /data/user/0/[appname]/app_flutter/

[JaffaKetchup]

Ok, I will try to do that, and get back to you soon. Do you know how I can view /data/user/0 on an Android Pixel 3a emulator on Windows?

[ibrierley]

I don't have that to test, but maybe this will help https://stackoverflow.com/questions/4847654/how-to-access-local-files-of-the-filesystem-in-the-android-emulator

…

On Wed, Jan 6, 2021 at 4:45 PM Luka S ***@***.***> wrote: Ok, I will try to do that, and get back to you soon. Do you know how I can view /data/user/0 on an Android Pixel 3a emulator on Windows? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#795 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA5YN5IQXQZ4I434HRQSCLTSYSHTVANCNFSM4VTAHQDA> .

[JaffaKetchup]

I have dug very deep and this is my findings:

There is no image there. So now I guess I track down the error in the library itself. I will not close this issue yet, until fully resolved, but thanks a lot for your help and insight! 👍

[ibrierley]

I'm just wondering if it's something to do with the nested directory on the url getting confused with the filename. . Eg you download path/some/nested/directory.file.png instead of filename_x_y_z.png or something ....is it treating that url path directory and failing because it needs the path/some/nested/directory already created. Shout if that doesn't make sense. I'm not sure of the solution if it is. Ian

…

On Wed, Jan 6, 2021 at 5:25 PM Luka S ***@***.***> wrote: I have dug very deep and this is my findings: [image: image] <https://user-images.githubusercontent.com/58115698/103800196-fea21c80-5043-11eb-879f-e558c6913d40.png> There is no image there. So now I guess I track down the error in the library itself. I will not close this issue yet, until fully resolved, but thanks a lot for your help and insight! 👍 Luka S — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#795 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA5YN5JG53N22JDDUCQGCE3SYSMJRANCNFSM4VTAHQDA> .

[JaffaKetchup]

Hello.
I reached that conclusion about 10 minutes before your comment! Almost beat me to it 👍 :).

It is that. It was trying to be stored as a directory that didn't exist. I haven't fully tested, but I have implemented a solution that appears to work:

As you can see, they are now stored, and I am testing to see if they are read back. Will close once test complete.

Thanks!
Luka S

[JaffaKetchup]

The test has completed successfully. Once again, thanks a bunch!

Closing this issue.