Mapbox Tiling Problems (Possibly Since Changes to MercatorTiledImageLayer)

[robmilton]

Description

We had created a copy of MercatorTiledImageLayer from WWJ v2.1.0 with the only change being that we made the "splitScale" variable public. This allowed us to influence the "needToSplit" resulting in better looking maps from Mapbox.

We have several Mapbox layers containing raster data and several containing vector data. ALL of these layer worked with WWJ v2.1.0, but after the change in the MercatorTiledImageLayer in WWJ-CE we are having problems with the layers that contain vector data (streets, boundaries, etc.).

When the layers that contain vector data load it looks like there are missing tiles in various locations as you zoom in and out. We know the tiles aren't missing, so I'm guessing there's something about the way the tiles are being requested that's changed.

If I revert to WWJ v2.1.0 ALL of these same Mapbox layers display without any problems. I'm new to Git/GitHub, but I'm wondering if there's a way for me to revert my local copy of WWJ-CE to a point before the change to the MercatorTiledImageLayer was merged in order to confirm if this is when the issue appeared? I'll poke around and see if I can figure out how to do that.

If there's another/better way to display all our Mapbox maps in WWJ I'd love some pointers. Otherwise I need to figure out what happened and get this corrected one way or another.

Edit
I figured out how to revert to just before the change to MercatorTiledImageLayer, but this did not fix the issue. If I use WWJ v2.1.0 the problem goes away. I guess I can keep reverting until I discover the point at which the problem goes away. Any thoughts on what may have caused this and how to fix it are certainly welcome.

Expected behavior:
All of our Mapbox maps should load as they did while using WWJ v2.1.0.

Actual behavior:
Mapbox maps containing vector data (streets, boundaries, etc.) look like they are missing tiles at various locations as the map is zoomed in and out.

Steps to Reproduce

I suppose this would be hard for anyone that doesn't use Mapbox maps. However, simply set up a map from Mapbox that contains vector data (using the OSMMapnikLayer as an example).

Reproduces how often:
100% of the time

Operating System and Version

Windows 10

Additional Information

[EMaksymenko]

Hello.
Could you provide a class which you are using to connect MapBox, so we can try to reproduce a bug.
Or at least url builder.

[robmilton]

I've temporarily made a very simple "black" map available with a temporary access token (included in the code below). I just used the OSMMapnikLayer as a template.

If you zoom in and out particularly along the coast line around the US you'll easily observe the issue (although you can see the issue all over the globe as you zoom in and out). A great place to zoom in is around San Francisco.

Here's the class:

public class BN_Mapbox_Black extends BasicMercatorTiledImageLayer {
    static String tempToken = "pk.eyJ1IjoiYWlycG9ydG1hc3RlciIsImEiOiJjand4azM3M2MwMXMzNGFxcjZwMXR5MjJqIn0.rtQ97ju2aDx_TZ_1rl-o7g";
    static String strName = "BN_Mapbox_Black";

    public BN_Mapbox_Black() {
        super(strName, "Earth/" + strName, 20, 256, false, ".png", new URLBuilder());
    }

    private static class URLBuilder extends MercatorTileUrlBuilder {

        @Override
        protected URL getMercatorURL(int x, int y, int z) throws MalformedURLException {
            String key = "?access_token=" + tempToken;
            return new URL("https://api.mapbox.com/styles/v1/airportmaster/cjhjerqz72m0e2smwjmlzm5sm/tiles/256/" + z + "/" + x + "/" + y + key);
        }
    }

    @Override
    public String toString() {
        return strName;
    }
}

I you need it I can also provide the other classes we used (before WWJ-CE) to display these layers. As I mentioned we literally copied the MercatorTiledImageLayer in WWJ v2.1.0 created by tag and the only change we made was to make the variable "splitScale" public. We then created a copy of BasicMercatorTiledImageLayer that extends our copy of MercatorTiledImageLayer. And with WWJ-CE we had to add a copy of MercatorTextureTile from WWJ v2.1.0. Then the map layer classes look like this:

public class BN_Mapbox_BlackX extends BN_BasicMercatorTiledImageLayer {
    static String tempToken = "pk.eyJ1IjoiYWlycG9ydG1hc3RlciIsImEiOiJjand4azM3M2MwMXMzNGFxcjZwMXR5MjJqIn0.rtQ97ju2aDx_TZ_1rl-o7g";
    static String strName = "BN_Mapbox_Black";
    
    public BN_Mapbox_BlackX() {
        super(makeLevels());
        this.splitScale = 1.4;
        
        this.setPickEnabled(false);
        this.setMinActiveAltitude(500.0);
    }

    private static LevelSet makeLevels() {
        AVList params = new AVListImpl();

        int intTileWidth = 256;
        int intTileHeight = 256;
        int intNumLevels = 20;

        params.setValue(AVKey.TILE_WIDTH, intTileWidth);
        params.setValue(AVKey.TILE_HEIGHT, intTileHeight);
        params.setValue(AVKey.DATA_CACHE_NAME, "Earth/" + strName);
        params.setValue(AVKey.SERVICE, "https://api.mapbox.com/styles/v1/airportmaster/cjhjerqz72m0e2smwjmlzm5sm/tiles/256/");
        params.setValue(AVKey.DATASET_NAME, strName);
        params.setValue(AVKey.FORMAT_SUFFIX, ".png");
        params.setValue(AVKey.NUM_LEVELS, intNumLevels);
        params.setValue(AVKey.NUM_EMPTY_LEVELS, 0);
        params.setValue(AVKey.LEVEL_ZERO_TILE_DELTA, new LatLon(Angle.fromDegrees(22.5d), Angle.fromDegrees(45d)));
        params.setValue(AVKey.SECTOR, new MercatorSector(-1.0, 1.0, Angle.NEG180, Angle.POS180));
        params.setValue(AVKey.TILE_URL_BUILDER, new URLBuilder());

        return new LevelSet(params);
    }

    private static class URLBuilder implements TileUrlBuilder {

        @Override
        public URL getURL(Tile tile, String imageFormat)
                throws MalformedURLException {
            // z/x/y
            return new URL(tile.getLevel().getService()
                    + (tile.getLevelNumber() + 3) + "/" + tile.getColumn() + "/"
                    + ((1 << (tile.getLevelNumber()) + 3) - 1 - tile.getRow())
                    + "?access_token=" + tempToken);
        }
        
    }

    @Override
    public String toString() {
        return strName;
    }

}

Using either of the classes/methods above produces the same results.

Let me know if you need me to post the Mercator classes I described above. And thanks for any help with this!

[EMaksymenko]

Have you checked an application log during browsing that regions? What result message of image retriever do you see?

[robmilton]

@Sufaev is there a log that prints the image retriever messages that I can enable/find? Or are you talking about me adding print statements somewhere?

[EMaksymenko]

When you run application in Idea or Eclipse it shows error messages in console among other messages if image was not retrieved.

Are you able to review this log when viewing the bug area? It will give an answer if retrieval failed or it even not request required tiles.

I plan to do it myself to analyse the issue, but i have no time nearest several days to do it immediatelly.

@wcmatthysen are you able to debug? I can do it probably at the end of next week.

[gbburkhardt]

To see more retriever messages, run your application with JVM arg "-Djava.util.logging.config.file=logging.properties", and set ".level=FINEST" in 'logging.properties'.

For Netbeans 8.2, using the Gradle run task, I needed to set "jvm-line-args=-Djava.util.logging.config.file=logging.properties" in Project Properties|Custom Variables.

You can limit the FINEST logging by adding a line to 'logging.properties' that applies only to Worldwind:
"gov.nasa.worldwind.level=FINEST".

[robmilton]

Thanks for the instructions. So far I haven't been able to turn on logging. I'm using NetBeans 11, and the project is an Ant project.

I added VM option -Djava.util.logging.config.file=logging.properties through the Project Properties dialog. I also tried -Djava.util.logging.config.file="logging.properties".

My project did not have a logging.properties file, so I created one under the nbproject folder. I added .level=FINEST. I also tried level=FINEST.

Any additional tips on how to get the logger working? I'll keep experimenting with it.

[gbburkhardt]

There is a 'logging.properties' file that's part of the repository, and my references to making changes are to it.
I didn't find a place to set the JVM arguments in Netbeans 11. My opinion at the moment is that Netbeans 11 isn't ready for prime time.

It did work for me using Netbeans 8.2, the Gradle project, and setting a custom variable:

So when running the WorldWindow sample application, one sees in the output pane:
Executing: gradle :run
Arguments: [-PmainClass=gov.nasa.worldwindx.applications.worldwindow.WorldWindow, -PcmdLineArgs=, -PjvmLineArgs=-Djava.util.logging.config.file=logging.properties, -c, C:\Users\glenn\Documents\repos\WorldWindJava\settings.gradle]

and then retriever messages:

Jun 16, 2019 1:55:38 PM sun.net.www.protocol.http.HttpURLConnection writeRequests
FINE: sun.net.www.MessageHeader@556c39be5 pairs: {GET /wms?EXCEPTIONS=application/vnd.ogc.se_xml&REQUEST=GetCapabilities&SERVICE=WMS&VERSION=1.3.0 HTTP/1.1: null}{User-Agent: Java/1.8.0_202}{Host: worldwind25.arc.nasa.gov}{Accept: text/html, image/gif, image/jpeg, *; q=.2, /; q=.2}{Connection: keep-alive}
Jun 16, 2019 1:55:38 PM sun.net.www.protocol.http.HttpURLConnection writeRequests
FINE: sun.net.www.MessageHeader@284a07e65 pairs: {GET /elev?EXCEPTIONS=application/vnd.ogc.se_xml&REQUEST=GetCapabilities&SERVICE=WMS&VERSION=1.3.0 HTTP/1.1: null}{User-Agent: Java/1.8.0_202}{Host: worldwind26.arc.nasa.gov}{Accept: text/html, image/gif, image/jpeg, *; q=.2, /; q=.2}{Connection: keep-alive}

[robmilton]

Ok. I was able to get logging working (my path to the logging.properties file was incorrect).

Side Note
Not sure if this should be taking place, but without any interaction whatsoever every 10 seconds the following messages get printed to the console:

Jun 16, 2019 3:46:02 PM sun.net.www.protocol.http.HttpURLConnection writeRequests
FINE: sun.net.www.MessageHeader@45b41a395 pairs: {GET / HTTP/1.1: null}{User-Agent: Java/1.8.0_201}{Host: worldwind.arc.nasa.gov}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
Jun 16, 2019 3:46:02 PM sun.net.www.protocol.http.HttpURLConnection getInputStream0
FINE: sun.net.www.MessageHeader@418b865714 pairs: {null: HTTP/1.1 200 OK}{Date: Sun, 16 Jun 2019 20:46:01 GMT}{Server: Apache}{Strict-Transport-Security: max-age=31536000;}{X-Frame-Options: sameorigin}{Last-Modified: Wed, 22 May 2019 19:30:22 GMT}{ETag: "280f-5897eff14ef80"}{Accept-Ranges: bytes}{Content-Length: 10255}{Vary: Accept-Encoding}{Access-Control-Allow-Origin: *}{Keep-Alive: timeout=5, max=100}{Connection: Keep-Alive}{Content-Type: text/html}

Map Problem
When I enable the layer that is having problems and begin to zoom in I get a bunch of messages like this:

Jun 16, 2019 3:48:15 PM sun.net.www.protocol.http.HttpURLConnection writeRequests
FINE: sun.net.www.MessageHeader@2e85ffe15 pairs: {GET /styles/v1/airportmaster/<mapId>/tiles/256/7/23/46?access_token=<token> HTTP/1.1: null}{User-Agent: Java/1.8.0_201}{Host: api.mapbox.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
Jun 16, 2019 3:48:15 PM sun.net.www.protocol.http.HttpURLConnection getInputStream0
FINE: sun.net.www.MessageHeader@445e706f16 pairs: {null: HTTP/1.1 200 OK}{Content-Type: image/png}{Content-Length: 1606}{Connection: keep-alive}{Date: Sun, 16 Jun 2019 14:41:58 GMT}{X-Powered-By: Express}{Access-Control-Allow-Origin: *}{Access-Control-Allow-Methods: GET}{Cache-Control: max-age=43200,s-maxage=43200}{X-Rate-Limit-Limit: 2000}{X-Rate-Limit-Interval: 60}{ETag: W/"646-odBij72xGoJMLxxVxDHxshKsUNk"}{Age: 21976}{X-Cache: Hit from cloudfront}{Via: 1.1 db929e55bb40b085896b8e336fba2ab7.cloudfront.net (CloudFront)}{X-Amz-Cf-Id: ynP3zjp2r5wusTTcppPXu2z8-ucQ17scB71daOabf4jfQ70XcSQsFg==}
Jun 16, 2019 3:48:15 PM gov.nasa.worldwind.retrieve.HTTPRetriever doRead
FINE: Response code 200, Content length 1,606, Content type image/png, retrieving https://api.mapbox.com/styles/v1/airportmaster/<mapId>/tiles/256/7/23/46?access_token=<token>

As I zoom in to an area where I'm getting blank or missing tiles i see the following messages in the log:

Jun 16, 2019 4:02:34 PM sun.net.www.protocol.http.HttpURLConnection writeRequests
FINE: sun.net.www.MessageHeader@559fc04c5 pairs: {GET /styles/v1/airportmaster/<mapId>/tiles/256/10/158/396?access_token=<token> HTTP/1.1: null}{User-Agent: Java/1.8.0_201}{Host: api.mapbox.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
Jun 16, 2019 4:02:34 PM sun.net.www.protocol.http.HttpURLConnection getInputStream0
FINE: sun.net.www.MessageHeader@672d4c0116 pairs: {null: HTTP/1.1 200 OK}{Content-Type: image/png}{Content-Length: 103}{Connection: keep-alive}{Date: Sun, 16 Jun 2019 21:02:33 GMT}{X-Powered-By: Express}{Access-Control-Allow-Origin: *}{Access-Control-Allow-Methods: GET}{Cache-Control: max-age=43200,s-maxage=43200}{X-Rate-Limit-Limit: 2000}{X-Rate-Limit-Interval: 60}{ETag: W/"67-1JzMAbxe/4kPVm67cU7inUOVtfg"}{X-Cache: Miss from cloudfront}{Via: 1.1 fc96a34d255c927c49d016d510d83ef9.cloudfront.net (CloudFront)}{X-Amz-Cf-Pop: IAH50-C2}{X-Amz-Cf-Id: fnTPJhNAkw-L4jNAD1ST-4iyE5x42FGLH2Cltt18BzXArQ9mCLav3g==}
Jun 16, 2019 4:02:34 PM sun.net.www.protocol.http.HttpURLConnection getInputStream0
FINE: sun.net.www.MessageHeader@5a52ee2a16 pairs: {null: HTTP/1.1 200 OK}{Content-Type: image/png}{Content-Length: 103}{Connection: keep-alive}{Date: Sun, 16 Jun 2019 21:02:33 GMT}{X-Powered-By: Express}{Access-Control-Allow-Origin: *}{Access-Control-Allow-Methods: GET}{Cache-Control: max-age=43200,s-maxage=43200}{X-Rate-Limit-Limit: 2000}{X-Rate-Limit-Interval: 60}{ETag: W/"67-1JzMAbxe/4kPVm67cU7inUOVtfg"}{X-Cache: Miss from cloudfront}{Via: 1.1 93818a791c595a1ca7a7d31c28fbcb86.cloudfront.net (CloudFront)}{X-Amz-Cf-Pop: IAH50-C2}{X-Amz-Cf-Id: 385x775IhIaCFPus0cmCUXqcfDZQQXDVGMvTuyPsZqaO9_5onXbS3A==}
Jun 16, 2019 4:02:34 PM gov.nasa.worldwind.retrieve.HTTPRetriever doRead
FINE: Response code 200, Content length 103, Content type image/png, retrieving https://api.mapbox.com/styles/v1/airportmaster/<mapId>/tiles/256/10/156/393?access_token=<token>

I did a little searching about the {X-Cache: Miss from cloudfront} which seems to relate to CloudFront caching. But I don't know that this makes sense as a real issue since our other (raster) Mapbox maps load without issue.

Then after a short while I start getting messages like this:

Jun 16, 2019 4:10:00 PM gov.nasa.worldwind.retrieve.BasicRetrievalService$RetrievalExecutor beforeExecute
FINER: Cancelling request too long on the retrieval queue for https://api.mapbox.com/styles/v1/airportmaster/<mapId>/tiles/256/7/18/48?access_token=<token>
Jun 16, 2019 4:10:00 PM gov.nasa.worldwind.retrieve.BasicRetrievalService$RetrievalExecutor afterExecute
FINE: Retrieval of https://api.mapbox.com/styles/v1/airportmaster/<mapId>/tiles/256/7/21/47?access_token=<token> was cancelled

I'll continue to look through the log to see if I come across any other messages that add to the story. I appreciate any thoughts/advice/help.

[EMaksymenko]

https://api.mapbox.com/styles/v1/airportmaster//tiles/256/7/18/48?access_token=

Are you sure that you correctly pass token and map id in code?

It looks like you view map from local cache.

[robmilton]

Yes, I'm sure about the token and mapId. I removed the actual mapId and token from each of the log entries I posted and replace them with <mapId> and <token> (since I was running the application using our actual map and token... not the temp map and token I included in the post above).

Again, if I switch back to WWJ v2.1.0 all of these layers load and display correctly. And the layers are identical (one of them is the " BN_Mapbox_BlackX extends BN_BasicMercatorTiledImageLayer" posted earlier in this thread). The only change I had to make in our WWJ-CE-based project was to bring back a copy of "MercatorTextureTile" from v2.1.0 that returns MercatorTextureTile arrays from the "createSubTiles" function instead of TextureTile arrays.

It looks like you view map from local cache.

I have intended to delete my map cache every time I run the application during this testing, but I may have forgotten to do so before one of these runs that I grabbed logger data from. However, I have deleted the map cache many times.

I'm confident these layers that retrieve Mapbox maps haven't changed. And if I use WWJ v2.1.0 they all load/display correctly. So everything point to some change between WWJ v2.1.0 and WWJ-CE that is causing this behavior.

[robmilton]

So far I haven't seen anything else in the log that is helpful. I don't understand why the other Mapbox layers still load/display without problem and why ALL the Mapbox layers load/display without problems if I go back to WWJ v2.1.0.

[wcmatthysen]

@robmilton, we can roll this change back and keep it on a separate branch until it is stable. @Sufaev, I don't know if that is ok for now?

[robmilton]

@wcmatthysen, I'm not 100% sure that the MercatorTiledImageLayer change is the issue. I'm new to Git, but I believe I reset my clone of WWJ-CE to just before that change was merged. But when I rebuilt WW and placed the new jar in my project I still had the same problems. So possibly an earlier change in WWJ-CE is the actual cause?

I don't know if I should continue to reset my copy of WWJ-CE at various points further back to try to discover which merge coincides with the problem?

EDIT
I just looked at resetting to an earlier commit when I realized that I had reset to "563f37f - Merge pull request #35 from wcmatthysen /macos-pixel-scale-fix" which was the commit just before "a840f19 - Merge pull request #41 from Sufaev /feature/refactor-mercator-layer". However, as I look down the list of commits I see earlier commits that have to do with the Mercator layer.

So it does seem possible that the MercatorTiledImageLayer change is the issue. I will try to reset to a point prior to any commits related to the Mercator layer change. Although I'm not entirely sure how to identify a good point to try. Any suggestions?