Skip to content

Support caching remote assets#148

Merged
crazytonyli merged 15 commits intotrunkfrom
fetched-assets-editor-cache
Jun 30, 2025
Merged

Support caching remote assets#148
crazytonyli merged 15 commits intotrunkfrom
fetched-assets-editor-cache

Conversation

@crazytonyli
Copy link
Contributor

@crazytonyli crazytonyli commented Jun 23, 2025

What?

This is an alternative solution to #107.

Happy path

Here are the steps to see the "happy path" scenario on the Demo iOS app:

  1. Create a self-hosted site.
  2. Install Jetpack and fully connect to WordPress.com. This is necessary to verify jetpack plugins block types are loaded into the editor.
  3. Update the EditorConfiguration.template to use your test site.
  4. Run make build.
  5. Make sure the GUTENBERG_EDITOR_REMOTE_URL launch argument is unchecked. Launch the Demo iOS app.
  6. You should see your site in the "Remote Editors" section of the Demo app.
  7. (Optional) Tap the Refresh button to pre-fetch the editor assets.
  8. Tap your test site to launch the editor.
  9. Open the block inserter and verify the Jetpack blocks (i.e. AI assistant)
Example

Simulator Screenshot - iPhone 16 - 2025-06-23 at 22 21 00

Known issues

When enabling GUTENBERG_EDITOR_REMOTE_URL, the fetched assets can be used by the editor, and the editor can be launched. However, the Jetpack blocks (see the happy path scenario) are not available in the block inserter.

Why?

How?

Here is how the caching mechanism works:

  1. The GBK editor invokes the fetchEditorAssets JavaScript function to retrieve the manifest from the wpcom/v2/editor-assets endpoint, and eventually renders the returned scripts and styles HTML on screen.
  2. The fetchEditorAssets JavaScript function internally calls the iOS native code EditorAssetsProvider to get the manifest content. However, the manifest content is not verbatim from the wpcom/v2/editor-assets endpoint.
  3. EditorAssetsProvider calls EditorAssetsMainifest.renderForEditor to swap the HTTP links in the script and link tags with gbk-cache-http(s):// links.
  4. After the modified manifest content is returned to the GBK editor, the script and link tags are rendered on screen, and the web view attempts to load those gbk-cache-http(s):// links.
  5. The CachedAssetSchemeHandler is triggered to load content for the gbk-cache-http(s):// links. This is where we can fetch and cache the assets from the original HTTP URL.

Difference with #107

The major difference is that #107 caches the editor assets as a whole. But this PR caches the asset individually.

In #107, it stores the hash value (which is proposed in wordpress-mobile/block-editor-assets-endpoint#1) returned by the editor-assets API. When a new download operation is triggered, it gets the hash value from the editor-assets API response. If the value changes, which happens if assets are added or removed, it re-downloads all the assets again, even if some have already been downloaded. Since the editor assets are treated as a complete bundle, the GBK editor can only be loaded after all assets are downloaded.

This PR works with the existing wpcom/v2/editor-assets API, and does not require the hash value. There is an API (EditorAssetsLibrary.fetchAssets) to start downloading the editor assets before launching the GBK editor. But you don't have to wait for the caching to complete; you can launch the editor anytime, and the cache can continue to get to work when users work on their posts. That's because this PR caches assets individually via the gbk-cache-http(s) links (see the "How" section above). The "identifier" of the asset is its URL (the hash used in #107 also uses asset URLs as cache identifiers). A new cache file is created for each asset URL. When new assets are included in the editor-assets API response, they will be cached either by a call to EditorAssetsLibrary.fetchAssets, or when the new assets are rendered in the GBK editor.

Testing Instructions

Verify that the solution works with sites that are

  • located under a subdirectory: https://example.com/blog
  • using a non-English locale for the whole site and/or the logged-in user.
  • The editor functions the same with or without the GUTENBERG_EDITOR_REMOTE_URL launch argument.

Accessibility Testing Instructions

Screenshots or screencast

@crazytonyli crazytonyli requested a review from dcalhoun June 23, 2025 10:37
Copy link
Member

@dcalhoun dcalhoun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. The "happy path" succeed for me. 🎉

When enabling GUTENBERG_EDITOR_REMOTE_URL, the fetched assets can be used by the editor, and the editor can be launched. However, the Jetpack blocks (see the happy path scenario) are not available in the block inserter.

This known issue no longer occurs for me. I am able to utilize Jetpack blocks while using the remote editor dev server.

@crazytonyli crazytonyli force-pushed the fetched-assets-editor-cache branch from 021b8cb to d15254a Compare June 26, 2025 02:06
@crazytonyli crazytonyli requested a review from dcalhoun June 26, 2025 03:24
@crazytonyli crazytonyli marked this pull request as ready for review June 26, 2025 03:24
dcalhoun and others added 3 commits June 26, 2025 14:49
Utilize JS fetch until we implement fetching and caching editor assets
for Android.
WP-iOS relies upon an exact version of 2.7.5. This caused a conflict:

```
Failed to resolve dependencies Dependencies could not be resolved because root depends on 'swiftsoup' 2.7.5 and 'gutenbergkit' depends on 'swiftsoup' 2.8.8..<3.0.0.
```
@crazytonyli
Copy link
Contributor Author

crazytonyli commented Jun 27, 2025

The font on the web editor and GBK remote editor is not the same. ⬇️

Screenshot 2025-06-27 at 3 37 23 PM

The remote edtiro in the trunk branch also uses a different font. ⬇️

Screenshot 2025-06-27 at 3 47 16 PM

@dcalhoun I guess this is a known issue?

@dcalhoun
Copy link
Member

The font on the web editor and GBK remote editor is not the same. ⬇️
[...]
@dcalhoun I guess this is a known issue?

Yes, #81 tracks loading fonts.

@dcalhoun dcalhoun mentioned this pull request Jun 27, 2025
2 tasks
Copy link
Member

@dcalhoun dcalhoun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works great from my testing. Thank you for helping design and implement this!

@dcalhoun
Copy link
Member

  • located under a subdirectory: https://example.com/blog
  • using a non-English locale for the whole site and/or the logged-in user.

I'm unsure why these were originally listed in #107 or they might impact this functionality. Presumably any impact would be present in the existing, non-caching fetching mechanism. It likely safe to merge this. WDYT?

At the moment, the l10n is handled in the app, the site/user setting has no impact on GBK. We plan to expand that l10n to include third-party strings later.

@crazytonyli
Copy link
Contributor Author

I'm unsure why these were originally listed in #107

I added those while reviewing the PR 😄.

I tested with a "subdirectory site", and the editor did not load. I don't think it's related to this PR, though. I have created a Linear issue under the "GutenbergKit supports self-hosted sites" project to track it.

@crazytonyli crazytonyli merged commit 6ce39be into trunk Jun 30, 2025
11 checks passed
@crazytonyli crazytonyli deleted the fetched-assets-editor-cache branch June 30, 2025 00:37
oguzkocer added a commit that referenced this pull request Jul 8, 2025
Implements the Android counterpart to iOS PR #148 for caching remote editor assets.
Unlike iOS, Android doesn't require URL scheme modification - it uses the existing
GutenbergRequestInterceptor interface to intercept and cache JS/CSS assets directly.

Key changes:
- AssetCacheManager: Handles cache storage with 7-day expiration
- CachedAssetRequestInterceptor: Intercepts requests for cacheable assets
- EditorConfiguration: Adds enableAssetCaching and cachedAssetHosts options
- GutenbergView: Integrates caching when enabled, with proper cleanup

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
oguzkocer added a commit that referenced this pull request Jul 8, 2025
Implements the Android counterpart to iOS PR #148 for caching remote editor assets.
Unlike iOS, Android doesn't require URL scheme modification - it uses the existing
GutenbergRequestInterceptor interface to intercept and cache JS/CSS assets directly.

Key changes:
- AssetCacheManager: Handles cache storage with 7-day expiration
- CachedAssetRequestInterceptor: Intercepts requests for cacheable assets
- EditorConfiguration: Adds enableAssetCaching and cachedAssetHosts options
- GutenbergView: Integrates caching when enabled, with proper cleanup

🤖 Generated with [Claude Code](https://claude.ai/code)
oguzkocer added a commit that referenced this pull request Jul 8, 2025
This commit adds asset caching support for Android that mirrors the iOS implementation from PR #148, while leveraging Android's native capabilities to avoid URL scheme modification.

- **EditorAssetsLibrary**: Manages fetching manifest from `wpcom/v2/editor-assets` endpoint and caching individual JS/CSS assets
- **EditorAssetsManifest**: Parses HTML content from manifest using JSoup to extract asset URLs
- **EditorAssetsProvider**: JavaScript interface that provides manifest to WebView via `loadFetchedEditorAssets`
- **CachedAssetRequestInterceptor**: Intercepts and caches asset requests using the existing `GutenbergRequestInterceptor` interface

- Added `editorAssetsEndpoint` property to `EditorConfiguration`
- Added `enableAssetCaching` flag to enable/disable caching
- Added `cachedAssetHosts` to specify which hosts to cache from

- Updated `GutenbergView` to set up caching when enabled
- Added `warmup()` method for preloading assets
- Integrated JavaScript bridge support for both iOS and Android in `bridge.js`
- Added JSoup dependency for HTML parsing

- No URL scheme modification needed - Android intercepts requests directly
- Uses existing `GutenbergRequestInterceptor` infrastructure
- Simpler implementation while maintaining feature parity

- 7-day cache expiration (matching iOS)
- Cache directory structure matches iOS pattern
- Only caches successful responses (2xx status codes)
- Supports .js, .css, and .js.map files

🤖 Generated with [Claude Code](https://claude.ai/code)
oguzkocer added a commit that referenced this pull request Jul 9, 2025
This commit adds asset caching support for Android that mirrors the iOS implementation from PR #148, while leveraging Android's native capabilities to avoid URL scheme modification.

- **EditorAssetsLibrary**: Manages fetching manifest from `wpcom/v2/editor-assets` endpoint and caching individual JS/CSS assets
- **EditorAssetsManifest**: Parses HTML content from manifest using JSoup to extract asset URLs
- **EditorAssetsProvider**: JavaScript interface that provides manifest to WebView via `loadFetchedEditorAssets`
- **CachedAssetRequestInterceptor**: Intercepts and caches asset requests using the existing `GutenbergRequestInterceptor` interface

- Added `editorAssetsEndpoint` property to `EditorConfiguration`
- Added `enableAssetCaching` flag to enable/disable caching
- Added `cachedAssetHosts` to specify which hosts to cache from

- Updated `GutenbergView` to set up caching when enabled
- Added `warmup()` method for preloading assets
- Integrated JavaScript bridge support for both iOS and Android in `bridge.js`
- Added JSoup dependency for HTML parsing

- No URL scheme modification needed - Android intercepts requests directly
- Uses existing `GutenbergRequestInterceptor` infrastructure
- Simpler implementation while maintaining feature parity

- 7-day cache expiration (matching iOS)
- Cache directory structure matches iOS pattern
- Only caches successful responses (2xx status codes)
- Supports .js, .css, and .js.map files

🤖 Generated with [Claude Code](https://claude.ai/code)
oguzkocer added a commit that referenced this pull request Jul 18, 2025
* Implement Android asset caching to match iOS functionality

This commit adds asset caching support for Android that mirrors the iOS implementation from PR #148, while leveraging Android's native capabilities to avoid URL scheme modification.

- **EditorAssetsLibrary**: Manages fetching manifest from `wpcom/v2/editor-assets` endpoint and caching individual JS/CSS assets
- **EditorAssetsManifest**: Parses HTML content from manifest using JSoup to extract asset URLs
- **EditorAssetsProvider**: JavaScript interface that provides manifest to WebView via `loadFetchedEditorAssets`
- **CachedAssetRequestInterceptor**: Intercepts and caches asset requests using the existing `GutenbergRequestInterceptor` interface

- Added `editorAssetsEndpoint` property to `EditorConfiguration`
- Added `enableAssetCaching` flag to enable/disable caching
- Added `cachedAssetHosts` to specify which hosts to cache from

- Updated `GutenbergView` to set up caching when enabled
- Added `warmup()` method for preloading assets
- Integrated JavaScript bridge support for both iOS and Android in `bridge.js`
- Added JSoup dependency for HTML parsing

- No URL scheme modification needed - Android intercepts requests directly
- Uses existing `GutenbergRequestInterceptor` infrastructure
- Simpler implementation while maintaining feature parity

- 7-day cache expiration (matching iOS)
- Cache directory structure matches iOS pattern
- Only caches successful responses (2xx status codes)
- Supports .js, .css, and .js.map files

🤖 Generated with [Claude Code](https://claude.ai/code)

* Fix InputStream.readText() compilation error                                                              │

Replace direct readText() call on InputStream with bufferedReader().readText() to properly convert the stream to text content when fetching the manifest.

* Simplify Android asset caching by removing JavaScript bridge

Removes the JavaScript interface approach in favor of direct request interception, making the Android implementation cleaner while maintaining the same functionality.

- Remove EditorAssetsProvider JavaScript interface
- Update CachedAssetRequestInterceptor to intercept manifest endpoint requests
- Implement fetchEditorAssets() in bridge.js for Android using apiFetch
- Simplify remote-editor.js to use fetchEditorAssets() for both platforms
- Add header forwarding support for manifest requests

Now Android intercepts the /wpcom/v2/editor-assets API call directly instead of using a JavaScript bridge, leveraging Android's native request interception capabilities while maintaining feature parity with iOS.

* Remove unnecessary HTML parsing from Android asset caching

Simplifies the Android implementation by removing HTML parsing since Android doesn't need to modify URLs like iOS does. Android can intercept requests directly without URL scheme changes.

- Remove EditorAssetsManifest.kt class and JSoup dependency
- Simplify manifestContentForEditor() to return raw JSON without parsing
- Remove fetchAssets() method - assets are cached on-demand when requested
- Update warmup() to only preload manifest, not individual assets
- Remove Gson import and JSON parsing logic

- Significantly reduced complexity and dependencies
- More efficient - no unnecessary HTML parsing or JSON manipulation
- Leverages Android's native request interception capabilities
- Assets are cached lazily as they're actually requested by the editor

The Android implementation now simply:
1. Intercepts /wpcom/v2/editor-assets → returns raw JSON
2. Intercepts asset requests → caches and serves them
3. No URL modification or HTML parsing required

* Fix thread safety in asset caching by removing blocking operations

Replace runBlocking calls in WebView request interceptor with non-blocking approach:
- Only serve assets that are already cached
- Start background caching for future requests when asset not found
- Let WebView handle manifest and uncached requests normally
- Prevents ANR issues while maintaining caching functionality

* Fix resource leaks by ensuring HTTP connections are properly closed

Add try-finally blocks around HttpURLConnection usage to guarantee
connection.disconnect() is called in all scenarios including:
- Network timeouts and errors
- HTTP error responses
- IOException during data reading
- Any other exceptions

Prevents accumulation of unclosed connections that could exhaust
system resources, especially important on mobile devices.

* Update Kotlin to 2.0.21

* Replace GlobalScope with bounded CoroutineScope for better lifecycle management

- Replace deprecated GlobalScope.launch with proper CoroutineScope
- Use SupervisorJob + Dispatchers.IO for isolated error handling and IO optimization
- Add cancelWarmup() method for explicit control over background operations
- Add explicit coroutines dependency and replace star imports with specific imports
- Prevents memory leaks and provides proper lifecycle management for warmup operations

* Add time-based cache cleanup for versioned editor assets

Implements periodic cleanup of old cached assets to prevent unlimited storage growth. Since assets are versioned with URLs that change when updated, old versions become unused naturally. The cleanup removes files older than 7 days and provides logging for monitoring.

* Remove unused runBlocking import to prevent accidental blocking operations

* Add missing apiFetch import to bridge.js for Android fetchEditorAssets implementation

* Fix Android bridge to use editorAssetsEndpoint configuration

- Update fetchEditorAssets() to use editorAssetsEndpoint if configured
- Add editorAssetsEndpoint to GBKit JavaScript configuration
- Match the native Android implementation's fallback logic
- Ensure consistent URL construction between native and bridge code

* Use 'application/javascript' content type for cached '.css?inline' assets

* Add a TODO to make sure the editor asset caching is site specific

* Fix mime type detection for CSS inline assets and remove ordering dependency

Replaced order-dependent map lookup with explicit `when` expression to determine mime types. This eliminates the risk of incorrect mime type matching when `.css` would match before `.css?inline`.

- Remove `MIME_TYPES` constant in favor of `getMimeType()` function
- Add comment explaining why `.css?inline` files are served as `application/javascript`
- Reference `use-editor-styles.js` where Vite transforms CSS files with `?inline` parameter into JavaScript modules that export CSS as strings

* Add `.claude/settings.local.json` to `.gitignore`

* Fix js lint errors
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants