feat: Hookless API by jakmro · Pull Request #82 · software-mansion/react-native-executorch

jakmro · 2025-01-14T15:51:08Z

Description

This PR introduces hookless API and restructures the ./src directory.

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update (improves or adds clarity to existing documentation)

Tested on

iOS
Android

Checklist

I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have updated the documentation accordingly
My changes generate no new warnings

NorbertKlockiewicz · 2025-01-22T08:23:44Z

Why are we changing it, UseClassification as a name for a type doesn't seem too clear for me

Changes were made to avoid name conflicts with the new classes introduced in this PR.

NorbertKlockiewicz · 2025-01-22T08:23:57Z

  score: number;
 }

-interface ObjectDetectionModule {


same as above

NorbertKlockiewicz · 2025-01-22T08:24:06Z

 <summary>Type definitions</summary>

 ```typescript
-interface StyleTransferModule {


same as above

NorbertKlockiewicz · 2025-01-22T08:27:21Z

+import { ETInput, getTypeIdentifier } from '../../types/common';
+
+export class ExecutorchModule {
+  static async load(modelSource: string) {


can't we use load function from BaseModule here?

@jakmro To do this let's craete BaseModule in src/modules/BaseModule.ts liek so:

export class BaseModule { static module: | _StyleTransferModule | _ObjectDetectionModule | _ClassificationModule | _ETModule; static async load(modelSource: string | number) { if (!modelSource) return; let path = typeof modelSource === 'number' ? Image.resolveAssetSource(modelSource).uri : modelSource; try { await this.module.loadModule(path as string); } catch (e) { throw new Error(getError(e)); } } static async forward(..._: any[]): Promise<any> { throw new Error("Method 'myMethod()' must be implemented."); } }

and then just extend both ExecutorchModule and BaseCVModule.
The change in let path is necessary for typescript to correctly infer path type as string.

Also I think we should use type ResourceSource like in LLMModule

mkopcins · 2025-01-22T09:17:46Z

@@ -72,44 +72,44 @@ const StyleTransfer = StyleTransferSpec
    );



I think we should keep it as it was if we ever want to create separate instances per call instead of treating each module as singleton.

mkopcins · 2025-01-22T10:48:56Z

+import { Image } from 'react-native';
+import { getError } from '../../Error';
+
+export class BaseModule {


Suggested change

export class BaseModule {

export class BaseCVModule {

mkopcins · 2025-01-22T10:52:06Z

+    }
+
+    try {
+      await _ETModule.loadModule(path);


Suggested change

await _ETModule.loadModule(path);

await this.module.loadModule(path);

mkopcins · 2025-01-22T10:52:14Z

+
+    try {
+      const numberArray = [...input] as number[];
+      return await _ETModule.forward(numberArray, shape, inputType);


Suggested change

return await _ETModule.forward(numberArray, shape, inputType);

return await this.module.forward(numberArray, shape, inputType);

mkopcins · 2025-01-22T12:01:17Z

-}: Props): StyleTransferModule => {
-  const [module, _] = useState(() => new _StyleTransferModule());
+export const useStyleTransfer = ({ modelSource }: Props): UseStyleTransfer => {
+  const [module, _] = useState(() => _StyleTransferModule);


mkopcins · 2025-01-22T12:11:56Z

 }

-interface ClassificationModule {
+interface UseClassification {


let's remove this naming completely and move this as inline type below. This would solve naming issues as well as give end user more clarity of what the type actually is instead of it being a cryptic UseClassification

mkopcins · 2025-01-22T12:12:06Z

 }

-interface ObjectDetectionModule {
+interface UseObjectDetection {


mkopcins · 2025-01-22T12:12:12Z

 }

-interface StyleTransferModule {
+interface UseStyleTransfer {


mkopcins · 2025-01-22T12:18:34Z

@@ -72,44 +72,44 @@ const StyleTransfer = StyleTransferSpec
    );

 class _ObjectDetectionModule {


Let's propagate return types from turbo modules specs like so:

Suggested change

class _ObjectDetectionModule {

class _ObjectDetectionModule {

static async forward(input: string) : ReturnType<ObjectDetectionInterface["forward"]> {

import { Spec as ObjectDetectionInterface } from './NativeObjectDetection';

Do this for all methods for other classes in this file too.

mkopcins · 2025-01-23T12:45:16Z

+      let modelUrl = modelSource;
+      let tokenizerUrl = tokenizerSource;
+
+      if (typeof modelSource === 'number') {
+        modelUrl = Image.resolveAssetSource(modelSource).uri;
+      }
+
+      if (typeof tokenizerSource === 'number') {
+        tokenizerUrl = Image.resolveAssetSource(tokenizerSource).uri;
+      }
+
+      await LLM.loadLLM(
+        modelUrl as string,
+        tokenizerUrl as string,


Suggested change

let modelUrl = modelSource;

let tokenizerUrl = tokenizerSource;

if (typeof modelSource === 'number') {

modelUrl = Image.resolveAssetSource(modelSource).uri;

}

if (typeof tokenizerSource === 'number') {

tokenizerUrl = Image.resolveAssetSource(tokenizerSource).uri;

}

await LLM.loadLLM(

modelUrl as string,

tokenizerUrl as string,

let modelUrl = typeof modelSource === 'number' ? Image.resolveAssetSource(modelSource).uri : modelSource;

let tokenizerUrl = typeof tokenizerSource === 'number' ? Image.resolveAssetSource(tokenizerSource).uri : tokenizerSource;

await LLM.loadLLM(

modelUrl,

tokenizerUrl,

This allows typescript to infer types correctly

## Description This PR introduces hookless API and restructures the `./src` directory. ### Type of change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation update (improves or adds clarity to existing documentation) ### Tested on - [x] iOS - [x] Android ### Checklist - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation accordingly - [x] My changes generate no new warnings

jakmro added 9 commits January 14, 2025 14:32

Move computer vision hooks to separate directory

851b396

Move hooks to appropriate directories

2e71bc4

Add executorch bindings to hookless api

656a447

Move getTypeIdentifier to common types file

1f95ad6

Add LLM hookless api

390c5d1

Rename directory and add exports

70c4d99

Fix lint warnings

2c531ca

Make classes static

41d6b42

Fix llm hookles api

031f399

jakmro requested review from NorbertKlockiewicz and mkopcins January 20, 2025 09:29

jakmro marked this pull request as ready for review January 20, 2025 09:30

jakmro changed the title ~~feat: Hookless API~~ draft: feat: Hookless API Jan 21, 2025

jakmro changed the title ~~draft: feat: Hookless API~~ feat: Hookless API Jan 21, 2025

jakmro marked this pull request as draft January 21, 2025 08:40

jakmro added 2 commits January 21, 2025 11:29

Rename TS interfaces

46e9957

Update docs

8a12781

jakmro marked this pull request as ready for review January 21, 2025 11:25

NorbertKlockiewicz reviewed Jan 22, 2025

View reviewed changes

mkopcins requested changes Jan 22, 2025

View reviewed changes

jakmro added 3 commits January 23, 2025 09:56

Add suggested changes

8703a56

Fix typing issue

6ecffa5

Update docs

09995c5

jakmro requested a review from mkopcins January 23, 2025 10:07

mkopcins approved these changes Jan 23, 2025

View reviewed changes

Add suggested changes v2

6f51f6c

jakmro requested a review from NorbertKlockiewicz January 27, 2025 08:28

mkopcins approved these changes Jan 27, 2025

View reviewed changes

jakmro merged commit cbe2b55 into main Jan 27, 2025

jakmro deleted the @jakmro/hookless-api branch January 27, 2025 09:45

		@@ -72,44 +72,44 @@ const StyleTransfer = StyleTransferSpec
		);

	await _ETModule.loadModule(path);
	await this.module.loadModule(path);

	return await _ETModule.forward(numberArray, shape, inputType);
	return await this.module.forward(numberArray, shape, inputType);

		@@ -72,44 +72,44 @@ const StyleTransfer = StyleTransferSpec
		);

		class _ObjectDetectionModule {

	class _ObjectDetectionModule {
	class _ObjectDetectionModule {
	static async forward(input: string) : ReturnType<ObjectDetectionInterface["forward"]> {

Conversation

jakmro commented Jan 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Tested on

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jakmro commented Jan 14, 2025 •

edited

Loading