Skip to content

Allow tensors for graph constants. #830

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
huningxin merged 8 commits into from
Apr 22, 2025
Merged

Allow tensors for graph constants. #830

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
673b9cd
Allow tensors for graph constants.
bbernhar Mar 31, 2025
18fb614
Add [[constantTensor]] slot to check if destroyed before build.
bbernhar Mar 31, 2025
3b69c13
Change to constant MLTensor.
bbernhar Apr 15, 2025
4fccbdd
Blank line before dfn
fdwr Apr 18, 2025
261c93b
Comma before then
fdwr Apr 18, 2025
7bd2ae3
Added non-normative note for @huningxin.
bbernhar Apr 21, 2025
63089ff
fix nit by @reillyeon
bbernhar Apr 21, 2025
8b972ff
fix unit by @inexorabletash
bbernhar Apr 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 107 additions & 1 deletion index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -869,6 +869,8 @@ interface MLContext {
undefined dispatch(MLGraph graph, MLNamedTensors inputs, MLNamedTensors outputs);

Promise<MLTensor> createTensor(MLTensorDescriptor descriptor);
Promise<MLTensor> createConstantTensor(
MLOperandDescriptor descriptor, AllowSharedBufferSource inputData);

Promise<ArrayBuffer> readTensor(MLTensor tensor);
Promise<undefined> readTensor(MLTensor tensor, AllowSharedBufferSource outputData);
Expand Down Expand Up @@ -939,6 +941,7 @@ The <dfn>context type</dfn> is the type of the execution context that manages th
</summary>
1. If |namedTensors|'s [=map/size=] is not equal to |namedDescriptors|'s [=map/size=], then return false.
1. [=map/For each=] |name| → |tensor| of |namedTensors|:
1. If |tensor|.{{MLTensor/[[isConstant]]}} is true, then return false.
1. If |namedDescriptors|[|name|] does not [=map/exist=], then return false.
1. If |tensor|.{{MLTensor/[[descriptor]]}} is not [=MLOperandDescriptor/equal=] to |namedDescriptors|[|name|], then return false.
1. Return true.
Expand Down Expand Up @@ -981,6 +984,11 @@ Note: `dispatch()` itself provides no signal that graph execution has completed.
1. Return {{undefined}}.
</details>

<p class="note">
When a constant operand is created using a tensor, it is legal for that tensor to be destroyed after build completes.
Implementations are expected to ensure that the compiled graph remains valid and unaffected by such destruction.
</p>

#### Examples #### {#api-mlcontext-dispatch-examples}
<div class="example">
<details open>
Expand Down Expand Up @@ -1065,6 +1073,42 @@ Creates an {{MLTensor}} associated with this {{MLContext}}.
1. Return |promise|.
</details>

### {{MLContext/createConstantTensor()}} ### {#api-mlcontext-createconstanttensor}

Creates a constant {{MLTensor}} associated with this {{MLContext}}.

<div dfn-for="MLContext/createConstantTensor(descriptor, inputData)" dfn-type=argument>
**Arguments:**
- <dfn>descriptor</dfn>: an {{MLOperandDescriptor}}.
- <dfn>inputData</dfn>: an {{AllowSharedBufferSource}}. The buffer whose bytes will be written into the tensor.

**Returns:** {{Promise}}<{{MLTensor}}>.
</div>

<details open algorithm>
<summary>
The <dfn method for=MLContext>createConstantTensor(|descriptor|, |inputData|)</dfn> method steps are:
</summary>
1. Let |global| be [=this=]'s [=relevant global object=].
1. Let |realm| be [=this=]'s [=relevant realm=].
1. If [=this=] [=MLContext/is lost=], then return [=a new promise=] in |realm| [=rejected=] with an "{{InvalidStateError}}" {{DOMException}}.
1. If [=MLOperandDescriptor/checking dimensions=] given |descriptor| returns false, then return [=a new promise=] in |realm| [=rejected=] with a {{TypeError}}.
1. If [=validating buffer with descriptor=] given |inputData| and |descriptor| returns false, then return [=a new promise=] in |realm| [=rejected=] with a {{TypeError}}.
1. Let |bytes| be the result of [=getting a copy of the bytes held by the buffer source=] given |inputData|.
1. [=Assert=]: |bytes|'s [=byte sequence/length=] is equal to |descriptor|'s [=MLOperandDescriptor/byte length=].
1. Let |tensor| be the result of [=creating a constant MLTensor=] given [=this=], and |descriptor|.
1. Let |promise| be [=a new promise=] in |realm|.
1. Enqueue the following steps to [=this=].{{MLContext/[[timeline]]}}:
1. Run these steps, but [=/abort when=] [=this=] [=MLContext/is lost=]:
1. Create |tensor|.{{MLTensor/[[data]]}} given |descriptor|.
1. If that fails, then [=queue an ML task=] with |global| to [=reject=] |promise| with an "{{UnknownError}}" {{DOMException}}, and abort these steps.
1. Copy |bytes| to |tensor|.{{MLTensor/[[data]]}}.
1. If that fails, then [=queue an ML task=] with |global| to [=reject=] |promise| with an "{{UnknownError}}" {{DOMException}}, and abort these steps.
1. Otherwise, [=queue an ML task=] with |global| to [=resolve=] |promise| with |tensor|.
1. [=/If aborted=], then [=queue an ML task=] with |global| to [=reject=] |promise| with an "{{InvalidStateError}}" {{DOMException}}.
1. Return |promise|.
</details>

### {{MLContext/readTensor(tensor)}} ### {#api-mlcontext-readtensor}

Reads back the {{MLTensor/[[data]]}} of an {{MLTensor}} from the {{MLContext}}.{{MLContext/[[timeline]]}} to script.
Expand Down Expand Up @@ -1454,6 +1498,10 @@ typedef (bigint or unrestricted double) MLNumber;
: <dfn>\[[operator]]</dfn> of type [=operator=]
::
Reference to {{MLOperand}}'s corresponding [=operator=].

: <dfn>\[[constantTensor]]</dfn> of type {{MLTensor}}
::
The {{MLOperand}}'s tensor (only for constant operands).
</dl>
</div>

Expand Down Expand Up @@ -1553,6 +1601,7 @@ interface MLTensor {
readonly attribute FrozenArray<unsigned long> shape;
readonly attribute boolean readable;
readonly attribute boolean writable;
readonly attribute boolean constant;

undefined destroy();
};
Expand All @@ -1576,6 +1625,10 @@ interface MLTensor {
: <dfn>\[[data]]</dfn> of an [=implementation-defined=] type
::
The bytes backing the {{MLTensor}}. This data may only be accessed or modified from the {{MLTensor/[[context]]}}.{{MLContext/[[timeline]]}}.

: <dfn>\[[isConstant]]</dfn> of type {{boolean}}
::
Whether the {{MLTensor}} was created by [=create a constant MLTensor=].
</dl>
</div>

Expand All @@ -1591,6 +1644,8 @@ The <dfn attribute for=MLTensor>readable</dfn> [=getter steps=] are to return [=

The <dfn attribute for=MLTensor>writable</dfn> [=getter steps=] are to return [=this=].{{MLTensor/[[descriptor]]}}.{{MLTensorDescriptor/writable}}.

The <dfn attribute for=MLTensor>constant</dfn> [=getter steps=] are to return [=this=]'s {{MLTensor/[[isConstant]]}}.

### Creating an {{MLTensor}} ### {#api-mltensor-create}

An {{MLTensor}} is created by its associated {{MLContext}}.
Expand All @@ -1604,6 +1659,7 @@ An {{MLTensor}} is created by its associated {{MLContext}}.
1. Set |tensor|.{{MLTensor/[[context]]}} to |context|.
1. Set |tensor|.{{MLTensor/[[descriptor]]}} to |descriptor|.
1. Set |tensor|.{{MLTensor/[[isDestroyed]]}} to false.
1. Set |tensor|.{{MLTensor/[[isConstant]]}} to false.
1. Return |tensor|.
</details>

Expand All @@ -1627,6 +1683,28 @@ Releases the resources associated with the {{MLTensor}}. This method is idempote

Note: Since no further operations can be enqueued using this tensor, implementations can free any additional resource allocations associated with this tensor once all previously submitted operations using it are complete.

### Creating a constant {{MLTensor}} ### {#api-mlconstant-tensor-create}

A constant {{MLTensor}} is created by its associated {{MLContext}}.

<details open algorithm>
<summary>
To <dfn>create a constant MLTensor</dfn> given {{MLContext}} |context|, {{MLOperandDescriptor}} |inputDescriptor|, run the following steps:
</summary>
1. Let |realm| be |context|'s [=relevant realm=].
1. Let |tensor| be a new {{MLTensor}} in |realm|.
1. Set |tensor|.{{MLTensor/[[context]]}} to |context|.
1. Let |tensorDescriptor| be a new {{MLTensorDescriptor}}.
1. Set |tensorDescriptor|.{{MLTensorDescriptor/readable}} to false.
1. Set |tensorDescriptor|.{{MLTensorDescriptor/writable}} to false.
1. Set |tensorDescriptor|.{{MLOperandDescriptor/dataType}} to |inputDescriptor|.{{MLOperandDescriptor/dataType}}.
1. Set |tensorDescriptor|.{{MLOperandDescriptor/shape}} to |inputDescriptor|.{{MLOperandDescriptor/shape}}.
1. Set |tensor|.{{MLTensor/[[descriptor]]}} to |tensorDescriptor|.
1. Set |tensor|.{{MLTensor/[[isDestroyed]]}} to false.
1. Set |tensor|.{{MLTensor/[[isConstant]]}} to true.
1. Return |tensor|.
</details>

## {{MLGraphBuilder}} interface ## {#api-mlgraphbuilder}

The {{MLGraphBuilder}} interface defines a set of operations as identified by the [[#usecases]] that can be composed into a computational graph. It also represents the intermediate state of a graph building session.
Expand All @@ -1649,6 +1727,9 @@ interface MLGraphBuilder {
// Create a scalar operand from the specified number of the specified type.
MLOperand constant(MLOperandDataType type, MLNumber value);

// Create an operand from a specified constant tensor.
MLOperand constant(MLTensor tensor);

// Compile the graph up to the specified output operands asynchronously.
Promise<MLGraph> build(MLNamedOperands outputs);
};
Expand Down Expand Up @@ -1749,6 +1830,30 @@ Create a constant {{MLOperand}} of the specified data type and shape that contai
1. Return |operand|.
</details>

#### {{MLGraphBuilder/constant(tensor)}} #### {#api-mlgraphbuilder-constant-tensor}
Create a constant {{MLOperand}} of the specified data type and shape that contains the initialized data.

<div dfn-for="MLGraphBuilder/constant(tensor)" dfn-type=argument>
**Arguments:**
- <dfn>tensor</dfn>: an {{MLTensor}}. The constant tensor containing the initialized data.
**Returns:** an {{MLOperand}}. The constant output tensor.
</div>

<details open algorithm>
<summary>
The <dfn method for=MLGraphBuilder>constant(|tensor|)</dfn> method steps are:
</summary>
1. If |tensor|.{{MLTensor/[[context]]}} is not [=this=].{{MLGraphBuilder/[[context]]}}, then [=exception/throw=] a {{TypeError}}.
1. If |tensor|.{{MLTensor/[[isDestroyed]]}} is true, then [=exception/throw=] a {{TypeError}}.
1. If |tensor|.{{MLTensor/[[isConstant]]}} is false, then [=exception/throw=] a {{TypeError}}.
1. If [=this=] [=MLGraphBuilder/can not build=], then [=exception/throw=] an "{{InvalidStateError}}" {{DOMException}}.
1. *Make graph connections:*
1. Let |operand| be the result of [=creating an MLOperand=] given [=this=] and |tensor|.{{MLTensor/[[descriptor]]}}.
1. Set |operand|.{{MLOperand/[[constantTensor]]}} to |tensor|.
1. Add |operand| to [=this=]'s [=MLGraphBuilder/graph=]'s [=computational graph/constants=] with |tensor| as value.
1. Return |operand|.
</details>

#### {{MLGraphBuilder/constant(type, value)}} #### {#api-mlgraphbuilder-constant-type-value}
Create a scalar constant {{MLOperand}} of the specified value and data type.

Expand Down Expand Up @@ -1796,6 +1901,7 @@ Build a composed graph up to a given output operand into a computational graph a
1. If |name| is empty, then return [=a new promise=] in |realm| [=rejected=] with a {{TypeError}}.
1. If [=MLGraphBuilder/validating operand=] given [=this=] and |operand| returns false, then return [=a new promise=] in |realm| [=rejected=] with a {{TypeError}}.
1. If |operand| is in [=this=]'s [=MLGraphBuilder/graph=]'s [=computational graph/inputs=] or [=computational graph/constants=], then return [=a new promise=] in |realm| [=rejected=] with a {{TypeError}}.
1. If |operand|.{{MLOperand/[[constantTensor]]}} exists and |operand|.{{MLOperand/[[constantTensor]]}}.{{MLTensor/[[isDestroyed]]}} is true, then return [=a new promise=] in |realm| [=rejected=] with a {{TypeError}}.
1. Let |operands| be a new empty [=/set=].
1. Let |operators| be a new empty [=/set=].
1. Let |inputs| be a new empty [=/set=].
Expand All @@ -1817,7 +1923,7 @@ Build a composed graph up to a given output operand into a computational graph a
1. Set |graph|.{{MLGraph/[[outputDescriptors]]}}[|name|] to |operand|.{{MLOperand/[[descriptor]]}}.
1. Set [=this=].{{MLGraphBuilder/[[hasBuilt]]}} to true.
1. Let |promise| be [=a new promise=] in |realm|.
1. Run the following steps [=in parallel=]:
1. Enqueue the following steps to |graph|.{{MLGraph/[[context]]}}.{{MLContext/[[timeline]]}}:
1. Run these steps, but [=/abort when=] |graph|.{{MLGraph/[[context]]}} [=MLContext/is lost=]:
1. Let |graphImpl| be the result of converting [=this=]'s [=MLGraphBuilder/graph=] with |operands|, |operators|, |inputs|, and |outputs|'s [=map/values=] into an [=implementation-defined=] format which can be interpreted by the underlying platform.
1. If the previous step failed, then [=queue an ML task=] with |global| to [=reject=] |promise| with an "{{OperationError}}" {{DOMException}}, and abort these steps.
Expand Down