get_output_shape draft

[lambda7xx]

Description of changes:

• get_output_shape for each OP
• AggregateSpecAttrs
• AggregateAttrs
• MultiHeadAttentionAttrs
• BatchMatmulAttr
• BatchNormAttrs
• BroadcastAttrs
• CastAttrs
• CombineAttrs
• ConcatAttrs
• Conv2DAttrs
• DropoutAttrs
• ElementBinaryAttr
• ElementUnaryAttrs
• EmbeddingAttrs
• FlatAttrs
• GatherAttrs
• Group_byAttrs
• InputAttrs
• LayerNormAttrs
• LinearAttrs
• NoopAttrs
• Pool2DAttrs
• ReduceAttrs
• ReductionAttrs
• RepartitionAttrs
• ReplicateAttrs
• ReshapeAttrs
• ReverseAttrs
• SoftmaxAttrs
• SplitAttrs
• TopKAttrs
• TransposeAttrs

Related Issues:

Linked Issues:

• Issue #

Issues closed by this PR:

• Closes #

---

This change is 

[wmdi]

Reviewed 19 of 48 files at r3, 30 of 39 files at r4, all commit messages.
Reviewable status: 50 of 59 files reviewed, 55 unresolved discussions (waiting on @jiazhihao, @lambda7xx, and @lockshaw)

---

lib/op-attrs/src/replicate.cc line 24 at r4 (raw file):

  ParallelTensorShape output = input;
  output.at(ff_dim_t(0)).is_replica_dim = true;
  output.at(ff_dim_t(0)).size *= attrs.replicate_degree;

I think you also have to update the degree

---

lib/op-attrs/src/reshape.cc line 43 at r4 (raw file):

      data[1].size = attrs.shape.at(ff_dim_t(0));
      data[2].size = input_volume / attrs.shape.at(ff_dim_t(0));
      for (int i = 1; i < 3; i++) {

I am a little confused of this piece of codes. What does 3 standard for?

---

lib/op-attrs/src/split.cc line 30 at r4 (raw file):

std::vector<ParallelTensorShape>
    get_output_shape(SplitAttrs const &attrs,

Why?

[lambda7xx]

lib/op-attrs/src/reshape.cc line 43 at r4 (raw file):

Previously, wmdi (Mengdi Wu) wrote…

I am a little confused of this piece of codes. What does 3 standard for?

Code snippet:

if the `attrs.shape.dims.num_dims() == 1` and `attrs.shape.at(ff_dim_t(0)) != -1` , i think it will be a tensor with 2 dimensison. And its ParallelTensorShape will be 3 dimension. 0-dimension represents the replicate num, the 1-dimention will be the attrs.shape.at(ff_dim_t(0)), and we know its input_volume,
ffor example, the pytorch tensor x = torch.randn((3,4)), its input_volume its 12, and the attrs.shape.at(ff_dim_t(0)) =2, so the new tensor will be the (2,6)

[lambda7xx]

lib/op-attrs/src/replicate.cc line 24 at r4 (raw file):

Previously, wmdi (Mengdi Wu) wrote…

I think you also have to update the degree

yeah, you are right. because it's replicate operator, so the degree of output is same as input.

[lockshaw]

FYI @lambda7xx: @wmdi will be handling finishing off and merging this PR to free you up to start to work on more runtime-related things

[lockshaw]

Superseded by #1394.