Skip to content

Conversation

@AndrewZhaoLuo
Copy link
Contributor

@AndrewZhaoLuo AndrewZhaoLuo commented Sep 28, 2022

This makes layer_norm relay op dispatch to new topi committed in PR #12864.

It removes LayerNorm from simplify inference pass now that we actually have compute.

Using TIR CSE elimination pass with FP16 layernorm also necessitates the handling of FP16 type when packing args for CUDA.

Copy link
Member

@vinx13 vinx13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering removing simplification for later norm affects non-metaschedule workflow. We didn't implement topi schedule for layer norm, I'm not sure if the default schedule_injective works well.

Btw there are also some unrelated (.csv, .md) files in this PR added by mistake.

@AndrewZhaoLuo
Copy link
Contributor Author

I have removed the unneeded files.

Good point about non-metaschedule workflows. I will see what I can do here.

@areusch areusch added needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it and removed needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it labels Oct 19, 2022
@masahi
Copy link
Member

masahi commented Dec 12, 2022

Any update? Also interested in using fused layer norm.

@AndrewZhaoLuo
Copy link
Contributor Author

AndrewZhaoLuo commented Dec 12, 2022

@masahi

Ah yes, I will try to get to this during the next week where I will have time.

  1. Figure out the behavior for autotvm and autoscheduler.
  2. The topi kernel for fused layernorm is known to be numerically unstable. We should replace it with a more stable version e.g. https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Parallel_algorithm first.
  3. [Packed Func] Pack args fp16 support #13532 this PR is included here to support fp16 layernorm, but needs some refactoring to avoid casting cost. Now fixed.

@AndrewZhaoLuo AndrewZhaoLuo force-pushed the aluo/hook-up-layernorm-topi branch from 332a388 to 21b8e2a Compare January 4, 2023 21:39
@tqchen tqchen closed this Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants