Since your model didn't use the identity embedding, How your model find the output belongs to search sequence?

[JananiKugaa]

[LPXTT]

Hi, we know the position of search tokens because we concatenate them with the target tokens in the first place. For example, if we have 40 target tokens + 60 search tokens, after transformer, the last 60 search tokens are still search tokens. I am not sure if I understand your question correctly.