the code for attention maps visualisations

[hmbb123]

Hello, thank you for your excellent work. How did you create the attention maps visualisations in Figures 4 and 5? Can you provide the code for this?

[LPXTT]

Hello, thanks for your interests. I am afraid I can not find the code now. It is actually not complex. The visualization maps are calculated based on the attention maps. First, we save the attention maps from different layers. Then, we calculate the target-related attention map. Suppose that we have 16 target tokens and 64 search tokens, the attention map will be 8080. We select a 6416 attention map from it, 64 is the search tokens from the first 80 and 16 is the target tokens from the last 80 (if we put the target tokens in front of the search tokens). After that, we calculate the average values along the second dimension of the 6416 attention map. Then, we will get 641 maps which can be reshaped to 8*8. That is our heat map.

[hmbb123]

Thanks for your reply, let me try it.

[hmbb123]

Thanks for your reply. But I think there may be something wrong with the phrase "64 is the search tokens from the first 80 and 16 is the target tokens from the last 80", I think it should be "64 is the search tokens from the last 80 and 16 is the target tokens from the first 80".
Am I understanding this correctly?
@LPXTT

[LPXTT]

Yes. Your understanding is correct. Sorry

…

在 2023年2月16日，下午10:16，hmbb123 ***@***.***> 写道： Thanks for your reply. But I think there may be something wrong with the phrase "64 is the search tokens from the first 80 and 16 is the target tokens from the last 80", I think it should be "64 is the search tokens from the last 80 and 16 is the target tokens from the first 80". Am I understanding this correctly? @LPXTT <https://github.com/LPXTT> — Reply to this email directly, view it on GitHub <#10 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AICKQEYSKII465FL3JQEPMTWXYD7VANCNFSM6AAAAAAU5ZO76Y>. You are receiving this because you were mentioned.