fix llama

[yandai]

Hi,

In current llama implement, post_decoder_embedding.bias seems uninitialized then cause crash .
Try to fix this problem and please help to take a look.

Thanks

[void-main]

Hi @yandai , thanks for the contribution, may I ask when we need post_decoder_embedding.bias? Which model are you using?

[yandai]

As my understanding，llama don't use bias. So gpt_weights->post_decoder_embedding.bias and padded_embedding_bias_ are not initialized correctly. My change is also compatible with "llama with post_decoder_embedding.bias".

ref:
https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L620

[void-main]

@yandai , I agree that llama doesn't use bias, that's why I wonder why we need to add that if/else test here?

[yandai]

So there's no need to keep padded_embedding_bias_ so I think removing the padded_embedding_bias_ member is better choice.

[void-main]

So there's no need to keep padded_embedding_bias_ so I think removing the padded_embedding_bias_ member is better choice.

Hi @yandai , agree! So, do you think it's a good idea that you change your commit by simply removing lines 823-827 from Llama.cc, since we don't care bias anyways.

[yandai]

okay, padded_embedding_bias_ is removed and please take a look.

[void-main]

merged, thanks!

[ryxli]

Should also remove post_decoder_embedding.bias = nullptr added in LlamaWeight.cc?