visual features

[tanzheen]

Hi, thank you for your work! Just want to ask how are your visual features extracted? Saw in the paper that you used a combination of depthwise and pointwise convolutions. However, the image features are already extracted in the dataset that you have downloaded. Is there any script for how you extracted the features and trained the convolutional module?

[EtaEnding]

Hi, the visual features are obtained from SL-Transformer as we followed their training pipeline, the depth wise separable convolutions we utilized in encoders are for latent signs differentiating from Gloss Attention and better features processing to the decoder

[tanzheen]

I see! where did you trained the weights of the convolutional module? Did you train it together with the encoder and decoder

[EtaEnding]

If you are referring to the visual features extractor, we didn’t because these features are provided by the SL-Transformer, our whole model is trained upon the visual features from scratch

[tanzheen]

I see! I think I might have misinterpreted the paper then. Can i ask what the convolution module is for then?