FutureWarning: This config doesn't use attention memories, a core feature of XLNet even though I'm using mem_len

[zainsarwar865]

So every time I try to run xl-net using the run_generation.py script, I get the warning : FutureWarning: This config doesn't use attention memories, a core feature of XLNet. Consider setting mem_len to a non-zero value, for example xlnet = XLNetLMHeadModel.from_pretrained('xlnet-base-cased'', mem_len=1024), for accurate training performance as well as an order of magnitude faster inference.

The issue is that I am using mem_len, I'm setting it to 384 (have tried other values as well) but I still get this warning.

[zainsarwar865]

Any solutions to this?

Thanks!

[TevenLeScao]

Hey, how are you setting mem_len=384? I am under the impression that the default run_generation script doesn't allow you to change the model from the default configuration, which would explain the warning.

[zainsarwar865]

Here is how I am setting it
model = model_class.from_pretrained(args.model_name_or_path,cache_dir=args.cache_dir,mem_len=args.mem_len)

[TevenLeScao]

Ok, I've understood the issue - when using this call, the config first gets created with mem_len=0 (hence the warning) then mem_len gets changed. I'll open a PR to move the warning to the forward call. In the meantime, you can suppress the warnings; memories are actually getting used.

[zainsarwar865]

Okay great!
Thanks a lot. Closing the issue.