About generating pseudo samples

[DianeYuu]

Thank you for sharing your wonderful work and codes!

I encountered some problems while trying to generate pseudo samples using the codes in the folder /pseudo_data. During the process of training, the loss doesn't decrease and remains around 12. Should I adjust any parameters or make other changes to obtain the pseudo samples in the folder /generated_data ?

Thank you very much for your assistance!

[circle-hit]

Thank you for sharing your wonderful work and codes!

I encountered some problems while trying to generate pseudo samples using the codes in the folder /pseudo_data. During the process of training, the loss doesn't decrease and remains around 12. Should I adjust any parameters or make other changes to obtain the pseudo samples in the folder /generated_data ?

Thank you very much for your assistance!

For hyperparameter settings, please refer to the penultimate paragraph of Appendix C of our paper.

[x-zb]

Thank you for your nice work.
I have a few questions below:

Why is max_new_tokens=512-300 in

SAPT/pseudo_data/src/run_uie_lora.py

Line 601 in 588cca7

max_new_tokens=512-300,

?

Besides, it seems codes in pseudo_data/src/run_uie_lora.py only generate token ids and save them to lookback_inference_result.json. Are there codes that transform these results to the pseudo samples in the folder generated_data ?

Also, in your paper you said you use 0.02 pseudo samples and the dataset size for each task is 1000. But the train set in generated_data seems to have 6 samples for SuperNI tasks. Wondering if I misunderstood this.

[circle-hit]

Thank you for your nice work. I have a few questions below:

Why is max_new_tokens=512-300 in

SAPT/pseudo_data/src/run_uie_lora.py

Line 601 in 588cca7

max_new_tokens=512-300,

?
Besides, it seems codes in pseudo_data/src/run_uie_lora.py only generate token ids and save them to lookback_inference_result.json. Are there codes that transform these results to the pseudo samples in the folder generated_data ?

Also, in your paper you said you use 0.02 pseudo samples and the dataset size for each task is 1000. But the train set in generated_data seems to have 6 samples for SuperNI tasks. Wondering if I misunderstood this.

Thank you for your attention.

For the issue regarding max_new_tokens, it's the generation hyper-parameter and we set it as the max_target_length.

And we also upload after_inference_transfer_lookback_format.py and sample_instances.py to transform results to the pseudo samples and perform sampling.

The reason for "the train set in generated_data seems to have 6 samples for some SuperNI tasks" is that the total number of training instances in these datasets is less than 1000. Thus keeping the sampling ratio of 0.02 would results in that the pseudo samples are less than 20.

[x-zb]

Thank you for your prompt response. I also have a questions on the method decoder_call() in pseudo_data/src/uie_collator.py.

I'm wondering if self.max_source_length in the following two lines should be replaced by limit_input_len? When subset=='train', source should be "input+label", whose length will be around "max_source_length + max_target_length", not "max_source_length". (and 'max_source_length' is set to5 in the script Ours_CL_lookback.sh, which seems small for "input+label")

SAPT/pseudo_data/src/uie_collator.py

Line 188 in 52a52b9

max_length=self.max_source_length,

SAPT/pseudo_data/src/uie_collator.py

Line 174 in 52a52b9

max_len = self.max_source_length