Add FAQ.md to provide sample estimation guidelines #408
Conversation
mergify
bot
added
documentation
relyt0925
force-pushed
the
faq-size
branch
5 times, most recently
from
b31ff34 to
1f568eb
Compare
| For each knowledge leaf node: the formula to estimate the number of produced synthetic samples in the training dataset is: | ||
|
|
||
| ```text | ||
| (total cumulative size of knowledge documents / max document chunk size) * number of qna pairs in the knowledge file leaf node * 30 synthetic samples per qna pair |
There was a problem hiding this comment.
Does this factor in the amount of samples that will be filtered out?
There was a problem hiding this comment.
@RobotSail this is an excellent point: it does not since that part is non detemenistic to a certain extent: but folks have been looking to be able to get some guidance on some ball park numbers/ answers to general questions around how the taxonomy is processed through SDG
I should call that out as a disclaimer and will do that
relyt0925
force-pushed
the
faq-size
branch
3 times, most recently
from
b03fc41 to
9cbd2ac
Compare
The new FAQ.md file includes detailed explanations and examples on how to estimate the number of synthetic samples produced at various stages of the SDG training process. This addition aims to enhance user understanding of the sample generation methodology. Signed-off-by: Tyler Lisowski <lisowski@us.ibm.com>
|
These are a good initial set of FAQs that we have seen pop up with what I feel (based on my studies of the codebase) are the appropriate answers. More than happy to adjust anything that is inaccurate though based on expert opinion! |
bbrowning
left a comment
bbrowning
left a comment
There was a problem hiding this comment.
Thanks for expanding our docs! I have a couple of comments, but also a more general question. Do you think it would be better to document most of these things in the instructlab/instructlab repository instead of directly in SDG? The reason I ask is that the questions touch on taxonomy, SDG, training, and the intersection of all these. And the target users here would likely be using the ilab CLI to run these workflows as opposed to SDG directly?
|
@bbrowning sorry for delay! Thank you so much for review! I agree with you that maybe there is a better home for this page. I will go ahead and get the comments addressed, we can make sure we are all comfortable with the content: and then we can think on where we want it's home to be! |
Signed-off-by: Ben Browning <bbrownin@redhat.com>
bbrowning
left a comment
bbrowning
left a comment
There was a problem hiding this comment.
I added in the requested changes (cleaning up links, fixing typo) as an additional commit and will go ahead and approve to get this merged.
|
Thank you so much @bbrowning for the assistance! Sorry for the delay hope everything is wonderful with you! |
|
All is well, and you're welcome! Sorry it took us so long to get this in, but thank you for the contribution! |
3 participants
The new FAQ.md file includes detailed explanations and examples on how to estimate the number of synthetic samples produced at various stages of the SDG training process. This addition aims to enhance user understanding of the sample generation methodology.
I believe it's an MVP to resolving: #307