Implement automated code coverage for CI

[tiffzhao5]

Pull Request Summary

Used diff-cover for automated code coverage. In my temp commit, I wrote an endpoint that wasn't covered by unit tests and this is what I got:

Test Plan and Usage Guide

How did you validate that your PR works correctly? How do you run or demo the code? Provide enough detail so a reviewer can reasonably reproduce the testing procedure. Paste example command line invocations if applicable.

[yixu34]

Thanks for doing this! Much needed. Can you paste the initial output from this in the PR description? Just to give us an idea of how much ground to 80% we need to cover before CI is green again 🟢

[tiffzhao5]

Thanks for doing this! Much needed. Can you paste the initial output from this in the PR description? Just to give us an idea of how much ground to 80% we need to cover before CI is green again 🟢

@yixu34 Pasted the initial output! Also, just to clarify this PR is only checking that the code coverage of the incremental changes, not the entire llm-engine repo, is 80%. Let me know if this is 80% is too high -- alternatively, I was thinking of adding CI PR comments instead of failing CI.

[yunfeng-scale]

Thanks for doing this! Much needed. Can you paste the initial output from this in the PR description? Just to give us an idea of how much ground to 80% we need to cover before CI is green again 🟢

@yixu34 Pasted the initial output! Also, just to clarify this PR is only checking that the code coverage of the incremental changes, not the entire llm-engine repo, is 80%. Let me know if this is 80% is too high -- alternatively, I was thinking of adding CI PR comments instead of failing CI.

it'll be nice to add an extra step on master branch to get coverage for all files

[yixu34]

Thanks for doing this! Much needed. Can you paste the initial output from this in the PR description? Just to give us an idea of how much ground to 80% we need to cover before CI is green again 🟢

@yixu34 Pasted the initial output! Also, just to clarify this PR is only checking that the code coverage of the incremental changes, not the entire llm-engine repo, is 80%. Let me know if this is 80% is too high -- alternatively, I was thinking of adding CI PR comments instead of failing CI.

Nope failing CI sounds good! Especially since it's only incremental for now.

it'll be nice to add an extra step on master branch to get coverage for all files

Agree, this can be a follow-up?