Support review comments on large diffs

[amit1rrr]

Problem

GitHub has a limitation where it does not render diffs greater than 1mb or 20k lines. For large diffs, they also do not accept review comments (i.e. comments on the diff) via API or UI.

We at ReviewNB, offer code reviews for Jupyter notebooks. Due to the nature of notebooks (embedded outputs), it's common for notebook diffs to go over the GitHub limit. So I came up with a clever solution (or glorious hack?) described below.

Solution

While there are limits on review comments, there are no such limits for issue comments (overall PR comment under the conversation tab). What issue comment lacks is,

• Context (which notebook cell the comment is meant for?)
• Discussion (reply to comment, resolve/unresolve conversations)

Our solution is that we post discussion thread as an issue comment on GitHub with all the metadata (context, author, created_at) as custom data attributes (see example below).

Example

• Say you wrote a comment on large notebook diff on ReviewNB.

• We can't post this as a review to GitHub (the API won't accept it). So we post this as issue comment with all the metadata in it

• It looks like this PR comment on GitHub

• But actually appears as a conversation on ReviewNB (along with notebook context)

Your colleagues can then write a reply or resolve/unresolve conversation and so on. All changes to the conversation are reflected in the corresponding issue comment. That's how we worked around GitHub's limitation and enabled conversations for large notebook diffs (yayy!).

Feel free to ask any questions you might have.