feature request: improve page-limits handling

[gabor]

we have some hardcoded page limits in the code, for example:

• github-datasource/pkg/github/constants.go

  Line 4 in 4452e59

  const PageNumberLimit = 2

• github-datasource/pkg/github/projects/projects.go

  Line 15 in 4452e59

  const PageNumberLimit = 2

we should increase these, but if we increase them too much we'll hit rate limits. some ideas:

1. make these user-configurable
2. warn the user if they are trying to fetch too much data

(see #232 for more context)

[natellium]

I ran into this yesterday. The project I was querying has hundreds of closed issues, so no matter what I did I only got the first bunch. It also seems the filters are applied after the query, so I couldn't work around the limit either 😞

[trevjonez]

Possibly related given apps have much higher rate limits compared to a user PAT.
#88

Another thing that could help is using the logged in users auth since that could scale better for concurrent users compared to using app auth

https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28