Added pagination for predictions.list + a new predictions.list_before_date function

[andreemic]

We (https://www.virtualstagingai.app) use replicate for production inference and wanted to keep an overview over our costs.

Since the Replicate dashboard is very simplistic, I added two functions here which we routinely use to aggregate and analyze inference data, to understand where our costs come from.

The changes are pretty simple and backwards-compatible, let me know if you have questions or change requests.

Thanks for Replicate!

[iveskins]

I wanted to use this but i was getting some parsing errors.

like

Exception: 2 validation errors for Prediction
logs
  Field required [type=missing, input_value={'completed_at': '2023-10...ebhook_completed': None}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.0.3/v/missing
version
  Field required [type=missing, input_value={'completed_at': '2023-10...ebhook_completed': None}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.0.3/v/missing

to get it going i had to set

class Prediction(BaseModel):
    id: str
    error: Optional[str]
    input: Optional[Dict[str, Any]]
    logs: Optional[str] = None
    output: Optional[Any]
    status: str
    version: Optional[Version] = None
    started_at: Optional[str]
    created_at: Optional[str]
    completed_at: Optional[str]

[mattt]

Hi @andreemic. Thank you for opening this PR. Sorry for not responding sooner. I just merged #189, which is now available in version 0.17.0.

Currently, Replicate's API doesn't provide a way to specify a date range for predictions (though that's something I'd like to add). Fetching all predictions before a given date will have the effect of fetching all of you predictions and discarding all of the leading entries before you reach that target date, so I wouldn't recommend doing this unless absolutely necessary.

Using the new list API, here's how you could get a list of all predictions up to a given date (say, YTD):

from datetime import datetime

import replicate

predictions = []
page = replicate.predictions.list()
target_date = datetime.strptime('2023-01-01', '%Y-%m-%d')

while page:
    predictions.extend(page.results)
    if page.next and predictions and datetime.strptime(predictions[-1].created_at, '%Y-%m-%dT%H:%M:%S.%fZ') > target_date:
       page = replicate.predictions.list(page.next)
    else:
       break

# Filter predictions to only include those created before or at the target date
predictions = [p for p in predictions if datetime.strptime(p.created_at, '%Y-%m-%dT%H:%M:%S.%fZ') <= target_date]