diff --git a/docs/cloud/features/observability/model_freshness.md b/docs/cloud/features/observability/model_freshness.md index 47c11718ca..67389f8b6c 100644 --- a/docs/cloud/features/observability/model_freshness.md +++ b/docs/cloud/features/observability/model_freshness.md @@ -1,35 +1,52 @@ # Model Freshness -Model freshness indicators on the homepage give at a glance understanding if production environment is correct and up to date with the latest data. These features help track whether models are being backfilled and created on time. -When investigating freshness issues, you can view a detailed list of models and their current status (complete, pending, or behind) in production, identify problematic models, and check if CI/CD processes have stopped running. Keep in mind that if a model shows red (behind) in the past, it doesn't necessarily reflect its current status. +Model freshness indicators on the homepage allow you to immediately determine whether the production environment is correct and up to date. + +Additional information on the page, such as lists of models and their current status, helps you investigate any freshness issues, identify problematic models, and check if CI/CD processes have stopped running. + +![tcloud model freshness](./model_freshness/find_model_freshness.png) ## When you might use this -The model freshness chart serves several key purposes in understanding production correctness. +The model freshness chart answers the question "how is the production environment right now?" It summarizes the recent history of production models and whether they were backfilled on time. + +When the chart is all green, everything is running smoothly and you're good to go! -It provides visibility into the history of production models that need to be backfilled on time, and helps monitor the current state of production. When all indicators are green, this signals that everything is running smoothly. While red indicators in historical data don't require immediate action, they provide valuable lessons to prevent similar issues in the future. For data engineers, this acts as a comprehensive report card on system health and performance. +Red indicators in the past don't require immediate action, but they may provide lessons that can help prevent similar issues in the future. -## Finding model freshness indicators +Red indicators now mean it's time to take action and debug the issue. -From the homescreen of Tobiko Cloud we have the graph on historical freshness. +## Finding the model freshness chart + +The model freshness chart is near the top of the Tobiko Cloud homepage. ![tcloud model freshness](./model_freshness/find_model_freshness.png) -**Data model freshness** here refers to the timeliness and relevance of the data used in a data model, ensuring that it reflects the most current and accurate state of the underlying system or domain. In other words, it measures how up-to-date and synchronized the model is with the real-world data. -Zooming into that data, the model freshness chart shows you the freshness of your models within your data warehouse relative to the model's configured cron. -![tcloud model freshness](./model_freshness/tcloud_model_freshness.png) +## Model freshness indicators + +Model freshness is the timeliness of the data most recently processed by a model. In other words, it measures how up-to-date each model is relative to its `cron`. -The chart displays historical data, showing freshness levels across time (shown on the `x-axis`). This historical view helps when troubleshooting reported data issues—you can quickly check if problems were caused by delayed data runs or other underlying issues. +The chart displays historical data, showing the percentage of models that were fresh (y-axis) across time (x-axis). -The chart uses three colors to show the percentage of models in different states: +This historical view helps when troubleshooting data issues — you can quickly check if the issue is associated with delayed model runs. + +![tcloud model freshness](./model_freshness/tcloud_model_freshness.png) + +The chart uses color to show the percentage of models in different states: 1. Models that have run for all previous cron periods are "complete" (green). - - All green indicates the data warehouse is fully up-to-date with model crons. + - All green indicates the data warehouse is fully up-to-date 2. Models that haven't run for the most recent cron period are "pending" (yellow). 3. Models that haven't run for multiple previous cron periods are "behind" (red). - - Red signals potential issues that need investigation. + - Red signals potential issues that need investigation + +Keep in mind that if a model shows red (behind) in the past, that doesn't necessarily reflect its current status. It may have caught up by now! + +The chart is interactive — hovering reveals the distribution of model freshness at a specific time point. + +![Tobiko Cloud model freshness chart tooltip](./model_freshness/tcloud_model-freshness_tooltip.png) -To make your life easier, the chart is interactive—you can click any point in time to see which specific models were complete, pending, or behind. +Click a time point to open a list of the models that were complete, pending, or behind at that time. -![Tobiko Cloud model freshness chart tooltip](./model_freshness/tcloud_model-freshness_tooltip.png) \ No newline at end of file +![Tobiko Cloud model freshness list](./model_freshness/tcloud_model_freshness_list.png) diff --git a/docs/cloud/features/observability/model_freshness/tcloud_model_freshness_list.png b/docs/cloud/features/observability/model_freshness/tcloud_model_freshness_list.png new file mode 100644 index 0000000000..794c8d2d57 Binary files /dev/null and b/docs/cloud/features/observability/model_freshness/tcloud_model_freshness_list.png differ