From 60816961bb8fee6a553f6fbda0717b6beba4da44 Mon Sep 17 00:00:00 2001 From: zachll Date: Fri, 12 Apr 2024 13:43:46 -0400 Subject: [PATCH 1/4] Delete priorization_algorithms directory --- .../motivational_potential_model.csv | 10 ---------- 1 file changed, 10 deletions(-) delete mode 100644 priorization_algorithms/motivational_potential_model.csv diff --git a/priorization_algorithms/motivational_potential_model.csv b/priorization_algorithms/motivational_potential_model.csv deleted file mode 100644 index 902a094..0000000 --- a/priorization_algorithms/motivational_potential_model.csv +++ /dev/null @@ -1,10 +0,0 @@ -Causal_pathway,Comparison_size,Trend_slope,Measure_achievement_recency,Loss_recency,Message_recency,Message_received_count,Measure_recency -Better,0.5,--,--,--,-0.1,-0.1,-0.5 -Worse,0.5,--,--,--,-0.1,-0.5,-0.5 -Improving,--,0.8,--,--,-0.1,-0.1,0 -Worsening,--,0.8,--,--,-0.1,-0.5,0 -Achievement,0.5,0.8,-0.5,--,-0.1,-0.1,-0.5 -Loss,0.5,0.8,--,-0.5,-0.1,-0.5,-0.5 -Approach,0.5,0.8,-0.5,--,-0.1,-0.1,0 -Sustain better,0.5,--,--,--,-0.1,-0.1,-0.5 -Sustain worse,0.5,--,--,--,-0.1,-0.5,-0.5 \ No newline at end of file From c4da122faddeb8726ffdd03dcb3426e4ea2571e5 Mon Sep 17 00:00:00 2001 From: Zach Landis-Lewis Date: Fri, 12 Apr 2024 13:58:32 -0400 Subject: [PATCH 2/4] comparator description --- signal_detectors/Comparison_detect.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/signal_detectors/Comparison_detect.md b/signal_detectors/Comparison_detect.md index cc9984f..c52dc90 100644 --- a/signal_detectors/Comparison_detect.md +++ b/signal_detectors/Comparison_detect.md @@ -1,6 +1,6 @@ # Comparison Signal Detector ## Concept -The comparison signal detector function compares the performance level of a feedback recipient for a specfic measure against the level of a benchmark, such as the peer performance benchmarks or the goal value for that particular measure. This function evaluates differently for both types of gaps, both positive and negative, relative to the performance level and the list of comparator values that are pre-defined by the pipeline. When the recipient's performance level is above that of a comparator, the comparison _detect method denotes that there exists a positive gap in performance level, which is a kind of motivating information. When this detector is activated, the positive or negative gap motivating information is used to influence the rank of corresponding precision feedback message templates. +The comparison signal detector function compares the performance level of a feedback recipient for a specfic measure against the level of a comparator, such as a peer-based benchmark or a goal set by an organization. This function evaluates differently for both types of gaps, both positive and negative, relative to the performance level and the list of comparator values that are pre-defined by the pipeline. When the recipient's performance level is above that of a comparator, the comparison _detect method denotes that there exists a positive gap in performance level, which is a kind of motivating information. When this detector is activated, the positive or negative gap motivating information is used to influence the rank of corresponding precision feedback message templates. ## Example Below are examples of performance data that leads to the detection of a comparison signal. From 4ab841c97c593bb57c20962526f09b76d496ec61 Mon Sep 17 00:00:00 2001 From: Zach Landis-Lewis Date: Fri, 12 Apr 2024 14:01:34 -0400 Subject: [PATCH 3/4] file name case changes --- signal_detectors/{Comparison_detect.md => comparison_detect.md} | 0 signal_detectors/{Trend_detect.md => trend_detect.md} | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename signal_detectors/{Comparison_detect.md => comparison_detect.md} (100%) rename signal_detectors/{Trend_detect.md => trend_detect.md} (100%) diff --git a/signal_detectors/Comparison_detect.md b/signal_detectors/comparison_detect.md similarity index 100% rename from signal_detectors/Comparison_detect.md rename to signal_detectors/comparison_detect.md diff --git a/signal_detectors/Trend_detect.md b/signal_detectors/trend_detect.md similarity index 100% rename from signal_detectors/Trend_detect.md rename to signal_detectors/trend_detect.md From fb1c4d977100813fe400a1342c97935b26b61509 Mon Sep 17 00:00:00 2001 From: Zach Landis-Lewis Date: Tue, 16 Apr 2024 13:01:13 -0400 Subject: [PATCH 4/4] trend documentation --- signal_detectors/trend_detect.md | 68 ++++++++++++++++++++++++++++---- 1 file changed, 61 insertions(+), 7 deletions(-) diff --git a/signal_detectors/trend_detect.md b/signal_detectors/trend_detect.md index 777a7b5..dbd3ad8 100644 --- a/signal_detectors/trend_detect.md +++ b/signal_detectors/trend_detect.md @@ -1,10 +1,22 @@ -# Trend Motivating Information Detector -## Concept -The trend motivating information detector function compares the performance level of a feedback recipient for a specfic measure as it changes over time. The trend detector currently evaluates over a three month window, and detects monotonic and non-monotonic performance trend information. The signal detects when the performance level has a positive or negative slope over time, and extracts the magnitude of teh trend as motivating information to be used as a moderator. +# Trend Detector + +## Introduction +The trend detector function identifies changes in the performance level of a feedback recipient for a single measure or metric across time intervals. This function interprets trends as changes in the absolute rate of performance. + +The trend detector function receives performance data as input, and produces the following outputs: +1. Identification: Indicates the existence of a trend +2. Monotonicity: Classifies a trend as simple monotonic or nonmonotonic +3. Slope: + - Classifies the slope as negative or positive + - Indicates magnitude of the trend (Range: 0 to 1) +4. Duration: Indicates the number of time intervals (i.e. months) that the trend spans + +The output of the trend detector function is used by a precision feedback system to prioritize candidate messages. + ## Example -Below are examples of performance data that leads to the detection of a trend signal. +The performance data below contains an example trend that can be detected by the Trend Signal Detector function: -|Month|Performance Level|Peer Average|75th Percentile Benchmark|90th Percentile Benchmark|MPOG Goal| +|Month|Performance Level|Peer Average|75th Percentile Benchmark|90th Percentile Benchmark|Organizational Goal| |-----|-----------------|------------|-------------------------|-------------------------|--| |Oct | 85%| 85| 88| 92|90| |Nov | ***85%***| 85| 88| 92|90| @@ -15,5 +27,47 @@ Below are examples of performance data that leads to the detection of a trend si - trend magnitude = **(91 - 85)/2 = 3** ## Implementation -The method operates with the current formula: -Month t0 performance - Month t2 performance / 2 \ No newline at end of file +The method operates with the following formula: +time1 performance - time2 performance / 2 + +## Signal detection criteria +The following criteria must be met for a trend to be detected: + +1. The recipient's performance level must differ between the current month and the previous month. + + + +## Test cases +These test cases are based on the following performance dataset header: + + ["staff_number","measure","month","passed_count","flagged_count","denominator","peer_average_comparator","peer_75th_percentile_benchmark","peer_90th_percentile_benchmark", "MPOG_goal"], + +1. A trend is detected in these cases: + + [3,"PUL01","2023-01-01",1,0,0,1,2,9,90], + [3,"PUL01","2023-02-01",40,0,0,40,42,6,90], + [3,"PUL01","2023-03-01",100,0,0,100,33,99,90], + + [3,"PUL01","2023-01-01",99,0,0,18,2,9,90], + [3,"PUL01","2023-02-01",98,0,0,48,42,6,90], + [3,"PUL01","2023-03-01",97,0,0,1,33,99,90], + + [3,"PUL01","2024-02-01",78,0,0,100,88,9,90], + [3,"PUL01","2024-03-01",18,0,0,40,42,68,90], + [3,"PUL01","2024-04-01",17,0,0,100,33,99,90], + + +2. No trend is detected in these cases: + + [3,"PUL01","2023-01-01",1,0,0,1,22,19,90], + [3,"PUL01","2023-02-01",1,0,0,40,2,76,90], + [3,"PUL01","2023-03-01",1,0,0,10,33,9,90], + + [3,"PUL01","2023-01-01",99,0,0,1,2,9,90], + [3,"PUL01","2023-02-01",99,0,0,40,42,6,90], + [3,"PUL01","2023-03-01",99,0,0,100,33,99,90], + + [3,"PUL01","2024-02-01",78,0,0,1,2,9,90], + [3,"PUL01","2024-03-01",78,0,0,40,42,6,90], + [3,"PUL01","2024-04-01",78,0,0,100,33,99,90], +