Merged
Conversation
- before created race3_hispanic when needed but stopped if there was race3_hispanic. It was non-sensical. Now gives informative messages
- chi_calc checks to make sure that all CHI cat1_varname and cat2_varname are coded as would be expected based upon rads.data:: misc_chi_byvars. However, there are times we have non-CHI variables that are created alongside CHI (for example for BSK, COO, etc.). This allows the function to maintain strict standards for CHI variables but with flexibility if we are producing non-CHI estimates
- previously gave empty rows for trends when there were no real trends to calculate
- previously if there were any indicators that had trends,
then rows for trends would be created for all variables.
Now it will only create trend rows for specific
variables that should have trends calculated
- Wilson score gives better CI when 0% or 100% - Needed to add level because when using factor variables in calc, will get estimates for each level and need a way to filter out undesired levels from CHI output
- ACS does not need to have cat1_varname, cat2_varname, numerator, and denominator - RSE is legitimately NA (actually undefined) when numerator is zero and therefore the result would be zero, which puts zero in the denominator for calculating RSE
- properly deal with times when numerator == 0 - Wilson socre no longer depends on se == 0, but only that result is 0 or 1 - caution flag now noted when numerator == 0 (in addition to RSE > 30)
- now follows standard naming convention in this pacackage: chi _ verb _ noun
- now allows a specific indicator to be missing a specific bit of information. For example, in ACS medinc variable does not have latest_year_count
- does not allow race4 to be 'Ethnicity' ... another of many tweaks to deal with insanity of race3/race4
- previously only could read relevant tables from prod server - Now can use a data.frame/data.table in memory to make an anlysis set
- allow for proper rounding based on whether integer, proportion, or rate - ensures there are no gaps in single years for trends - warns if there appears to be more than one distinct multi year 'year' value - streamlined messaging and warnings depdenent upon verbose argument
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
chi_calcnow processes correctly when numerator == 0, uses WilsonScore for calculating the CI when numerator == 0, triggers caution flag
when numerator ==0 (or RSE > 30%)
chi_calccan now distinguish between CHI indicators that must meet allstandards and non-CHI indicators that are processed with CHI for
efficiency/convenience
chi_generate_analysis_set... allow to use a table in memory in additionto tables in prod server
chi_generate_metadataallows for missing data specified by the userchi_generate_tro_shell.. more tweaks to get race3/race4 to behavechi_generate_tro_shell... fix so the trends only created for specificindicator_keys
chi_qa_tro... intelligent roundning based on whether metadata designatesestimates as integers, rates, or proportions
chi_qa_tro... checks for missing years in datachi_qa_tro... allow exceptions for ACS's unique qualitieschi_qa_tro, allow for legitimate cases where RSE is NA, namely whennumerator == 0
chi_sql_updaterenamed to tochi_update_sqlto follow naming convention