Skip to content

testing is more robust to changes in base messages#4806

Merged
mattdowle merged 6 commits intomasterfrom
translate-base-R
May 13, 2021
Merged

testing is more robust to changes in base messages#4806
mattdowle merged 6 commits intomasterfrom
translate-base-R

Conversation

@MichaelChirico
Copy link
Copy Markdown
Member

@MichaelChirico MichaelChirico commented Nov 11, 2020

Related: #4770

To figure out which messages are produced by base:

First, I edited R/test.data.table.R to force foreign=FALSE, then I commented out the setnames call between tests 1904.5 and 1904.6 (it relies on an object that doesn't get created due to test failure), then I ran:

# note: `es` is a language where R has translations but data.table does not,
#   meaning DT messages are in English but base are in Spanish
LANGUAGE=es Rscript -e "library(data.table); test.data.table()" > test_failures.txt

There were two cases where the approach here wouldn't work:

  1. .checkTypos relies on literal messages in English #4989 -- because .checkTypos relies on English messages, in non-English .checkTypos can never return suggested fixes.
  2. test 1671 is testing an error that's generated in C, I don't see a way to generate that directly from R short of writing a short C function solely for this purpose which seems like a bit much. The gettext() approach here is robust to other languages, but not to R's own message changing.

@mattdowle
Copy link
Copy Markdown
Member

mattdowle commented May 11, 2021

@MichaelChirico Good idea to force foreign=FALSE. But LANGUAGE=zh_CN so our own messages are being translated. How about picking a language that data.table is not translated for (and so data.table will emit English and will pass our English tests) but base R is translated for. Then we will see the base R messages in non-English not matching our English tests which should be a much smaller list.

@MichaelChirico
Copy link
Copy Markdown
Member Author

@mattdowle that worked excellently. I got about halfway done changing things to work across languages before realizing I'd lost the plot of what we're trying to do exactly. Going back over it now. The goal is to make ourselves immune to base message changes, but in the process we can make base errors work regardless of session language as well.

@MichaelChirico MichaelChirico changed the title test commit showing reliance on base error messages testing is more robust to changes in base messages May 11, 2021
@codecov
Copy link
Copy Markdown

codecov bot commented May 11, 2021

Codecov Report

Merging #4806 (e44ed9b) into master (19c61c0) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4806      +/-   ##
==========================================
- Coverage   99.46%   99.46%   -0.01%     
==========================================
  Files          75       75              
  Lines       14707    14706       -1     
==========================================
- Hits        14629    14628       -1     
  Misses         78       78              
Impacted Files Coverage Δ
R/test.data.table.R 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19c61c0...e44ed9b. Read the comment docs.

@mattdowle mattdowle added this to the 1.14.1 milestone May 12, 2021
@mattdowle
Copy link
Copy Markdown
Member

It's especially nice that the base messages appear as base_messages in the test() call now. That conveys more, reads well, and can be searched. Great.

@mattdowle mattdowle merged commit 8f7ef88 into master May 13, 2021
@mattdowle mattdowle deleted the translate-base-R branch May 13, 2021 00:00
@jangorecki jangorecki modified the milestones: 1.14.9, 1.15.0 Oct 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants