Replace phantomjs with chrome as headless browser by Faye-yufan · Pull Request #82 · animint/animint2

Faye-yufan · 2023-03-12T08:53:23Z

This pull request replaces the previous use of PhantomJS as the headless testing browser with Chrome, which was done to resolve the issue of getting Dynamic HTML via JavaScript DOM nodes. Previously, the remote driver getPageSource() function returned the original skeletal HTML when attempting to retrieve XML.
Here is an example of the output returned by the PhantomJS remote driver when requesting a page:

> remDr$getPageSource()
[[1]]
[1] "<!DOCTYPE html><html><head>\n\n    <meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n    <title>Interactive animation</title>\n    <script type=\"text/javascript\" src=\"vendor/d3.v3.js\"></script>\n    <script type=\"text/javascript\" src=\"animint.js\"></script>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"styles.css\">\n    <!-- Selectize -->\n    <script type=\"text/javascript\" src=\"vendor/jquery-1.11.3.min.js\"></script>\n    <script type=\"text/javascript\" src=\"vendor/selectize.min.js\"></script>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"vendor/selectize.css\">\n  </head>\n\n  <body>\n\n    <div id=\"plot\"> </div>\n    <script>\n      var plot = new animint(\"#plot\",\"plot.json\");\n    </script>\n\n  \n\n\n</body></html>"

After switching to chrome and using the headless mode with wdman extra arguments, most of the tests that check for html node/property values were fixed. However, there are still a few failed tests remaining.

> tests_init("chrome")
...
> [ FAIL 19 | WARN 1 | SKIP 5 | PASS 1825 ]

tdhock · 2023-03-14T18:21:23Z

thanks, this is a great first step toward modernizing the test framework!

Faye-yufan · 2023-03-22T17:50:21Z

I opted to use the Chromedriver in the wdman package instead of Chromote as proposed in the previous PR, since there are numerous functions in the remoteDriver test suite that are not compatible with Chromote.

However, there are still a few tests that fail in my local environment (macOS, arm64, Chrome version: 111.0.5563.110):

In test-renderer1-axis-angle-rotate.R, here is one of them:

Failure (test-renderer1-axis-angle-rotate.R:48:3): axis.text.x=element_text(angle=90) means transform="rotate(-90)" ──
title_box$top >= tick_box$bottom is not TRUE

`actual`:   FALSE
`expected`: TRUE 
Backtrace:
    ▆
 1. └─animint2 (local) expect_rotate_anchor(info, "-90", "end") at test-renderer1-axis-angle-rotate.R:48:2
 2.   └─testthat::expect_true(title_box$top >= tick_box$bottom) at test-renderer1-axis-angle-rotate.R:33:2
── Failure (test-renderer1-axis-angle-rotate.R:56:3): axis.text.x=element_text(angle=70) means transform="rotate(-70)" ──
title_box$top >= tick_box$bottom is not TRUE

Interestingly, these tests pass perfectly in Firefox. The transform="translate(x, y)" values are different between Chrome and Firefox. This could be attributed to the differences in rendering engines and the implementation of CSS styles across browsers.

The following error occurs in test-shiny.R:

── Error (test-tt-inter.R:3:1): (code run outside of `test_that()`) ────────────
<packageNotFoundError/error/condition>
Error in `find.package(package, lib.loc, verbose = verbose)`: there is no package called ‘animint’

I suspect this is due to the old animint package not being supported in my current R version.

Faye-yufan · 2023-03-22T17:52:06Z

Also some concerns:

Using Chrome as the headless driver for testing may result in incompatibilities, if the local Chrome browser is not updated to the available Chrome driver versions. This imposes higher requirements for the local Chrome browser version.
I have not yet tested this on Linux and Windows systems. Any feedback or suggestions for these operating systems would be greatly appreciated.

tdhock · 2023-03-22T23:21:29Z

You wrote "The following error occurs in test-shiny.R" but are you sure that you were running test-shiny.R? The output says Error (test-tt-inter.R:3:1): and that file, test-tt-inter.R looks old, I will remove.

tdhock · 2023-03-22T23:23:15Z

tests_init('chrome') gives an error on windows, any idea how to fix?

> tests_init("chrome")

HKEY_CURRENT_USER\Software\Google\Chrome\BLBeacon
    version    REG_SZ    111.0.5563.65

Error: C:\Users\th798\AppData\Local/binman/binman_chromedriver app directory not found.
> Serving the directory C:\Users\th798\R\animint2\tests\testthat at http://127.0.0.1:4848

> traceback()
6: stop(assertError(attr(res, "msg")))
5: assert_that(app_dir_exists(appdir))
4: app_dir(appname)
3: binman::list_versions("chromedriver") at helper-HTML.R#71
2: rev(binman::list_versions("chromedriver")[[1]]) at helper-HTML.R#71
1: tests_init("chrome")
> devtools::session_info()
─ Session info ───────────────────────────────────────────────────────────────
 setting  value
 version  R version 4.2.2 (2022-10-31 ucrt)
 os       Windows 10 x64 (build 19042)
 system   x86_64, mingw32
 ui       RTerm
 language (EN)
 collate  English_United States.utf8
 ctype    English_United States.utf8
 tz       America/Phoenix
 date     2023-03-22
 pandoc   2.11.3.2 @ C:\\PROGRA~1\\Pandoc\\pandoc.exe

─ Packages ───────────────────────────────────────────────────────────────────
 package     * version   date (UTC) lib source
 animint2    * 2023.3.10 2023-03-22 [1] local
 assertthat    0.2.1     2019-03-21 [1] CRAN (R 4.2.0)
 binman        0.1.3     2022-09-01 [1] CRAN (R 4.2.1)
 bitops        1.0-7     2021-04-24 [1] CRAN (R 4.2.0)
 brio          1.1.3     2021-11-30 [1] CRAN (R 4.2.0)
 cachem        1.0.6     2021-08-19 [1] CRAN (R 4.2.0)
 callr         3.7.3     2022-11-02 [1] CRAN (R 4.2.2)
 caTools       1.18.2    2021-03-28 [1] CRAN (R 4.2.1)
 cli           3.6.0     2023-01-09 [1] CRAN (R 4.2.2)
 colorspace    2.1-0     2023-01-23 [1] CRAN (R 4.2.2)
 crayon        1.5.2     2022-09-29 [1] CRAN (R 4.2.2)
 data.table    1.14.9    2023-03-21 [1] local
 devtools      2.4.5     2022-10-11 [1] CRAN (R 4.2.2)
 digest        0.6.31    2022-12-11 [1] CRAN (R 4.2.2)
 ellipsis      0.3.2     2021-04-29 [1] CRAN (R 4.2.0)
 fastmap       1.1.0     2021-01-25 [1] CRAN (R 4.2.0)
 fs            1.6.1     2023-02-06 [1] CRAN (R 4.2.2)
 glue          1.6.2     2022-02-24 [1] CRAN (R 4.2.0)
 gtable        0.3.1     2022-09-01 [1] CRAN (R 4.2.2)
 htmltools     0.5.4     2022-12-07 [1] CRAN (R 4.2.2)
 htmlwidgets   1.6.1     2023-01-07 [1] CRAN (R 4.2.2)
 httpuv        1.6.9     2023-02-14 [1] CRAN (R 4.2.2)
 httr          1.4.4     2022-08-17 [1] CRAN (R 4.2.2)
 knitr         1.42      2023-01-25 [1] CRAN (R 4.2.2)
 later         1.3.0     2021-08-18 [1] CRAN (R 4.2.0)
 lifecycle     1.0.3     2022-10-07 [1] CRAN (R 4.2.2)
 magrittr      2.0.3     2022-03-30 [1] CRAN (R 4.2.0)
 memoise       2.0.1     2021-11-26 [1] CRAN (R 4.2.0)
 mime          0.12      2021-09-28 [1] CRAN (R 4.2.0)
 miniUI        0.1.1.1   2018-05-18 [1] CRAN (R 4.2.1)
 munsell       0.5.0     2018-06-12 [1] CRAN (R 4.2.0)
 pkgbuild      1.4.0     2022-11-27 [1] CRAN (R 4.2.2)
 pkgload       1.3.2     2022-11-16 [1] CRAN (R 4.2.2)
 plyr          1.8.8     2022-11-11 [1] CRAN (R 4.2.2)
 prettyunits   1.1.1     2020-01-24 [1] CRAN (R 4.2.0)
 processx      3.8.0     2022-10-26 [1] CRAN (R 4.2.2)
 profvis       0.3.7     2020-11-02 [1] CRAN (R 4.2.1)
 promises      1.2.0.1   2021-02-11 [1] CRAN (R 4.2.0)
 ps            1.7.2     2022-10-26 [1] CRAN (R 4.2.2)
 purrr         1.0.1     2023-01-10 [1] CRAN (R 4.2.2)
 R6            2.5.1     2021-08-19 [1] CRAN (R 4.2.0)
 rappdirs      0.3.3     2021-01-31 [1] CRAN (R 4.2.0)
 Rcpp          1.0.10    2023-01-22 [1] CRAN (R 4.2.2)
 remotes       2.4.2     2021-11-30 [1] CRAN (R 4.2.0)
 RJSONIO       1.3-1.8   2023-01-31 [1] CRAN (R 4.2.2)
 rlang         1.0.6     2022-09-24 [1] CRAN (R 4.2.2)
 RSelenium   * 1.7.9     2022-09-02 [1] CRAN (R 4.2.2)
 scales        1.2.1     2022-08-20 [1] CRAN (R 4.2.2)
 semver        0.2.0     2017-01-06 [1] CRAN (R 4.2.1)
 sessioninfo   1.2.2     2021-12-06 [1] CRAN (R 4.2.0)
 shiny         1.7.4     2022-12-15 [1] CRAN (R 4.2.2)
 stringi       1.7.12    2023-01-11 [1] CRAN (R 4.2.2)
 stringr       1.5.0     2022-12-02 [1] CRAN (R 4.2.2)
 testthat    * 3.1.6     2022-12-09 [1] CRAN (R 4.2.2)
 urlchecker    1.0.1     2021-11-30 [1] CRAN (R 4.2.1)
 usethis       2.1.6     2022-05-25 [1] CRAN (R 4.2.2)
 vctrs         0.5.2     2023-01-23 [1] CRAN (R 4.2.2)
 wdman         0.2.6     2022-09-01 [1] CRAN (R 4.2.3)
 xfun          0.37      2023-01-31 [1] CRAN (R 4.2.2)
 XML         * 3.99-0.13 2022-12-04 [1] CRAN (R 4.2.2)
 xtable        1.8-4     2019-04-21 [1] CRAN (R 4.2.1)

 [1] C:/Users/th798/AppData/Local/R/win-library/4.2
 [2] C:/Program Files/R/R-4.2.2/library

──────────────────────────────────────────────────────────────────────────────

Faye-yufan · 2023-03-22T23:36:55Z

You wrote "The following error occurs in test-shiny.R" but are you sure that you were running test-shiny.R? The output says Error (test-tt-inter.R:3:1): and that file, test-tt-inter.R looks old, I will remove.

Sorry for the mix-up, the below is the error from test-shiny.R:

── Error (test-shiny.R:14:1): (code run outside of `test_that()`) ──────────────
<packageNotFoundError/error/condition>
Error in `loadNamespace(x)`: there is no package called ‘animint’
Backtrace:
    ▆
 1. └─base::loadNamespace(x) at test-shiny.R:14:0
 2.   └─base::withRestarts(stop(cond), retry_loadNamespace = function() NULL)
 3.     └─base (local) withOneRestart(expr, restarts[[1L]])
 4.       └─base (local) doWithOneRestart(return(expr), restart)

And thanks for the feedback for the Windows environment, I'll look into that and work on a fix.

Faye-yufan · 2023-07-21T06:41:32Z

I'm still unable to get it to work due to incompatibility between the Chrome version and the Chrome Driver version on my machine.

[1] "115.0.5790.102" # the Chrome version on my local machine
[1] "112.0.5615.28" "111.0.5563.64" "111.0.5563.41" "111.0.5563.19" # the available Chrome Driver versions
[5] "106.0.5249.21" "105.0.5195.52" "105.0.5195.19"
Error in tests_init("chrome") : Cannot find compatible chrome driver.

tdhock · 2023-08-07T15:38:21Z

what do you think would be the way forward here? it seems like it should be possible to get a current chrome + compatible chromeDriver...

Faye-yufan · 2023-08-07T17:12:11Z

I have utilized WebDriver managers wdman to get the 'latest_compatible' version of ChromeDriver, in here, it should be working because I have successfully run the test before.

There are two most straightforward ways I can think of now:

Manually Downgrade/Upgrade Chrome: This is probably the most straightforward approach, but it’s not the most convenient one because it requires more effort to do so.
Wait for ChromeDriver Update, but this is a passive approach.

I think the real snag here here is the ChromeDriver has to work with the Chrome browser, even if in the headless mode, and the version of ChromeDriver typically lag behind the browser's.

tdhock · 2023-08-11T23:00:35Z

what do they do in other projects? we are not the only ones using chromeDriver on github....

Faye-yufan · 2023-08-12T00:49:49Z

I've looked into it and it seems that many packages like webdriver and webshot are now leaning towards using Chromote as their remote browser driver, maybe that's the right choice. I'm thinking of diving in and start updating our test suite functions to use Chromote.
Given that it'd be a significant update for our test suite, any thoughts on where to start?

tdhock · 2023-08-15T12:23:38Z

thanks for looking at other R packages like webdriver and webshot, and finding that Chromote is now recommended over chromedriver. That is useful.
To clarify my previous comment, I meant to investigate other projects (not necessarily R packages, python/js/etc packages also maybe) which are using chrome+chromedriver/chromote for testing, and try to figure out how they handle this version incompatibility issue. My guess is that instead of always trying to use the most recent version, we should try using a somewhat recent version (say released 1 year ago) of both chrome + chromote, which work together, and should be easy to install. I guess the issue is that our requirement for testing is that we want it to work on both (1) gh actions and (2) our own personal computers, I wonder if other projects have this requirement as well?

Faye-yufan · 2023-08-15T22:32:55Z

I see, I'll dive deeper into that.
Just found out the latest announcement on the chromedriver's site. Looks like Chrome and ChromeDriver are now release together. So, I'm thinking we might not face those annoying lag issues anymore.

But here's the problem: in our tests_init(), we use binman to fetch available chromedriver versions, and identify the compatible chromedriver version based on the system's Chrome version. Use wdman to start the identified chromedriver version. And use RSelenium to interact with the started chromedriver for running tests. However, I haven't seen any updates on these packages, specially with this new ChromeDriver release adjustment.

Also, ChromeDriver used to be released over here: https://chromedriver.storage.googleapis.com/index.html. But they've changed the urls to https://googlechromelabs.github.io/chrome-for-testing/#stable. I'm thinking that this might be why the binman and wdman fail to get the latest compatible chromedriver.

tdhock · 2024-08-29T17:13:30Z

this PR is no longer relevant because #126 implemented chromote for testing.

Replace phantomjs with chrome as headless browser

8604e1d

Faye-yufan added 2 commits March 22, 2023 09:50

Add OS-specific Chrome version detection and update a test

5617530

Fix test: ungroup data to resolve grouped_df compatibility issue

2509ce0

Faye-yufan marked this pull request as ready for review March 22, 2023 17:52

use animint()

ee40589

siddhesh195 mentioned this pull request May 30, 2024

Replace phantomjs with chromote as headless browser #124

Closed

tdhock closed this Aug 29, 2024

Conversation

Faye-yufan commented Mar 12, 2023

Uh oh!

tdhock commented Mar 14, 2023

Uh oh!

Faye-yufan commented Mar 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Faye-yufan commented Mar 22, 2023

Uh oh!

tdhock commented Mar 22, 2023

Uh oh!

tdhock commented Mar 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Faye-yufan commented Mar 22, 2023

Uh oh!

Faye-yufan commented Jul 21, 2023

Uh oh!

tdhock commented Aug 7, 2023

Uh oh!

Faye-yufan commented Aug 7, 2023

Uh oh!

tdhock commented Aug 11, 2023

Uh oh!

Faye-yufan commented Aug 12, 2023

Uh oh!

tdhock commented Aug 15, 2023

Uh oh!

Faye-yufan commented Aug 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tdhock commented Aug 29, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Faye-yufan commented Mar 22, 2023 •

edited

Loading

tdhock commented Mar 22, 2023 •

edited

Loading

Faye-yufan commented Aug 15, 2023 •

edited

Loading