From e8564559037d1924c21cf88a07727959ad84f2e5 Mon Sep 17 00:00:00 2001 From: Edgars Eglitis Date: Tue, 11 Jun 2024 11:30:49 +0300 Subject: [PATCH 1/4] feat: add scroll_vertically_until_visible method --- README.md | 69 +++++++++++++++++++++++++---- lib/core/device.rb | 108 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 168 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index b8a8c907..49c9a386 100644 --- a/README.md +++ b/README.md @@ -334,15 +334,16 @@ Roles are ALWAYS defined at the begining of the cases. You have to write always 7. [press](#press) 8. [click_and_hold](#click_and_hold) 9. [swipe_up/swipe_down](#swipe_up/swipe_down) -10. [swipe_on_element](#swipe_on_element) -11. [swipe_elements](#swipe_elements) -12. [swipe_coord](#swipe_coord) -13. [click_coord](#click_coord) -14. [clipboard](#clipboard) -15. [handle_ios_alert](#handle_ios_alert) -16. [notifications](#notifications) -17. [back](#back) -18. [update_settings](#update_settings) +10. [scroll_vertically_until_visible](#scroll_vertically_until_visible) +11. [swipe_on_element](#swipe_on_element) +12. [swipe_elements](#swipe_elements) +13. [swipe_coord](#swipe_coord) +14. [click_coord](#click_coord) +15. [clipboard](#clipboard) +16. [handle_ios_alert](#handle_ios_alert) +17. [notifications](#notifications) +18. [back](#back) +19. [update_settings](#update_settings) ## API @@ -789,6 +790,56 @@ It works simillar as click, but it holds the pressing. The labels and options th Id: //some/path (Element from where to start the swipe) NoRaise: false/true (Default - false -> will rise error on fail) +### scroll_vertically_until_visible + +Scroll vertically (up or down) until the specified element is visible on the screen. This function accepts many optional parameters, but the simplest form is as follows: + + - Type: scroll_vertically_until_visible + Role: role1 (Optional. if not specified will use the first one defined in the case Roles) + Strategy: id/css/xpath/uiautomator/class_chain/... + Id: //some/path + +Set a specific background element to scroll on (the scroll target). If not specified, the scroll target is set to the entire visible window. + + - Type: scroll_vertically_until_visible + Role: role1 + Strategy: id/css/xpath/uiautomator/class_chain/... + Id: //some/path + ScrollTarget: + Strategy: id/css/xpath/uiautomator/class_chain/... + Id: //some/path + RecheckAfterScrolls: 2 (Sometimes the scroll target dimensions may change, e.g. in mobile browsers, the URL bar may auto-hide. Setting this value recalculates the scroll target dimensions after the specified number of swipes) + +Customize the swipe action: + + - Type: scroll_vertically_until_visible + Role: role1 + Strategy: id/css/xpath/uiautomator/class_chain/... + Id: //some/path + SwipeAction: + OffsetFractionX: 0.3 (Fraction of the scroll target width where the swipe action will execute. Default 0.5, i.e. the midpoint of the scroll target width) + StartFractionY: 0.2 (Fraction of the scroll target height where the swipe action will start. Default 0.7) + EndFractionY: 0.8 (Fraction of the scroll target height where the swipe action will end. Default 0.3) + SwipeSpeedMultiplier: 1.2 (Speed multiplier to apply to the default scroll speed) + SwipePauseDuration: 0.5 (Time in seconds to wait in between swipes. Default 0.2 for iOS, otherwise 0.1) + +Set the scrolling timeout: + + - Type: scroll_vertically_until_visible + Role: role1 + Strategy: id/css/xpath/uiautomator/class_chain/... + Id: //some/path + ScrollTimeout: 90 (In seconds - default is 60) + +Instruct the target element to be scrolled into full view. This means that once the element is visible, one additional swipe will be executed from the element location, to either the top or the bottom of the scroll target (depending on the scroll direction configured in the swipe action). + + - Type: scroll_vertically_until_visible + Role: role1 + Strategy: id/css/xpath/uiautomator/class_chain/... + Id: //some/path + FullView: true/false (Default false) + FullViewOffsetY: 150 (Optional - offset in pixels added to either the top or bottom of the scroll target, only when executing the full view swipe action. This can be used to ensure that the element is not scrolled outside the scroll target dimensions. If this is set, FullView is assumed to be true and can be omitted) + ### swipe_elements - Type: swipe_elements diff --git a/lib/core/device.rb b/lib/core/device.rb index 7220463c..a2aaea59 100644 --- a/lib/core/device.rb +++ b/lib/core/device.rb @@ -707,6 +707,114 @@ def swipe_down(action) .perform end + # Scrolls vertically until the specified element is visible or a timeout is reached + # Accepts: + # Strategy + # Id + # SwipeAction + # OffsetFractionX + # StartFractionY + # EndFractionY + # SwipeSpeedMultiplier + # SwipePauseDuration + # ScrollTarget + # Strategy + # Id + # RecheckAfterScrolls + # ScrollTimeout + # FullView + # FullViewOffsetY + def scroll_vertically_until_visible(action) + # the element will be checked in a loop, so its search should be fast and never fail + original_noraise = action["NoRaise"] + action["NoRaise"] = true + action["Time"] = 0.5 + # configure swipe action properties + x_frac = 0.5 + y_start_frac = 0.7 + y_end_frac = 0.3 + scroll_mul = @platform == "iOS" ? 3 : 1.5 + scroll_pause = @platform == "iOS" ? 0.2 : 0.1 + if action.key?("SwipeAction") + x_frac = convert_value(action["OffsetFractionX"]) if action.key?("OffsetFractionX") + y_start_frac = convert_value(action["StartFractionY"]) if action.key?("StartFractionY") + y_end_frac = convert_value(action["EndFractionY"]) if action.key?("EndFractionY") + scroll_mul *= convert_value(action["SwipeSpeedMultiplier"]) if action.key?("SwipeSpeedMultiplier") + scroll_pause = convert_value(action["SwipePauseDuration"]) if action.key?("SwipePauseDuration") + end + scroll_timeout = action.key?("ScrollTimeout") ? convert_value(action["ScrollTimeout"]) : 60 + # calculate the exact coordinates for swiping, + # depending on whether a specific element to swipe on is provided + if action.key?("ScrollTarget") + recheck_after_scrolls = action.key?("RecheckAfterScrolls") ? convert_value(action["RecheckAfterScrolls"]) : nil + bg_el = wait_for(action["ScrollTarget"]) + y_top = bg_el.location.y + y_bottom = bg_el.location.y + bg_el.size.height + x_point = bg_el.location.x + (bg_el.size.width * x_frac) + y_start = y_top + (bg_el.size.height * y_start_frac) + y_end = y_top + (bg_el.size.height * y_end_frac) + else + screen_size = @driver.window_size + y_top = 0 + y_bottom = screen_size.height + x_point = screen_size.width * x_frac + y_start = screen_size.height * y_start_frac + y_end = screen_size.height * y_end_frac + end + # configure FullView parameters + # if FullViewOffsetY is provided, assume that FullView is requested + action["FullView"] = true if action.key?("FullViewOffsetY") + y_fullview_offset = action.key?("FullViewOffsetY") ? convert_value(action["FullViewOffsetY"]).to_f : 0 + + # start the scrolling/checking loop + scrolls = 0 + start = Time.now + while (Time.now - start) < scroll_timeout + el = wait_for(action) + if el&.displayed? + return unless action.key?("FullView") && convert_value(action["FullView"]) + el_y_loc = el.location.y # save position so it is not retrieved every time + # the condition for full view depends on the scrolling direction (up or down) + el_in_full_view = if y_start_frac > y_end_frac + el_y_loc <= y_top + y_fullview_offset + else + el_y_loc >= y_bottom + y_fullview_offset + end + return if el_in_full_view + # if element is visible but not in full view, change the scroll endpoints + if el_y_loc > y_top && el_y_loc < y_bottom + y_end = y_start_frac > y_end_frac ? y_top : y_bottom + y_start = el_y_loc + end + end + # element not displayed or not in full view - execute swipe action + duration = (y_start - y_end) * scroll_mul / 1000.0 + @driver.action + .move_to_location(x_point, y_start) + .pointer_down(:left) + .move_to_location(x_point, y_end, duration: duration) + .pause(device: @driver.action.key_inputs[0], duration: scroll_pause) + .release + .perform + scrolls += 1 + # recalculate the scroll target properties if the recheck variable is set + if action.key?("ScrollTarget") && scrolls == recheck_after_scrolls + bg_el = wait_for(action["ScrollTarget"]) + y_top = bg_el.location.y + y_bottom = bg_el.location.y + bg_el.size.height + x_point = bg_el.location.x + (bg_el.size.width * x_frac) + y_start = y_top + (bg_el.size.height * y_start_frac) + y_end = y_top + (bg_el.size.height * y_end_frac) + end + end + # raise error if timeout exceeded + if !original_noraise + path = take_error_screenshot + raise "\n#{@role}: Element '#{action["Id"]}' is not visible after scrolling for #{scroll_timeout} " + + "seconds\nError Screenshot: #{path}" + end + end + def swipe_elements(action) # swipe from element1 to element2 el1 = wait_for(action["Element1"]) From e7875ada349978a494c2203ba0b6771bcf7f20d8 Mon Sep 17 00:00:00 2001 From: Edgars Eglitis Date: Tue, 11 Jun 2024 14:49:18 +0300 Subject: [PATCH 2/4] fix: update behavior after testing --- README.md | 16 +++++++------- lib/core/device.rb | 52 ++++++++++++++++++++++++---------------------- 2 files changed, 35 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index 49c9a386..59aa0bd2 100644 --- a/README.md +++ b/README.md @@ -334,7 +334,7 @@ Roles are ALWAYS defined at the begining of the cases. You have to write always 7. [press](#press) 8. [click_and_hold](#click_and_hold) 9. [swipe_up/swipe_down](#swipe_up/swipe_down) -10. [scroll_vertically_until_visible](#scroll_vertically_until_visible) +10. [scroll_until_element_visible](#scroll_until_element_visible) 11. [swipe_on_element](#swipe_on_element) 12. [swipe_elements](#swipe_elements) 13. [swipe_coord](#swipe_coord) @@ -790,18 +790,18 @@ It works simillar as click, but it holds the pressing. The labels and options th Id: //some/path (Element from where to start the swipe) NoRaise: false/true (Default - false -> will rise error on fail) -### scroll_vertically_until_visible +### scroll_until_element_visible Scroll vertically (up or down) until the specified element is visible on the screen. This function accepts many optional parameters, but the simplest form is as follows: - - Type: scroll_vertically_until_visible + - Type: scroll_until_element_visible Role: role1 (Optional. if not specified will use the first one defined in the case Roles) Strategy: id/css/xpath/uiautomator/class_chain/... Id: //some/path Set a specific background element to scroll on (the scroll target). If not specified, the scroll target is set to the entire visible window. - - Type: scroll_vertically_until_visible + - Type: scroll_until_element_visible Role: role1 Strategy: id/css/xpath/uiautomator/class_chain/... Id: //some/path @@ -812,7 +812,7 @@ Set a specific background element to scroll on (the scroll target). If not speci Customize the swipe action: - - Type: scroll_vertically_until_visible + - Type: scroll_until_element_visible Role: role1 Strategy: id/css/xpath/uiautomator/class_chain/... Id: //some/path @@ -821,11 +821,11 @@ Customize the swipe action: StartFractionY: 0.2 (Fraction of the scroll target height where the swipe action will start. Default 0.7) EndFractionY: 0.8 (Fraction of the scroll target height where the swipe action will end. Default 0.3) SwipeSpeedMultiplier: 1.2 (Speed multiplier to apply to the default scroll speed) - SwipePauseDuration: 0.5 (Time in seconds to wait in between swipes. Default 0.2 for iOS, otherwise 0.1) + SwipePauseDuration: 0.5 (Time in seconds to wait after every swipe action. Default 0.2 for iOS, otherwise 0.1) Set the scrolling timeout: - - Type: scroll_vertically_until_visible + - Type: scroll_until_element_visible Role: role1 Strategy: id/css/xpath/uiautomator/class_chain/... Id: //some/path @@ -833,7 +833,7 @@ Set the scrolling timeout: Instruct the target element to be scrolled into full view. This means that once the element is visible, one additional swipe will be executed from the element location, to either the top or the bottom of the scroll target (depending on the scroll direction configured in the swipe action). - - Type: scroll_vertically_until_visible + - Type: scroll_until_element_visible Role: role1 Strategy: id/css/xpath/uiautomator/class_chain/... Id: //some/path diff --git a/lib/core/device.rb b/lib/core/device.rb index a2aaea59..d5a21def 100644 --- a/lib/core/device.rb +++ b/lib/core/device.rb @@ -711,20 +711,20 @@ def swipe_down(action) # Accepts: # Strategy # Id + # ScrollTarget + # Strategy + # Id + # RecheckAfterScrolls # SwipeAction # OffsetFractionX # StartFractionY # EndFractionY # SwipeSpeedMultiplier # SwipePauseDuration - # ScrollTarget - # Strategy - # Id - # RecheckAfterScrolls # ScrollTimeout # FullView # FullViewOffsetY - def scroll_vertically_until_visible(action) + def scroll_until_element_visible(action) # the element will be checked in a loop, so its search should be fast and never fail original_noraise = action["NoRaise"] action["NoRaise"] = true @@ -736,17 +736,21 @@ def scroll_vertically_until_visible(action) scroll_mul = @platform == "iOS" ? 3 : 1.5 scroll_pause = @platform == "iOS" ? 0.2 : 0.1 if action.key?("SwipeAction") - x_frac = convert_value(action["OffsetFractionX"]) if action.key?("OffsetFractionX") - y_start_frac = convert_value(action["StartFractionY"]) if action.key?("StartFractionY") - y_end_frac = convert_value(action["EndFractionY"]) if action.key?("EndFractionY") - scroll_mul *= convert_value(action["SwipeSpeedMultiplier"]) if action.key?("SwipeSpeedMultiplier") - scroll_pause = convert_value(action["SwipePauseDuration"]) if action.key?("SwipePauseDuration") - end - scroll_timeout = action.key?("ScrollTimeout") ? convert_value(action["ScrollTimeout"]) : 60 + sw_action = action["SwipeAction"] + x_frac = convert_value(sw_action["OffsetFractionX"]).to_f if sw_action.key?("OffsetFractionX") + y_start_frac = convert_value(sw_action["StartFractionY"]).to_f if sw_action.key?("StartFractionY") + y_end_frac = convert_value(sw_action["EndFractionY"]).to_f if sw_action.key?("EndFractionY") + scroll_mul /= convert_value(sw_action["SwipeSpeedMultiplier"]).to_f if sw_action.key?("SwipeSpeedMultiplier") + scroll_pause = convert_value(sw_action["SwipePauseDuration"]).to_f if sw_action.key?("SwipePauseDuration") + end + scroll_timeout = action.key?("ScrollTimeout") ? convert_value(action["ScrollTimeout"]).to_f : 60 # calculate the exact coordinates for swiping, # depending on whether a specific element to swipe on is provided if action.key?("ScrollTarget") - recheck_after_scrolls = action.key?("RecheckAfterScrolls") ? convert_value(action["RecheckAfterScrolls"]) : nil + recheck_after_scrolls = nil + if action["ScrollTarget"].key?("RecheckAfterScrolls") + recheck_after_scrolls = convert_value(action["ScrollTarget"]["RecheckAfterScrolls"]).to_i + end bg_el = wait_for(action["ScrollTarget"]) y_top = bg_el.location.y y_bottom = bg_el.location.y + bg_el.size.height @@ -764,7 +768,7 @@ def scroll_vertically_until_visible(action) # configure FullView parameters # if FullViewOffsetY is provided, assume that FullView is requested action["FullView"] = true if action.key?("FullViewOffsetY") - y_fullview_offset = action.key?("FullViewOffsetY") ? convert_value(action["FullViewOffsetY"]).to_f : 0 + y_fullview_offset = action.key?("FullViewOffsetY") ? convert_value(action["FullViewOffsetY"]).to_i : 0 # start the scrolling/checking loop scrolls = 0 @@ -772,23 +776,21 @@ def scroll_vertically_until_visible(action) while (Time.now - start) < scroll_timeout el = wait_for(action) if el&.displayed? - return unless action.key?("FullView") && convert_value(action["FullView"]) + return unless action.key?("FullView") && convert_value(action["FullView"]) == "true" el_y_loc = el.location.y # save position so it is not retrieved every time - # the condition for full view depends on the scrolling direction (up or down) - el_in_full_view = if y_start_frac > y_end_frac - el_y_loc <= y_top + y_fullview_offset - else - el_y_loc >= y_bottom + y_fullview_offset - end - return if el_in_full_view + # save the target point depending on scroll direction + full_view_target = y_start_frac > y_end_frac ? y_top + y_fullview_offset : y_bottom + y_fullview_offset + # if the following condition is true, the full view swipe has already been executed, so we can return + return if y_end == full_view_target # if element is visible but not in full view, change the scroll endpoints if el_y_loc > y_top && el_y_loc < y_bottom - y_end = y_start_frac > y_end_frac ? y_top : y_bottom + y_end = full_view_target y_start = el_y_loc end end # element not displayed or not in full view - execute swipe action - duration = (y_start - y_end) * scroll_mul / 1000.0 + duration = (y_start - y_end).abs * scroll_mul / 1000.0 + log_info("#{@role}: Scrolling from [#{x_point}, #{y_start}] to [#{x_point}, #{y_end}] for #{duration}s") @driver.action .move_to_location(x_point, y_start) .pointer_down(:left) @@ -810,7 +812,7 @@ def scroll_vertically_until_visible(action) # raise error if timeout exceeded if !original_noraise path = take_error_screenshot - raise "\n#{@role}: Element '#{action["Id"]}' is not visible after scrolling for #{scroll_timeout} " + + raise "\nRole #{@role}: Element '#{action["Id"]}' is not visible after scrolling for #{scroll_timeout} " + "seconds\nError Screenshot: #{path}" end end From 6213e4e9b23f6e8fcfe0a500e69d28b578e4ee66 Mon Sep 17 00:00:00 2001 From: Edgars Eglitis Date: Tue, 11 Jun 2024 14:54:57 +0300 Subject: [PATCH 3/4] minor linting fixes --- lib/core/device.rb | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/lib/core/device.rb b/lib/core/device.rb index d5a21def..310f7927 100644 --- a/lib/core/device.rb +++ b/lib/core/device.rb @@ -800,21 +800,19 @@ def scroll_until_element_visible(action) .perform scrolls += 1 # recalculate the scroll target properties if the recheck variable is set - if action.key?("ScrollTarget") && scrolls == recheck_after_scrolls - bg_el = wait_for(action["ScrollTarget"]) - y_top = bg_el.location.y - y_bottom = bg_el.location.y + bg_el.size.height - x_point = bg_el.location.x + (bg_el.size.width * x_frac) - y_start = y_top + (bg_el.size.height * y_start_frac) - y_end = y_top + (bg_el.size.height * y_end_frac) - end + next unless action.key?("ScrollTarget") && scrolls == recheck_after_scrolls + bg_el = wait_for(action["ScrollTarget"]) + y_top = bg_el.location.y + y_bottom = bg_el.location.y + bg_el.size.height + x_point = bg_el.location.x + (bg_el.size.width * x_frac) + y_start = y_top + (bg_el.size.height * y_start_frac) + y_end = y_top + (bg_el.size.height * y_end_frac) end # raise error if timeout exceeded - if !original_noraise - path = take_error_screenshot - raise "\nRole #{@role}: Element '#{action["Id"]}' is not visible after scrolling for #{scroll_timeout} " + - "seconds\nError Screenshot: #{path}" - end + return if original_noraise + path = take_error_screenshot + raise "\nRole #{@role}: Element '#{action["Id"]}' is not visible after scrolling for #{scroll_timeout} " + + "seconds\nError Screenshot: #{path}" end def swipe_elements(action) From 2684b9195ad29f3367ec800cbfb558be98f79d47 Mon Sep 17 00:00:00 2001 From: Edgars Eglitis Date: Fri, 14 Jun 2024 14:39:03 +0300 Subject: [PATCH 4/4] test: add simple scenario for scroll_until_element_visible --- examples/tests/cases/case_swipe.yaml | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/examples/tests/cases/case_swipe.yaml b/examples/tests/cases/case_swipe.yaml index 35db4b50..cbe636d5 100644 --- a/examples/tests/cases/case_swipe.yaml +++ b/examples/tests/cases/case_swipe.yaml @@ -20,3 +20,18 @@ SwipeElementsTest: Id: '//android.view.View[1]' - Type: sleep Time: 5 + +ScrollUntilElementVisibleTest: + Vars: + USER: localAndroid + Roles: + - Role: $AND_CLI_USER$ + App: PlayStore + Actions: + - Type: click + Role: $AND_CLI_USER$ + Strategy: uiautomator + Id: text("Top charts") + - Type: scroll_until_element_visible + Strategy: uiautomator + Id: text("25")