[AIRFLOW-6055] Option for exponential backoff in Sensors

[msumit]

Make sure you have checked all steps below.

Jira

• My PR addresses the following Airflow Jira issue https://issues.apache.org/jira/browse/AIRFLOW-6055

Description

• Adding an option in sensors to do exponential backoff before poke or schedule again.

Tests

• Added a couple of unit tests around the new function

Commits

• My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters (not including Jira issue reference)
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

Documentation

• In case of new functionality, my PR adds documentation that describes how to use it.
  • All the public functions and the classes in the PR contain docstrings that explain what it does
  • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

[codecov-io]

Codecov Report

Merging #6654 into master will decrease coverage by 0.32%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master   #6654      +/-   ##
=========================================
- Coverage   83.82%   83.5%   -0.33%     
=========================================
  Files         672     672              
  Lines       37560   37576      +16     
=========================================
- Hits        31483   31376     -107     
- Misses       6077    6200     +123

Impacted Files	Coverage Δ
airflow/sensors/base_sensor_operator.py	98.57% <100%> (+0.42%)	⬆️
airflow/kubernetes/volume_mount.py	44.44% <0%> (-55.56%)	⬇️
airflow/kubernetes/volume.py	52.94% <0%> (-47.06%)	⬇️
airflow/kubernetes/pod_launcher.py	45.25% <0%> (-46.72%)	⬇️
airflow/kubernetes/refresh_config.py	50.98% <0%> (-23.53%)	⬇️
...rflow/contrib/operators/kubernetes_pod_operator.py	78.2% <0%> (-20.52%)	⬇️
airflow/configuration.py	89.13% <0%> (-3.63%)	⬇️
airflow/utils/dag_processing.py	57.99% <0%> (-0.5%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db4be19...a1eab04. Read the comment docs.

[ashb]

Here too.

[msumit]

problem is that this method calculates sha1, which is based upon the started_at, which is relative to the current time.

[ashb]

In which case use mocking to make timezone.utcnow return a known constant value in the tests.

Or if the "jitter" caused by this is small then

self.assertGreaterThanEqual(interval1, 5)
self.assertLessThanEqual(interval1, 5.5)

self.assertGreaterThanEqual(interval2, 6)
self.assertLessThanEqual(interval2, 7)

Those values are not right, but this is the sort of thing I'd like to see.

[msumit]

mocked timezone

[ashb]

You mocked the timezone, but didn't add the extra tests that I asked for. Please could you add them?