Adopt Black and isort (SC-700)

[TheRealFalcon]

Proposed Commit Message

Adopt Black and isort

Applied Black and isort, fixed any linting issues, updated tox.ini
and CI.

Additional Context

Black and isort are their own commits, so surrounding changes are easier to review.
Black, isort, flake8, and pylint checks are all green. Unit tests are also green. (at least lcoally)

I actually ran black twice...first with the --experimental-string-processing flag, then without arguments. The flag has been around for a while and should be made default soon. The referenced issues around it seem to be all be fixed, and it seemed to produce better initial results.
psf/black#2188

Checklist:

• My code follows the process laid out in the documentation
• I have updated or added any unit tests accordingly
• I have updated or added any documentation accordingly

[TheRealFalcon]

@blackboxsw , @holmanb only 441 files to review 😬

[blackboxsw]

This looks good @TheRealFalcon. The only oversight black had was for lines that already contained # noqa: E501 and got re-formatted to be less than that max width.

I also think we can change cloudinit/stages.py to avoid to from typing import Dict, Set # noqa: F401 by declaring the type ahead of time. We also need to format the landed distro/debian.py changes. Here's a combined diff of the additional changes I think we can make here:

diff --git a/cloudinit/distros/debian.py b/cloudinit/distros/debian.py
index 399194ca..0105a383 100644
--- a/cloudinit/distros/debian.py
+++ b/cloudinit/distros/debian.py
@@ -50,10 +50,10 @@ LOCALE_CONF_FN = "/etc/default/locale"
 # More context:
 #   https://github.com/canonical/cloud-init/pull/1034#issuecomment-986971376
 APT_LOCK_FILES = [
-    '/var/lib/dpkg/lock-frontend',
-    '/var/lib/dpkg/lock',
-    '/var/cache/apt/archives/lock',
-    '/var/lib/apt/lists/lock',
+    "/var/lib/dpkg/lock-frontend",
+    "/var/lib/dpkg/lock",
+    "/var/cache/apt/archives/lock",
+    "/var/lib/apt/lists/lock",
 ]
 
 
diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py
index 2de2feb2..a8b403e8 100755
--- a/cloudinit/sources/DataSourceAzure.py
+++ b/cloudinit/sources/DataSourceAzure.py
@@ -466,7 +466,7 @@ class DataSourceAzure(sources.DataSource):
             )
             crawled_data["metadata"][
                 "disable_password"
-            ] = imds_disable_password  # noqa: E501
+            ] = imds_disable_password
 
         if metadata_source == "IMDS" and not crawled_data["files"]:
             try:
@@ -1620,7 +1620,7 @@ def _disable_password_from_imds(imds_data):
         return (
             imds_data["compute"]["osProfile"]["disablePasswordAuthentication"]
             == "true"
-        )  # noqa: E501
+        )
     except KeyError:
         return None
 
diff --git a/cloudinit/sources/__init__.py b/cloudinit/sources/__init__.py
index a3463325..9083f399 100644
--- a/cloudinit/sources/__init__.py
+++ b/cloudinit/sources/__init__.py
@@ -745,7 +745,7 @@ class DataSource(CloudInitPickleMixin, metaclass=abc.ABCMeta):
             for (
                 update_scope,
                 update_events,
-            ) in self.supported_update_events.items():  # noqa: E501
+            ) in self.supported_update_events.items():
                 if event in update_events:
                     if not supported_events.get(update_scope):
                         supported_events[update_scope] = set()
diff --git a/cloudinit/sources/helpers/digitalocean.py b/cloudinit/sources/helpers/digitalocean.py
index 0537885f..dcff110c 100644
--- a/cloudinit/sources/helpers/digitalocean.py
+++ b/cloudinit/sources/helpers/digitalocean.py
@@ -16,8 +16,9 @@ LOG = logging.getLogger(__name__)
 
 
 def assign_ipv4_link_local(distro, nic=None):
-    """Bring up NIC using an address using link-local (ip4LL) IPs. On
-    DigitalOcean, the link-local domain is per-droplet routed, so there
+    """Bring up NIC using an address using link-local (ip4LL) IPs.
+
+    On DigitalOcean, the link-local domain is per-droplet routed, so there
     is no risk of collisions. However, to be more safe, the ip4LL
     address is random.
     """
diff --git a/cloudinit/stages.py b/cloudinit/stages.py
...skipping...
 from collections import namedtuple
-from typing import Dict, Set  # noqa: F401
+from typing import Dict, Set
 
 from cloudinit import cloud, config, distros, handlers, helpers, importer
 from cloudinit import log as logging
@@ -57,12 +57,11 @@ def update_event_enabled(
     case, we only have the data source's `default_update_events`,
     so an event that should be enabled in userdata may be denied.
     """
-    default_events = (
-        datasource.default_update_events
-    )  # type: Dict[EventScope, Set[EventType]]    # noqa: E501
-    user_events = userdata_to_events(
-        cfg.get("updates", {})
-    )  # type: Dict[EventScope, Set[EventType]]  # noqa: E501
+    default_events: Dict[EventScope, Set[EventType]]
+    default_events = datasource.default_update_events
+    user_events: Dict[EventScope, Set[EventType]]
+    user_events = userdata_to_events(cfg.get("updates", {}))
+
     # A value in the first will override a value in the second
     allowed = util.mergemanydict(
         [

[TheRealFalcon]

I also think we can change cloudinit/stages.py to avoid to from typing import Dict, Set # noqa: F401 by declaring the type ahead of time.

Not 3.5 compatible, so let's leave it for now.

I believe I addressed all of your other comments.