ui: dvc startup became slow

[Suor]

$ time dvc --version
0.57.0+e3f6c7

real	0m2.186s
user	0m1.834s
sys	0m0.288s

[efiop]

Might be imports again. Need to check cprofile report, it will make it clear.

[Suor]

So it's imports:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
4903/2532    0.004    0.000    0.471    0.000 <frozen importlib._bootstrap>:1009(_handle_fromlist)
     1802    0.003    0.000    0.004    0.000 <frozen importlib._bootstrap>:103(release)
     1330    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:143(__init__)
     1330    0.001    0.000    0.018    0.000 <frozen importlib._bootstrap>:147(__enter__)
     1330    0.001    0.000    0.004    0.000 <frozen importlib._bootstrap>:151(__exit__)
     1802    0.012    0.000    0.015    0.000 <frozen importlib._bootstrap>:157(_get_module_lock)
     1317    0.002    0.000    0.002    0.000 <frozen importlib._bootstrap>:176(cb)
      472    0.001    0.000    0.003    0.000 <frozen importlib._bootstrap>:194(_lock_unlock_module)
   1597/4    0.001    0.000    1.011    0.253 <frozen importlib._bootstrap>:211(_call_with_frames_removed)
    17493    0.004    0.000    0.004    0.000 <frozen importlib._bootstrap>:222(_verbose_message)
       23    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:232(_requires_builtin_wrapper)
      169    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:269(_module_repr)
     1242    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:307(__init__)
     1242    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:311(__enter__)
     1242    0.002    0.000    0.005    0.000 <frozen importlib._bootstrap>:318(__exit__)
     4954    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:321(<genexpr>)
     1190    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:35(_new_module)
     2461    0.002    0.000    0.002    0.000 <frozen importlib._bootstrap>:369(__init__)
     2405    0.001    0.000    0.012    0.000 <frozen importlib._bootstrap>:403(cached)
     2554    0.001    0.000    0.002    0.000 <frozen importlib._bootstrap>:416(parent)

[efiop]

Need to research which modules are causing us the most delay (e.g. boto, paramiko and other remote libs or something else?). And how to solve it? (e.g. simple dynamic imports or writing our own lazy_import implementation or what else?)

[Suor]

Timed imports:

553.1 ms dvc.cli
534.1 ms dvc.repo
161.7 ms dvc.remote.oss
123.8 ms dvc.remote.hdfs
92.6 ms oss2.api
90.3 ms oss2.crypto
88.6 ms dvc.remote.azure
84.4 ms azure.storage.blob
83.5 ms dvc.remote.gs
83.2 ms azure.storage.blob.appendblobservice
66.7 ms pyarrow
65.8 ms oss2.utils
62.8 ms azure.storage.common.tokencredential
54.5 ms pyarrow.lib
44.4 ms google.cloud.storage.client
44.1 ms google.cloud.client
42.4 ms urllib3.connectionpool
38.0 ms dvc.remote.s3
37.4 ms boto3.session
35.5 ms Crypto.IO
31.8 ms google.cloud.storage.batch
25.2 ms google.cloud
24.0 ms Crypto.Cipher._mode_ecb
23.4 ms urllib3.packages.rfc3986.normalizers
23.2 ms urllib3.packages.rfc3986.api
22.5 ms urllib3.packages.rfc3986.iri
21.8 ms Crypto.Util._raw_api
21.7 ms Crypto.Math.Numbers
21.3 ms Crypto.Math._IntegerGMP
18.1 ms cryptography.hazmat.backends.openssl.backend
17.0 ms cryptography.x509.base
16.6 ms dvc.remote.ssh
16.4 ms cryptography.x509.extensions
15.8 ms Crypto.IO._PBES
15.3 ms urllib3.connection
13.8 ms paramiko.transport
12.5 ms urllib3.util.ssl_
12.0 ms botocore.compat
11.6 ms asn1crypto.keys
10.7 ms Crypto.Cipher._mode_siv
10.6 ms botocore.exceptions
10.5 ms Crypto.Protocol.KDF