RFC/WIP: adopt ruff and its code upgrade/format methods

[RonnyPfannschmidt]

this starts with the isort and pyugprade rules + flake8 based rules
some warnings are resolved - about 141 more to go

i stumbled across a number of leftovers from historic python2 support

i'd like to get some input on whether adding type annotations would be appreciated or frowned upon
i'd like to get input on whether this type of project modernization would be appreciated or not before digging into the leftover linter warnings

[tobixen]

I haven't looked through all of it, but I give it thumbs up. I should probably modernize my python skills myself one of those days :-)

[RonnyPfannschmidt]

lovely, i'll gradually work trough the rest then as i want to use the library for modification of caldav calendars and want to give back

[tobixen]

... but I should add, I find backward compatibility very important, as well as support for old python versions. I even believe it's a nice thing to support python versions that are already officially "EOL", as those python versions often are still included in distros that are not EOL. Though, python2 is officially not supported anymore, and there was a pull request removing python 3.6 from the test matrix slipping through some weeks ago :-)

[RonnyPfannschmidt]

While personally I prefer to shed dead snakes as dropping support code and adopting new features tends to clean up and Speed up code, I'm absolutely on board with keeping them for projects that consider them important

In those cases I strongly appreciate a definition of the intended support range in order to avoid endless support for dead snakes (I love the win some of the newer versions enable)

[tobixen]

Here are my thoughts:

• As a bare minimum, all python versions that are officially supported by upstream should be supported
• Shedding of old python version should be well announced in the CHANGELOG.md file
• Preferably, shedding of old python versions should only be done when changing the major version number of the library (I did have great plans for the 2.0-version of this library, but perhaps I will need to procrastinate it until 8.0 or something) :-)
• It's a nice thing to keep an overview of python versions used by the popular Linux distros that aren't EOL'ed yet, and consider that before shedding support for old python versions
• We try to avoid "schrödingers support". It's a nice thing to toss a warning or error if the python version is not officially supported, rather than something suddenly breaking out of the blue in a production environment.

[tobixen]

So this will go into a 2.0-release, which will officially break with 3.7 (and possibly 3.8 as well). I cannot give any ETA on a 2.0-release currently, but possibly in some few months.

[tobixen]

Now that 1.4.0 has been released, we may consider shedding support for python 3.7 and python 3.8 and work towards a 2.0-release.

There are references to python 3.7 in docs/source/index.rst, pyproject.toml and .github/workflows/tests.yaml. If those are fixed, I believe the tests will pass, and this work may be merged into main.

Post-edit: there is also tox.ini.

[RonnyPfannschmidt]

i started picking this up again - there are still dozens of warnings, some dead code and more

[RonnyPfannschmidt]

im particularly confused around the numbered variables in tests, of which some are unused. some are missed all together

[tobixen]

im particularly confused around the numbered variables in tests, of which some are unused. some are missed all together

The functional tests are more or less a complete mess, developed organically over time. I've considered doing something with it, but ... it's not a high priority. An idea could be to rename it into test_legacy_ftest.py or something, and then make new test code from scratch (and deleting the legacy tests once the new tests are well enough done).

One of the problems with the tests is that many of them are not really testing the library code per se, but rather the servers. I started today writing up some new code in the test directory, check_server_compatibility.py, I will push it in a separate branch as soon as it's doing anything useful. The purpose is to make a tool that can be run from the command line towards any server and spit out a list of "quirks". The tests today work the other way around, one have to provide the expected "quirks" and then the tests will break if the list is incorrect.

[tobixen]

Ubuntu 20.04 Focal uses Python 3.8, but it goes EOL by the end of May. RHEL 8 is supported until 2029 and ships with Python 3.6 - while RHEL 9 comes with Python 3.9. At least that's what I find through a five minute web search, no warranties given that it's correct.

At some point support for 3.6 was already shedded (I was not careful enough when accepting the pull request), meaning that we already do not support RHEL 8.

I think I'm ready to shed support for python 3.7 and python 3.8 in the main/master branch now, and make a 2.0-release around the end of May.

[tobixen]

@RonnyPfannschmidt - you

im particularly confused around the numbered variables in tests, of which some are unused. some are missed all together

Is this the data in tests/test_caldav.py like ev1, broken_ev1, ev2, etc?

I see that in some other python projects handling icalendar data, test ics data was provided as separate files with descriptive file names, perhaps that's a better way to handle it? Perhaps a starting point could be to give the variables better names and do some search'n'replace here. Do you have any thoughts on it?

[RonnyPfannschmidt]

I haven't researched how to clean it up nicely

I'll shedule something for today

[tobixen]

I'm getting more and more annoyed with the objects.py being really large, so I'd like to split it up into several files. However, that would mess up all outstanding pull requests - so I'm working hard on closing them. What's the status with this one, @RonnyPfannschmidt ? :-)

[RonnyPfannschmidt]

Break me i can reamake it easily

[tobixen]

thanks

[tobixen]

I just released version 1.6 of the caldav library - this will definitively be the last minor release before 2.0. I will remove python 3.7 and python 3.8 from the testing matrix in the master branch now.

[RonnyPfannschmidt]

so i can rebase/recreate this now?

[tobixen]

Yes. Next on my list is documentation, so most likely I won't be touching the code base a lot over the next days.

[tobixen]

A significant amount of the documentation is fetched from the docstrings and even from the type annotations. I'm intending to go over the docstrings now. Please indicate when/if you start working with this so I can freeze my work.

[RonnyPfannschmidt]

Im stretched ATM

I'll restart with a more minimal approach and then land small non disruptive bits instead of big bang

[tobixen]

I think I also prefer to chew on many small pull requests rather than one big :-)

[tobixen]

I will handle this myself, already set up ruff with the icalendar-searcher subproject. An issue #589 has been set up for it.