Do we need to track cython .c files?

[mnmelo]

Whenever I build MDAnalysis cython regenerates some *.c files in /lib/ and /lib/formats/. This messes up my commits as a number of files I didn't specifically change appear as modified.

The question is: do we even need these files around? Can we just leave them out (and untracked) and let cython recreate them every time a build is issued?

[hainm]

Given that it is super easy to install cython, I dont think cythonized files are needed.

(And given that mdanalysis requires other packages too)

[jbarnoud]

For the record, the issue was discussed in issue #66 and cython files got included to remove the dependency to cython.

In my opinion, you should now what you are doing if you install MDAnalysis from the develop branch. I would remove the C files from the develop branch but keep them in the pip package, and perhaps in the master branch.

[mnmelo]

I knew I should have dug deeper in the old issues... Anyway, my question then is, if they're distributed so that cython isn't needed what triggers their regeneration by cython? Shouldn't setup.py just rely on them being there?

Or is it then a facultative dependency thing, where if we have cython we can use it to get some improvement otherwise we fallback to the distributed .c files? (I saw that some of the changes to the .c files seem to be architecture-dependent optimizations.)

[richardjgowers]

You can definitely force the rebuilding just by touching the files, but yeah sometimes it seems that they're rebuilt because of versions of Cython being different too. FWIW, I'd rather keep the c files just to not have Cython as a hard dependency.

[mnmelo]

Yup, I agree on the dependency part, but wouldn't it be better then to leave them untracked, and then rebuild them for each release? (Like the docs regeneration?)

[dotsdl]

I agree with @hainm ; these days Cython isn't a scary dependency, and since most users are likely installing with Anaconda anyway, they already have it. I say we cut the amount of generated stuff we're committing to the repo's main branches and leave out the cython-generated code.

[mnmelo]

I'm not sure @orbeckst will agree... I still vote for not tracking but rebuilding and releasing, like the docs.

[dotsdl]

@mnmelo I like that solution. I think rebuilding is something the package maintainer / release manager has to do anyway.

[mnmelo]

@richardjgowers, would you be ok with adding the cython rebuild to the release workflow?

[orbeckst]

On 28 Jan, 2016, at 11:26, David Dotson wrote:

@mnmelo I like that solution. I think rebuilding is something the package maintainer / release manager has to do anyway.

I also find the small changes in the cythonized files annoying. In principle, anyone using develop can be expected to install cython; as @hainm points out, it's not a big deal anymore.

I would be ok with building the cython files as part of release. They should also end up on master, don't you think?

It will change our "minimal builds" in that they are not THAT minimal anymore.

Oliver

[mnmelo]

@orbeckst, yup: master and releases makes sense.

This will also make cythonized files more consistent. I just opened a PR (#678) where I change code in one of the .pyx files. I'm also committing the respective cythonized .c file. Only now its Cython metadata block differs from all other cythonized files.

[kain88-de]

yes having them in the release tarball is very useful. I wouldn't want to give that up. It makes sure that we have some control over the cython generated files

[hainm]

@orbeckst if mdanalysis can be released from conda build, you don't even care about cython. 👍
(install should take only 10s for user)