Skip to content

Conversation

@stes
Copy link
Member

@stes stes commented Oct 20, 2024

Update: The root issue was numpy 2. This is a recent upgrade (June 2024), and it seems like many of the packages required for cebra do not easily work with npy2 (yet). Errors included pandas/pytables, but also torch. It would be probably possible to work around these errors, but I think it is not worth yet --- in half a year or so, the situation is probably more stable, and we can remove the pinning.

This PR now does the following changes:

  • Pin numpy below v2.
  • Remove nlb_tools to allow use of more recent pandas versions. This will only affect users who want to use the monkey dataset. We provide guidance on how to post-hoc install the package directly via a warning for these users.
  • Fix a minor issue in the plotly code, which arises because we now use more recent versions of plotly.

I propose to merge as-is. From the user perspective, it is slightly annoying that CEBRA does not run with numpy 2 for now, but given the errors caused in other upstream packages, chances are that no one uses numpy 2 (yet) in the first place. We just still aim to remove the pinning again as soon as possible.


Old description:

This PR aims to solve some of the broken tests we saw in recent PRs (e.g. #166 #183 #175 #180 ). My hypothesis is that the nlb_tools package which we require for the test workflow right now is the root cause, which cases pandas to be downgraded to 1.3.4 (https://github.com/neurallatents/nlb_tools/blob/main/requirements.txt#L1), while the current pandas version is 2.2.3 (https://pypi.org/project/pandas/).

The build issue was a pandas <> numpy version mismatch (numpy was >2, pandas at 1.3.4 before), so hopefully this fixes the build.

I removed this depedency which causes more recent versions of all packages to be installed. This caused some plotly tests to break, which now do stricter checking on the colormaps. I also updated one of the helper functions there.

Let's see if this passes as-is on github, local tests look good.

@cla-bot cla-bot bot added the CLA signed label Oct 20, 2024
@stes
Copy link
Member Author

stes commented Oct 20, 2024

relevant packages in failing build:

h5py-3.12.1 (latest: 3.12.1)
pandas-2.2.3 (latest: 2.2.3)
numpy-2.0.2 (latest: 2.1.2)

@stes
Copy link
Member Author

stes commented Oct 20, 2024

Our particular error was also discussed here, it seems like some of the packages (pandas, pytables, ...) simply do not work yet with numpy 2 which was released this june. testing what happens if we include this as a constraint.

@stes
Copy link
Member Author

stes commented Oct 20, 2024

Related issue from the pytorch docs: pytorch/pytorch#107302 (seems merged though)

@stes stes requested a review from MMathisLab October 20, 2024 15:13
@stes
Copy link
Member Author

stes commented Oct 20, 2024

@MMathisLab this is ready for review, and can be merged from my end. I updated the PR description with my line of thinking, if you agree feel free to merge.

Copy link
Member

@MMathisLab MMathisLab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much @stes

@MMathisLab MMathisLab merged commit 7b2bdb4 into main Oct 20, 2024
@stes stes deleted the stes/fix-build branch October 20, 2024 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants