Skip to content

Conversation

@booxter
Copy link
Contributor

@booxter booxter commented Sep 15, 2025

To reproduce (not tested, just the gist):

$ NIXPKGS=~/src/nixpkgs
$ gh repo clone technowledgy/refactor-tractor
$ cd refactor-tractor
$ gh pr checkout 15
$ devenv shell
(devenv) remove-meta-with-lib $NIXPKGS --write
$ cd $NIXPKGS && git commit -a -m "treewide: clean up 'meta = with' pattern"

A machine with lots of cores is recommended because nixpkgs-aware variable analysis demands significant compute power to plow through all .nix files in-tree in reasonable time.

Things done

  • Built on platform:
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested, as applicable:
  • Ran nixpkgs-review on this PR. See nixpkgs-review usage.
  • Tested basic functionality of all binary files, usually in ./result/bin/.
  • Nixpkgs Release Notes
    • Package update: when the change is major or breaking.
  • NixOS Release Notes
    • Module addition: when adding a new NixOS module.
    • Module update: when the change is significant.
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

Add a 👍 reaction to pull requests you find important.

@booxter
Copy link
Contributor Author

booxter commented Sep 15, 2025

Not sure why nixf-diagnose --auto-fix doesn't pick up a lot of unused meta = with ... patterns. Also eval seems to fail, will take a look later.

@wolfgangwalther
Copy link
Contributor

Not sure why nixf-diagnose --auto-fix doesn't pick up a lot of unused meta = with ... patterns. Also eval seems to fail, will take a look later.

I think you'll need to specify a long disable list for nixf-diagnose otherwise, it will run all the other checks (and auto-fixes) as well, right? Maybe some other check broke something.

I wonder whether we can pin the right version of nixf-diagnose in the refactor-tractor repo and add it to the lib-prefix command (turning it into remove-meta-with-lib or so), aka doing both steps at once?

Then it would be fully reproducible, and runnable via a single flake reference - which, importantly, is outside the nixpkgs repo, which makes it much easier to use when doing the staging merges later.

@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 15, 2025
@booxter booxter force-pushed the add-lib-prefixes-in-meta branch from 1d7df12 to 797cbf4 Compare September 18, 2025 01:02
@nixpkgs-ci nixpkgs-ci bot added 2.status: merge conflict This PR has merge conflicts with the target branch and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Sep 18, 2025
@booxter booxter force-pushed the add-lib-prefixes-in-meta branch 2 times, most recently from 0c5fb39 to aab635e Compare September 20, 2025 16:00
@nixpkgs-ci nixpkgs-ci bot added 2.status: merge conflict This PR has merge conflicts with the target branch and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Sep 20, 2025
@booxter booxter force-pushed the add-lib-prefixes-in-meta branch from aab635e to a521ebe Compare September 29, 2025 02:20
@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 6.topic: emacs Text editor 6.topic: vim Advanced text editor 6.topic: nodejs Node.js is a free, open-source, cross-platform JavaScript runtime environment 6.topic: hardware Drivers, Firmware and Kernels 6.topic: coq A formal proof management system 6.topic: pantheon The Pantheon desktop environment 6.topic: cinnamon Desktop environment 6.topic: docker tools Open-source software for deploying and running of containerized applications 6.topic: R R is a programming language for statistical computing and data visualization. 6.topic: vscode A free and versatile code editor that supports almost every major programming language. 6.topic: jupyter Interactive computing tooling: kernels, notebook, jupyterlab and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Sep 29, 2025
@booxter booxter force-pushed the add-lib-prefixes-in-meta branch from d6ab4be to 8676f84 Compare November 25, 2025 03:56
@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 25, 2025
@booxter
Copy link
Contributor Author

booxter commented Nov 25, 2025

Generated using 80c42444a7613782290df3ab5d2e346805620071 commit from: https://github.com/technowledgy/refactor-tractor/pull/15/commits

Changes:

  • hydraPlatforms and badPlatforms are now also handled;
  • variables bound inside lets are now considered, avoiding spurious overrides when their names match our heuristic patterns like platforms;
  • all files with meta = with lib; are now processed, even when no ast-grep modifications happened;
  • nixpkgs used for variable resolution is now pulled from the same directory as the target for modification (not from system <nixpkgs>).
$ git grep 'meta = with lib;' | grep '.nix:' | wc -l
90

$ ./diff_all_meta.sh pre post
== Diffing x86_64-linux ==

== Diffing x86_64-darwin ==

== Diffing aarch64-linux ==

== Diffing aarch64-darwin ==

No meta changes detected.

@booxter
Copy link
Contributor Author

booxter commented Nov 25, 2025

I've checked ~half of remaining pattern occurrences and all of them had a good explanation for why they were not picked up by automation (mostly just actual usage of functions from lib that we'll clean up manually). I got tired so I haven't checked all.

@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 26, 2025
@philiptaron
Copy link
Contributor

@wolfgangwalther, are we getting close to merge? I think this would be nice to land.

@wolfgangwalther
Copy link
Contributor

Not quite at the top of my list, yet, but it's getting there. I certainly have not forgotten and would like to finish it off soon.

This commit was created by a combination of scripts and tools:
- an ast-grep script to prefix things in meta with `lib.`,
- a modified nixf-diagnose / nixf combination to remove unused `with
lib;`, and
- regular nixfmt.

Co-authored-by: Wolfgang Walther <walther@technowledgy.de>
@wolfgangwalther wolfgangwalther force-pushed the add-lib-prefixes-in-meta branch from 8676f84 to 567e8df Compare December 10, 2025 17:10
@wolfgangwalther wolfgangwalther removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 10, 2025
@wolfgangwalther
Copy link
Contributor

wolfgangwalther commented Dec 10, 2025

Thank you, @booxter, for dealing with all my requests. This is great work, I'll follow through with it now.

Not doing it for 25.05 anymore, because that's close to EOL.

@wolfgangwalther wolfgangwalther added this pull request to the merge queue Dec 10, 2025
Merged via the queue into NixOS:master with commit af6d3ec Dec 10, 2025
24 of 25 checks passed
@booxter
Copy link
Contributor Author

booxter commented Dec 10, 2025

image

@wolfgangwalther thanks a lot for support with the PR. It was a pleasure working with you on the change and I learned a lot. ❤️

@philiptaron
Copy link
Contributor

Wow! This is awesome! Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: cinnamon Desktop environment 6.topic: coq A formal proof management system 6.topic: COSMIC COSMIC is a software platform for designing beautiful user experiences 6.topic: docker tools Open-source software for deploying and running of containerized applications 6.topic: emacs Text editor 6.topic: hardware Drivers, Firmware and Kernels 6.topic: jupyter Interactive computing tooling: kernels, notebook, jupyterlab 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: nodejs Node.js is a free, open-source, cross-platform JavaScript runtime environment 6.topic: pantheon The Pantheon desktop environment 6.topic: R R is a programming language for statistical computing and data visualization. 6.topic: vim Advanced text editor 6.topic: vscode A free and versatile code editor that supports almost every major programming language. 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants