Pruning the dependency tree for initial Pairwise comparisons

[ccerv1]

Problem

The current dependency graph is too large for efficient Pairwise voting, so we need to prune it back to something more manageable for a person putting in, say, up to 5 minutes per comparison.

---

Proposed Solution

We can use heuristics to prune the dependency graph to a more manageable size, capping the number of dependencies to a maximum of 30 per seed node.

The pruning process relies on the following heuristics:

1. Funding History
   Prioritizes repositories associated with projects that have received funding from platforms such as Gitcoin, Optimism, or Open Collective.

2. Language Matching
   Retains dependencies written in the same programming language as the dependent repository.

   • Most effective for languages like TypeScript/JavaScript, Rust, Go, and Python.
   • Currently less effective for languages like C, Java, and Nim.

3. Maintainer Identity
   Prioritizes repositories within a relevant namespace that align closely with the seed node’s dependencies.

   • Example: In the libp2p namespace, both libp2p/go-libp2p and libp2p/go-libp2p-asn-util exist.
     In effect, this should retain libp2p/go-libp2p as the more relevant repo and prune a less central one like libp2p/go-libp2p-asn-util.

4. Star Counts
   As a fallback, repositories are ranked by star counts when the above heuristics are insufficient to reduce dependencies below the desired cap.

[ccerv1]

Results

#6

This is the result from applying the methodology described above. You can see the output here.

• ethereum/
  • ethereum/solidity | C++ | 3/3 dependencies | 3 pairwise combinations (vs 3 previously)
    • [1] readthedocs/sphinx_rtd_theme | n/a (not indexed yet)
    • [2] sphinx-doc/sphinx | Python
    • [3] taminomara/sphinx-a4doc | n/a (not indexed yet)
  • vyperlang/vyper | Python | 9/9 dependencies | 36 pairwise combinations (vs 36 previously)
    • [1] agronholm/cbor2 | n/a (not indexed yet)
    • [2] executablebooks/sphinx-copybutton | n/a (not indexed yet)
    • [3] gristlabs/asttokens | n/a (not indexed yet)
    • [4] legrandin/pycryptodome | n/a (not indexed yet)
    • [5] lepture/shibuya | n/a (not indexed yet)
    • [6] pypa/packaging | Python
    • [7] pypa/wheel | Python
    • [8] python/importlib_metadata | n/a (not indexed yet)
    • [9] sphinx-doc/sphinx | Python
  • ethereum/py-evm | Python | 11/11 dependencies | 55 pairwise combinations (vs 55 previously)
    • [1] ethereum/eth-hash | Python
    • [2] hypothesisworks/hypothesis | n/a (not indexed yet)
    • [3] ipython/ipython | n/a (not indexed yet)
    • [4] peritus/bumpversion | n/a (not indexed yet)
    • [5] pycqa/flake8 | n/a (not indexed yet)
    • [6] pytest-dev/pytest | Python
    • [7] pytest-dev/pytest-asyncio | Python
    • [8] pytest-dev/pytest-cov | Python
    • [9] pytest-dev/pytest-xdist | Python
    • [10] python/mypy | n/a (not indexed yet)
    • [11] tox-dev/tox | n/a (not indexed yet)
  • ethereum/web3.py | Python | 13/13 dependencies | 78 pairwise combinations (vs 78 previously)
    • [1] aaugustin/websockets | n/a (not indexed yet)
    • [2] aio-libs/aiohttp | n/a (not indexed yet)
    • [3] ethereum/eth-abi | Python
    • [4] ethereum/eth-account | Python
    • [5] ethereum/eth-hash | Python
    • [6] ethereum/eth-typing | Python
    • [7] ethereum/eth-utils | Python
    • [8] ethereum/hexbytes | Python
    • [9] mlodewijck/pyunormalize | n/a (not indexed yet)
    • [10] psf/requests | n/a (not indexed yet)
    • [11] pydantic/pydantic | n/a (not indexed yet)
    • [12] python/typeshed | n/a (not indexed yet)
    • [13] python/typing_extensions | n/a (not indexed yet)
  • paradigmxyz/reth | Rust | 30/463 dependencies | 435 pairwise combinations (vs 106953 previously)
    • [1] alloy-rs/core [+7 other relevant repos in same namespace] | Rust
    • [2] amanieu/parking_lot [+2 other relevant repos in same namespace] | Rust
    • [3] bheisler/criterion.rs [+1 other relevant repos in same namespace] | Rust
    • [4] bluealloy/revm | Rust
    • [5] boa-dev/boa [+1 other relevant repos in same namespace] | Rust
    • [6] clap-rs/clap | Rust
    • [7] crossbeam-rs/crossbeam | Rust
    • [8] dtolnay/thiserror [+15 other relevant repos in same namespace] | Rust
    • [9] getsentry/symbolic [+1 other relevant repos in same namespace] | Rust
    • [10] google/zerocopy | Rust
    • [11] hyperium/hyper [+6 other relevant repos in same namespace] | Rust
    • [12] jeltef/derive_more | Rust
    • [13] libp2p/rust-libp2p | Rust
    • [14] microsoft/windows-rs | Rust
    • [15] peternator7/strum | Rust
    • [16] plotters-rs/plotters | Rust
    • [17] quinn-rs/quinn | Rust
    • [18] rayon-rs/rayon | Rust
    • [19] retep998/winapi-rs | Rust
    • [20] rust-lang/cargo [+14 other relevant repos in same namespace] | Rust
    • [21] rust-phf/rust-phf | Rust
    • [22] rust-random/rand [+2 other relevant repos in same namespace] | Rust
    • [23] rustcrypto/hashes [+14 other relevant repos in same namespace] | Rust
    • [24] rustwasm/wasm-bindgen [+1 other relevant repos in same namespace] | Rust
    • [25] serde-rs/serde [+1 other relevant repos in same namespace] | Rust
    • [26] servo/rust-smallvec [+4 other relevant repos in same namespace] | Rust
    • [27] tikv/pprof-rs [+1 other relevant repos in same namespace] | Rust
    • [28] tokio-rs/tokio [+8 other relevant repos in same namespace] | Rust
    • [29] tower-rs/tower [+1 other relevant repos in same namespace] | Rust
    • [30] unicode-org/icu4x | Rust
  • consensys/teku | Java | 30/137 dependencies | 435 pairwise combinations (vs 9316 previously)
    • [1] acornjs/acorn [+1 other relevant repos in same namespace] | JavaScript
    • [2] ajv-validator/ajv | TypeScript
    • [3] alexei/sprintf.js | JavaScript
    • [4] babel/babel | TypeScript
    • [5] caolan/async | JavaScript
    • [6] chalk/chalk [+5 other relevant repos in same namespace] | JavaScript
    • [7] colorjs/color-name | JavaScript
    • [8] debug-js/debug | JavaScript
    • [9] enquirer/enquirer | JavaScript
    • [10] epoberezkin/fast-deep-equal [+2 other relevant repos in same namespace] | JavaScript
    • [11] eslint/eslint [+4 other relevant repos in same namespace] | JavaScript
    • [12] gulpjs/glob-parent | JavaScript
    • [13] humanwhocodes/object-schema [+1 other relevant repos in same namespace] | JavaScript
    • [14] isaacs/node-glob [+9 other relevant repos in same namespace] | TypeScript
    • [15] jprichardson/node-fs-extra [+1 other relevant repos in same namespace] | JavaScript
    • [16] jquery/esprima | TypeScript
    • [17] kaelzhang/node-ignore | JavaScript
    • [18] ljharb/concat-map | JavaScript
    • [19] lodash/lodash | JavaScript
    • [20] mathiasbynens/emoji-regex [+1 other relevant repos in same namespace] | JavaScript
    • [21] mysticatea/regexpp [+1 other relevant repos in same namespace] | TypeScript
    • [22] node-fetch/node-fetch [+1 other relevant repos in same namespace] | JavaScript
    • [23] nodeca/js-yaml [+1 other relevant repos in same namespace] | JavaScript
    • [24] npm/node-semver [+3 other relevant repos in same namespace] | JavaScript
    • [25] qix-/color-convert | JavaScript
    • [26] sindresorhus/type-fest [+31 other relevant repos in same namespace] | TypeScript
    • [27] tj/commander.js | JavaScript
    • [28] vercel/ms | TypeScript
    • [29] visionmedia/node-progress | JavaScript
    • [30] webreflection/flatted | JavaScript
  • prysmaticlabs/prysm | Go | 30/244 dependencies | 435 pairwise combinations (vs 29646 previously)
    • [1] crate-crypto/go-kzg-4844 | Go
    • [2] etcd-io/bbolt | Go
    • [3] ethereum/go-ethereum [+1 other relevant repos in same namespace] | Go
    • [4] fatih/color | Go
    • [5] fsnotify/fsnotify | Go
    • [6] getsentry/sentry-go | Go
    • [7] go-playground/validator [+2 other relevant repos in same namespace] | Go
    • [8] golang-jwt/jwt | Go
    • [9] golang/groupcache [+3 other relevant repos in same namespace] | Go
    • [10] google/pprof [+4 other relevant repos in same namespace] | Go
    • [11] gorilla/websocket | Go
    • [12] grpc/grpc-go | Go
    • [13] ipfs/go-cid [+1 other relevant repos in same namespace] | Go
    • [14] json-iterator/go | Go
    • [15] kubernetes/client-go [+3 other relevant repos in same namespace] | Go
    • [16] libp2p/go-libp2p [+11 other relevant repos in same namespace] | Go
    • [17] miekg/dns | Go
    • [18] mitchellh/mapstructure [+2 other relevant repos in same namespace] | Go
    • [19] onsi/ginkgo [+1 other relevant repos in same namespace] | Go
    • [20] patrickmn/go-cache | Go
    • [21] pion/webrtc [+15 other relevant repos in same namespace] | Go
    • [22] pkg/errors | Go
    • [23] prysmaticlabs/gohashtree [+4 other relevant repos in same namespace] | Assembly
    • [24] quic-go/quic-go [+2 other relevant repos in same namespace] | Go
    • [25] shirou/gopsutil | Go
    • [26] sirupsen/logrus | Go
    • [27] stretchr/testify [+1 other relevant repos in same namespace] | Go
    • [28] supranational/blst | Assembly
    • [29] uber-go/zap [+4 other relevant repos in same namespace] | Go
    • [30] urfave/cli | Go
  • ethereum/go-ethereum | Go | 30/112 dependencies | 435 pairwise combinations (vs 6216 previously)
    • [1] aws/aws-sdk-go-v2 [+1 other relevant repos in same namespace] | Go
    • [2] cockroachdb/pebble [+5 other relevant repos in same namespace] | Go
    • [3] crate-crypto/go-ipa [+1 other relevant repos in same namespace] | Go
    • [4] davecgh/go-spew | Go
    • [5] deckarep/golang-set | Go
    • [6] dop251/goja | Go
    • [7] fatih/color | Go
    • [8] fsnotify/fsnotify | Go
    • [9] getsentry/sentry-go | Go
    • [10] goccy/go-json | Go
    • [11] gogo/protobuf | Go
    • [12] golang-jwt/jwt | Go
    • [13] golang/protobuf [+1 other relevant repos in same namespace] | Go
    • [14] google/pprof [+3 other relevant repos in same namespace] | Go
    • [15] gorilla/websocket | Go
    • [16] graph-gophers/graphql-go | Go
    • [17] klauspost/compress [+1 other relevant repos in same namespace] | Go
    • [18] mitchellh/mapstructure [+1 other relevant repos in same namespace] | Go
    • [19] olekukonko/tablewriter | Go
    • [20] opentracing/opentracing-go | Go
    • [21] pkg/errors | Go
    • [22] prometheus/client_golang [+3 other relevant repos in same namespace] | Go
    • [23] rs/cors | Go
    • [24] russross/blackfriday | Go
    • [25] shirou/gopsutil | Go
    • [26] stretchr/testify | Go
    • [27] supranational/blst | Assembly
    • [28] syndtr/goleveldb | Go
    • [29] uber-go/automaxprocs | Go
    • [30] urfave/cli | Go
  • erigontech/erigon | Go | 30/253 dependencies | 435 pairwise combinations (vs 31878 previously)
    • [1] crate-crypto/go-ipa [+1 other relevant repos in same namespace] | Go
    • [2] definitelytyped/definitelytyped | TypeScript
    • [3] erigontech/interfaces [+6 other relevant repos in same namespace] | Go
    • [4] etcd-io/bbolt | Go
    • [5] ethers-io/ethers.js | TypeScript
    • [6] fsnotify/fsnotify | Go
    • [7] go-chi/chi [+1 other relevant repos in same namespace] | Go
    • [8] google/pprof [+5 other relevant repos in same namespace] | Go
    • [9] gorilla/websocket | Go
    • [10] grpc/grpc-go | Go
    • [11] ipfs/go-cid [+1 other relevant repos in same namespace] | Go
    • [12] json-iterator/go | Go
    • [13] julienschmidt/httprouter | Go
    • [14] libp2p/go-libp2p [+11 other relevant repos in same namespace] | Go
    • [15] microsoft/typescript [+1 other relevant repos in same namespace] | TypeScript
    • [16] miekg/dns | Go
    • [17] mitchellh/mapstructure [+2 other relevant repos in same namespace] | Go
    • [18] onsi/ginkgo | Go
    • [19] pion/webrtc [+16 other relevant repos in same namespace] | Go
    • [20] pkg/errors | Go
    • [21] prysmaticlabs/gohashtree [+1 other relevant repos in same namespace] | Assembly
    • [22] quic-go/quic-go [+2 other relevant repos in same namespace] | Go
    • [23] shirou/gopsutil | Go
    • [24] sirupsen/logrus | Go
    • [25] spf13/cobra [+3 other relevant repos in same namespace] | Go
    • [26] stretchr/testify | Go
    • [27] supranational/blst | Assembly
    • [28] uber-go/zap [+4 other relevant repos in same namespace] | Go
    • [29] urfave/cli | Go
    • [30] websockets/ws | JavaScript
  • web3/web3.js | TypeScript | 30/1709 dependencies | 435 pairwise combinations (vs 1459486 previously)
    • [1] ajv-validator/ajv [+1 other relevant repos in same namespace] | TypeScript
    • [2] babel/babel [+3 other relevant repos in same namespace] | TypeScript
    • [3] definitelytyped/definitelytyped | TypeScript
    • [4] ethereum/solc-js [+2 other relevant repos in same namespace] | TypeScript
    • [5] ethereumjs/ethereumjs-monorepo [+9 other relevant repos in same namespace] | TypeScript
    • [6] ethers-io/ethers.js | TypeScript
    • [7] facebook/docusaurus [+6 other relevant repos in same namespace] | TypeScript
    • [8] fb55/entities [+8 other relevant repos in same namespace] | TypeScript
    • [9] floating-ui/floating-ui | TypeScript
    • [10] getsentry/sentry-javascript | TypeScript
    • [11] gregberge/svgr | TypeScript
    • [12] immutable-js/immutable-js | TypeScript
    • [13] import-js/eslint-import-resolver-typescript [+1 other relevant repos in same namespace] | TypeScript
    • [14] inikulin/parse5 | TypeScript
    • [15] jestjs/jest | TypeScript
    • [16] lerna/lerna | TypeScript
    • [17] lukeed/mrmime [+8 other relevant repos in same namespace] | TypeScript
    • [18] microsoft/typescript [+7 other relevant repos in same namespace] | TypeScript
    • [19] mysticatea/abort-controller [+3 other relevant repos in same namespace] | TypeScript
    • [20] nrwl/nx | TypeScript
    • [21] pnpm/components [+1 other relevant repos in same namespace] | TypeScript
    • [22] postcss/postcss [+3 other relevant repos in same namespace] | TypeScript
    • [23] reactivex/rxjs | TypeScript
    • [24] remix-run/react-router [+1 other relevant repos in same namespace] | TypeScript
    • [25] salesforce/tough-cookie | TypeScript
    • [26] sindresorhus/type-fest [+183 other relevant repos in same namespace] | TypeScript
    • [27] typescript-eslint/typescript-eslint | TypeScript
    • [28] vercel/swr [+4 other relevant repos in same namespace] | TypeScript
    • [29] wevm/viem [+2 other relevant repos in same namespace] | TypeScript
    • [30] yarnpkg/berry [+1 other relevant repos in same namespace] | TypeScript
  • ethereumjs/ethereumjs-monorepo | TypeScript | 30/796 dependencies | 435 pairwise combinations (vs 316410 previously)
    • [1] ajv-validator/ajv | TypeScript
    • [2] babel/babel | TypeScript
    • [3] definitelytyped/definitelytyped | TypeScript
    • [4] ethereum/solc-js [+1 other relevant repos in same namespace] | TypeScript
    • [5] ethereumjs/ethereumjs-monorepo [+2 other relevant repos in same namespace] | TypeScript
    • [6] ethers-io/ethers.js | TypeScript
    • [7] fb55/entities | TypeScript
    • [8] isaacs/node-glob [+15 other relevant repos in same namespace] | TypeScript
    • [9] jquery/esprima | TypeScript
    • [10] lukeed/mrmime [+5 other relevant repos in same namespace] | TypeScript
    • [11] microsoft/typescript [+3 other relevant repos in same namespace] | TypeScript
    • [12] mysticatea/abort-controller [+4 other relevant repos in same namespace] | TypeScript
    • [13] open-telemetry/opentelemetry-js | TypeScript
    • [14] pillarjs/path-to-regexp [+3 other relevant repos in same namespace] | TypeScript
    • [15] postcss/postcss | TypeScript
    • [16] puppeteer/puppeteer | TypeScript
    • [17] reactivex/rxjs | TypeScript
    • [18] salesforce/tough-cookie | TypeScript
    • [19] sboudrias/inquirer.js | TypeScript
    • [20] shikijs/shiki [+1 other relevant repos in same namespace] | TypeScript
    • [21] sindresorhus/type-fest [+83 other relevant repos in same namespace] | TypeScript
    • [22] streetsidesoftware/cspell [+3 other relevant repos in same namespace] | TypeScript
    • [23] swc-project/pkgs [+1 other relevant repos in same namespace] | TypeScript
    • [24] typescript-eslint/typescript-eslint | TypeScript
    • [25] typestrong/typedoc | TypeScript
    • [26] uuidjs/uuid | TypeScript
    • [27] vercel/ms | TypeScript
    • [28] vitest-dev/vitest | TypeScript
    • [29] webdriverio/webdriverio [+1 other relevant repos in same namespace] | TypeScript
    • [30] yarnpkg/berry | TypeScript
  • grandinetech/grandine | Rust | 30/435 dependencies | 435 pairwise combinations (vs 94395 previously)
    • [1] amanieu/parking_lot [+1 other relevant repos in same namespace] | Rust
    • [2] bheisler/criterion.rs [+1 other relevant repos in same namespace] | Rust
    • [3] burntsushi/quickcheck [+8 other relevant repos in same namespace] | Rust
    • [4] clap-rs/clap | Rust
    • [5] crossbeam-rs/crossbeam | Rust
    • [6] dtolnay/thiserror [+17 other relevant repos in same namespace] | Rust
    • [7] google/zerocopy | Rust
    • [8] heim-rs/darwin-libproc | Rust
    • [9] hickory-dns/hickory-dns | Rust
    • [10] hyperium/hyper [+7 other relevant repos in same namespace] | Rust
    • [11] jeltef/derive_more | Rust
    • [12] libp2p/rust-libp2p | Rust
    • [13] microsoft/windows-rs | Rust
    • [14] peternator7/strum | Rust
    • [15] plotters-rs/plotters | Rust
    • [16] quinn-rs/quinn | Rust
    • [17] rayon-rs/rayon | Rust
    • [18] retep998/winapi-rs | Rust
    • [19] rusqlite/rusqlite | Rust
    • [20] rust-lang/cargo [+15 other relevant repos in same namespace] | Rust
    • [21] rust-phf/rust-phf | Rust
    • [22] rust-random/rand [+2 other relevant repos in same namespace] | Rust
    • [23] rustcrypto/hashes [+16 other relevant repos in same namespace] | Rust
    • [24] rustwasm/wasm-bindgen [+1 other relevant repos in same namespace] | Rust
    • [25] serde-rs/serde [+1 other relevant repos in same namespace] | Rust
    • [26] servo/rust-smallvec [+5 other relevant repos in same namespace] | Rust
    • [27] sfackler/rust-openssl [+5 other relevant repos in same namespace] | Rust
    • [28] tokio-rs/tokio [+6 other relevant repos in same namespace] | Rust
    • [29] tower-rs/tower [+1 other relevant repos in same namespace] | Rust
    • [30] unicode-org/icu4x | Rust
  • chainsafe/lodestar | TypeScript | 30/1514 dependencies | 435 pairwise combinations (vs 1145341 previously)
    • [1] ajv-validator/ajv [+1 other relevant repos in same namespace] | TypeScript
    • [2] babel/babel [+3 other relevant repos in same namespace] | TypeScript
    • [3] chainsafe/lodestar [+17 other relevant repos in same namespace] | TypeScript
    • [4] definitelytyped/definitelytyped | TypeScript
    • [5] electron/get [+1 other relevant repos in same namespace] | TypeScript
    • [6] ethereumjs/ethereumjs-monorepo [+1 other relevant repos in same namespace] | TypeScript
    • [7] ethers-io/ethers.js | TypeScript
    • [8] facebook/docusaurus [+4 other relevant repos in same namespace] | TypeScript
    • [9] fb55/entities [+8 other relevant repos in same namespace] | TypeScript
    • [10] gregberge/svgr | TypeScript
    • [11] inikulin/parse5 | TypeScript
    • [12] ipfs/protons [+2 other relevant repos in same namespace] | TypeScript
    • [13] jestjs/jest | TypeScript
    • [14] libp2p/js-libp2p | TypeScript
    • [15] lukeed/mrmime [+12 other relevant repos in same namespace] | TypeScript
    • [16] microsoft/typescript [+5 other relevant repos in same namespace] | TypeScript
    • [17] mysticatea/abort-controller [+2 other relevant repos in same namespace] | TypeScript
    • [18] pnpm/components [+1 other relevant repos in same namespace] | TypeScript
    • [19] postcss/postcss [+3 other relevant repos in same namespace] | TypeScript
    • [20] puppeteer/puppeteer | TypeScript
    • [21] reactivex/rxjs | TypeScript
    • [22] remix-run/react-router [+1 other relevant repos in same namespace] | TypeScript
    • [23] salesforce/tough-cookie | TypeScript
    • [24] sindresorhus/type-fest [+156 other relevant repos in same namespace] | TypeScript
    • [25] swc-project/pkgs [+1 other relevant repos in same namespace] | TypeScript
    • [26] vercel/ms [+2 other relevant repos in same namespace] | TypeScript
    • [27] vitest-dev/vitest | TypeScript
    • [28] vuejs/vue [+2 other relevant repos in same namespace] | TypeScript
    • [29] webdriverio/webdriverio [+1 other relevant repos in same namespace] | TypeScript
    • [30] yarnpkg/berry [+1 other relevant repos in same namespace] | TypeScript
  • sigp/lighthouse | Rust | 30/451 dependencies | 435 pairwise combinations (vs 101475 previously)
    • [1] alloy-rs/core [+2 other relevant repos in same namespace] | Rust
    • [2] amanieu/parking_lot [+1 other relevant repos in same namespace] | Rust
    • [3] bheisler/criterion.rs [+1 other relevant repos in same namespace] | Rust
    • [4] burntsushi/quickcheck [+9 other relevant repos in same namespace] | Rust
    • [5] clap-rs/clap | Rust
    • [6] crossbeam-rs/crossbeam | Rust
    • [7] diesel-rs/diesel | Rust
    • [8] dtolnay/thiserror [+17 other relevant repos in same namespace] | Rust
    • [9] gakonst/ethers-rs [+1 other relevant repos in same namespace] | Rust
    • [10] google/zerocopy | Rust
    • [11] heim-rs/darwin-libproc | Rust
    • [12] hickory-dns/hickory-dns | Rust
    • [13] hyperium/hyper [+7 other relevant repos in same namespace] | Rust
    • [14] libp2p/rust-libp2p | Rust
    • [15] microsoft/windows-rs | Rust
    • [16] mozilla/lmdb-rs | Rust
    • [17] peternator7/strum | Rust
    • [18] plotters-rs/plotters | Rust
    • [19] quinn-rs/quinn | Rust
    • [20] rayon-rs/rayon | Rust
    • [21] retep998/winapi-rs | Rust
    • [22] rusqlite/rusqlite | Rust
    • [23] rust-lang/cargo [+16 other relevant repos in same namespace] | Rust
    • [24] rust-phf/rust-phf | Rust
    • [25] rustcrypto/hashes [+15 other relevant repos in same namespace] | Rust
    • [26] rustwasm/wasm-bindgen | Rust
    • [27] serde-rs/serde [+1 other relevant repos in same namespace] | Rust
    • [28] servo/rust-smallvec [+4 other relevant repos in same namespace] | Rust
    • [29] tokio-rs/tokio [+7 other relevant repos in same namespace] | Rust
    • [30] tower-rs/tower | Rust
  • status-im/nimbus-eth2 | Nim | 8/96 dependencies | 28 pairwise combinations (vs 4560 previously)
    • [1] microsoft/debugpy | Python
    • [2] mozilla/bleach | Python
    • [3] numpy/numpy | Python
    • [4] pandas-dev/pandas | Python
    • [5] prometheus/client_python | Python
    • [6] pypa/build | Python
    • [7] pypa/packaging | Python
    • [8] pypa/wheel | Python
  • ethereum/remix-project | TypeScript | 30/2277 dependencies | 435 pairwise combinations (vs 2591226 previously)
    • [1] ajv-validator/ajv [+1 other relevant repos in same namespace] | TypeScript
    • [2] babel/babel [+6 other relevant repos in same namespace] | TypeScript
    • [3] coinbase/coinbase-wallet-sdk | TypeScript
    • [4] electron/forge [+9 other relevant repos in same namespace] | TypeScript
    • [5] ethereum/remix-project [+4 other relevant repos in same namespace] | TypeScript
    • [6] ethereum/solc-js [+4 other relevant repos in same namespace] | TypeScript
    • [7] ethereumjs/ethereumjs-monorepo [+3 other relevant repos in same namespace] | TypeScript
    • [8] ethers-io/ethers.js | TypeScript
    • [9] fb55/entities [+8 other relevant repos in same namespace] | TypeScript
    • [10] floating-ui/floating-ui | TypeScript
    • [11] formatjs/formatjs | TypeScript
    • [12] getsentry/sentry-javascript | TypeScript
    • [13] gregberge/svgr | TypeScript
    • [14] inikulin/parse5 | TypeScript
    • [15] ipfs/protons [+2 other relevant repos in same namespace] | TypeScript
    • [16] lukeed/mrmime [+14 other relevant repos in same namespace] | TypeScript
    • [17] mysticatea/abort-controller [+6 other relevant repos in same namespace] | TypeScript
    • [18] openzeppelin/openzeppelin-upgrades [+4 other relevant repos in same namespace] | TypeScript
    • [19] pnpm/cmd-shim | TypeScript
    • [20] postcss/postcss [+5 other relevant repos in same namespace] | TypeScript
    • [21] reactivex/rxjs | TypeScript
    • [22] rollup/rollup-pluginutils [+2 other relevant repos in same namespace] | TypeScript
    • [23] salesforce/tough-cookie | TypeScript
    • [24] sindresorhus/type-fest [+209 other relevant repos in same namespace] | TypeScript
    • [25] typescript-eslint/typescript-eslint | TypeScript
    • [26] vercel/ms [+2 other relevant repos in same namespace] | TypeScript
    • [27] vuejs/core | TypeScript
    • [28] walletconnect/walletconnect-monorepo [+2 other relevant repos in same namespace] | TypeScript
    • [29] wevm/abitype | TypeScript
    • [30] yarnpkg/berry [+1 other relevant repos in same namespace] | TypeScript
  • safe-global/safe-smart-account | TypeScript | 30/519 dependencies | 435 pairwise combinations (vs 134421 previously)
    • [1] ajv-validator/ajv | TypeScript
    • [2] babel/babel | TypeScript
    • [3] cosmiconfig/cosmiconfig | TypeScript
    • [4] definitelytyped/definitelytyped | TypeScript
    • [5] ethereum/js-ethereum-cryptography [+1 other relevant repos in same namespace] | TypeScript
    • [6] ethereum/solc-js [+1 other relevant repos in same namespace] | TypeScript
    • [7] ethereumjs/ethereumjs-monorepo | TypeScript
    • [8] ethers-io/ethers.js | TypeScript
    • [9] gajus/table | TypeScript
    • [10] gcanti/fp-ts [+1 other relevant repos in same namespace] | TypeScript
    • [11] getsentry/sentry-javascript | TypeScript
    • [12] immutable-js/immutable-js | TypeScript
    • [13] isaacs/node-glob [+17 other relevant repos in same namespace] | TypeScript
    • [14] jaredwray/keyv [+3 other relevant repos in same namespace] | TypeScript
    • [15] jquery/esprima | TypeScript
    • [16] jshttp/cookie [+4 other relevant repos in same namespace] | TypeScript
    • [17] microsoft/typescript [+1 other relevant repos in same namespace] | TypeScript
    • [18] nomicfoundation/hardhat [+3 other relevant repos in same namespace] | TypeScript
    • [19] omnidan/node-emoji | TypeScript
    • [20] pillarjs/path-to-regexp | TypeScript
    • [21] pnpm/components [+1 other relevant repos in same namespace] | TypeScript
    • [22] safe-global/safe-singleton-factory | TypeScript
    • [23] sindresorhus/type-fest [+65 other relevant repos in same namespace] | TypeScript
    • [24] tootallnate/proxy-agents [+1 other relevant repos in same namespace] | TypeScript
    • [25] tsconfig/bases | TypeScript
    • [26] typescript-eslint/typescript-eslint | TypeScript
    • [27] typestrong/ts-node | TypeScript
    • [28] uuidjs/uuid | TypeScript
    • [29] vercel/ms [+1 other relevant repos in same namespace] | TypeScript
    • [30] wighawag/hardhat-deploy | TypeScript
  • eth-infinitism/account-abstraction | Solidity | 30/854 dependencies | 435 pairwise combinations (vs 364231 previously)
    • [1] ajv-validator/ajv | TypeScript
    • [2] babel/babel [+2 other relevant repos in same namespace] | TypeScript
    • [3] broofa/mime | TypeScript
    • [4] cosmiconfig/cosmiconfig | TypeScript
    • [5] definitelytyped/definitelytyped | TypeScript
    • [6] dividab/tsconfig-paths | TypeScript
    • [7] eemeli/yaml | TypeScript
    • [8] ethereum/solc-js [+2 other relevant repos in same namespace] | TypeScript
    • [9] ethereumjs/ethereumjs-monorepo [+11 other relevant repos in same namespace] | TypeScript
    • [10] ethers-io/ethers.js | TypeScript
    • [11] gcanti/fp-ts [+1 other relevant repos in same namespace] | TypeScript
    • [12] getsentry/sentry-javascript | TypeScript
    • [13] immutable-js/immutable-js | TypeScript
    • [14] isaacs/node-glob [+18 other relevant repos in same namespace] | TypeScript
    • [15] jaredwray/keyv [+3 other relevant repos in same namespace] | TypeScript
    • [16] jquery/esprima | TypeScript
    • [17] jshttp/cookie [+18 other relevant repos in same namespace] | TypeScript
    • [18] microsoft/typescript [+1 other relevant repos in same namespace] | TypeScript
    • [19] mysticatea/regexpp [+3 other relevant repos in same namespace] | TypeScript
    • [20] nomicfoundation/hardhat [+2 other relevant repos in same namespace] | TypeScript
    • [21] openzeppelin/openzeppelin-contracts | Solidity
    • [22] pillarjs/path-to-regexp [+4 other relevant repos in same namespace] | TypeScript
    • [23] salesforce/tough-cookie | TypeScript
    • [24] sindresorhus/type-fest [+86 other relevant repos in same namespace] | TypeScript
    • [25] tsconfig/bases | TypeScript
    • [26] typescript-eslint/typescript-eslint | TypeScript
    • [27] typestrong/ts-node | TypeScript
    • [28] uuidjs/uuid | TypeScript
    • [29] vercel/ms [+1 other relevant repos in same namespace] | TypeScript
    • [30] wighawag/hardhat-deploy [+1 other relevant repos in same namespace] | TypeScript

[davidgasquez]

Heya @ccerv1! Had a question around this issue.

The current dependency graph is too large for efficient Pairwise voting, so we need to prune it back to something more manageable for a person putting in, say, up to 5 minutes per comparison.

Since jurors will be only looking at a subset of these pairwise samples, why is it "too large"? Is it because we want a big percentage of labeled edges, and having the previous amount of repositories will force jurors to "label" tons of edges?

[ccerv1]

Yes, I think that's the crux of it. We only have a limited amount of time we can expect from high-context jurors, and ideally there is more than one juror reviewing each edge. I personally have not been as close to that part of the process, but napkin math would be ~20 jurors reviewing 30-50 edges total with at least 2 jurors weighting each comparison.