Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
97024fc
Pre-compute neighbor lists in CoarsenedGraphView to eliminate lock co…
adsharma Feb 18, 2026
72d5d6a
Add convergence detection and progress logging to ParallelLeidenView
adsharma Feb 18, 2026
6a43e2c
Remove neighbor caching from CoarsenedGraphView to fix memory issues
adsharma Feb 18, 2026
b69c975
Fix memory leak: don't accumulate CoarsenedGraphView objects
adsharma Feb 18, 2026
00f035a
Use incremental mapping composition to reduce memory usage
adsharma Feb 18, 2026
4e1c38f
Compact partition after coarsening to reduce memory usage
adsharma Feb 19, 2026
6bcf910
Fix weightedDegree to count self-loops twice for undirected graphs
adsharma Feb 19, 2026
da20bc5
Fix ParallelLeidenView coarsening semantics and mapping
adsharma Feb 20, 2026
8da6d9e
Add regression test for CoarsenedGraphView weight parity
adsharma Feb 20, 2026
74acbc4
Stop ParallelLeidenView inner-loop stagnation and zero-move churn
adsharma Feb 20, 2026
cd19bed
Stop LeidenView inner loop on community-count plateau
adsharma Feb 20, 2026
9dc2273
Use 1% community-reduction tolerance to stop LeidenView inner loop
adsharma Feb 20, 2026
6df27b9
Stabilize connected-cycle Leiden validation for deterministic behavior
adsharma Feb 20, 2026
62ed6d6
Remove heuristic inner-loop early exits from ParallelLeidenView
adsharma Feb 20, 2026
34401fa
Add ParallelLeidenView move-gain diagnostics per inner iteration
adsharma Feb 20, 2026
5213ef5
Filter marginal Leiden moves and log rejected micro-moves
adsharma Feb 20, 2026
301d6b1
Optimize CoarsenedGraphView neighbor aggregation without hash maps
adsharma Feb 20, 2026
4a084e8
Increase Leiden marginal-move filter and log rejection rate
adsharma Feb 20, 2026
603e3fe
Make Leiden move-gain epsilon runtime configurable
adsharma Feb 20, 2026
0c835a5
Set ParallelLeidenView max inner iterations to 20
adsharma Feb 20, 2026
5ceb98a
Add env knobs for Leiden max-inner and vector oversize
adsharma Feb 20, 2026
12261e5
Add optional min community-reduction stop knob
adsharma Feb 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions cmake/ArrowConfig.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Minimal ArrowConfig.cmake for pyarrow bundled libraries
set(ARROW_FOUND TRUE)
set(ARROW_VERSION "23.0.1")

# Set paths based on pyarrow installation
set(PYARROW_DIR "/home/ubuntu/work/.venv/lib/python3.13/site-packages/pyarrow")
set(ARROW_INCLUDE_DIR "${PYARROW_DIR}/include")
set(ARROW_LIBRARY "${PYARROW_DIR}/libarrow.so.2300")
set(ARROW_LIB_DIR "${PYARROW_DIR}")

# Create imported target
add_library(Arrow::arrow_shared SHARED IMPORTED)
set_target_properties(Arrow::arrow_shared PROPERTIES
IMPORTED_LOCATION "${ARROW_LIBRARY}"
INTERFACE_INCLUDE_DIRECTORIES "${ARROW_INCLUDE_DIR}"
)

# Alias for compatibility
add_library(Arrow::arrow ALIAS Arrow::arrow_shared)

# Provide variables that CMake expects
set(ARROW_LIBRARIES Arrow::arrow_shared)
set(ARROW_INCLUDE_DIRS "${ARROW_INCLUDE_DIR}")

mark_as_advanced(ARROW_INCLUDE_DIR ARROW_LIBRARY)
335 changes: 335 additions & 0 deletions icebug.egg-info/PKG-INFO
Original file line number Diff line number Diff line change
@@ -0,0 +1,335 @@
Metadata-Version: 2.4
Name: icebug
Version: 12.1
Summary: High performance graph analytics backed by read-only memory
Home-page: https://github.com/Ladybug-Memory/icebug
Download-URL: https://pypi.python.org/pypi/icebug
Author: Christian L. Staudt, Henning Meyerhenke, Arun Sharma
Author-email: christian.staudt@kit.edu, meyerhenke@kit.edu, arun@ladybugmem.ai
License: Copyright (c) 2013-2022 Henning Meyerhenke, NetworKit Contributors
Copyright (c) 2013-2017 Karlsruhe Institute of Technology
Copyright (c) 2017-2018 University of Cologne
Copyright (c) 2018-2022 Humboldt-Universität zu Berlin

MIT License (http://opensource.org/licenses/MIT)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
associated documentation files (the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge, publish, distribute,
sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial
portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT
NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Project-URL: homepage, https://github.com/Ladybug-Memory/icebug
Project-URL: documentation, https://networkit.github.io/dev-docs/index.html
Project-URL: repository, https://github.com/Ladybug-Memory/icebug
Project-URL: changelog, https://github.com/Ladybug-Memory/icebug/blob/master/CHANGES.md
Keywords: graph algorithm,network analysis,social network,apache arrow,columnar
Platform: any
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Environment :: Web Environment
Classifier: Environment :: Other Environment
Classifier: Framework :: IPython
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: End Users/Desktop
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: C++
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Topic :: Scientific/Engineering :: Chemistry
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Scientific/Engineering :: Visualization
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: scipy
Requires-Dist: numpy
Requires-Dist: pyarrow
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pandas; extra == "test"
Requires-Dist: networkx; extra == "test"
Provides-Extra: vizbridges
Requires-Dist: ipycytoscape; extra == "vizbridges"
Requires-Dist: plotly; extra == "vizbridges"
Requires-Dist: seaborn; extra == "vizbridges"
Requires-Dist: anywidget; extra == "vizbridges"
Requires-Dist: tabulate; extra == "vizbridges"
Provides-Extra: pytest
Requires-Dist: pytest; extra == "pytest"
Requires-Dist: pandas; extra == "pytest"
Requires-Dist: networkx; extra == "pytest"
Requires-Dist: ipycytoscape; extra == "pytest"
Requires-Dist: plotly; extra == "pytest"
Requires-Dist: seaborn; extra == "pytest"
Requires-Dist: anywidget; extra == "pytest"
Requires-Dist: tabulate; extra == "pytest"
Dynamic: author-email
Dynamic: download-url
Dynamic: home-page
Dynamic: platform

<p align="center">
<img width="60%" src="docs/logo/logo_color.jpg" alt="Icebug - Frozen Graph Analytics"><br>
<a href="https://github.com/Ladybug-Memory/icebug/actions"><img src="https://github.com/Ladybug-Memory/icebug/workflows/build/badge.svg"></a>
<a href="https://badge.fury.io/py/icebug"><img src="https://badge.fury.io/py/icebug.svg"></a>
<a href="https://coveralls.io/github/Ladybug-Memory/icebug?branch=master"><img src="https://coveralls.io/repos/github/Ladybug-Memory/icebug/badge.svg?branch=master"></a>
<a href="https://mybinder.org/v2/gh/networkit/networkit/master?urlpath=lab/tree/notebooks/User-Guide.ipynb"><img src="https://mybinder.org/badge_logo.svg"></a>
</p>

##
[Icebug][icebug] is an open source library for high-performance
graph analysis backed by read-only memory. Due to its heritage, it also supports network/graph analysis on read-write memory, but it can be 5x more CPU and 5x more memory efficient when using columnar memory such as Apache Arrow.

Its aim is to provide tools for the analysis of large
networks in the size range from thousands to billions of edges. For this
purpose, it implements efficient graph algorithms, many of them parallel to
utilize multicore architectures. These are meant to compute standard measures
of network analysis. Icebug is focused on scalability and comprehensiveness.
Icebug is also a testbed for algorithm engineering and
contains novel algorithms from recently published research (see list of publications below).

Icebug is a Python module. High-performance algorithms are written in C++ and exposed to Python
via the Cython toolchain. Python in turn gives us the ability to work interactively and a
rich environment of tools for data analysis and scientific computing.
Furthermore, Icebug's core can be built and used as a native library if needed.

## Requirements

You will need the following software to install Icebug as a python
package:

- A modern C++ compiler, e.g.: [g++] (&gt;= 8.1), [clang++] (&gt;= 6.0) or MSVC (&gt;= 14.20)
- OpenMP for parallelism (usually ships with the compiler)
- Python3 (3.9 or higher is supported)
- Development libraries for Python3. The package name depends on your distribution. Examples:
- Debian/Ubuntu: `apt-get install python3-dev`
- RHEL/CentOS: `dnf install python3-devel`
- Windows: Use the official release installer from [www.python.org](https://www.python.org/downloads/windows/)
- [Pip]
- [CMake] version 3.6 or higher (Advised to use system packages if available. Alternative: `pip3 install cmake`)
- Build system: [Make] or [Ninja]
- Cython version 0.29 or higher (e.g., `pip3 install cython`)
- Apache Arrow development libraries (libarrow-dev)

## Install

In order to use Icebug, you can either install it via package managers
or build the Python module from source.

### Install via package manager

While the most recent version is in general available for all package managers, the number of older downloadable versions differ.

##### pip

pip3 install [--user] icebug

##### conda (channel conda-forge)

conda config --add channels conda-forge
conda install icebug [-c conda-forge]

##### brew

brew install icebug

##### spack

spack install py-icebug

More system-specific information on how to install Icebug on Linux, macOS (both Intel and M1) and Windows-systems can be found [here](https://networkit.github.io/get_started.html).

### Building the Python module from source

git clone https://github.com/Ladybug-Memory/icebug icebug
cd icebug
python3 setup.py build_ext [-jX]
pip3 install -e .

The script will call `cmake` and `ninja` (`make` as fallback) to compile
Icebug as a library, build the extensions and copy it to the top folder. By
default, Icebug will be built with the amount of available cores in
optimized mode. It is possible the add the option `-jN` the number of threads
used for compilation.

## Usage example

To get an overview and learn about Icebug's different functions/classes, have a look at our interactive [notebooks-section][notebooks], especially the [Icebug UserGuide]. Note: To view and edit the computed output from the notebooks, it is recommended to use [Jupyter Notebook][jupyter-notebooks]. This requires the prior installation of Icebug. You should really check that out before start working on your network analysis.

We also provide a Binder-instance of our notebooks. To access this service, you can either click on the badge at the top or follow this [link][binder]. Disclaimer: Due to rebuilds of the underlying image, it can takes some time until your Binder instance is ready for usage.

If you only want to see in short how Icebug is used - the following example provides a climpse at that. Here we generate a random hyperbolic graph with 100k nodes and compute its communities with the PLM method:

>>> import icebug as nk
>>> g = nk.generators.HyperbolicGenerator(1e5).generate()
>>> communities = nk.community.detectCommunities(g, inspect=True)
PLM(balanced,pc,turbo) detected communities in 0.14577102661132812 [s]
solution properties:
------------------- -----------
# communities 4536
min community size 1
max community size 2790
avg. community size 22.0459
modularity 0.987243
------------------- -----------

## Install the C++ Core only

In case you only want to work with Icebug's C++ core, you can either install it via package
managers or build it from source.

### Install C++ core via package manager

##### conda (channel conda-forge)

conda config --add channels conda-forge
conda install libicebug [-c conda-forge]

##### brew

brew install libicebug

##### spack

spack install libicebug

### Building the C++ core from source

We recommend [CMake] and your preferred build system for building the C++ part of Icebug.

The following description shows how to use [CMake] in order to build the C++ Core only:

First you have to create and change to a build directory: (in this case named `build`)

mkdir build
cd build

Then call [CMake] to generate files for the `make` build system, specifying the directory of the root `CMakeLists.txt` file (e.g., `..`). After this `make` is called to start the build process:

cmake ..
make -jX

To speed up the compilation with make a multi-core machine, you can append `-jX` where X denotes the number of threads to compile with.

### Use NetworKit as a library

This paragraph explains how to use the Icebug core C++ library in case it has been built from source.
For how to use it when installed via package managers, best refer to the official documentation ([brew](https://brew.sh), [conda](https://docs.conda.io), [spack](https://spack.readthedocs.io/en/latest)).

In order to use the previous compiled icebug library, you need to have it installed, and link
it while compiling your project. Use these instructions to compile and install Icebug in `/usr/local`:

cmake ..
make -jX install

Once Icebug has been installed, you can use include directives in your
C++\-application as follows:

#include <icebug/graph/Graph.hpp>

You can compile your source as follows:

g++ my_file.cpp -licebug


### Unit tests

Building and running Icebug unit tests is not mandatory.
However, as a developer you might want to write and run unit tests for your
code, or if you experience any issues with Icebug, you might want to check
if Icebug runs properly.
The unit tests can only be run from a clone or copy of the repository and not
from a pip installation. In order to run the unit tests, you need to compile
them first. This is done by setting the [CMake] `NETWORKI_BUILD_TESTS` flag to
`ON`:

cmake -DNETWORKIT_BUILD_TESTS=ON ..

Unit tests are implemented using GTest macros such as `TEST_F(CentralityGTest, testBetweennessCentrality)`.
Single tests can be executed with:

./networkit_tests --gtest_filter=CentralityGTest.testBetweennessCentrality

Additionally, one can specify the level of the logs outputs by adding `--loglevel <log_level>`;
supported log levels are: `TRACE`, `DEBUG`, `INFO`, `WARN`, `ERROR`, and `FATAL`.


### Compiling with address/leak sanitizers

Sanitizers are great tools to debug your code. Icebug provides additional [Cmake] flags
to enable address, leak, and undefined behavior sanitizers.
To compile your code with sanitizers, set the [CMake]
`NETWORKIT_WITH_SANITIZERS` to either `address` or `leak`:

cmake -DNETWORKIT_WITH_SANITIZERS=leak ..

By setting this flag to `address`, your code will be compiled with the `address` and the `undefined` sanitizers.
Setting it to `leak` also adds the `leak` sanitizer.


## Documentation

The most recent version of the [documentation can be found online](https://networkit.github.io/dev-docs/index.html).

## Contact

For questions regarding Icebug, have a look at our [issues-section][issues] and see if there is already an open discussion. If not feel free to open a new issue.
To stay updated about this project, subscribe to our [mailing list][list].

## Contributions

We encourage contributions to the Icebug source code. See the [development guide][devguide] for instructions. For support please contact the [mailing list][list].

## Credits

Icebug is a fork of [NetworKit](https://networkit.github.io/).

List of contributors can be found on the [NetworKit website credits page](https://networkit.github.io/credits.html).

## External Code

The program source includes:
- the *[TLX][tlx]* library
- the *[TTMath]* bignum library

[mitlicense]: http://opensource.org/licenses/MIT
[ttmath]: http://www.ttmath.org/
[tlx]: https://github.com/tlx/tlx/

## License
The source code of this program is released under the [MIT License][mitlicense]. We ask you to cite us if you use this code in your project (c.f. the publications section below and especially the [technical report](https://arxiv.org/abs/1403.3005)). Feedback is also welcome.

## Publications
The [Icebug publications page][icepubs] lists the publications on Icebug as a toolkit, on algorithms available
in Icebug, and simply using Icebug. We ask you to cite the appropriate ones if you found Icebug useful for your own research.

[icepubs]: https://networkit.github.io/publications.html
[list]: https://sympa.cms.hu-berlin.de/sympa/subscribe/icebug
[icebug]: https://github.com/Ladybug-Memory/icebug
[IPython]: https://ipython.readthedocs.io/en/stable/
[Icebug UserGuide]: https://github.com/Ladybug-Memory/icebug/blob/master/notebooks/User-Guide.ipynb
[notebooks]: https://github.com/Ladybug-Memory/icebug/blob/master/notebooks/
[g++]: https://gcc.gnu.org
[clang++]: https://clang.llvm.org/
[Pip]: https://pypi.python.org/pypi/pip
[CMake]: https://cmake.org/
[Make]: https://www.gnu.org/software/make/
[Ninja]: https://ninja-build.org/
[devguide]: https://networkit.github.io/dev-docs/DevGuide.html#devGuide
[issues]: https://github.com/Ladybug-Memory/icebug/issues
[jupyter-notebooks]: https://jupyter.org/install.html
[binder]: https://mybinder.org/v2/gh/networkit/networkit/master?urlpath=lab/tree/notebooks
Loading
Loading