diff --git a/.github/workflows/draft-pdf.yml b/.github/workflows/draft-pdf.yml new file mode 100644 index 00000000..9f738cd9 --- /dev/null +++ b/.github/workflows/draft-pdf.yml @@ -0,0 +1,29 @@ +name: draft-pdf + +on: + pull_request: + branches: + - main + - development + +jobs: + paper: + runs-on: ubuntu-latest + name: Paper Draft + steps: + - name: Checkout + uses: actions/checkout@v3 + - name: Build draft PDF + uses: openjournals/openjournals-draft-action@master + with: + journal: joss + # This should be the path to the paper within your repo. + paper-path: paper/paper.md + - name: Upload + uses: actions/upload-artifact@v1 + with: + name: paper + # This is the output path where Pandoc will write the compiled + # PDF. Note, this should be the same directory as the input + # paper.md + path: paper/paper.pdf diff --git a/.gitignore b/.gitignore index d10fc33a..a569f3e8 100644 --- a/.gitignore +++ b/.gitignore @@ -13,3 +13,4 @@ boxkit/depends/boost/* .sphinx/build/* !.sphinx/source/index.rst !README.rst +!DESIGN.rst diff --git a/.pylintrc b/.pylintrc new file mode 100644 index 00000000..05fefd3e --- /dev/null +++ b/.pylintrc @@ -0,0 +1,3 @@ +[DESIGN] +disable= + duplicate-code diff --git a/DESIGN.rst b/DESIGN.rst new file mode 100644 index 00000000..d26b3eec --- /dev/null +++ b/DESIGN.rst @@ -0,0 +1,30 @@ +############################################## + |icon| Software Design and Developer's Guide +############################################## + +***** + API +***** + +BoxKit API details here + +********* + Library +********* + +BoxKit library details here + +******* + C-Box +******* + +C++ backend detailes here + +*************** + Miscellaneous +*************** + +Miscellaneous details about how to develop frontend methods, etc. + +.. |icon| image:: ./media/book.svg + :width: 25 diff --git a/README.rst b/README.rst index 354b01af..5c5f92d3 100644 --- a/README.rst +++ b/README.rst @@ -1,87 +1,113 @@ -.. |icon| image:: ./media/icon.svg - :width: 30 - -============= -|icon| BoxKit -============= +############### + |icon| BoxKit +############### |Code style: black| |FlashX| |FlowX| |Minimal| |Publish| -BoxKit is a library that provides building blocks to parallelize and scale data science, high performance computing, and machine learning applications for block-structured datasets. Spatial data from simulations and experiments can be accessed and managed using tools available in this library when working with more data analysis oriented packages like SciKit (https://github.com/scikit-learn/scikit-learn) and FlowNet (https://github.com/NVIDIA/flownet2-pytorch) +BoxKit is a library that provides building blocks to parallelize and +scale data science, high performance computing, and machine learning +applications for block-structured datasets. Spatial data from +simulations and experiments can be accessed and managed using tools +available in this library when working with more data analysis oriented +packages like SciKit (https://github.com/scikit-learn/scikit-learn) and +FlowNet (https://github.com/NVIDIA/flownet2-pytorch) -Installation -============ +************** + Installation +************** -Stable releases of BoxKit are hosted on Python Package Index website (``_) and can be installed by executing, +Stable releases of BoxKit are hosted on Python Package Index website +(https://pypi.org/project/BoxKit/) and can be installed by executing, -:: +.. code:: pip install BoxKit --user - + export CXX=$(CPP_COMPILER) pip install BoxKit --user --install-option="--enable-testing" --install-option="--with-cbox" - -Note that ``pip`` should point to ``python3+`` installation package ``pip3``. -Upgrading and uninstallation is easily managed through this interface using, +Note that ``pip`` should point to ``python3+`` installation package +``pip3``. + +Upgrading and uninstallation is easily managed through this interface +using, -:: +.. code:: pip install --upgrade BoxKit --user pip install --upgrade BoxKit --user --install-option="--enable-testing" --install-option="--with-cbox" pip uninstall BoxKit -There maybe situations where users may want to install BoxKit in development mode $\\textemdash$ to design new features, debug, or customize classes/methods to their needs. This can be easily accomplished using the ``setup`` script located in the project root directory and executing, +There maybe situations where users may want to install BoxKit in +development mode $\\textemdash$ to design new features, debug, or +customize classes/methods to their needs. This can be easily +accomplished using the ``setup`` script located in the project root +directory and executing, -:: +.. code:: ./setup develop -Development mode enables testing of features/updates directly from the source code and is an effective method for debugging. Note that the ``setup`` script relies on ``click``, which can be installed using, +Development mode enables testing of features/updates directly from the +source code and is an effective method for debugging. Note that the +``setup`` script relies on ``click``, which can be installed using, -:: +.. code:: - pip install click - -Usage -===== + pip install click -BoxKit is undergoing active development and therefore design changes are frequent, however, the library is divided into two broad categories: +******* + Usage +******* -- **Create**: Containing interface for classes/methods to store spatial data in a rectangular/cubic frame, along with auxillary tools to manage irregular geometries composed of unstructured triangular mesh. +BoxKit is undergoing active development and therefore design changes are +frequent, however, the library is divided into two broad categories: -- **Utilities**: Containing interface for classes/method to improve memory managment of data on NUMA nodes. +- **Create**: Containing interface for classes/methods to store spatial + data in a rectangular/cubic frame, along with auxillary tools to + manage irregular geometries composed of unstructured triangular mesh. -We are currently setting up use cases for BoxKit, and will update this section when we are able to demonstrate proof-of-concept. +- **Utilities**: Containing interface for classes/method to improve + memory managment of data on NUMA nodes. -Citation -======== +We are currently setting up use cases for BoxKit, and will update this +section when we are able to demonstrate proof-of-concept. -:: +********** + Citation +********** - @software{akash_dhruv_2022_7255632, - author = {Akash Dhruv}, - title = {akashdhruv/BoxKit: October 2022}, - month = oct, - year = 2022, - publisher = {Zenodo}, - version = {22.10}, - doi = {10.5281/zenodo.7255632}, - url = {https://doi.org/10.5281/zenodo.7255632} - } +.. code:: -Help & Support -============== + @software{akash_dhruv_2022_7255632, + author = {Akash Dhruv}, + title = {akashdhruv/BoxKit: October 2022}, + month = oct, + year = 2022, + publisher = {Zenodo}, + version = {22.10}, + doi = {10.5281/zenodo.7255632}, + url = {https://doi.org/10.5281/zenodo.7255632} + } -Please file an issue on the repository page +**************** + Help & Support +**************** +Please file an issue on the repository page .. |Code style: black| image:: https://img.shields.io/badge/code%20style-black-000000.svg :target: https://github.com/psf/black - + .. |FlashX| image:: https://github.com/akashdhruv/BoxKit/workflows/FlashX/badge.svg + .. |FlowX| image:: https://github.com/akashdhruv/BoxKit/workflows/FlowX/badge.svg + .. |Minimal| image:: https://github.com/akashdhruv/BoxKit/workflows/Minimal/badge.svg + .. |Publish| image:: https://github.com/akashdhruv/BoxKit/workflows/Publish/badge.svg + +.. |icon| image:: ./media/icon.svg + :width: 30 diff --git a/boxkit/__meta__.py b/boxkit/__meta__.py index 65489fe7..14ef0ddc 100644 --- a/boxkit/__meta__.py +++ b/boxkit/__meta__.py @@ -1,7 +1,7 @@ """Metadata for BoxKit package""" __pkgname__ = "BoxKit" -__version__ = "3.0" +__version__ = "2023.06" __authors__ = "Akash Dhruv" __license__ = "Apache License" __copyright__ = "Copyright (c) Akash Dhruv 2021. All Rights Reserved." diff --git a/boxkit/api/_mean.py b/boxkit/api/_mean.py index 9a30310f..e029e5b7 100644 --- a/boxkit/api/_mean.py +++ b/boxkit/api/_mean.py @@ -17,7 +17,7 @@ def mean_temporal(datasets, varlist, backend="serial", nthreads=1, monitor=False if isinstance(varlist, str): varlist = [varlist] - # TODO: Add more error handling here to account + # TODO: Add more error handling here to account # pylint: disable=fixme # for consistency between multiple datasets # # Handle errors, compute level of the first @@ -33,6 +33,13 @@ def mean_temporal(datasets, varlist, backend="serial", nthreads=1, monitor=False # Create an mean dataset mean_dataset = datasets[0].clone(storage="numpy-memmap") + # loop over varlist append values to + # add it to the mean dataset and perform mean + for varkey in varlist: + mean_dataset.addvar(varkey) + + sample_size = len(datasets) + # Create a block list for reduction, first add # blocks from average_dataset and then loop over # datasets to add blocks from their respective blocklist @@ -42,12 +49,8 @@ def mean_temporal(datasets, varlist, backend="serial", nthreads=1, monitor=False for block, blk_list in zip(dataset.blocklist, blk_reduce_list): blk_list.append(block) - # loop over varlist append values to - # add it to the mean dataset and perform mean for varkey in varlist: - mean_dataset.addvar(varkey) - for varkey in varlist: if monitor: time_reduction = library.Timer("[boxkit.mean_temporal.mean_blk_list]") @@ -58,7 +61,9 @@ def mean_temporal(datasets, varlist, backend="serial", nthreads=1, monitor=False # pass list of parallel objects (blk_list for blk_list in blk_reduce_list), varkey, + sample_size, ) + if monitor: del time_reduction @@ -68,12 +73,11 @@ def mean_temporal(datasets, varlist, backend="serial", nthreads=1, monitor=False return mean_dataset -def mean_blk_list(blk_list, varkey): +def mean_blk_list(blk_list, varkey, sample_size): """ Reduce dataset / compute average """ mean_blk = blk_list[0] - sample_size = len(blk_list[1:]) for work_blk in blk_list[1:]: mean_blk[varkey] = mean_blk[varkey] + work_blk[varkey] / sample_size diff --git a/boxkit/api/_mergeblocks.py b/boxkit/api/_mergeblocks.py index ca079f52..25710b54 100644 --- a/boxkit/api/_mergeblocks.py +++ b/boxkit/api/_mergeblocks.py @@ -1,15 +1,13 @@ """Module with implemenetation of api methods""" -import sys - -from .. import library -from .. import api -from ..library import Action +from boxkit import library # pylint: disable=cyclic-import +from boxkit import api # pylint: disable=cyclic-import +from boxkit.library import Action # pylint: disable=cyclic-import def mergeblocks( dataset, varlist, nthreads=1, batch="auto", monitor=False, backend="serial" -): +): # pylint: disable=too-many-arguments disable=too-many-locals """ Reshaped dataset at a level """ @@ -34,9 +32,15 @@ def mergeblocks( ) # Compute number of blocks in each direction - nblockx = int((dataset.xmax - dataset.xmin) / dataset.blocklist[0].dx / dataset.nxb) - nblocky = int((dataset.ymax - dataset.ymin) / dataset.blocklist[0].dy / dataset.nyb) - nblockz = int((dataset.zmax - dataset.zmin) / dataset.blocklist[0].dz / dataset.nzb) + nblockx = round( + (dataset.xmax - dataset.xmin) / dataset.blocklist[0].dx / dataset.nxb + ) + nblocky = round( + (dataset.ymax - dataset.ymin) / dataset.blocklist[0].dy / dataset.nyb + ) + nblockz = round( + (dataset.zmax - dataset.zmin) / dataset.blocklist[0].dz / dataset.nzb + ) nblockx, nblocky, nblockz = [ value if value > 0 else 1 for value in [nblockx, nblocky, nblockz] @@ -76,10 +80,6 @@ def mergeblocks( if monitor: resources.display() - print( - f"[mem_dataset]: {round(sys.getsizeof(dataset._data.variables[varlist[0]][:])/(2**20),2)} MB" - ) - map_blk_to_merged_dset.nthreads = nthreads map_blk_to_merged_dset.monitor = monitor map_blk_to_merged_dset.backend = backend diff --git a/boxkit/api/_read.py b/boxkit/api/_read.py index 1abcdef3..41881b67 100644 --- a/boxkit/api/_read.py +++ b/boxkit/api/_read.py @@ -13,7 +13,7 @@ def read_dataset( force_memmap=False, nthreads=1, backend="serial", -): +): # pylint: disable=too-many-arguments disable=too-many-locals """ Create a dataset from a file diff --git a/boxkit/api/_regionprops.py b/boxkit/api/_regionprops.py index 826b5d63..004c59c4 100644 --- a/boxkit/api/_regionprops.py +++ b/boxkit/api/_regionprops.py @@ -3,8 +3,8 @@ import itertools import skimage.measure as skimage_measure -from .. import api -from ..library import Action +from boxkit import api # pylint: disable=cyclic-import +from boxkit.library import Action # pylint: disable=cyclic-import def regionprops(dataset, lsetkey, backend="serial", nthreads=1, monitor=False): diff --git a/boxkit/api/_resfilter.py b/boxkit/api/_resfilter.py index d41be69f..6793b2be 100644 --- a/boxkit/api/_resfilter.py +++ b/boxkit/api/_resfilter.py @@ -1,12 +1,13 @@ """Module with implemenetation of api reshape methods""" -from .. import library -from .. import api +from boxkit import library # pylint: disable=cyclic-import + +# from boxkit import api # pylint: disable=cyclic-import def resfilter( dataset, varlist=None, level=1, nthreads=1, monitor=False, backend="serial" -): +): # pylint: disable=too-many-arguments disable=too-many-locals disable=unused-argument """ Build a pseudo UG dataset from AMR dataset at a level @@ -51,20 +52,21 @@ def resfilter( for value in [nblockx_level, nblocky_level, nblockz_level] ] - filtered_dataset = api.create_dataset( - nblockx=nblockx_level, - nblocky=nblocky_level, - nblockz=nblockz_level, - nxb=dataset.nxb, - nyb=dataset.nyb, - nzb=dataset.nzb, - xmin=dataset.xmin, - ymin=dataset.ymin, - zmin=dataset.zmin, - xmax=dataset.xmax, - ymax=dataset.ymax, - zmax=dataset.zmax, - ) + # TODO: uncomment this # pylint: disable=fixme + # filtered_dataset = api.create_dataset( + # nblockx=nblockx_level, + # nblocky=nblocky_level, + # nblockz=nblockz_level, + # nxb=dataset.nxb, + # nyb=dataset.nyb, + # nzb=dataset.nzb, + # xmin=dataset.xmin, + # ymin=dataset.ymin, + # zmin=dataset.zmin, + # xmax=dataset.xmax, + # ymax=dataset.ymax, + # zmax=dataset.zmax, + # ) if monitor: del time_resfilter diff --git a/boxkit/library/_action.py b/boxkit/library/_action.py index e6607a84..fcffd628 100644 --- a/boxkit/library/_action.py +++ b/boxkit/library/_action.py @@ -1,8 +1,8 @@ """Module with implementation of Action utility""" - +from types import GeneratorType import copy -from .. import library +from boxkit import library # pylint: disable=cyclic-import class Action: # pylint: disable=too-many-arguments @@ -68,9 +68,15 @@ def execute(self, *args, **kwargs): """Custom call signature""" toparg, args = Action.toparg(*args) + + if not isinstance(toparg, GeneratorType) and not isinstance(toparg, list): + raise ValueError( + "[boxkit.library.Action] First argument " + + f"must be {GeneratorType!r} or {list!r} not {type(toparg)!r}" + ) + obj_list = list(toparg) del toparg - self.__class__.chk_obj_list(obj_list) return library.exectask(self, obj_list, *args, **kwargs) @@ -79,15 +85,11 @@ def execute(self, *args, **kwargs): def chk_obj_list(obj_list): """Check if obj_list matches the parallel_obj type""" - if not isinstance(obj_list, list): - raise ValueError( - "[boxkit.library.Action] Top argument must be a list of parallel_objs" - ) - first_obj = obj_list[0] for index, parallel_obj in enumerate(obj_list): if not isinstance(parallel_obj, type(first_obj)): raise ValueError( - "[boxkit.library.Action] Inconsistent type at index " + "[boxkit.library.Action] Inconsistent type " + + f"{type(parallel_obj)} vs {type(first_obj)} at index " + f'"{index}" in parallel object list' ) diff --git a/boxkit/library/_block.py b/boxkit/library/_block.py index 6d5fb0c5..7159624e 100644 --- a/boxkit/library/_block.py +++ b/boxkit/library/_block.py @@ -1,6 +1,5 @@ """Module with implementation of the Block class.""" -import math import numpy import pymorton @@ -252,9 +251,9 @@ def get_relative_loc(self, origin): Get offset from origin """ iloc, jloc, kloc = [ - math.ceil((self.xmin - origin[0]) / (self.xmax - self.xmin + 1e-13)), - math.ceil((self.ymin - origin[1]) / (self.ymax - self.ymin + 1e-13)), - math.ceil((self.zmin - origin[2]) / (self.zmax - self.zmin + 1e-13)), + round((self.xmin - origin[0]) / (self.xmax - self.xmin + 1e-13)), + round((self.ymin - origin[1]) / (self.ymax - self.ymin + 1e-13)), + round((self.zmin - origin[2]) / (self.zmax - self.zmin + 1e-13)), ] return iloc, jloc, kloc diff --git a/boxkit/library/_data.py b/boxkit/library/_data.py index efc0bd99..0f12e894 100644 --- a/boxkit/library/_data.py +++ b/boxkit/library/_data.py @@ -4,11 +4,11 @@ import string import random import shutil - +import h5py import h5pickle import numpy -from .. import options +from boxkit import options if options.dask: import dask.array as dsarray @@ -211,7 +211,7 @@ def _create_h5_datasets(self): pass for varkey in emptykeys: - outputfile = h5pickle.File(os.path.join(self.boxmem, varkey), "w") + outputfile = h5py.File(os.path.join(self.boxmem, varkey), "w") outputshape = [ self.nblocks, @@ -224,9 +224,10 @@ def _create_h5_datasets(self): varkey, data=numpy.zeros(outputshape, dtype=self.dtype[varkey]) ) outputfile.close() + del outputfile self.outfile[varkey] = h5pickle.File( - os.path.join(self.boxmem, varkey), "r+", skip_cache=False + os.path.join(self.boxmem, varkey), "r+", skip_cache=True ) self.variables[varkey] = self.outfile[varkey][varkey] @@ -265,6 +266,12 @@ def _create_numpy_memmap(self): self.variables[varkey] = numpy.memmap( outputfile, dtype=self.dtype[varkey], shape=outputshape, mode="w+" ) + self.variables[varkey].flush() + del self.variables[varkey] + + self.variables[varkey] = numpy.memmap( + outputfile, dtype=self.dtype[varkey], shape=outputshape, mode="r+" + ) self.dtype[varkey] = type(self.variables[varkey]) @@ -312,6 +319,20 @@ def _create_zarr_objects(self): ), dtype=self.dtype[varkey], ) + del self.variables[varkey] + + self.variables[varkey] = zarr.open( + outputfile, + mode="r+", + shape=outputshape, + chunks=( + 1, + self.nzb + 2 * self.zguard, + self.nyb + 2 * self.yguard, + self.nxb + 2 * self.xguard, + ), + dtype=self.dtype[varkey], + ) self.dtype[varkey] = type(self.variables[varkey]) diff --git a/boxkit/library/_dataset.py b/boxkit/library/_dataset.py index 80e4b329..c72f544a 100644 --- a/boxkit/library/_dataset.py +++ b/boxkit/library/_dataset.py @@ -1,8 +1,8 @@ """Module with implemenetation of Dataset class""" -from . import Block -from . import Data -from . import Action +from boxkit.library import Block # pylint: disable=cyclic-import +from boxkit.library import Data # pylint: disable=cyclic-import +from boxkit.library import Action # pylint: disable=cyclic-import class Dataset: # pylint: disable=too-many-instance-attributes diff --git a/boxkit/library/_execute.py b/boxkit/library/_execute.py index 6c875fd7..103ed41b 100644 --- a/boxkit/library/_execute.py +++ b/boxkit/library/_execute.py @@ -5,7 +5,7 @@ import joblib import tqdm -from .. import options +from boxkit import options if options.dask: import dask # pylint: disable=unused-import diff --git a/boxkit/library/_monitor.py b/boxkit/library/_monitor.py index b7382360..3915c74d 100644 --- a/boxkit/library/_monitor.py +++ b/boxkit/library/_monitor.py @@ -2,7 +2,7 @@ from progress.bar import Bar, ChargingBar # pylint: disable=unused-import -from .. import options +from boxkit import options if options.cbox: from ..cbox.lib import boost as cbox # pylint: disable=c-extension-no-member diff --git a/boxkit/library/_slice.py b/boxkit/library/_slice.py index 851003dd..af5a829d 100644 --- a/boxkit/library/_slice.py +++ b/boxkit/library/_slice.py @@ -1,6 +1,6 @@ """Module with implementation of the Slice class.""" -from . import Region +from boxkit.library import Region # pylint: disable=cyclic-import class Slice(Region): # pylint: disable=too-few-public-methods @@ -8,7 +8,7 @@ class Slice(Region): # pylint: disable=too-few-public-methods type_ = "derived" - def __init__(self, blocklist, **attributes): + def __init__(self, blocklist, **attributes): # pylint: disable=W0235 """Initialize the Slice object and allocate the data. Parameters diff --git a/boxkit/resources/read/_sample.py b/boxkit/resources/read/_sample.py index f967c419..196c99f1 100644 --- a/boxkit/resources/read/_sample.py +++ b/boxkit/resources/read/_sample.py @@ -3,7 +3,7 @@ import h5pickle as h5py -def read_test_sample(filename, server): +def read_test_sample(filename, server): # pylint: disable=too-many-locals """ Read dataset from BoxKit test sample @@ -34,9 +34,9 @@ def read_test_sample(filename, server): xmax = inputfile["boundbox/max"][:, 0] ymax = inputfile["boundbox/max"][:, 1] zmax = inputfile["boundbox/max"][:, 2] - dx = inputfile["deltas"][0] - dy = inputfile["deltas"][1] - dz = inputfile["deltas"][2] + dx = inputfile["deltas"][0] # pylint: disable=invalid-name + dy = inputfile["deltas"][1] # pylint: disable=invalid-name + dz = inputfile["deltas"][2] # pylint: disable=invalid-name variables = {} variables.update(inputfile["quantities"]) diff --git a/media/book.svg b/media/book.svg new file mode 100644 index 00000000..a440aa83 --- /dev/null +++ b/media/book.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/paper/.gitignore b/paper/.gitignore new file mode 100644 index 00000000..af2a6303 --- /dev/null +++ b/paper/.gitignore @@ -0,0 +1,2 @@ +paper.pdf +paper.jats diff --git a/paper/Makefile b/paper/Makefile new file mode 100644 index 00000000..c762cda7 --- /dev/null +++ b/paper/Makefile @@ -0,0 +1,2 @@ +all: + docker run --rm --volume $(PWD):/data --user $(id -u):$(id -g) --env JOURNAL=joss openjournals/inara diff --git a/paper/paper.bib b/paper/paper.bib new file mode 100644 index 00000000..3c529537 --- /dev/null +++ b/paper/paper.bib @@ -0,0 +1,23 @@ +@article{DUBEY2022, + Title = {Flash-{X}: A multiphysics simulation software instrument}, + Journal = {SoftwareX}, + Volume = {19}, + Pages = {101168}, + Year = {2022}, + issn = {2352-7110}, + doi = {https://doi.org/10.1016/j.softx.2022.101168}, + url = {https://www.sciencedirect.com/science/article/pii/S2352711022001030}, + Author = {Anshu Dubey and Klaus Weide and Jared O’Neal and Akash Dhruv and Sean Couch and J. Austin Harris and Tom Klosterman and Rajeev Jain and Johann Rudi and Bronson Messer and Michael Pajkos and Jared Carlson and Ran Chu and Mohamed Wahib and Saurabh Chawdhary and Paul M. Ricker and Dongwook Lee and Katie Antypas and Katherine M. Riley and Christopher Daley and Murali Ganapathy and Francis X. Timmes and Dean M. Townsley and Marcos Vanella and John Bachan and Paul M. Rich and Shravan Kumar and Eirik Endeve and W. Raphael Hix and Anthony Mezzacappa and Thomas Papatheodore} +} + +@unpublished{DHRUV2023, + Author = {Akash Dhruv}, + Title = {A Vortex Damping Outflow Forcing for Multiphase Flows with Sharp Interfacial Jumps}, + Year = {2023} +} + +@unpublished{HASSAN2023, + Author = {Shakeel Hassan and Arthur Feeney and Akash Dhruv and Jihoon Kim and Youngjoon Suh and Jaiyoung Ryu and Yoonjin Won and Aparna Chandramowlishwaran}, + Title = {{B}ubble{ML}: A Multi-Physics Dataset and Benchmarks for Machine Learning}, + Year = {2023} +} diff --git a/paper/paper.md b/paper/paper.md new file mode 100644 index 00000000..3649d7c4 --- /dev/null +++ b/paper/paper.md @@ -0,0 +1,80 @@ +--- +title: 'BoxKit: A Python library to manage analysis of block-structured simulation datasets' +tags: + - Python + - block structured datasets + - simulation analysis + - machine learning + - performance optimization +authors: + - name: Akash Dhruv + orcid: 0000-0003-4997-321X + affiliation: 1 +affiliations: + - name: Argonne National Laboratory, USA + index: 1 +date: 15 June 2023 +bibliography: paper.bib +--- + +# Summary + +BoxKit is a library that provides building blocks to parallelize and +scale data science, high performance computing, and machine learning +applications for block-structured datasets. Spatial data from +simulations and experiments can be accessed and managed using tools +available in this library when working with more data analysis oriented +packages like SciKit, FlowNet, and OpticalFlow + + +# Statement of need + +Details about why there is software addresses references like +[@DUBEY2022], [@DHRUV2023], and [@HASSAN2023] + +# Mathematics + +Single dollars ($) are required for inline mathematics e.g. $f(x) = e^{\pi/x}$ + +Double dollars make self-standing equations: + +$$\Theta(x) = \left\{\begin{array}{l} +0\textrm{ if } x < 0\cr +1\textrm{ else} +\end{array}\right.$$ + +You can also use plain \LaTeX for equations +\begin{equation}\label{eq:fourier} +\hat f(\omega) = \int_{-\infty}^{\infty} f(x) e^{i\omega x} dx +\end{equation} +and refer to \autoref{eq:fourier} from text. + +# Citations + +Citations to entries in paper.bib should be in +[rMarkdown](http://rmarkdown.rstudio.com/authoring_bibliographies_and_citations.html) +format. + +If you want to cite a software repository URL (e.g. something on GitHub without a preferred +citation) then you can do it with the example BibTeX entry below for @fidgit. + +For a quick reference, the following citation commands can be used: +- `@author:2001` -> "Author et al. (2001)" +- `[@author:2001]` -> "(Author et al., 2001)" +- `[@author1:2001; @author2:2001]` -> "(Author1 et al., 2001; Author2 et al., 2002)" + +# Figures + +Figures can be included like this: +![Caption for example figure.\label{fig:example}](figure.png) +and referenced from text using \autoref{fig:example}. + +Figure sizes can be customized by adding an optional second parameter: +![Caption for example figure.](figure.png){ width=20% } + +# Acknowledgements + +We acknowledge contributions from Brigitta Sipocz, Syrtis Major, and Semyeong +Oh, and support from Kathryn Johnston during the genesis of this project. + +# References diff --git a/setup.py b/setup.py index 0981ff80..66015a57 100644 --- a/setup.py +++ b/setup.py @@ -11,16 +11,16 @@ # Import bin from current working directory # sys.path.insert makes sure that current file path is searched # first to find this module -import bin.cmd as bin_cmd +import bin.cmd as bin_cmd # pylint: disable=wrong-import-position # Parse README and get long # description -with open("README.rst", mode="r") as readme: +with open("README.rst", mode="r", encoding="ascii") as readme: long_description = readme.read() # Open metadata file to extract package information -with open("boxkit/__meta__.py", mode="r") as source: +with open("boxkit/__meta__.py", mode="r", encoding="ascii") as source: content = source.read().strip() metadata = { key: re.search(key + r'\s*=\s*[\'"]([^\'"]*)[\'"]', content).group(1) @@ -34,8 +34,8 @@ } # core dependancies for the package -with open('requirements/core.txt') as reqs: - DEPENDENCIES = reqs.read() +with open("requirements/core.txt", mode="r", encoding="ascii") as core_reqs: + DEPENDENCIES = core_reqs.read() # Call setup command with necessary arguments # replace existing build and develop commands diff --git a/tests/cache/trivial.py b/tests/cache/trivial.py index a1cd7567..7c6a39fc 100644 --- a/tests/cache/trivial.py +++ b/tests/cache/trivial.py @@ -2,7 +2,7 @@ from boxkit.library import Action, Timer -@Action(parallel_obj=float, backend="serial", nthreads=1, monitor=True) +@Action(backend="serial", nthreads=1, monitor=True) def multiply(parallel_obj): return parallel_obj * 10