Add pymeep build instructions to docs

[ChristopherHogan]

@stevengj @oskooi @HomerReid

[oskooi]

Replacing --with-pic and LIBS=-ldl with --enable-shared will result in a Meep failure to link with the MPB library (i.e., the linker is unable to find mpb.h). This will in turn cause the wvg_src.py test of the Meep make check suite to fail. The original flags produce no errors and all tests pass.

[ChristopherHogan]

Ardavan is right. The problem is that using just --enable-shared causes make check to fail when running ./mpb ../examples/check.ctl, so MPB never gets installed, which is why meep can't find it. The test is looking for the installed mpb.scm but this is before make install is run:

make  check-local
make[3]: Entering directory '/root/install/mpb/mpb'
./mpb ../examples/check.ctl
Backtrace:
In ice-9/boot-9.scm:
 157: 4 [catch #t #<catch-closure 1403a80> ...]
In unknown file:
   ?: 3 [apply-smob/1 #<catch-closure 1403a80>]
In ice-9/eval.scm:
 432: 2 [eval # #]
 432: 1 [eval # #]
In unknown file:
   ?: 0 [primitive-load "/usr/local/share/mpb/mpb.scm"]

ERROR: In procedure primitive-load:
ERROR: In procedure open-file: No such file or directory: "/usr/local/share/mpb/mpb.scm"
Makefile:972: recipe for target 'check-local' failed
make[3]: *** [check-local] Error 1

[stevengj]

As a general rule, when you encounter a problem like this, it is best to fix it at the source rather than hack around it. If ./configure --enable-shared is not working for mpb, then we should fix it.

It should be fixed now with stevengj/libctl@51f7a7b

[stevengj]

I would no longer recommend using environment variables for these things (397c693). It is better to pass flags directly to configure. (That way the configure script will "remember" them if you run make again, for example.)

[stevengj]

In particular, it is especially better to use -Wl,-rpath,... rather than LD_LIBRARY_FLAGS, since with -rpath the correct path will be embedded directly into the meep executable.

[stevengj]

It is dangerous to include dist-upgrade in the script. This forces users to update their entire operating system — our installation script should not do this for them.

[stevengj]

I would skip the make check in the build script.

[stevengj]

Why can't we use Ubuntu's libhdf5-openmpi-dev package?

Note that you can still build the serial versions of Meep and MPB and link to the MPI version of HDF5, you just need to build with CC=mpicc CXX=mpicxx.

[oskooi]

Building parallel hdf5 from source after setting the environment variables export CC=mpicc and export CC=mpicc ensures that the correct OpenMPI compilers are used throughout the entire build. In previous tests, I found that using Ubuntu's libhdf5-openmpi-dev package often lead to linking errors when building Meep (i.e., the hdf5 libraries could not be found, etc.).

[stevengj]

Use CC=mpicc.openmpi and CXX=mpic++.openmpi in case the user installed other MPI versions too.

[oskooi]

This install script uses libhdf5-openmpi-dev and builds successfully. The only other packages built from source are: harminv, libctl, h5utils, and mpb.

There are two minor issues: (1) h5ls is missing, and (2) MPI jobs produce the following warning:

--------------------------------------------------------------------------
[[47997,1],0]: A high-performance Open MPI point-to-point messaging module
was unable to find any relevant network interfaces:

Module: OpenFabrics (openib)
  Host: ip-XXX-XX-XX-XXX

Another transport will be used instead, although this may result in
lower performance.
--------------------------------------------------------------------------

This can be circumvented by adding a flag to prevent searching for Infiniband hardware.

All that is left now is to omit the environment variables using -rpath...

[stevengj]

h5ls is in the hdf5-tools Ubuntu package — see the Ubuntu package search.

[oskooi]

The updated install scripts (serial and parallel) have now been validated with --enable-shared replacing --with-pic and LIBS=-ldl.

[oskooi]

Here is the working install script without environment variables. Users must still add export PYTHONPATH="${HOME}/install/meep/python" to their .bashrc, etc file.

#!/bin/bash                                                                                                                                                                            

set -e

sudo apt-get update
sudo apt-get -y install libblas-dev liblapack-dev libgmp-dev libunistring-dev libmatheval-dev swig libgsl-dev autoconf pkg-config libpng16-dev git guile-2.0-dev libfftw3-dev libhdf5-openmpi-dev hdf5-tools libpython3.5-dev python3-numpy python3-pip

mkdir ~/install

cd ~/install
git clone https://github.com/stevengj/harminv.git
cd harminv/
sh autogen.sh --enable-shared
make && sudo make install

cd ~/install
git clone https://github.com/stevengj/libctl.git
cd libctl/
sh autogen.sh --enable-shared
make && sudo make install

cd ~/install
git clone https://github.com/stevengj/h5utils.git
cd h5utils/
CC=mpicc LDFLAGS="-L/usr/local/lib -L/usr/lib/x86_64-linux-gnu/hdf5/openmpi -Wl,-rpath,/usr/local/lib:/usr/lib/x86_64-linux-gnu/hdf5/openmpi" CPPFLAGS="-I/usr/local/include -I/usr/include/hdf5/openmpi" sh autogen.sh
make && sudo make install

cd ~/install
git clone https://github.com/stevengj/mpb.git
cd mpb/
CC=mpicc LDFLAGS="-L/usr/local/lib -L/usr/lib/x86_64-linux-gnu/hdf5/openmpi -Wl,-rpath,/usr/local/lib:/usr/lib/x86_64-linux-gnu/hdf5/openmpi" CPPFLAGS="-I/usr/local/include -I/usr/include/hdf5/openmpi" sh autogen.sh --enable-shared
make && make check && sudo make install

pip3 install --upgrade pip
pip3 install --user --no-cache-dir mpi4py
export HDF5_MPI="ON"
pip3 install --user --no-binary=h5py h5py

cd ~/install
git clone https://github.com/stevengj/meep.git
cd meep/
CC=mpicc CXX=mpic++ LDFLAGS="-L/usr/local/lib -L/usr/lib/x86_64-linux-gnu/hdf5/openmpi -Wl,-rpath,/usr/local/lib:/usr/lib/x86_64-linux-gnu/hdf5/openmpi" CPPFLAGS="-I/usr/local/include -I/usr/include/hdf5/openmpi" sh autogen.sh --enable-shared --with-python --with-mpi
make && make check && sudo make install

[ChristopherHogan]

I replaced the separate serial and parallel build instructions with just the parallel version, using Ardavan's latest script.

[stevengj]

You shouldn't write CC=foo sh autogen.sh (which passes CC via temporarily setting an environment variable), you should use sh autogen.sh CC=foo (which passes CC by a command-line argument).

To do this, however, the Meep autogen.sh script should be modified. The whole configure_args business is problematic because it only allows a certain whitelist of names.

I already changed MPB's autoconf.sh script (NanoComp/mpb@ee35372) to be better in this way. Let's just change the autoconf.sh scripts for Meep etcetera.

[stevengj]

Okay, I updated the autogen.sh scripts so that you can now pass CC=foo etcetera directly as arguments.

[ChristopherHogan]

PR 35 is needed for MPB to pass the args to configure. The same would need to be done for meep. See h5utils PR 5 for the explanation.

[ChristopherHogan]

I believe this is correct now.

[stevengj]

Do you have to pass PYTHON=python3 if you want Meep to use Python 3?

[ChristopherHogan]

Automake goes through a list of possible interpreters and subs in the first one found into PYTHON. In our case, it picks python3 because that's the only one we have installed. Sicne most systems will likely have a python that points to python 2 though, I set PYTHON=python3 explicitly, with a note that the user can choose 2 or 3.

[stevengj]

Needs sudo?

[ChristopherHogan]

Done.

[oskooi]

To add sudo to the pip3 install lines, the --user flag needs to be removed, i.e.

sudo pip3 install --upgrade pip
sudo pip3 install --no-cache-dir mpi4py
export HDF5_MPI="ON"
sudo pip3 install --no-binary=h5py h5py

[oskooi]

Minor correction: change the line mkdir ~/install to [ ! -d ~/install ] && mkdir ~/install

[ChristopherHogan]

To add sudo to the pip3 install lines, the --user flag needs to be removed

I only added sudo to the line upgrading pip. mpi4py and h5py can still be installed with --user without sudo. We might want to install the latest pip with --user too in case the user might not want to upgrade the system pip. Then we wouldn't need sudo.

[ChristopherHogan]

change the line mkdir ~/install to [ ! -d ~/install ] && mkdir ~/install

Done.

[stevengj]

You can just do mkdir -p ~/install ...

[ChristopherHogan]

You can just do mkdir -p ~/install

Done.