Add wiki and a few more docs

docs/additional-resources/developer-guidelines-for-addding-new-dependent-libraries.md

@@ -0,0 +1,68 @@
+Developer Guidelines For Adding New Dependent Libraries
+======================================================
+
+# Our dependency library philosophy
+
+In short, requirements for adding a new library are:
+- indispensable functionality
+- license compatibility
+- availability for all platforms
+
+## Indispensable functionality
+
+In general, adding a new dependency library (of which we currently have more than a handful, e.g. Xerces-C or ZLib)
+imposes a significant integration and maintenance effort. Thus, the new library should add **indispensable functionality**.
+If the added value does not compensate for the overhead, alternative solutions encompass:
+
+- write it yourself and add to the OpenMS library (i.e. its repository) directly
+- write a TOPPAdapter which calls an external executable (placing the burden on the user to supply the executable)
+
+## License compatibility
+
+OpenMS has a BSD-3 clause license and we try hard to remove dependencies of GSL-like libraries. Therefore, a new library
+with e.g. LGPL-2 would be prohibitive.
+
+
+## C++ standard compatibility
+
+Needs to be compatible and therefore compilable with the same C++ standard as OpenMS.
+
+## Availability for all platforms
+
+OpenMS is meant to run and behave the same on the three main operating systems, i.e., Windows, macOS and Linux. Adding
+a new dependent library is thus required to be available on these platforms
+
+- on **WindowsOS** this usually means, adding the new library to the Contrib in debug and release variants. In short all
+  recent versions of Visual Studio (VS2008 and onwards) must be supported (or support must be added). This encompasses
+  - a solution file (which can be either statically present or generated by a meta-system like CMake) is available
+  - the lib actually compiles and is linked to the **dynamic** VS-C++ runtime lib (since this is what the OpenMS lib will
+  link to as well - combining static and dynamic links will lead to linker errors or segfaults).
+
+- on **macOS** it should be ensured that the library can be build on recent macOS versions (> 10.10) compiled using the
+  mac specific _libc++_. Ideally the package should be available via **HomeBrew** or **MacPorts** so we can directly use
+  those libraries instead of shipping them via the contrib. Additionally, the MacPorts and HomeBrew formulas for building
+  the libraries can serve as blueprints on how to compile the library in a generic setting inside the contrib which should
+  also be present.
+
+- on **Linux** since we (among other distributions) feature an OpenMS Debian package which requires that all dependencies
+  of OpenMS are available as Debian package as well, the new library must be available (or made available) as Debian
+  package or linked statically during the OpenMS packaging build.
+
+## How to add it to the contrib build
+
+Add a CMake file to `OpenMS/contrib` into the `libraries.cmake` folder on how to build the library. Preferably of course
+the library supports building with CMake (see Xerces) which makes the script really easy. It should support static and
+dynamic builds on every platform. Add the compile flag for position independent code (e.g. `-fpic`) in the static version.
+Add patches in the *patches* folder and call them with the macros in the `macros.cmake` file. Create patches with
+`diff -Naur original_file my_file > patch.txt`. If there are problems during applying a patch, make sure to double check
+filepaths in the head of the patch and the call of the patching macro in CMake.
+
+- All the libraries need to go into (e.g. copied/installed/moved) to `$buildfolder/lib`
+- All the headers under `$buildfolder/include/$libraryname` (the only exception to leave out the library name subfolder
+  is when the `Find$libraryname.cmake` does not support this subfolder e.g. because system libraries are not structured
+  like this, see boost).
+- All needed files into `$buildfolder/share/$libraryname`
+
+Then test the build on your platform including a subsequent build of OpenMS using that library. Submit a pull request to
+`OpenMS/contrib`. Submit a pull request to `OpenMS/OpenMS` that updates the contrib submodule. Make sure the libraries
+are correctly shipped in pyOpenMS and the packages (especially dynamic libraries and especially on Windows).

docs/additional-resources/external-code-using-openms.md

@@ -0,0 +1,120 @@
+External Code using OpenMS
+==========================
+
+If OpenMS' TOPP and UTILS tools are not enough in a certain scenario, you can either request a change to OpenMS, if you
+feel this functionality is useful for others as well, or modify/extend OpenMS privately. For the latter, there are multiple
+ways to do this:
+
+- Modify the developer version of OpenMS by changing existing tools or adding new ones.
+- Use an **External Project** to write a new tool, while not touching OpenMS itself (see below on how to do that).
+
+Once you've finished your new tool, and it only needs to run on the development machine. To ship it to a new client machine,
+see, read further in this document.
+
+# Compiling external code
+
+It is very easy to set up an environment to write your own programs using OpenMS. Make sure to downloaded and installed
+the source package of OpenMS/TOPP properly.
+
+```{note}
+You cannot use the `install` target when working with the development version of OpenMS, it must be built and used within
+the build tree.
+```
+
+All important compiler settings and preprocessor definitions along with the OpenMS library are available. The most
+important variables are:
+
+- `OpenMS_INCLUDE_DIRECTORIES`:  all include directories containing OpenMS headers
+- `OPENMS_ADDCXX_FLAGS`: preprocessor macros we require written as `(-DMACRO1 -DMACRO2)`
+
+and the OpenMS target itself (which you can link against).
+
+The example that follows will be explained in details:
+
+```cpp
+### example CMakeLists.txt to develop C++ programs using OpenMS
+project("Example_Project_using_OpenMS")
+cmake_minimum_required(VERSION 3.0)
+
+## list all your executables here (a corresponding .cpp file should exist, e.g. Main.cpp)
+set(my_executables
+  Main
+)
+
+## list all classes here, which are required by your executables
+## (all these classes will be linked into a library)
+set(my_sources
+  ExampleLibraryFile.cpp
+)
+
+## find OpenMS configuration and register target "OpenMS" (our library)
+find_package(OpenMS)
+## if the above fails you can try calling cmake with -D OpenMS_DIR=/path/to/OpenMS/
+## or modify the find_package() call accordingly
+## find_package(OpenMS PATHS "</path/to/OpenMS//")
+
+# check whether the OpenMS package was found
+if (OpenMS_FOUND)
+  message(STATUS "\nFound OpenMS at ${OpenMS_DIR}\n")
+
+  ## library with additional classes from above
+  add_library(my_custom_lib STATIC ${my_sources})
+
+  ## add targets for the executables
+  foreach(i ${my_executables})
+    add_executable(${i} ${i}.cpp)
+    ## link executables against OpenMS
+    target_link_libraries(${i} OpenMS my_custom_lib)
+  endforeach(i)
+
+
+else(OpenMS_FOUND)
+  message(FATAL_ERROR "OpenMSConfig.cmake file not found!")
+endif(OpenMS_FOUND)
+```
+
+The command `project` defines the name of the project, the name is only of interest of you're working in an IDE or want
+to export this project's targets. To compile the program, append it to the `my_executables` list. If you use object files
+(classes which do not contain a main program), append them to the `my_sources` list. In the next step CMake creates a
+statically linked library of the object files, listed in `my_sources`. This simple CMakeLists.txt example can be
+extended to also build shared libraries, include other external libraries and so on.
+
+An example external project can be found in `OpenMS/share/OpenMS/examples/external_code`. Copy these files to a separate
+directory and use CMake to configure it (here as an in-source build).
+
+```bash
+cd <path_to_external_project>
+cmake -G "<generator>" .
+```
+
+For more information visit the website of cmake at cmake.org and consult the documentation.
+
+```{important}
+Have fun coding with OpenMS!
+```
+
+# Shipping external code to a new machine
+
+If you've modified OpenMS itself and not used an external project use our installer scripts, to build your own OpenMS
+installer for your platform (see our internal FAQ which is built using "make doc_internal") and ship that to a client
+machine.
+
+If you've used an external project and have a new executable (+ an optional new library), use the installer approach as
+well, and manually copy the new executable to the `TOPP/UTILS` binary directory (e.g. on Windows this could be
+`c:/program files/OpenMS/bin`, on Linux it could be `/bin`.
+
+If you do NOT use the installer, copy all required files manually, plus a few extra steps, see below. What needs to be
+done is a little platform dependent, thus very cumbersome to explain. Look at the cmake installer scripts, to see whats
+required (for Mac and Linux see `OpenMS/cmake/package*.cmake`).
+
+In short:
+
+- copy the `OpenMS/share/OpenMS` directory to the client machine (e.g `<client/my_dir>/share`) and set the environment
+  variable `OPENMS_DATA_PATH` to this directory
+- copy the OpenMS library (`OpenMS.dll` for Windows or `OpenMS.so/.dylib` for Linux/Mac) to `<client/my_dir>/bin`.
+- copy all Qt4 libraries to the client `<client/my_dir>/bin` or on Linux/Mac make sure you have installed the Qt4 package
+- [Windows only] copy Xerces dll (see `contrib/lib`) to `<client/my_dir>/bin`
+- [Windows only] install the VS redistributable package (see Microsoft Homepage) on the client machine which corresponds
+  to the VS version that was used to compile your code (use the correct redistributable package!, i.e., architecture
+  32|64bit, VS version, VS Service Pack version). If you choose the wrong redistributable package, you will get
+  "Application failed to initialize properly..." error messages.

docs/advanced-resources/build-custom-openms-knime-package.md

@@ -0,0 +1,115 @@
+Build Custom OpenMS KNIME package
+================================
+
+The following guidelines should help in preparing your own KNIME package. If you still encounter any errors, please
+[contact us](../contact-us.md).
+
+## Prerequisites
+
+- **KNIME SDK**: Download it from the [KNIME Download Site](https://www.knime.com/downloads) (at the end of the page).
+  We will use Version 2.9.2 (We assume that you have installed it to ~/Development/knime/eclipse_knime_2.9.2 but it could
+  be anywhere).
+- **Apache Ant**: The Generic KNIME Plugins project uses Apache Ant as the build system. On Linux and Mac, install it
+  using your package manager. For Windows, see the [Apache Ant Downloads](https://ant.apache.org/bindownload.cgi). For
+  macOS, `brew install ant`.
+- A clone of OpenMS and a compiled contrib should be available on your system. Please check the OpenMS documentation for
+  instructions how to compile OpenMS. For simplicity we will assume that the OpenMS build tree is located in
+  `~/Development/OpenMS/build`.
+
+## Prepare your build environment and create the plugin directory
+
+To generate the source code of the OpenMS KNIME plugin, prepare the build environment. For this, download a recent version
+of the search engines distributed with OpenMS. Checkout the tested versions of the binaries matching your system from the
+OpenMS subversion repository at sourceforge. Let's assume that the search engines are located in
+`~/Development/knime/SEARCHENGINES/`.
+
+```bash
+svn export --force https://github.com/OpenMS/THIRDPARTY/trunk/All ~/Development/knime/SEARCHENGINES_ALL
+svn export --force https://github.com/OpenMS/THIRDPARTY/trunk/Linux/64bit ~/Development/knime/SEARCHENGINES
+mv "~/Development/knime/SEARCHENGINES_ALL/LuciPHOr2" "~/Development/knime/SEARCHENGINES_ALL/MSGFPlus" "~/Development/knime/SEARCHENGINES/"
+```
+
+to get the search engines for 64 bit linux build. Supported builds can be found in the [thirdparty tool repository](https://github.com/OpenMS/THIRDPARTY). Ensure that in `~/Development/knime/SEARCHENGINES` ou now have multiple folders (one per searchengine) that
+each immediately contain the respective binaries (and metafiles) for all searchengines.
+
+After downloading the search engines we can activate the knime preparation in the OpenMS build system by calling cmake
+with some specific arguments in the build directory.
+
+```
+$ cd ~/Development/OpenMS/build
+$ cmake -D SEARCH_ENGINES_DIRECTORY=$HOME/Development/knime/SEARCHENGINES/ -D ENABLE_PREPARE_KNIME_PACKAGE=On .
+```
+
+now, if you used the Makefile generator, execute the `prepare_knime_package` target, e.g.,
+
+```
+$ make prepare_knime_package
+```
+
+This should recompile OpenMS and construct all the necessary input files for the KNIME plugin generation in the directory
+`~/Development/OpenMS/build/ctds`.
+
+## Generate the plugin source code
+
+Based on the files generated in the previous step, now, generate the source code of the KNIME plugin using the
+GenericKNIMENodes NodeGenerator tool. Start by cloning the latest version of the GenericKNIMENodes into the directory
+`~/Development/knime/GenericKnimeNodes`:
+
+```bash
+$ cd ~/Development/knime
+$ git clone git://github.com/genericworkflownodes/GenericKnimeNodes.git
+```
+
+The node generator can easily be called using `ant`:
+
+```bash
+$ cd ~/Development/knime/GenericKnimeNodes
+$ ant -Dplugin.dir=$HOME/Development/OpenMS/build/ctds -Dcustom.plugin.generator.target=$HOME/Development/knime/openms_plugin/
+```
+
+This will generate the source code of the OpenMS plugin in the directory `~/Development/knime/openms_plugin/`.
+
+
+## Compiling and running the OpenMS plugin
+
+Before compiling and testing the OpenMS KNIME plugin we have to install some additional plugins necessary for building
+the OpenMS nodes. For this we start the previously downloaded KNIME SDK and click on **Help** > **Install New Software...**.
+
+ In the now open dialog we select the **KNIME Desktop Update Site**.
+
+ ![](../images/advanced-resources/KNIME-Desktop-Update-Site.png)
+
+ From the list below we select the `KNIME File Handling Nodes`, the `KNIME Build System`, and the `KNIME Testing Framework`.
+
+ ```{tip}
+KNIME Build System and KNIME Testing Framework (ships with KNIME SDK) are probably not necessary to install.
+ ```
+
+![](../images/advanced-resources/KNIME-File-Handling-Nodes.png)
+
+![](../images/advanced-resources/KNIME-Build-System.png)
+
+Now follow the installation procedure by clicking on **Next**.
+
+Now that the KNIME SDK is properly setup import the GenericKNIMENodes plugin and the generated OpenMS plugin by clicking
+on **File** > **Import...**. First select `Existing Projects into Workspace` and select the GenericKNIMENodes directory
+(`~/Development/knime/GenericKnimeNodes`). Repeat those steps with the OpenMS plugin directory
+(`~/Development/knime/openms_plugin/`).
+
+![](../images/advanced-resources/Existing-Projects-into-Workspace.png)
+
+![](../images/advanced-resources/select-root-directory.png)
+
+This should add the base plugin and the OpenMS plugin to your build environment. Note that it is necessary to copy both
+`de.openms.*` and `com.genericworkflownodes.*` into the same subfolder.
+
+Now, start a KNIME instance from within the SDK by clicking on **Run** > **Run Configurations...**. Double click on
+**Eclipse Application** to create a new run configuration.
+
+![](../images/advanced-resources/eclipse-application.png)
+
+Rename the run configuration (e.g., KNIME Testing) and select `org.knime.product.KNIME_PRODUCT` in `Run a product`.
+
+![](../images/advanced-resources/new-configuration.png)
+
+After clicking **Run** a new KNIME instance should start containing the OpenMS KNIME nodes.

docs/advanced-resources/custom-compilation.md

@@ -0,0 +1,36 @@
+Custom Compilation of OpenMS
+===========================
+
+To compile with self built compilers and non default standard libraries, follow listed steps.
+
+To choose any specific compiler, instead of the system default, add the whole path to these options for the cmake call:
+
+- GCC:
+  ```bash
+  cmake -DCMAKE_C_COMPILER=/path/to/c-compiler/binary/gcc -DCMAKE_CXX_COMPILER=/path/to/c++-compiler/binary/g++
+  ```
+
+- Clang:
+  ```bash
+  cmake -DCMAKE_C_COMPILER=/path/to/c-compiler/binary/clang -DCMAKE_CXX_COMPILER=/path/to/c++-compiler/binary/clang++
+  ```
+
+To compile OpenMS with clang and a specific GCC stdlib, instead of the system default one:
+
+Use this cmake option to specify an additional compiling option for clang:
+
+```bash
+cmake -DMY_CXX_FLAGS="--gcc-toolchain=/path/to/gcc"
+```
+
+with the path to the top gcc directory (containing the directory lib64) to the cmake call.
+
+```{warning}
+This combination does not work for all versions of clang and gcc.
+- Clang 9.0.0 and GCC 4.8.5 stdlib does not work!
+- Clang 9.0.0 and GCC 9.2.0 stdlib does not work!
+- Clang 9.0.0 and GCC 8.3.0 stdlib compiles, but some tests fail.
+- Clang 6.0.0 and GCC 7.4.0 stdlib (Ubuntu 18.04 default versions) works
+```
+
+

docs/index.rst

@@ -62,6 +62,7 @@ Contents
    :caption: OpenMS TOPP Tools
 
    topp/topp.md
+   topp/adding-new-tool-to-topp.md
 
 .. toctree::
    :maxdepth: 2
@@ -70,10 +71,19 @@ Contents
    developer-faq.md
    contributor-faq.md
 
+.. toctree::
+   :maxdepth: 2
+   :caption: Advanced Resources
+
+   advanced-resources/custom-compilation.md
+   advanced-resources/build-custom-openms-knime-package.md
+
 .. toctree::
    :maxdepth: 2
    :caption: Additional Resources
 
+   additional-resources/developer-guidelines-for-addding-new-dependent-libraries.md
+   additional-resources/external-code-using-openms.md
    additional-resources/openms-git-workflow.md
    additional-resources/reporting-bugs-and-issues.md
    additional-resources/write-and-label-github-issues.md