diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 14296434..9c192a88 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -5,14 +5,14 @@ Hi there! Thank you for thinking about enhancing OpenMS Documentation further. -Please feel free: +Please feel free to: -1. Creating a bug report or feature request in OpenMS Documentation, [here](https://github.com/OpenMS/OpenMS-docs/issues). -2. Creating a pull request in [OpenMS Documentation](https://github.com/OpenMS/OpenMS-docs) with the change you're proposing. +1. Create a bug report or feature request in OpenMS Documentation, [here](https://github.com/OpenMS/OpenMS-docs/issues). +2. Create a pull request in [OpenMS Documentation](https://github.com/OpenMS/OpenMS-docs) with the change you're proposing. -If you need help while doing any of these, drop us a ping in [OpenMS Gitter](https://gitter.im/OpenMS/OpenMS). +For any questions, drop us a ping in [OpenMS Gitter](https://gitter.im/OpenMS/OpenMS). -## Creating a Pull Request +## Create a Pull Request(PR) 1. Fork this repository. 2. Add the change in your fork. @@ -27,7 +27,22 @@ If you need help while doing any of these, drop us a ping in [OpenMS Gitter](htt 2. Restrict the line length to 120 characters, including space(s). 3. Title and H1 heading should be in `Title Case`. Follow `Sentence case`, until otherwise stated. 4. Write as OpenMS is writing for itself as an object and subject. -5. Be nice, polite, and respectful. +5. Use American English. +6. Use `backtick`(s) for formatting code, library name, files, and path. +7. Use **bold** for product name, object name, an independent entity. +8. Use **bold** for menu title in an application. +9. Link to glossary terms using {term}`this is a glossary term`. +10. OpenMS documentation uses following Admonitions + - Hint + - Important + - Note + - Warning + - Tip + - See Also + Example of these are present in documentation, please follow them. +11. Always specify lexers for code blocks. +12. Format keyboard strokes using `qwerty-keyboard-button`. +13. Be nice, polite, and respectful. ### Naming files @@ -36,13 +51,13 @@ If you need help while doing any of these, drop us a ping in [OpenMS Gitter](htt ### Images and figures -For images and figures, follow: +For images and figures: -1. Add screenshot of the window. +1. Add a screenshot of the window. 2. In tutorial, align images in center. Other instructions, should have alighment to left. 3. Please set the size to `500px` of images added in step-by-step guides or instructions. -## OpenMS Documentation Contributors +## OpenMS documentation contributors Thank you for your contribution! diff --git a/.github/ISSUE_TEMPLATE/feature-request.md b/.github/ISSUE_TEMPLATE/feature-request.md index 6a4ad6b0..ebdb28fd 100644 --- a/.github/ISSUE_TEMPLATE/feature-request.md +++ b/.github/ISSUE_TEMPLATE/feature-request.md @@ -13,14 +13,14 @@ Thanks for suggesting a new feature for the OpenMS Documentation! Please delete this text and anything that's not relevant from the template below: --> -## Is your feature request related to a OpenMS Documentation problem? Please describe +## Is your feature request related to a OpenMS API Reference Documentation? Please describe. - -- [ ] I have attached screenshot describing the problem, if applicable +- [ ] I have attached screenshot describing the problem, if applicable. ## Describe the documentation enhancement you'd like diff --git a/conf.py b/conf.py index b5029ca3..d2493add 100644 --- a/conf.py +++ b/conf.py @@ -76,9 +76,9 @@ html_theme = 'furo' html_logo = 'assets/OpenMS_transparent_background.png' -pygments_style = "sphinx" -pygments_dark_style = "monokai" +pygments_style = 'sas' +pygments_dark_style = 'rrt' # Add any paths that contain custom static files (such as style sheets) here, diff --git a/docs/additional-resources/developer-guidelines-for-addding-new-dependent-libraries.md b/docs/additional-resources/developer-guidelines-for-addding-new-dependent-libraries.md new file mode 100644 index 00000000..436bf1f4 --- /dev/null +++ b/docs/additional-resources/developer-guidelines-for-addding-new-dependent-libraries.md @@ -0,0 +1,68 @@ +Developer Guidelines For Adding New Dependent Libraries +====================================================== + +# Our dependency library philosophy + +In short, requirements for adding a new library are: +- indispensable functionality +- license compatibility +- availability for all platforms + +## Indispensable functionality + +In general, adding a new dependency library (of which we currently have more than a handful, e.g. Xerces-C or ZLib) +imposes a significant integration and maintenance effort. Thus, the new library should add **indispensable functionality**. +If the added value does not compensate for the overhead, alternative solutions encompass: + +- write it yourself and add to the OpenMS library (i.e. its repository) directly +- write a TOPPAdapter which calls an external executable (placing the burden on the user to supply the executable) + +## License compatibility + +OpenMS has a BSD-3 clause license and we try hard to remove dependencies of GSL-like libraries. Therefore, a new library +with e.g. LGPL-2 would be prohibitive. + + +## C++ standard compatibility + +Needs to be compatible and therefore compilable with the same C++ standard as OpenMS. + +## Availability for all platforms + +OpenMS is meant to run and behave the same on the three main operating systems, i.e., Windows, macOS and Linux. Adding +a new dependent library is thus required to be available on these platforms + +- on **WindowsOS** this usually means, adding the new library to the Contrib in debug and release variants. In short all + recent versions of Visual Studio (VS2008 and onwards) must be supported (or support must be added). This encompasses + - a solution file (which can be either statically present or generated by a meta-system like CMake) is available + - the lib actually compiles and is linked to the **dynamic** VS-C++ runtime lib (since this is what the OpenMS lib will + link to as well - combining static and dynamic links will lead to linker errors or segfaults). + +- on **macOS** it should be ensured that the library can be build on recent macOS versions (> 10.10) compiled using the + mac specific _libc++_. Ideally the package should be available via **HomeBrew** or **MacPorts** so we can directly use + those libraries instead of shipping them via the contrib. Additionally, the MacPorts and HomeBrew formulas for building + the libraries can serve as blueprints on how to compile the library in a generic setting inside the contrib which should + also be present. + +- on **Linux** since we (among other distributions) feature an OpenMS Debian package which requires that all dependencies + of OpenMS are available as Debian package as well, the new library must be available (or made available) as Debian + package or linked statically during the OpenMS packaging build. + +## How to add it to the contrib build + +Add a CMake file to `OpenMS/contrib` into the `libraries.cmake` folder on how to build the library. Preferably of course +the library supports building with CMake (see Xerces) which makes the script really easy. It should support static and +dynamic builds on every platform. Add the compile flag for position independent code (e.g. `-fpic`) in the static version. +Add patches in the *patches* folder and call them with the macros in the `macros.cmake` file. Create patches with +`diff -Naur original_file my_file > patch.txt`. If there are problems during applying a patch, make sure to double check +filepaths in the head of the patch and the call of the patching macro in CMake. + +- All the libraries need to go into (e.g. copied/installed/moved) to `$buildfolder/lib` +- All the headers under `$buildfolder/include/$libraryname` (the only exception to leave out the library name subfolder + is when the `Find$libraryname.cmake` does not support this subfolder e.g. because system libraries are not structured + like this, see boost). +- All needed files into `$buildfolder/share/$libraryname` + +Then test the build on your platform including a subsequent build of OpenMS using that library. Submit a pull request to +`OpenMS/contrib`. Submit a pull request to `OpenMS/OpenMS` that updates the contrib submodule. Make sure the libraries +are correctly shipped in pyOpenMS and the packages (especially dynamic libraries and especially on Windows). diff --git a/docs/additional-resources/external-code-using-openms.md b/docs/additional-resources/external-code-using-openms.md new file mode 100644 index 00000000..770b6cb5 --- /dev/null +++ b/docs/additional-resources/external-code-using-openms.md @@ -0,0 +1,120 @@ +External Code using OpenMS +========================== + +If OpenMS' TOPP and UTILS tools are not enough in a certain scenario, you can either request a change to OpenMS, if you +feel this functionality is useful for others as well, or modify/extend OpenMS privately. For the latter, there are multiple +ways to do this: + +- Modify the developer version of OpenMS by changing existing tools or adding new ones. +- Use an **External Project** to write a new tool, while not touching OpenMS itself (see below on how to do that). + +Once you've finished your new tool, and it only needs to run on the development machine. To ship it to a new client machine, +see, read further in this document. + +# Compiling external code + +It is very easy to set up an environment to write your own programs using OpenMS. Make sure to downloaded and installed +the source package of OpenMS/TOPP properly. + +```{note} +You cannot use the `install` target when working with the development version of OpenMS, it must be built and used within +the build tree. +``` + +All important compiler settings and preprocessor definitions along with the OpenMS library are available. The most +important variables are: + +- `OpenMS_INCLUDE_DIRECTORIES`: all include directories containing OpenMS headers +- `OPENMS_ADDCXX_FLAGS`: preprocessor macros we require written as `(-DMACRO1 -DMACRO2)` + +and the OpenMS target itself (which you can link against). + +The example that follows will be explained in details: + +```cpp +### example CMakeLists.txt to develop C++ programs using OpenMS +project("Example_Project_using_OpenMS") +cmake_minimum_required(VERSION 3.0) + +## list all your executables here (a corresponding .cpp file should exist, e.g. Main.cpp) +set(my_executables + Main +) + +## list all classes here, which are required by your executables +## (all these classes will be linked into a library) +set(my_sources + ExampleLibraryFile.cpp +) + +## find OpenMS configuration and register target "OpenMS" (our library) +find_package(OpenMS) +## if the above fails you can try calling cmake with -D OpenMS_DIR=/path/to/OpenMS/ +## or modify the find_package() call accordingly +## find_package(OpenMS PATHS " +cmake -G "" . +``` + +For more information visit the website of cmake at cmake.org and consult the documentation. + +```{important} +Have fun coding with OpenMS! +``` + +# Shipping external code to a new machine + +If you've modified OpenMS itself and not used an external project use our installer scripts, to build your own OpenMS +installer for your platform (see our internal FAQ which is built using "make doc_internal") and ship that to a client +machine. + +If you've used an external project and have a new executable (+ an optional new library), use the installer approach as +well, and manually copy the new executable to the `TOPP/UTILS` binary directory (e.g. on Windows this could be +`c:/program files/OpenMS/bin`, on Linux it could be `/bin`. + +If you do NOT use the installer, copy all required files manually, plus a few extra steps, see below. What needs to be +done is a little platform dependent, thus very cumbersome to explain. Look at the cmake installer scripts, to see whats +required (for Mac and Linux see `OpenMS/cmake/package*.cmake`). + +In short: + +- copy the `OpenMS/share/OpenMS` directory to the client machine (e.g `/share`) and set the environment + variable `OPENMS_DATA_PATH` to this directory +- copy the OpenMS library (`OpenMS.dll` for Windows or `OpenMS.so/.dylib` for Linux/Mac) to `/bin`. +- copy all Qt4 libraries to the client `/bin` or on Linux/Mac make sure you have installed the Qt4 package +- [Windows only] copy Xerces dll (see `contrib/lib`) to `/bin` +- [Windows only] install the VS redistributable package (see Microsoft Homepage) on the client machine which corresponds + to the VS version that was used to compile your code (use the correct redistributable package!, i.e., architecture + 32|64bit, VS version, VS Service Pack version). If you choose the wrong redistributable package, you will get + "Application failed to initialize properly..." error messages. diff --git a/docs/additional-resources/openms-git-workflow.md b/docs/additional-resources/openms-git-workflow.md new file mode 100644 index 00000000..0272eb98 --- /dev/null +++ b/docs/additional-resources/openms-git-workflow.md @@ -0,0 +1,197 @@ +OpenMS Git Workflow +=================== + +Before getting started, install latest version of git to avoid problems like GitHub https authentication errors +(see [Troubleshooting cloning errors](https://docs.github.com/en/repositories/creating-and-managing-repositories/troubleshooting-cloning-errors) and a solution using [ssh](https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories)). + +OpenMS follows the [git flow workflow](https://nvie.com/posts/a-successful-git-branching-model/). The difference is that +merge commits are managed via pull requests instead of creating merge commits locally. + +## Naming conventions + +Naming conventions for the following apply: + +* A **local repository** is the repository that lies on your hard drive after cloning. +* A **remote repository** is a repository on a git server such as GitHub. +* A **fork** is a copy of a repository. Forking a repository allows you to freely experiment with changes without affecting the original project. +* **Origin** refers to a remote repository that you have forked. Call this repository `https://github.com/_YOURUSERNAME_/OpenMS`. +* **Upstream** refers to the original remote OpenMS repository. Call this repository `https://github.com/OpenMS/OpenMS`. + +## Create fork + +Start by [forking](https://docs.github.com/en/get-started/quickstart/fork-a-repo) the OpenMS repository. + +To create a fork, click **Fork** under the main menu as shown below. + +![image info](../images/additional-resources/click-fork.png) + +## Clone your fork + +To obtain a local repository copy, clone your fork using: + +```bash +$ git clone https://github.com/_YOURUSERNAME_/OpenMS.git +``` + +This will clone your fork (correctly labelled `origin` by default) into a local copy on your computer. + +## Link remote branches to your local working repository + +After cloning your fork, your local repository should be named origin. Validate this by executing: + +```bash +$ git remote -v + origin https://github.com/_YOURUSERNAME_/OpenMS.git (fetch) + origin https://github.com/_YOURUSERNAME_/OpenMS.git (push) +``` + +Sync data between your local copy, your fork (`origin`) and the remote original OpenMS/OpenMS repository (`upstream`) +by using the following command: + +```bash +$ git remote add upstream https://github.com/OpenMS/OpenMS.git +``` +Verify that upstream was added correctly by calling: + +```bash +$ git remote -v + origin https://github.com/_YOURUSERNAME_/OpenMS.git (fetch) + origin https://github.com/_YOURUSERNAME_/OpenMS.git (push) + upstream https://github.com/OpenMS/OpenMS.git (fetch) + upstream https://github.com/OpenMS/OpenMS.git (push) + +``` + +Fetch changes and new branches from your fork (`origin`) as well as from the central, upstream OpenMS repository by executing: + +```bash +$ git fetch upstream +$ git fetch origin +``` +or + +```bash +$ git fetch --all +``` + +Create a local branch using the following: + +```bash +$ git checkout -b +``` + +Call `git branch -va` to display the status of local and remote branches. You should see an output that looks like this: + +```bash +$ git branch -va +* develop 349ec48 Merge pull request #691 from cbielow/MGF_fix + feature/my_shiny_new_feature 3c05538 [FEATURE] added option to keep, ensure or reassign UIDs during conversion + remotes/origin/SILACAnalyzer 3ceae38 Fixed test. + remotes/origin/antilope 3fe5aa3 git-svn-id: https://open-ms.svn.sourceforge.net/svnroot/open-ms/branches/antilope@12117 6adb6e08-d915-0410-941f-83917bcadc18 + remotes/origin/develop 349ec48 Merge pull request #691 from cbielow/MGF_fix + remotes/origin/master b182ba5 [NOP] first commit after SVN import to git + remotes/origin/msnovogen 93a5e4c [OPT] For faster access to specific amino acids a ResidueServer was added. + remotes/upstream/HEAD -> upstream/develop + remotes/upstream/SILACAnalyzer 3ceae38 Fixed test. + remotes/upstream/antilope 3fe5aa3 git-svn-id: https://open-ms.svn.sourceforge.net/svnroot/open-ms/branches/antilope@12117 6adb6e08-d915-0410-941f-83917bcadc18 + remotes/upstream/develop 349ec48 Merge pull request #691 from cbielow/MGF_fix + remotes/upstream/master b182ba5 [NOP] first commit after SVN import to git + remotes/upstream/msnovogen 93a5e4c [OPT] For faster access to specific amino acids a ResidueServer was added. +``` + +## Keep your fork in sync + +Keep your fork (`origin`) in sync with the OpenMS repository (`upstream`) by following the [GitHub instructions](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork). +In summary, to keep your fork in sync: +1. Fetch changes from upstream and update your local branch. +2. Push your updated local branch to your fork (`origin`). + +```{tip} +To keep track of others repositories, use `git fetch --all --prune` to update them as well. The option `--prune` tells +git to automatically remove tracked branches if they got removed in the remote repository. +``` + +```bash +$ git fetch --all --prune +$ git checkout develop +$ git merge --ff-only upstream/develop +$ git push origin develop +``` +Feel free to experiment within your fork. However, for your code needs to meet OpenMS quality standards to be merged +into the OpenMS repository, + +Follow these rules: +* Never commit directly to the `develop` or `master` branches as it will complicate the merge. +* Try to start every feature from develop and not base features on other features. +* Name the OpenMS remote `upstream` and always push directly to `origin` (`git push origin `). +* When updating your fork, consider using `git fetch upstream` followed by `git merge --ff-only upstream/develop` to + avoid creating merge commits in `develop`. +* If you never commit to `develop` this should always succeed and (if a commit accidentally went to develop) warn you + instead of creating a merge commit. + +## Create new feature + +All features start from `develop`. + +```bash +$ git checkout develop +$ git checkout -b feature/your-cool-new-feature +``` +All commits related to this feature will then go into the branch `feature/your-cool-new-feature`. + +## Keeping your feature branch in sync with develop branch + +While working on your feature branch, it is usual that development continues and new features get integrated into the +main development branch. This means your feature branch lags behind `develop`. To get your feature branch up-to-date, +rebase your feature branch on `develop` using: + +```bash +$ git checkout feature/myfeaturebranch +$ git rebase develop +``` + +The above commands: + +1. Performs a rewind of your commits until the branching point. +2. Applies all commits that have been integrated into `develop`. +3. Reapplies your commits on top of the commits integrated into `develop`. + +For more information, refer to a [visual explanation of rebasing](http://git-scm.com/book/en/v2/Git-Branching-Rebasing). + +```{tip} +Do not rebase published branches (e.g. branches that are part of a pull request). If you created a pull request, you +should only add commits in your feature branch to fix things that have been discussed. After your pull request contains +all fixes, you are ready to merge the pull request into develop without rebasing (see e.g. rebase-vs-merge). +``` + +## Adding a feature to OpenMS + +Features that should go into the main development line of OpenMS should be integrated via a [pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests). This allows +the development community of OpenMS to discuss the changes and suggest possible improvements. + +After opening the pull request via the GitHub web site, GitHub will try to create the pull request against the branch +that you branched off from. Please check the branch that you are opening the pull request against before submitting the +pull request. If any changes are made, a new pull request is required. Select +**Allow others to make changes to this pull request** so that maintainers can directly help to solve problems. + +Open pull requests only after checking code-style, documentation and passing tests. Pull requests that do not pass CI +or code review will not be merged until the problems are solved. It is recommended that you read the +[pull request guidelines](pull-request-checklist.md) before you submit a pull request. + +## Update git submodules + +Start in your local `OpenMS/OpenMS` repository (on your feature/pull request branch). + +The following example uses a submodule called `THIRDPARTY`. + +```bash +$ git submodule update --init THIRDPARTY +$ cd THIRDPARTY +# yes, in the submodules the default remote is origin +# usually you want to pull the changes from master (e.g. after your pull request to OpenMS/THIRDPARTY has been merged) +$ git pull origin master +$ cd .. +$ git status +# Make sure that you see "modified: THIRDPARTY (new commits)" +$ git commit -am "updated submodule" +``` diff --git a/docs/additional-resources/pull-request-checklist.md b/docs/additional-resources/pull-request-checklist.md new file mode 100644 index 00000000..ccb6487c --- /dev/null +++ b/docs/additional-resources/pull-request-checklist.md @@ -0,0 +1,44 @@ +Pull Request Checklist +====================== + +Before opening a pull request, check the following: + +1. **Does the code build?** + Execute `make` (or your build system's equivalent, e.g., `cmake --build . --target ALL_BUILD --config Release` on Windows). +2. **Do all tests pass?** + To check if all tests have passed, execute `ctest`. + If a test that is unrelated to your changes fails, check the [nightly builds](http://cdash.openms.de/index.php?project=OpenMS) + to see if the error is also in `develop`. If the error is in develop, [create a github issue](write-and-label-github-issues.md). +3. **Is the code documented?** + Document all new classes, including their methods and parameters. + It is also recommended to document non-public members and methods. +4. **Does the code introduce changes to the API?** + If the code introduces changes to the API, make sure that the documentation is up-to-date and that the Python bindings + (pyOpenMS) still work. For each change in the C++ API, make a change in the Python API wrapper via the `pyOpenMS/pxds/` files. +5. **Have you completed regression testing?** + Make sure that you include a test in the test suite for: + - Public methods of a class + - TOPP tools + - Bug fixes + +Make sure to: + +- **Rebase before you open a pull request.** + To include all recent changes, rebase your branch on `develop` before opening a pull request. + If you pushed your branch to `origin` before rebasing, git will most likely tell you after the rebase that your + local branch and the remote branch have diverged. If you are sure that the remote branch does not contain any local + commits in the rebased version, you can safely push using `git push -f origin ` to enforce overwrite. If + not, contact your local git expert on how to get the changes into your local branch. + +- **Capture similar changes in a single commit** + Each commit should represent one logical unit. Consolidate multiple commits if they belong together or split single + commits if they are unrelated. For example, committing code formatting together with a one-line fix makes it very hard + to figure out what the fix was and which changes were inconsequential. + +* **Create a pull request for a single feature or bug** + If you have multiple features or fixes in a pull request, you might get asked to split your request and open multiple + pull requests instead. + +* **Describe what you have changed in your pull request.** + When opening the pull request, give a detailed overview of what has changed and why. Include a clear rationale for the + changes and add benchmark data if available. See [this request](https://github.com/bitly/dablooms/pull/19) for an example. diff --git a/docs/additional-resources/reporting-bugs-and-issues.md b/docs/additional-resources/reporting-bugs-and-issues.md new file mode 100644 index 00000000..3ac63aeb --- /dev/null +++ b/docs/additional-resources/reporting-bugs-and-issues.md @@ -0,0 +1,19 @@ +Reporting Bugs and Issues +========================= + +A list of known issues in the current OpenMS release can be found [here](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/known_dev_bugs.html). Please check if your OpenMS version matches the current version and if the bug +has already been reported. + +In order to report a new bug, please create a [GitHub issue](write-and-label-github-issues.md) or [contact us](../contact-us.md). + +Include the following information in your bug report: + +1. The command line (i.e. call) including the TOPP tool and the arguments you used, or the steps you followed in a GUI + tool (e.g. TOPPView) - e.g. `FeatureFinderCentroided -in myfile.mzML -out myfile.featureXML`. +2. The output of OpenMS/TOPP (or a screenshot in case of a GUI problem). +3. Operating system (e.g. "Windows XP 32 bit", "Win 7 64 bit", "Fedora 8 32 bit", "macOS 10.6 64 bit"). +4. OpenMS version (e.g. "OpenMS 1.11.1", "Revision 63082 from the SVN repository"). +5. OpenMS architecture ("32 bit" or "64 bit") + +Please provide files that we need to reproduce the bug (e.g. `TOPP INI` files, data files — usually mzML) via a download +link, via the mailing list or by directly contacting one of the developers. diff --git a/docs/additional-resources/write-and-label-github-issues.md b/docs/additional-resources/write-and-label-github-issues.md new file mode 100644 index 00000000..f1f43192 --- /dev/null +++ b/docs/additional-resources/write-and-label-github-issues.md @@ -0,0 +1,24 @@ +Write and Label GitHub Issues +============================= + +## Create an Issue + +To create an issue: + +1. Go to the [OpenMS codebase](https://github.com/OpenMS/OpenMS). +2. Submit an [issue](https://github.com/OpenMS/OpenMS/issues/new). + +The issue will be listed under **Issues**. + +## Label an Issue + +To label an issue: + +1. On the right of the screen, select the cog icon under **Labels**. +2. Choose a label from the list. Normally, an issue can have one or more of the following labels: + - **defect**: A defect refers to a bug in OpenMS. This is a high priority issue. + - **enhancement**: An enhancement refers to a feature idea to enhance the current OpenMS code. This is a medium + priority issue. + - **task**: A task refers to a single piece of work that a developer can undertake. This is a medium priority issue. + - **refactoring**: A refactoring issue refers to a suggestion to streamline the code without changing how the code function. + - **question**: A question could trigger to a discussion about tools, parameters and scientific tasks. diff --git a/docs/advanced-resources/build-custom-openms-knime-package.md b/docs/advanced-resources/build-custom-openms-knime-package.md new file mode 100644 index 00000000..24d45584 --- /dev/null +++ b/docs/advanced-resources/build-custom-openms-knime-package.md @@ -0,0 +1,115 @@ +Build Custom OpenMS KNIME package +================================ + +The following guidelines should help in preparing your own KNIME package. If you still encounter any errors, please +[contact us](../contact-us.md). + +## Prerequisites + +- **KNIME SDK**: Download it from the [KNIME Download Site](https://www.knime.com/downloads) (at the end of the page). + We will use Version 2.9.2 (We assume that you have installed it to ~/Development/knime/eclipse_knime_2.9.2 but it could + be anywhere). +- **Apache Ant**: The Generic KNIME Plugins project uses Apache Ant as the build system. On Linux and Mac, install it + using your package manager. For Windows, see the [Apache Ant Downloads](https://ant.apache.org/bindownload.cgi). For + macOS, `brew install ant`. +- A clone of OpenMS and a compiled contrib should be available on your system. Please check the OpenMS documentation for + instructions how to compile OpenMS. For simplicity we will assume that the OpenMS build tree is located in + `~/Development/OpenMS/build`. + +## Prepare your build environment and create the plugin directory + +To generate the source code of the OpenMS KNIME plugin, prepare the build environment. For this, download a recent version +of the search engines distributed with OpenMS. Checkout the tested versions of the binaries matching your system from the +OpenMS subversion repository at sourceforge. Let's assume that the search engines are located in +`~/Development/knime/SEARCHENGINES/`. + +```bash +svn export --force https://github.com/OpenMS/THIRDPARTY/trunk/All ~/Development/knime/SEARCHENGINES_ALL +svn export --force https://github.com/OpenMS/THIRDPARTY/trunk/Linux/64bit ~/Development/knime/SEARCHENGINES +mv "~/Development/knime/SEARCHENGINES_ALL/LuciPHOr2" "~/Development/knime/SEARCHENGINES_ALL/MSGFPlus" "~/Development/knime/SEARCHENGINES/" +``` + +to get the search engines for 64 bit linux build. Supported builds can be found in the [thirdparty tool repository](https://github.com/OpenMS/THIRDPARTY). Ensure that in `~/Development/knime/SEARCHENGINES` ou now have multiple folders (one per searchengine) that +each immediately contain the respective binaries (and metafiles) for all searchengines. + +After downloading the search engines we can activate the knime preparation in the OpenMS build system by calling cmake +with some specific arguments in the build directory. + +``` +$ cd ~/Development/OpenMS/build +$ cmake -D SEARCH_ENGINES_DIRECTORY=$HOME/Development/knime/SEARCHENGINES/ -D ENABLE_PREPARE_KNIME_PACKAGE=On . +``` + +now, if you used the Makefile generator, execute the `prepare_knime_package` target, e.g., + +``` +$ make prepare_knime_package +``` + +This should recompile OpenMS and construct all the necessary input files for the KNIME plugin generation in the directory +`~/Development/OpenMS/build/ctds`. + +## Generate the plugin source code + +Based on the files generated in the previous step, now, generate the source code of the KNIME plugin using the +GenericKNIMENodes NodeGenerator tool. Start by cloning the latest version of the GenericKNIMENodes into the directory +`~/Development/knime/GenericKnimeNodes`: + +```bash +$ cd ~/Development/knime +$ git clone git://github.com/genericworkflownodes/GenericKnimeNodes.git +``` + +The node generator can easily be called using `ant`: + +```bash +$ cd ~/Development/knime/GenericKnimeNodes +$ ant -Dplugin.dir=$HOME/Development/OpenMS/build/ctds -Dcustom.plugin.generator.target=$HOME/Development/knime/openms_plugin/ +``` + +This will generate the source code of the OpenMS plugin in the directory `~/Development/knime/openms_plugin/`. + + +## Compiling and running the OpenMS plugin + +Before compiling and testing the OpenMS KNIME plugin we have to install some additional plugins necessary for building +the OpenMS nodes. For this we start the previously downloaded KNIME SDK and click on **Help** > **Install New Software...**. + + In the now open dialog we select the **KNIME Desktop Update Site**. + + ![](../images/advanced-resources/KNIME-Desktop-Update-Site.png) + + From the list below we select the `KNIME File Handling Nodes`, the `KNIME Build System`, and the `KNIME Testing Framework`. + + ```{tip} +KNIME Build System and KNIME Testing Framework (ships with KNIME SDK) are probably not necessary to install. + ``` + +![](../images/advanced-resources/KNIME-File-Handling-Nodes.png) + +![](../images/advanced-resources/KNIME-Build-System.png) + +Now follow the installation procedure by clicking on **Next**. + +Now that the KNIME SDK is properly setup import the GenericKNIMENodes plugin and the generated OpenMS plugin by clicking +on **File** > **Import...**. First select `Existing Projects into Workspace` and select the GenericKNIMENodes directory +(`~/Development/knime/GenericKnimeNodes`). Repeat those steps with the OpenMS plugin directory +(`~/Development/knime/openms_plugin/`). + +![](../images/advanced-resources/Existing-Projects-into-Workspace.png) + +![](../images/advanced-resources/select-root-directory.png) + +This should add the base plugin and the OpenMS plugin to your build environment. Note that it is necessary to copy both +`de.openms.*` and `com.genericworkflownodes.*` into the same subfolder. + +Now, start a KNIME instance from within the SDK by clicking on **Run** > **Run Configurations...**. Double click on +**Eclipse Application** to create a new run configuration. + +![](../images/advanced-resources/eclipse-application.png) + +Rename the run configuration (e.g., KNIME Testing) and select `org.knime.product.KNIME_PRODUCT` in `Run a product`. + +![](../images/advanced-resources/new-configuration.png) + +After clicking **Run** a new KNIME instance should start containing the OpenMS KNIME nodes. diff --git a/docs/advanced-resources/custom-compilation.md b/docs/advanced-resources/custom-compilation.md new file mode 100644 index 00000000..0cba8b7e --- /dev/null +++ b/docs/advanced-resources/custom-compilation.md @@ -0,0 +1,35 @@ +Custom Compilation of OpenMS +=========================== + +To compile with self built compilers and non default standard libraries, follow listed steps. + +To choose any specific compiler, instead of the system default, add the whole path to these options for the cmake call: + +```{tab} GCC + +`cmake -DCMAKE_C_COMPILER=/path/to/c-compiler/binary/gcc -DCMAKE_CXX_COMPILER=/path/to/c++-compiler/binary/g++` + ``` +```{tab} Clang + +`cmake -DCMAKE_C_COMPILER=/path/to/c-compiler/binary/clang -DCMAKE_CXX_COMPILER=/path/to/c++-compiler/binary/clang++` +``` + +To compile OpenMS with clang and a specific GCC stdlib, instead of the system default one: + +Use this cmake option to specify an additional compiling option for clang: + +```bash +cmake -DMY_CXX_FLAGS="--gcc-toolchain=/path/to/gcc" +``` + +with the path to the top gcc directory (containing the directory lib64) to the cmake call. + +```{warning} +This combination does not work for all versions of clang and gcc. +- Clang 9.0.0 and GCC 4.8.5 stdlib does not work! +- Clang 9.0.0 and GCC 9.2.0 stdlib does not work! +- Clang 9.0.0 and GCC 8.3.0 stdlib compiles, but some tests fail. +- Clang 6.0.0 and GCC 7.4.0 stdlib (Ubuntu 18.04 default versions) works +``` + + diff --git a/docs/contact-us.md b/docs/contact-us.md index 07c4c860..3f450fd3 100644 --- a/docs/contact-us.md +++ b/docs/contact-us.md @@ -1,14 +1,15 @@ Contact Us ========= -To get in touch with us, please follow preferred medium/channel as listed below: +Join us on [Discord](https://discord.com/invite/swkxs72CHB)! -1. For [user and contributor real time Gitter chat](https://gitter.im/OpenMS/OpenMS). +You can also contact us: + +1. On the user and contributor real time [Gitter chat](https://gitter.im/OpenMS/OpenMS). 2. Drop us an email at user support [open-ms-general](https://sourceforge.net/projects/open-ms/lists/open-ms-general) mailing list. 3. To stay updated of new versions of OpenMS and releases, subscribe to [openms-announcements mailing list](https://sourceforge.net/projects/open-ms/lists/open-ms-announcements). -4. In ordered to report a new bug, please create an issue on GitHub [OpenMS](https://github.com/OpenMS/OpenMS/issues) - repository. +4. To report a new bug, create an issue on GitHub [OpenMS](https://github.com/OpenMS/OpenMS/issues) repository. -Say hi, on [OpenMS Twitter](https://twitter.com/openmsteam)! +Say hi on [OpenMS Twitter](https://twitter.com/openmsteam)! diff --git a/docs/downloads.md b/docs/downloads.md new file mode 100644 index 00000000..647f8de7 --- /dev/null +++ b/docs/downloads.md @@ -0,0 +1,42 @@ +# OpenMS Installers + +| Platform | Name | SHA256 Hash | +|----------|------|-------------| +| Windows | [OpenMS-2.8.0-Win64.exe](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/release/2.8.0/OpenMS-2.8.0-Win64.exe) | `d203985c7042b885ac1085c30a2d9f36d7609b47`| +| macOS | [OpenMS-2.8.0-macOS.dmg](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/release/2.8.0/OpenMS-2.8.0-macOS.dmg) | `d203985c7042b885ac1085c30a2d9f36d7609b47` | +| GNU/Linux | [OpenMS-2.8.0-Debian-Linux-x86_64.deb](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/release/2.8.0/OpenMS-2.8.0-Debian-Linux-x86_64.deb) | `d203985c7042b885ac1085c30a2d9f36d7609b47` | +| Source | [OpenMS-2.8.0-src.tar.gz](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/release/2.8.0/OpenMS-2.8.0-src.tar.gz) | `d203985c7042b885ac1085c30a2d9f36d7609b47` | + + +# Workflows + +| Workflow | Description | Download Link | +|----------|-------------|---------------| +|`ProteomicsLFQ_tool_and_MSstats_postprocessing` | Label-free identification and quantification using the comet search engine, the ProteomicsLFQ tool and statistical down-stream processing using MSstats. Compared to the other proteomics LFQ workflows, it is less complex as it combines quantification and inference steps in a single ProtemicLFQ tool. | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/ProteomicsLFQ_tool_and_MSstats_postprocessing.knwf?raw=true) | +|`basic_peptide_identification` | Label-free identification using the omssa search engine. | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/basic_peptide_identification.knwf?raw=true) | +|`DIAMetAlyzer` | Metabolomics assay library construction with decoy generation from DDA data and targeted DIA analysis using OpenSWATH and pyprophet for statistical validation. | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/DIAMetAlyzer.knwf?raw=true) | +|`Identification_quantification_with_inference_isobaric_epifany_MSstatsTMT` | Identification and quantification for isobaric experiments using MSGFPlus as search engine, epifany for inference and MSstatsTMT for statistical down-stream analysis. | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/Identification_quantification_with_inference_isobaric_epifany_MSstatsTMT.knwf?raw=true) | +|`labelfree_with_protein_quantification` | Label-free with protein quantification steps implemented using individual OpenMS tools | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/labelfree_with_protein_quantification.knwf?raw=true) | +|`Metabolite_Adduct_Grouping` | Quantification and identification via accurate mass based on multiple adduct grouping steps (adducts, neutral losses). | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/Metabolite_Adduct_Grouping.knwf?raw=true) | +|`Metabolite_DeNovoID` | Quantification and identification via adduct grouping and de-novo identification using SIRIUS/CSI:FingerID. | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/Metabolite_DeNovoID.knwf?raw=true) | +|`Metabolite_ID` | Quantification and identification via accurate mass based with downstream processing and visualisation. | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/Metabolite_ID.knwf?raw=true) | +|`Metabolite_SpectralID` | Identification via spectral library search for small molecules. | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/Metabolite_SpectralID.knwf?raw=true) | +|`MSstats_statPostProcessing_iPRG2015` | Post processing workflow for using MSstats based on "Example_OneTool_ProteomicsLFQ_MSstats.knwf" | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/MSstats_statPostProcessing_iPRG2015.knwf?raw=true) | +|`MSstatsTMT` | Post processing workflow for using MSstatsTMT based on "Identification_quantification_with_inference_isobaric_epifany_MSstatsTMT". | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/MSstatsTMT.knwf?raw=true) | +|`OpenSWATH` | Targeted extraction and scoring of transitions in DIA data based on an (iRT) assay library. | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/OpenSWATH.knwf?raw=true) | +|`Phosphoproteomics_ID` | Identification of Phosphorilation sites. | [Download](https://github.com/OpenMS/Tutorials/blob/master/Workflows/Phosphoproteomics_ID.knwf?raw=true) | + +# OpenMS Releases + +| Release | Installers | +|--------------------------------------------------------|-------------| +| Stable release | [Archive Link](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/release/) | +| Release candidates | [Archive Link](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/RC/) | +| Nightly release | [Archive Link](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/nightly/) | + +# Other Resources + +| Name | Description | Download Link | +|------|-------------|---------------| +| Schemas | Documented schemas of the OpenMS formats | [Download](https://github.com/OpenMS/OpenMS/tree/develop/share/OpenMS/SCHEMAS)| +| iPRG2016 data | Dataset mxMLs, Fasta database, Identification file (idXML), Big Data (idXML) | [Download](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Tutorials/Data/iPRG2016/) | diff --git a/docs/faqs/contributor-faq.md b/docs/faqs/contributor-faq.md new file mode 100644 index 00000000..71e930ce --- /dev/null +++ b/docs/faqs/contributor-faq.md @@ -0,0 +1,250 @@ +Contributor FAQ +=============== + +The following contains answers to typical questions from contributors about OpenMS. + +## General + +The following section provides general information to new contributors. + +### I am new to OpenMS. What should I do first? + +* Check out the development version of OpenMS (see OpenMS [releases and installers](downloads/releases.md)). +* Build OpenMS according to the installation instructions. +* Read the [OpenMS Coding Conventions](https://github.com/OpenMS/OpenMS/wiki/Coding-conventions). +* Read the [OpenMS Tutorial](../tutorials/user-tutorial.md). +* Create a GitHub account. +* Subscribe to the [open-ms-general](https://sourceforge.net/projects/open-ms/lists/open-ms-general) or [contact us](../contact-us.md). + +### What is the difference between an OpenMS tool and util? + +A tool starts its lifecycle in `UTILS` and may exist without being thoroughly tested. Tools may be promoted from `UTILS` +to `TOOLS` if they are stable enough, are fully tested, fully documented, and a test workflow exists. + +### I have written a class for OpenMS. What should I do? + +Follow the [OpenMS coding conventions](). + +Coding style (brackets, variable names, etc.) must conform to the conventions. + +* The class and all the members must be documented thoroughly. +* Check your code with the tool `tools/checker.php`. Call `php tools/checker.php` for detailed instructions. + +Please open a pull request and follow the [pull request guidelines](pull-request-checklist.md). + +## Troubleshooting + +The following section provides information about how to troubleshoot common OpenMS issues. + +### OpenMS complains about boost not being found + +`CMake` got confused. Set up a new build directory and try again. Build from source, deleting the `CMakeCache.txt` +and `cmake` directory might help. + +## Build System + +The following questions are related to the build system. + +### What is CMake? + +`CMake` builds BuildSystems for different platforms, e.g. VisualStudio Solutions on Windows, Makefiles on Linux etc. +This allows us to define in one central location (namely `CMakeLists.txt`) how OpenMS is build and have the platform +specific stuff handled by `CMake`. View the [cmake website](http://www.cmake.org) for more information. + +### How do I use CMake? + +See Installation instructions for your platform. +In general, call `CMake(.exe)` with some parameters to create the native build-system. +Afterwards, (but usually) don't have to edit the current configuration using a GUI named `ccmake` +(or `CMake-GUI` in Windows), which ships with `CMake`). + +```{note} +Whenever `ccmake` is mentioned in this document, substitute this by `CMake-GUI` if your OS is Windows. Edit +the `CMakeCache.txt` file directly. +``` + +### How do I generate a build-system for Eclipse, KDevelop, CodeBlocks etc? + +Type `cmake` into a console. This will list the available code generators available on your platform, pass them to `CMake` +using the `-G` option. + +### How do I switch to debug or release configuration? + +For Makefile generators (typically on Linux), set the `CMAKE_BUILD_TYPE` variable to either Debug or Release by calling +`ccmake`. For Visual Studio, this is not necessary as all configurations are generated and choose the one you like within +the IDE itself. The 'Debug' configuration enabled debug information. The 'Release' configuration disables debug +information and enables optimisation. + +### How do I add a new class to the build system? + +1. Create the new class in the corresponding sub-folder of the sub-project. The header has to be created in + `src//include/OpenMS` and the cpp file in `src//source`, e.g., + `src/openms/include/OpenMS/FORMAT/NewFileFormat.h` and `src/openms/source/FORMAT/NewFileFormat.cpp`. +2. Add both to the respective `sources.cmake` file in the same directory (e.g., `src/openms/source/FORMAT/` and + `src/openms/include/OpenMS/FORMAT/`). +3. Add the corresponding class test to `src/tests/class_tests//` + (e.g., `src/tests/class_tests/openms/source/NewFileFormat_test.cpp`). +4. Add the test to the `executables.cmake` file in the test folder (e.g., `src/tests/class_tests/openms/executables.cmake`). +5. Add them to git by using the command `git add`. + +### How do I add a new directory to the build system? + +1. Create two new `sources.cmake` files (one for `src//include/OpenMS/MYDIR`, one for + `src//source/MYDIR`), using existing `sources.cmake` files as template. +2. Add the new `sources.cmake` files to `src//includes.cmake` +3. If you created a new directory directly under `src/openms/source`, then have a look at `src/tests/class_tests/openms/executables.cmake`. +4. Add a new section that makes the unit testing system aware of the new (upcoming) tests. +5. Look at the very bottom and augment `TEST_executables`. +6. Add a new group target to `src/tests/class_tests/openms/CMakeLists.txt`. + +## Debugging + +The following section provides information about how to debug your code. + +### How do I run a single test? + +Execute an OpenMS class test using the CTest regular expressions: + +```bash + +$ ctest -V -R "^_test" + +# To build a class test, call the respective make target in ./source/TEST: + +$ make _test +``` +To run a TOPP test, use: + +```bash + +$ ctest -V -R "TOPP_" +``` + +To build the tool, use: + +```bash +$ make +``` +### How do I debug uncaught exceptions? + +Dump a core if an uncaught exception occurs, by setting the environment variable `OPENMS_DUMP_CORE`. + +Each time an uncaught exception occurs, the `OPENMS_DUMP_CORE` variable is checked and a segmentation fault is caused, +if it is set. + +### (Linux) Why is no core dumped, although a fatal error occured? + +The `ulimit -c` unlimited command. It sets the maximum size of a core to unlimited. + +```{attention} +We observed that, on some systems, no core is dumped even if the size of the core file is set to unlimited. We are not +sure what causes this problem +``` + +### (Linux) How can I set breakpoints in gdb to debug OpenMS? + +Debug the TOPPView application to stop at line 341 of SpectrumMDIWindow.C. + +1. Enter the following in your terminal: + + ```bash + Run gdb: + shell> gdb TOPPView +``` + +2. Start the application (and close it): + + ```bash + gdb> run [arguments] +``` + +3. Set the breakpoint: + ```bash + gdb> break SpectrumMDIWindow.C:341 +``` + +4. Start the application again (with the same arguments): + + ```bash + gdb> run + ``` + +## Cross-platform thoughts + +OpenMS runs on three major platforms. Here are the most prominent causes of "it runs on Platform A, but not on B. What now?" + +### Reading or writing binary files + +Reading or writing binary files causes different behaviour. Usually Linux does not make a difference between text-mode +and binary-mode when reading files. This is quite different on Windows as some bytes are interpreted as `EOF`, which +lead might to a premature end of the reading process. + +If reading binary files, make sure to explicitly state that the file is binary when opening it. + +During writing in text-mode on Windows a line-break (`\n`) is expanded to (`\r\n`). Keep this in mind or use the +`eol-style` property of subversion to ensure that line endings are correctly checked out on non-Windows systems. + +### `UInt` vs `Size` + +Both `unsigned int` vs `size_t` `UInt` and `Size` have the same size on Linux GCC (32 bit on 32 bit systems, 64 bit on +64 bit systems), however on Windows this only holds for 32 bit. On a 64 bit Windows, the `UInt` type is still 32 bit, +while the `Size` type is 64bit. This might lead to warnings (at best) or overflows and other drawbacks. + +Therefore, do not assume that `UInt` is equal to `Size`. + +### Paths and system functions + +Avoid hardcoding e.g.`String tmp_dir = "/tmp";`. This will fail on Windows. Use Qt's `QDir` to get a path to the systems +temporary directory if required. + +Avoid names like uname which are only available on Linux. + +When working with files or directories, it is usually safe to use "/" on all platforms. Take care of spaces in directory +names though. Always quote paths if they are used in a system call to ensure that the subsequent interpreter +takes the spaced path as a single entity. + +## Doxygen Documentation + +### Where can I find the definition of the main page? + +Find a definition of the main page [here](https://github.com/OpenMS/OpenMS/edit/develop/doc/doxygen/public/Main.doxygen). + +### Where can I add a new module? + +Add a new module [here](https://github.com/OpenMS/OpenMS/edit/develop/doc/doxygen/public/Modules.doxygen). + + +### How is the command line documentation for TOPP/UTILS tools created? + +The program `OpenMS/doc/doxygen/parameters/TOPPDocumenter.cpp` creates the command line documentation for all classes +that are included in the static `ToolHandler.cpp` tools list. It can be included in the documentation using the following `doxygen` command: + +`@verbinclude TOPP_.cli` + +Test if everything worked by calling `make doc_param_internal`. The command line documentation is written to +`OpenMS/doc/doxygen/parameters/output/`. + +### What are the important files for adding a new tutorial section? + +View the following OpenMS tutorials: + +* `OpenMS/doc/OpenMS_tutorial/refman_overwrite.tex.in` (for PDF tutorials) +* `OpenMS/doc/doxygen/public/OpenMS_Tutorial_html.doxygen` (for html tutorials) + +View the following TOPP and TOPPView tutorials: + +* `OpenMS/doc/TOPP_tutorial/refman_overwrite.tex.in` (for PDF tutorials) +* `OpenMS/doc/doxygen/public/TOPP_Tutorial_html.doxygen` (for html tutorials) + +## Bug Fixes + +### How do I contribute to a bug fix? + +To contribute to a bug fix: + +1. Submit the bug as a GitHub issue. +2. Create a feature branch (e.g. `feature/fix_missing_filename_issue_615`) from your (up-to-date) develop branch in your fork of OpenMS. +3. Fix the bug and add a test. +4. Create a pull request for your branch. +5. After approval and merge make sure the issue is closed. + diff --git a/docs/faqs/developer-faq.md b/docs/faqs/developer-faq.md new file mode 100644 index 00000000..81b46548 --- /dev/null +++ b/docs/faqs/developer-faq.md @@ -0,0 +1,399 @@ +Developer FAQ +============= + +The following contains answers to typical questions from developers about OpenMS. + +## General + +The following section provides general information to new contributors. + +### I am new to OpenMS. What should I do first? + +* Check out the development version of OpenMS (see website). +* Build OpenMS by following the [installation instructions]() or [from source](../installations/build-openms-from-source.md). +* Read the [OpenMS Coding Conventions]() +* Read the [OpenMS Tutorial](../tutorials/user-tutorial.md). +* Create a GitHub account. +* Subscribe to the [open-ms-general](https://sourceforge.net/projects/open-ms/lists/open-ms-general) or [contact-us](../contact-us.md). + +### What is the difference between an OpenMS tool and util? + +A tool starts its lifecycle in `UTILS` and may exist without being thoroughly tested. Tools may be promoted from `UTILS` +to `TOOLS` if they are stable enough, are fully tested, fully documented, and a test workflow exists. + +### I have written a class for OpenMS. What should I do? + +Follow the [OpenMS coding conventions](). + +Coding style (brackets, variable names, etc.) must conform to the conventions. + +* The class and all the members should be properly documented. +* Check your code with the tool `tools/checker.php`. Call `php tools/checker.php` for detailed instructions. + +Please open a pull request and follow the [pull request guidelines](pull-request-checklist.md). + +### Can I use QT designer to create GUI widgets? + +Yes. Create a class called `Widget: Create .ui-File` with `QT designer` and store it as `Widget.ui.`, add the class to +`sources.cmake`. From the .ui-File the file `include/OpenMS/VISUAL/UIC/ClassTemplate.h` is generated by the build system. + +```{note} +Do not check in this file, as it is generated automatically when needed. +``` + +Derive the class `Widget` from `WidgetTemplate`. For further details, see the `Widget.h` and `Widget.cpp` files. + +### Can the START_SECTION-macro not handle template methods that have two or more arguments? + +Insert round brackets around the method declaration. + +### Where can I find the binary installers created? + +View the binary installers at the [build archive](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/nightly/). +Please verify the creation date of the individual installers, as there may have been an error while creating the installer. + +## Troubleshooting + +The following section provides information about how to troubleshoot common OpenMS issues. + +### OpenMS complains about boost not being found but I'm sure its there + +`CMake` got confused. Set up a new build directory and try again. If you build from source (not recommended), deleting +the `CMakeCache.txt` and `cmake` directory might help. + +## Build System + +The following questions are related to the build system. + +### What is CMake? + +`CMake` builds BuildSystems for different platforms, e.g. VisualStudio Solutions on Windows, Makefiles on Linux etc. +This allows to define in one central location (namely `CMakeLists.txt`) how OpenMS is build and have the platform specific +stuff handled by `CMake`. + +View the [cmake website](http://www.cmake.org) for more information. + +### How do I use CMake? + +See Installation instructions for your platform. +In general, call `CMake(.exe)` with some parameters to create the native build-system. + +```{tip} +whenever `ccmake` is mentioned in this document, substitute this by `CMake-GUI` if your OS is Windows. Edit the +`CMakeCache.txt` file directly. +``` + +### How do I generate a build-system for Eclipse, KDevelop, CodeBlocks etc? + +Type `cmake` into a console. This will list the available code generators available on your platform; use them with +`CMake` using the `-G` option. + +### What are user definable CMake cache variables? + +They allow the user to pass options to `CMake` which will influence the build system. The most important option which +should be given when calling `CMake.exe` is: + +`CMAKE_FIND_ROOT_PATH`, which is where `CMake` will search for additional libraries if they are not found in the default +system paths. By default we add `OpenMS/contrib`. + +If you have installed all libraries on your system already, there is no need to change `CMAKE_FIND_ROOT_PATH`. For +`contrib` libraries, set the variable `CMAKE_FIND_ROOT_PATH`. + +On Windows, `contrib` folder is required, as there are no system developer packages. To pass this variable to +`CMake` use the `-D` switch e.g. `cmake -D CMAKE_FIND_ROOT_PATH:PATH="D:\\somepath\\contrib"`. + +Everything else can be edited using `ccmake` afterwards. + +The following options are of interest: + +- `CMAKE_BUILD_TYPE` To build Debug or Release version of OpenMS. Release is the default. +- `CMAKE_FIND_ROOT_PATH` The path to the `contrib` libraries. + ```{tip} + Provide more then one value here (e.g., `-D CMAKE_FIND_ROOT_PATH="/path/to/contrib;/usr/"` will search in your + `contrib` path and in `/usr` for the required libraries) + ``` +- `STL_DEBUG` Enables STL debug mode. +- `DB_TEST` (deprecated) Enables database testing. +- `QT_DB_PLUGIN` (deprecated) Defines the db plugin used by Qt. + +View the description for each option by calling `ccmake`. + +### Can I use another solver other than GLPK? + +Other solvers can be used, but by default, the build system only links to GLPK (this is how OpenMS binary packages must +be built). To to use another solver, use `cmake ... -D USE_COINOR=1 ....` and refer to the documentation of the +`LPWrapper` class. + +### How do I switch to debug or release configuration? + +For Makefile generators (typically on Linux), set the `CMAKE_BUILD_TYPE` variable to either Debug or Release by +calling `ccmake`. For Visual Studio, this is not necessary as all configurations are generated and choose the one you +like within the IDE itself. The 'Debug' configuration enabled debug information. The 'Release' configuration disables +debug information and enables optimisation. + +### I changed the `contrib` path, but re-running `CMake` won't change the library paths? + +Once a library is found and its location is stored in a cache variable, it will only be searched again if the +corresponding entry in the cache file is set to false. + +```{warning} +If you delete the `CMakeCache.txt`, all other custom settings will be lost. +``` + +The most useful targets will be shown to you by calling the targets target, i.e. make targets. + +### `CMake` can't seem to find a `Qt` library (usually `QtCore`). What now? + +`CMake` finds `QT` by looking for `qmake` in your `PATH` or for the Environment Variable `QTDIR`. Set these accordingly. + +Make sure there is no second installation of Qt (especially the MinGW version) in your local environment. +```{warning} +This might lead ``CMake`` to the wrong path (it's searching for the ``Qt*.lib`` files). +You should only move or delete the offending `Qt` version if you know what you are doing! +``` + +A save workaround is to edit the `CMakeCache` file (e.g. via `ccmake`) and set all paths relating to `QT` +(e.g. `QT_LIBRARY_DIR`) manually. + +### (Windows) What version of Visual Studio should I use? + +It is recommended to use the latest version. Get the latest `CMake`, as its generator needs to support your VS. If +your VS is too new and there is no `CMake` for that yet, you're gonna be faced with a lot of conversion issues. +This happens whenever the Build-System calls `CMake` (which can be quite often, e.g., after changes to `CMakeLists.txt`). + +### How do I add a new class to the build system? + +1. Create the new class in the corresponding sub-folder of the sub-project. The header has to be created in `src//include/OpenMS` and the `.cpp` file in `src//source`, e.g., `src/openms/include/OpenMS/FORMAT/NewFileFormat.h` and `src/openms/source/FORMAT/NewFileFormat.cpp`. +2. Add both to the respective `sources.cmake` file in the same directory (e.g., `src/openms/source/FORMAT/` and `src/openms/include/OpenMS/FORMAT/`). +3. Add the corresponding class test to `src/tests/class_tests//` (e.g., `src/tests/class_tests/openms/source/NewFileFormat_test.cpp`). +4. Add the test to the `executables.cmake` file in the test folder (e.g., `src/tests/class_tests/openms/executables.cmake`). +5. Add them to git by using the command `git add`. + +### How do I add a new directory to the build system? + +1. Create two new `sources.cmake` files (one for `src//include/OpenMS/MYDIR`, one for `src//source/MYDIR`), using existing `sources.cmake` files as template. +2. Add the new `sources.cmake` files to `src//includes.cmake` +3. If you created a new directory directly under `src/openms/source`, then have a look at `src/tests/class_tests/openms/executables.cmake`. +4. Add a new section that makes the unit testing system aware of the new (upcoming) tests. +5. Look at the very bottom and augment `TEST_executables`. +6. Add a new group target to `src/tests/class_tests/openms/CMakeLists.txt`. + +### How can I speed up the compile process of OpenMS? + +To speed up the compile process of OpenMS, use several threads. If you have several processors/cores, build OpenMS +classes/tests and `TOPP` tools in several threads. On Linux, use the `make option -j: make -j8 OpenMS TOPP test_build`. + +On Windows, Visual Studio solution files are automatically build with the `/MP` flag, such that Visual Studio uses all +available cores of the machine. + +## Release + +View [preparation of a new OpenMS release](https://github.com/OpenMS/OpenMS/wiki/Preparation-of-a-new-OpenMS-release#release_developer) to learn more about contributing to releases. + + +## Working in Integrated Development Environments (IDEs) + +### Why are there no `source/TEST` and `source/APPLICATIONS/TOPP|UTILS` folder? + +All source files added to an IDE are associated with their targets. Find the source files for each test within +its own subproject. The same is true for the `TOPP` and `UTILS` classes. + +### I'm getting the error "Error C2471: cannot update program database" + +This is a bug in Visual Studio and there is a [bug fix](http://code.msdn.microsoft.com/KB946040) Only apply it if you +encounter the error. The bug fix might have unwanted side effects! + +### Visual Studio can't read the clang-format file. + +Depending on the Visual Studio version it might get an error like `Error while formating with ClangFormat`. +This is because Visual Studio is using an outdated version of clang-format. Unfortunately there is no easy way to update +this using Visual Studio itself. There is a plugin provided by LLVM designed to fix this problem, but the plugin doesn't +work with every Visual Studio version. In that case, update clang-format manually using the pre-build clang-format binary. +Both the binary and a link to the plugin can be found [here](https://llvm.org/builds/). +To update clang-format download the binary and exchange it with the clang-format binary in your Visual Studio folder. +For Visual Studio 17 and 19 it should be located at: `C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\Llvm\bin`. + +### The indexer gets stuck at some file which ``#includes seqan`` + +It seems that SeqAn code is just too confusing for older eclipse C++ indexers. You should upgrade to eclipse galileo +(CDT 6.0.x). Also, increase the available memory limit in `eclipse.ini`, e.g. `-Xmx1024m` for one gig. + +### The parser is confused after OPENMS_DLLAPI and does not recognize standard C++ headers + +Go to ``Project -> Properties -> C/C++ Include Paths and Preprocessor Symbols -> Add Preprocessor symbol -> "OPENMS_DLLAPI="``. +This tells eclipse that the macro is defined empty. In the same dialog add an external include path to +e.g. ``/usr/include/c++/4.3.3/``, etc. The issue with C++ headers was fixed in the latest galileo release. + +Hints to resolve the OPENMS_DLLAPI issue using the ``cmake`` generator are welcome! + +## Debugging + +The following section provides information about how to debug your code. + +### How do I debug uncaught exceptions? + +Dump a core if an uncaught exception occurs, by setting the environment variable `OPENMS_DUMP_CORE`. + +Each time an uncaught exception occurs, the `OPENMS_DUMP_CORE` variable is checked and a segmentation fault is caused, +if it is set. + +### (Linux) Why is no core dumped, although a fatal error occured? + +The `ulimit -c` unlimited command. It sets the maximum size of a core to unlimited. + +```{warning} +We observed that, on some systems, no core is dumped even if the size of the core file is set to unlimited. We are not +sure what causes this problem. +``` + +### (Linux) How can I set breakpoints in gdb to debug OpenMS? + +Imagine you want to debug the TOPPView application and you want it to stop at line 341 of SpectrumMDIWindow.C. + +1. Enter the following in your terminal: + + ```bash + Run gdb: + shell> gdb TOPPView +``` + +2. Start the application (and close it): + + ```bash + gdb> run [arguments] +``` +3. Set the breakpoint: + ```bash + gdb> break SpectrumMDIWindow.C:341 +``` +4. Start the application again (with the same arguments): + + ```bash + gdb> run + ``` + +### How can I find out which shared libraries are used by an application? + +Linux: Use `ldd`. + +Windows (Visual studio console): See [Dependency Walker](http://www.dependencywalker.com/) (use x86 for 32 bit builds +and the x64 version for 64bit builds. Using the wrong version of depends.exe will give the wrong results) or +``dumpbin /DEPENDENTS OpenMS.dll``. + +### How can I get a list of the symbols defined in a (shared) library or object file? + +Linux: Use `nm `. + +Use `nm -C` to switch on demangling of low-level symbols into their C++-equivalent names. `nm` also accepts .a and .o files. + +Windows (Visual studio console): Use ``dumpbin /ALL ``. + +Use dumpbin on object files (.o) or (shared) library files (.lib) or the DLL itself e.g. `dumpbin /EXPORTS OpenMS.dll`. + +## Cross-platform thoughts + +OpenMS runs on three major platforms.. Here are the most prominent causes of "it runs on Platform A, but not on B. What now?" + +### Reading or writing binary files + +Reading or writing binary files causes different behaviour. Usually Linux does not make a difference between text-mode +and binary-mode when reading files. This is quite different on Windows as some bytes are interpreted as `EOF`, which lead +might to a premature end of the reading process. + +If reading binary files, make sure that you explicitly state that the file is binary when opening it. + +During writing in text-mode on Windows a line-break (`\n`) is expanded to (`\r\n`). Keep this in mind or use the +`eol-style` property of subversion to ensure that line endings are correctly checked out on non-Windows systems. + +### Paths and system functions + +Avoid hardcoding e.g.`String tmp_dir = "/tmp";`. This will fail on Windows. Use Qt's `QDir` to get a path to the systems +temporary directory if required. + +Avoid names like uname which are only available on Linux. + +When working with files or directories, it is usually safe to use "/" on all platforms. Take care of spaces in directory +names though. Quote paths if they are used in a system call to ensure that the subsequent interpreter +takes the spaced path as a single entity. + + +## Doxygen Documentation + +### Where can I find the definition of the main page? + +Find a definition of the main page [here](https://github.com/OpenMS/OpenMS/edit/develop/doc/doxygen/public/Main.doxygen). + +### Where can I add a new module? + +Add a new module [here](https://github.com/OpenMS/OpenMS/edit/develop/doc/doxygen/public/Modules.doxygen). + +### How is the parameter documentation for classes derived from DefaultParamHandler created? + +Add your class to the program ``OpenMS/doc/doxygen/parameters/DefaultParamHandlerDocumenter.cpp``. This program generates +a html table with the parameters. This table can then be included in the class documentation using the following +`doxygen` command:`@htmlinclude OpenMS_.parameters`. + +```{note} +Parameter documentation is automatically generated for `TOPP/UTILS` included in the static `ToolHandler.cpp` tools list. +``` + +To include TOPP/UTILS parameter documentation use following `doxygen` command: + + +`@htmlinclude TOPP_.parameters` + +or + +`@htmlinclude UTILS_.parameters` + +Test if everything worked by calling `make doc_param_internal`. The parameters documentation is written to +`OpenMS/doc/doxygen/parameters/output/`. + +### How is the command line documentation for TOPP/UTILS tools created? + +The program `OpenMS/doc/doxygen/parameters/TOPPDocumenter.cpp` creates the command line documentation for all classes +that are included in the static `ToolHandler.cpp` tools list. It can be included in the documentation using the following `doxygen` command: + +`@verbinclude TOPP_.cli` + +Test if everything worked by calling `make doc_param_internal`. The command line documentation is written to +`OpenMS/doc/doxygen/parameters/output/`. + +## Bug Fixes + +### How to contribute a bug fix? + +Read [contributor quickstart guide](../guides/contributors-quickstart-guide.md). + +### How can I profile my code? + +IBM's profiler, available for all platforms (and free for academic use): Purify(Plus) and/or Quantify. + +Windows: this is directly supported by Visual Studio (Depending on the edition: Team and above). Follow their documentation. + +Linux: + +1. Build OpenMS in debug mode (set `CMAKE_BUILD_TYPE` to `Debug`). +2. Call the executable with valgrind: `valgrind –tool=callgrind`. + ```{warning} + Other processes running on the same machine can influence the profiling. Make sure your application gets enough + resources (memory, CPU time). + ``` +3. Start and stop the profiling while the executable is running e.g. to skip initialization steps: +4. Start valgrind with the option `–instr-atstart=no`. +5. Call `callgrind -i [on|off]` to start/stop the profiling. +6. The output can be viewed with `kcachegrind callgrind.out`. + +### (Linux) How do I check my code for memory leaks? + +* Build OpenMS in debug mode (set ``CMAKE_BUILD_TYPE`` to ``Debug``). +* Call the executable with ``valgrind: valgrind --suppressions=OpenMS/tools/valgrind/openms_external.supp –leak-check=full ``. + +Common errors are: + +* ``'Invalid write/read ...'`` - Violation of container boundaries. +* ``'... depends on uninitialized variable'`` - Uninitialized variables: +* ``'... definitely lost'`` - Memory leak that has to be fixed +* ``'... possibly lost'`` - Possible memory leak, so have a look at the code + +For more information see the [`valgrind` documentation](http://valgrind.org/docs/manual/) . diff --git a/docs/glossary.md b/docs/glossary.md new file mode 100644 index 00000000..6a94a2bb --- /dev/null +++ b/docs/glossary.md @@ -0,0 +1,147 @@ +OpenMS Glossary +============== + +A glossary of common terms used throughout OpenMS documentation. + +```{glossary} +LC-MS + [Liquid Chromatography(LC)](introduction.md#liquid-chromatography-lc) and [Mass Spectrometry(MS)](introduction.md#mass-spectrometry). + +Peptides + A short chain of amino acids. + +Octadecyl(C18) + An alkyl radical C(18)H(37) derived from an octadecane by removal of one hydrogen atom. + +Mass + Mass is a measure of the amount of matter that an object contains. In comparison to often used term weight, which is + a measure of the force of gravity on that object. + +Ion + Any {term}`atom` or group of atoms that bears one or more positive or negative electrical charges. Positively charged are + cations, negavtively charged anions. + +Atom + An atom is the smallest unit of ordinary matter that forms a chemical element. + +Electrospray ionization + A technique used in mass spectrometry to produce ions using an electrospray in which a high voltage is applied to a + liquid to create an {term}`aerosol`. + +Aerosol + An aerosol is a suspension of fine solid particles or liquid droplets in air or another gas. + +Time-of-flight (TOF) + A measurement of the time taken by an object, particle of wave (be it acoustic, electromagnetic, e.t.c) to travel a + distance through a medium. + +Quadrupole mass filters + A mass filter allowing one mass channel at a time to reach the detector as the mass range is scanned. + +Orbitrap analyzers + In mass spectrometry, an ion trap mass analyzer consisting of an outer barrel-like electrode and a coaxial inner + spindle-like electrode that traps ions in an orbital motion around the spindle. + A high resoltion mass spectrometry analyzer. + +MS(1) + First stage to get a spectra. A sample is injected into the mass spectrometer, ionized, accelerated and analyzed by + mass spectrometry. + +MS(2) + Ions from MS1 spectra are then selectively fragmented and analyzed by a second stage of mass spectrometry (MS2) to + generate the spectra for the ion fragments. + +MS/MS + Tandem mass spectrometry, MS^2^, a technique where two or more mass analyzers are coupled together using an additional + reaction step to increase their abilities to analyse chemical samples. + +Collision-induced dissociation (CID) + A mass spectrometry technique to induce fragmentation of selected ions in the gas phase. Also known as Collision + induced dissociation. + +TOPP + The OpenMS Proteomics Pipeline. + +TOPPAS.app + TOPPAS allows to create, edit, open, save, and run TOPP workflows. + +MSGFPlusAdapter + Adapter for the MS-GF+ protein identification (database search) engine. More information is available [here](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_MSGFPlusAdapter.html). + +LuciphorAdapter + Adapter for the LuciPHOr2: a site localisation tool of generic post-translational modifications from tandem mass + spectrometry data. More information is available [here](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_LuciphorAdapter.html). + +pyOpenMS + pyOpenMS is an open-source Python library for mass spectrometry, specifically for the analysis of proteomics and + metabolomics data in Python. For pyOpenMS documentaion visit [this](https://pyopenms.readthedocs.io/en/latest/) link. + +TOPP Tools + All {term}`TOPP` tools can be found [here](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_documentation.html). + +UTILS + Besides TOPP, OpenMS offers a range of other tools. They are not included in TOPP as they are not part of typical + analysis pipelines. More information is present in [OpenMS UTILS Documentation)(https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/UTILS_documentation.html). + +TOPPView + TOPPView is a viewer for MS and HPLC-MS data. More information is available in [TOPPView documentation](topp/toppview.md). + +[Nightly Snapshot](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/index.html) + Untested installers and containers are known as the nightly snapshot. + +Proteomics + Proteomics is the large-scale study of proteins. + +Proteins + Proteins are vital parts of living organisms, with many functions, for example composing the structural fibers of + muscle to the enzymes that catalyze the digestion of food to synthesizing and replicating DNA. + +Mascot + Identifies peptides in MS/MS spectra via Mascot. Please find more information in the {term}`TOPP` [Documentation](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_MascotAdapter.html). + +HPLC-MS + Data produced by High performance liquid chromatography (HPLC) separates components of a mixture, whereas mass + spectrometry (MS) offers the detection tools to identify them. + +mzML + The mzML format is an open, XML-based format for mass spectrometer output files, developed with the full participation + of vendors and researchers in order to create a single open format that would be supported by all software. + +mzData + mzData was the first attempt by the Proteomics Standards Initiative (PSI) from the Human Proteome Organization (HUPO) + to create a standardized format for Mass Spectrometry data.[7] This format is now deprecated, and replaced by mzML. + +mzXML + mzXML is an open data format for storage and exchange of mass spectroscopy data, developed at the SPC/Institute for + Systems Biology. + +Spectra + Singluar of spectrum. + +Spectrum + A mass spectrum is a type of plot of the ion signal as a function of the mass-to-charge ratio. These spectra are used + to determine the elemental or isotopic signature of a sample, the masses of particles and of molecules, and to + elucidate the chemical identity or structure of molecules and other chemical compounds. + +m/z + mass to charge ratio. + +RT + Retention time (RT). + +ProteoWizard + ProteoWizard is a set of open-source, cross-platform tools and libraries for proteomics data analyses. It provides a + framework for unified mass spectrometry data file access and performs standard chemistry and LCMS dataset computations. + +OMSSA + The Open Mass Spectrometry Search Algorithm (OMSSA) is an efficient search engine for identifying {term}`MS/MS` + {term}`peptide` {term}`spectra` by searching libraries of known protein sequences. + +PepNovo + PepNovo is a de novo sequencing algorithm for {term}`MS/MS` {term}`spectra`. + +De novo peptide sequencing + A peptide’s amino acid sequence is inferred directly from the precursor peptide mass and tandem mass spectrum + ({term}`MS/MS` or {term}`MS^3`) fragment ions, without comparison to a reference proteome. +``` + diff --git a/docs/guides/contributors-quickstart-guide.md b/docs/guides/contributors-quickstart-guide.md new file mode 100644 index 00000000..81dd3732 --- /dev/null +++ b/docs/guides/contributors-quickstart-guide.md @@ -0,0 +1,75 @@ +Contribute to OpenMS +==================== + +To contribute to OpenMS: + +- Familiarise yourself with the [OpenMS online documentation](../index.rst). +- Learn how to [build OpenMS](../installations/build-openms-from-source.md). +- Check out the [OpenMS tutorial for developers](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/release/latest/html/OpenMS_tutorial.html). + +For any questions, please [contact us](../contact-us.md) at [open-ms-general](https://sourceforge.net/projects/open-ms/lists/open-ms-general) mailing list. + +## Technical documentation + +```{note} +Untested installers and containers are known as the {term}`nightly snapshot`, are released every night. They generally pass +automated continuous integration tests but no manual tests. +``` + +View the documentation for the nightly snapshot of [OpenMS develop branch](https://github.com/OpenMS/OpenMS/tree/develop) +at the [build archive](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/index.html). + +See the documentation for the [latest release](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/release/latest/html/index.html). + +## Contribution guidelines + +Before contributing to OpenMS, read information on the development model and conventions followed to maintain a coherent +code base. + +### Development model + +OpenMS follows the [Gitflow development workflow](http://nvie.com/posts/a-successful-git-branching-model/). + +Every contributor is encouraged to create their own fork (even if they are eligible to push directly to OpenMS). +To create a fork: + +1. Follow the documentation on [forking](https://help.github.com/articles/fork-a-repo). +2. Keep your fork [up-to-date](https://help.github.com/articles/syncing-a-fork). +3. Create a [pull request](https://help.github.com/articles/using-pull-requests). Before opening the pull request, please + view the [pull request guidelines](../additional-resources/pull-request-checklist.md). + +### Coding conventions + +See the manual for coding style recommended by OpenMS: [Coding conventions](https://github.com/OpenMS/OpenMS/wiki/Coding-conventions). + +```{seealso} +[C++ Guide](https://github.com/OpenMS/OpenMS/wiki/Cpp-Guide). +``` + +OpenMS automatically tests for common coding convention violations using a modified version of `cpplint`. +Style testing can be enabled using `cmake` options. We also provide a configuration file for `Uncrustify` for automated +style corrections (see `tools/uncrustify.cfg`). + +### Commit messages + +View the guidelines for commit messages: [How to write commit messages](https://github.com/OpenMS/OpenMS/wiki/HowTo---Write-Commit-Messages). + +### Automated unit tests + +Nightly tests run on different platforms. It is recommended to test on different platforms. + +```{tip} +This saves time and increases productivity during continuous integration tests. +``` + +Nightly tests: [CDASH](http://cdash.openms.de/index.php?project=OpenMS). + +## Further contributor resources + +Consider the following resources for further information: + +- **Guidelines for adding new dependency libraries**: View the guidelines for [adding new dependency libraries](). +- **Experimental installers**: We automatically build installers for different platforms. These usually contain + unstable or partially untested code. + The nightly (unstable) installers are available at the [build archive](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/nightly/). +- **Developer FAQ**: Visit the [Developer FAQ](../faqs/developer-faq.md) to get answers to frequently asked questions. diff --git a/docs/guides/user-guides/user-quickstart-guide.md b/docs/guides/user-guides/user-quickstart-guide.md index 1ec203fc..b611400a 100644 --- a/docs/guides/user-guides/user-quickstart-guide.md +++ b/docs/guides/user-guides/user-quickstart-guide.md @@ -2,65 +2,66 @@ User Quickstart Guide ==================== Read the User Quickstart guide to gain a brief understanding of key concepts and how to use the tools. For more in-depth -information, consult [OpenMS documentation](). +information, consult [OpenMS API Reference](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/index.html). ## What is OpenMS -[OpenMS.org](https://www.openms.de/) is a free, open-source C++ library with Python bindings. It is commonly used for liquid -chromatography-mass spectrometry (LC-MS) data management and analyses. OpenMS provides an infrastructure for the rapid +[OpenMS](https://www.openms.de/) is a free, open-source C++ library with Python bindings. It is commonly used for liquid +chromatography-mass spectrometry ({term}`LC-MS`) data management and analyses. OpenMS provides an infrastructure for the rapid development of mass spectrometry related software as well as a rich toolset built on top of it. OpenMS is available -under the three clause BSD licence and runs under Windows, macOS, and Linux operating systems. +under the [three clause BSD licence](https://github.com/OpenMS/OpenMS/blob/develop/LICENSE) and runs under Windows, macOS, and Linux operating systems. ## Background -Before using [OpenMS.org](https://www.openms.de/), you need to be familiar with the following terms: +Before using OpenMS, become familiar with the following terms: | Tool and Utilities | Description | |--------------------|-------------| -|**TOPPView** |A design tool that is used to view and explore LC-MS data, alignments, groups, peptide identifications, and more.| -|**TOPPAS** |A graphical workflow design tool that is used to create pipelines from all TOPP tools (and UTILS).| -|**TOPP tools** |A set of command line tools. Each of these command line tools is a building block of an analysis pipeline and are chained together in a way that fits the requirements of the user. The TOPP tools are accessible from a command prompt/shell or via TOPPAS. See also: [TOPP tutorial]() and [TOPP documentation](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_documentation.html)| -|**UTILS** |Similar to TOPP tools, but with more supporting character, which are rarely used in a productive pipeline, but rather during pipeline construction or parameter optimization. See also: [UTILS documentation](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/UTILS_documentation.html)| +|**TOPPView** |A design tool that is used to view and explore {term}`LC-MS` data, alignments, groups, peptide identifications, and more.| +|**TOPPAS** |A graphical workflow design tool that is used to create pipelines from all {term}`TOPP tools` (and {term}`UTILS`).| +|**TOPP tools** |A set of command line tools. Each of these command line tools is a building block of an analysis pipeline and are chained together in a way that fits the requirements of the user. The {term}`TOPP tools` are accessible from a command prompt/shell or via {term}`TOPPAS`. See also: [TOPP tutorial](../../tutorials/TOPP/TOPP-tutorial.md) and [TOPP documentation](../../topp/topp.md)| +|**UTILS** |Similar to {term}`TOPP tools`, but with more supporting character, which are rarely used in a productive pipeline, but rather during pipeline construction or parameter optimization. See also: [UTILS documentation](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/UTILS_documentation.html)| ## How to run a Tool -It is recommended to use TOPPAS. A good start are the example pipelines (select **File** > **Open example file** within TOPPAS). -In parallel read the documentation of the tools (see [TOPP tutorial](), [TOPP documentation](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_documentation.html)) and the one of TOPPAS ([TOPPAS tutorial](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_documentation.html)). +A good start are the example pipelines (select **File** > **Open example file** within {term}`TOPPAS`). -Alternatively, you can use the command line and call tools directly. In this case you'll probably want to use some type of shell +Read the documentation of the tools see [TOPP tutorial](../../tutorials/TOPP/TOPP-tutorial.md), [TOPP documentation](../../topp/topp.md) and the one of ([TOPPAS tutorial](../../tutorials/TOPPAS/TOPPAS-tutorial.md)). + +Alternatively, use the command line and call tools directly. In this case, you'll probably want to use some type of shell script for automation. -## Adapting your Pipeline Parameters +## Adapte pipeline parameters The default parameters of each tool can usually be tweaked to fit the data and improve results. -### Where do you change them? +### Where do you change pipeline parameters? 1. **TOPPAS**: Double-click the node of which you want to change the parameters of. A short docu for each parameter will show up once it is selected. All parameters which would be available on the command line and in the INI file are shown here as well. 2. **Command line**: Very basic parameters can be set on the command line, e.g. `FileFilter -rt 1000:2000 .....` -3. Doing 2 for all parameters would create a very long list, thus, use so-called ".ini" files to provide full parameter - sets to TOPP tools. If no INI file is given, default parameters are used. To get a default `.ini` use +3. Doing 2 for all parameters would create a very long list, thus, use so-called `.ini` files to provide full parameter + sets to {term}`TOPP tools`. If no INI file is given, default parameters are used. To get a default `.ini` use ` -write_ini ` e.g. `FileFilter -write_ini filefilter.ini` - Now, edit the INI file (which is a XML file) using the [INIFileEditor](), which is another GUI tool shipped with - OpenMS and similar to the one build into TOPPAS. + Now, edit the INI file (which is a XML file) using the [INIFileEditor](../../topp/ini-file-editor.md), which is another GUI tool shipped with + OpenMS and similar to the one build into {term}`TOPPAS`. ### How do I feed the INI file to a Tool? -1. **TOPPAS**: Once you changed the parameters of a node and clicked "Ok", the parameters are in effect. Because - they are part of the TOPPAS workflow, they are saved together with the workflow. -2. **Command line** : Simply supply the INI file via the `-ini` flag, +1. **TOPPAS**: Once you changed the parameters of a node and clicked **Ok**, the parameters are in effect. Because + they are part of the {term}`TOPPAS` workflow, they are saved together with the workflow. +2. **Command line** : Supply the INI file via the `-ini` flag, ` -ini ` e.g. `FileFilter -ini filefilter.ini` -### What parameters require to be changed and to what value? +### What parameters to set and to what value? -This is tricky and its not possible to give a general answer. In general, read the tool description, change the -parameters and compare the results using TOPPView if possible. If that does not help, drop us an email on the -[OpenMS mailing list]() and ask. Please include all the necessary details we need in order to help you. +The answer is complex, in general, read the tool description, change the parameters and compare the results using +{term}`TOPPView` if possible. If that does not help, [contact us](../../contact-us.md). Please include all the necessary +details we need in order to help you. diff --git a/docs/images/additional-resources/click-fork.png b/docs/images/additional-resources/click-fork.png new file mode 100644 index 00000000..0ca09e1e Binary files /dev/null and b/docs/images/additional-resources/click-fork.png differ diff --git a/docs/images/advanced-resources/Existing-Projects-into-Workspace.png b/docs/images/advanced-resources/Existing-Projects-into-Workspace.png new file mode 100644 index 00000000..245226c5 Binary files /dev/null and b/docs/images/advanced-resources/Existing-Projects-into-Workspace.png differ diff --git a/docs/images/advanced-resources/KNIME-Build-System.png b/docs/images/advanced-resources/KNIME-Build-System.png new file mode 100644 index 00000000..230c84a8 Binary files /dev/null and b/docs/images/advanced-resources/KNIME-Build-System.png differ diff --git a/docs/images/advanced-resources/KNIME-Desktop-Update-Site.png b/docs/images/advanced-resources/KNIME-Desktop-Update-Site.png new file mode 100644 index 00000000..d1d3727e Binary files /dev/null and b/docs/images/advanced-resources/KNIME-Desktop-Update-Site.png differ diff --git a/docs/images/advanced-resources/KNIME-File-Handling-Nodes.png b/docs/images/advanced-resources/KNIME-File-Handling-Nodes.png new file mode 100644 index 00000000..17b4d82d Binary files /dev/null and b/docs/images/advanced-resources/KNIME-File-Handling-Nodes.png differ diff --git a/docs/images/advanced-resources/eclipse-application.png b/docs/images/advanced-resources/eclipse-application.png new file mode 100644 index 00000000..ba79e668 Binary files /dev/null and b/docs/images/advanced-resources/eclipse-application.png differ diff --git a/docs/images/advanced-resources/new-configuration.png b/docs/images/advanced-resources/new-configuration.png new file mode 100644 index 00000000..d8aff95c Binary files /dev/null and b/docs/images/advanced-resources/new-configuration.png differ diff --git a/docs/images/advanced-resources/select-root-directory.png b/docs/images/advanced-resources/select-root-directory.png new file mode 100644 index 00000000..245226c5 Binary files /dev/null and b/docs/images/advanced-resources/select-root-directory.png differ diff --git a/docs/images/introduction/introduction_LC.png b/docs/images/introduction/introduction_LC.png new file mode 100644 index 00000000..49151fad Binary files /dev/null and b/docs/images/introduction/introduction_LC.png differ diff --git a/docs/images/introduction/introduction_MS.png b/docs/images/introduction/introduction_MS.png new file mode 100644 index 00000000..e5fd7f4e Binary files /dev/null and b/docs/images/introduction/introduction_MS.png differ diff --git a/docs/images/introduction/spectrum_peakmap.png b/docs/images/introduction/spectrum_peakmap.png new file mode 100644 index 00000000..6c89ee7f Binary files /dev/null and b/docs/images/introduction/spectrum_peakmap.png differ diff --git a/docs/images/research/whitepapers/design-summary.png b/docs/images/research/whitepapers/design-summary.png new file mode 100644 index 00000000..841be1c8 Binary files /dev/null and b/docs/images/research/whitepapers/design-summary.png differ diff --git a/docs/images/research/whitepapers/summary.png b/docs/images/research/whitepapers/summary.png new file mode 100644 index 00000000..097f5c90 Binary files /dev/null and b/docs/images/research/whitepapers/summary.png differ diff --git a/docs/images/tutorials/proteowizard.png b/docs/images/tutorials/proteowizard.png new file mode 100644 index 00000000..64733b22 Binary files /dev/null and b/docs/images/tutorials/proteowizard.png differ diff --git a/docs/images/tutorials/topp/MetaDataBrowser.png b/docs/images/tutorials/topp/MetaDataBrowser.png new file mode 100644 index 00000000..93d88edf Binary files /dev/null and b/docs/images/tutorials/topp/MetaDataBrowser.png differ diff --git a/docs/images/tutorials/topp/TOPPView_statistics.png b/docs/images/tutorials/topp/TOPPView_statistics.png new file mode 100644 index 00000000..84262818 Binary files /dev/null and b/docs/images/tutorials/topp/TOPPView_statistics.png differ diff --git a/docs/images/tutorials/topp/TOPPView_tools.png b/docs/images/tutorials/topp/TOPPView_tools.png new file mode 100644 index 00000000..c7fe7cd8 Binary files /dev/null and b/docs/images/tutorials/topp/TOPPView_tools.png differ diff --git a/docs/index.rst b/docs/index.rst index 4edcef5f..c8f5be64 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -36,33 +36,71 @@ Contents introduction .. toctree:: - :maxdepth: 3 + :maxdepth: 2 :caption: Getting Started installations/installation-on-gnu-linux installations/installation-on-windows installations/installation-on-macos + installations/build-openms-from-source .. toctree:: :maxdepth: 2 - :navigation_depth: 4 :caption: Quick Start Guides guides/user-guides/user-quickstart-guide + guides/contributors-quickstart-guide.md .. toctree:: - :maxdepth: 3 + :maxdepth: 2 :caption: Tutorials tutorials/TOPP/TOPP-tutorial tutorials/TOPPAS/TOPPAS-tutorial .. toctree:: - :maxdepth: 3 + :maxdepth: 2 :caption: OpenMS TOPP Tools topp/topp.md + topp/adding-new-tool-to-topp.md + +.. toctree:: + :maxdepth: 2 + :caption: Frequently Asked Questions + + faqs/developer-faq.md + faqs/contributor-faq.md + +.. toctree:: + :maxdepth: 2 + :caption: Advanced Resources + + advanced-resources/custom-compilation.md + advanced-resources/build-custom-openms-knime-package.md + +.. toctree:: + :maxdepth: 2 + :caption: Additional Resources + + additional-resources/developer-guidelines-for-addding-new-dependent-libraries.md + additional-resources/external-code-using-openms.md + additional-resources/openms-git-workflow.md + additional-resources/reporting-bugs-and-issues.md + additional-resources/write-and-label-github-issues.md + additional-resources/pull-request-checklist.md + +.. toctree:: + :maxdepth: 2 + :caption: Downloads + + downloads.md + +.. toctree:: + :maxdepth: 2 + :caption: Glossary + glossary.md .. toctree:: :maxdepth: 2 diff --git a/docs/installations/build-openms-from-source.md b/docs/installations/build-openms-from-source.md new file mode 100644 index 00000000..52298e1c --- /dev/null +++ b/docs/installations/build-openms-from-source.md @@ -0,0 +1,10 @@ +Build OpenMS From Source +======================== + +To build OpenMS, please follow the build instructions for: + +* [Linux](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/release/latest/html/install_linux.html) +* [macOS](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/release/latest/html/install_mac.html) +* [Windows](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/release/latest/html/install_win.html) + +Instructions to build {term}`pyOpenMS` can be found in the [pyOpenMS documentation](https://pyopenms.readthedocs.io/en/latest/build_from_source.html). diff --git a/docs/installations/installation-on-gnu-linux.md b/docs/installations/installation-on-gnu-linux.md index ed01626e..731833eb 100644 --- a/docs/installations/installation-on-gnu-linux.md +++ b/docs/installations/installation-on-gnu-linux.md @@ -1,10 +1,34 @@ Installation on GNU/Linux ========================= -## Install via Debian Package +## Install via Conda + +Use conda or bioconda to install OpenMS. + +1. Follow the instructions to [install conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/macos.html). +2. Install OpenMS using conda: + `conda install -c openms openms` +3. Other OpenMS packages can be installed using: + ``` + conda install -c openms pyopenms + conda install -c openms openms-thirdparty + conda install -c openms libopenms + ``` + +To install using bioconda: + +``` +conda install -c bioconda openms +conda install -c bioconda/label/cf201901 openms +conda install -c bioconda openms-tools +conda install -c bioconda libopenms +conda install -c bioconda openms-thirdparty +``` + +## Install via Debian package For Debian-based Linux users, it is suggested to use the [deb-package](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/release/latest/) provided. It is most easily installed with **[gdebi](https://launchpad.net/gdebi)** -which automatically resolves the dependencies available in the PPA Respositories. +which automatically resolves the dependencies available in the PPA Repositories. ```bash sudo apt-get install gdebi @@ -12,20 +36,21 @@ sudo gdebi /PATH/TO/OpenMS.deb ``` If you encounter errors with unavailable packages, troubleshoot using the following steps. -1. Qt5 (or one of its packages, e.g. qt5xbase) is missing - It might be becuase your Debian is too old to have a recent enough version in its official repositories. It is +1. Qt5 (or one of its packages, e.g. `qt5xbase`) is missing. + It might be because your Debian is too old to have a recent enough version in its official repositories. It is suggested to use the same packages that are used while building (make sure to adapt the Qt version and your Debian/Ubuntu version, here Xenial): - ``` + ```bash sudo add-apt-repository ppa:beineri/opt-qt59-xenial sudo apt-get update ``` Run the installation again. -2. ICU with its libicu is missing - You can find the missing version on pkgs.org and install it with gdebi, too. You can have + +2. ICU with its `libicu` is missing. + You can find the missing version on [pkgs.org](https://pkgs.org) and install it with `gdebi`, too. You can have multiple versions of ICU installed. 3. Error while executing a tool - To ensure the tool functionalry, make sure you add the `OPENMS_DATA_PATH` variable to your environmnet as follow + To ensure the tool functionality, make sure you add the `OPENMS_DATA_PATH` variable to your environment as follow `export OPENMS_DATA_PATH=/usr/share/OpenMS` 4. Thirdparty installation of Qt5 in 1 Make sure you source the provided environment file using: @@ -35,27 +60,28 @@ If you encounter errors with unavailable packages, troubleshoot using the follow `/usr/share/OpenMS/THIRDPARTY` 5. Add the folders in your `PATH` for a convenient use of the adapters. -## Install via Package Managers +## Install via package managers Packaged versions of **OpenMS** are provided for Fedora, OpenSUSE, Debian, and Ubuntu. You can find them to download [here](https://pkgs.org/download/openms). For other GNU/Linux distributions or to obtain the most recent version of the library, installation should be done via building from the source code. -```{note} - -Please note that these packages are not directly maintained by OpenMS and they can not be guaranteed the +```{important} +These packages are not directly maintained by OpenMS and they can not be guaranteed the same behaviour as when building it on from source code. Also, their availability and version is subject to change and support might be limited (due to unforeseen or untested behaviour). It is suggested not to install them parallel to our Debian package. ``` -## Installation via Building From Source Code +## Installation via building from source code The source code of OpenMS is available on GitHub. Follow [Building OpenMS on GNU/Linux](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/install_linux.html) instructions on how to build OpenMS from source. -> :memo: **_NOTE:_** Some thirdparty software used via adapter tools in OpenMS might also require an installed JavaVM. +```{note} +Some thirdparty software used via adapter tools in OpenMS might also require an installed JavaVM. +``` -## Run via a (Bio)Docker Image +## Run via a (Bio)Docker image Make sure you have [Docker installed](https://docs.docker.com/engine/install/). diff --git a/docs/installations/installation-on-macos.md b/docs/installations/installation-on-macos.md index a0624672..61f99282 100644 --- a/docs/installations/installation-on-macos.md +++ b/docs/installations/installation-on-macos.md @@ -1,59 +1,69 @@ Installation on macOS ==================== +## Install via macOS installer + To install OpenMS on macOS, run the following steps: 1. Download and install the macOS drag-and-drop installer from the [archive](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/release/latest/). 2. Double click on the dowloaded file. It will start to open the download `openms--macos.dmg` file -Verifying OpenMS-<version>-macOS.dmg +Verifying OpenMS-<version>-macOS.dmg 3. Verify the download. -Verifying OpenMS-<version>-macOS.dmg +Verifying OpenMS-<version>-macOS.dmg 4. Agree the license agreements. -License Agreement +License Agreement 5. Drag openms to applications. -License Agreement +License Agreement 6. It will start copying to applications. -Preparing to Copy to Applications +Preparing to Copy to Applications -Copying to Applications +Copying to Applications -To use TOPP as regular app in the shell, add the following lines to the `~/.profile` file. +To use {term}`TOPP` as regular app in the shell, add the following lines to the `~/.profile` file. -``` +```bash export OPENMS_TOPP_PATH= source ${OPENMS_TOPP_PATH}/.TOPP_bash_profile ``` Make sure `` points to the folder where OpenMS is installed locally (e.g., `/Applications/OpenMS-`) +## Install via Conda or Bioconda + +Follow [these](installation-on-gnu-linux.md#install-via-conda) instructions. + ## Known Issues 1. OpenMS software landing in quarantine since macOS Catalina after installation of the `.dmg`. + Since macOS Catalina (maybe also Mojave) notarized apps and executables are mandatory. - > **_NOTE:_** Although there is a lot of effort in signing and notarizing everything, it seems like openms software + ```{important} + Although there is a lot of effort in signing and notarizing everything, it seems like openms software still lands in quarantine on the above mentioned systems, after installation of the DMG (when downloading it from a browser). + ``` To have a streamlined experience without blocking popups, it is recommended to remove the quarantine flag manually, using the following steps: Open the Terminal.app and type the following (replace the first line with the actual installation directory): - ``` + ```bash cd /Applications/OpenMS- sudo xattr -r -d com.apple.quarantine * ``` -2. Bug with running Java based thirdparty tools like MSGFPlusAdapter and LuciphorAdapter from within TOPPAS.app - If you face issues while running Java based thirdparty tools from within `TOPPAS.app`, run the TOPPAS.app from within - the Terminal.app (e.g. with the `open` command) to get access to the `path` where Java is located. - Java is usually present in the `PATH` of the terminal. Adavanced users can set this path in the `Info.plist` of/inside - the `TOPPAS.app`. +2. Bug with running Java based thirdparty tools like {term}`MSGFPlusAdapter` and {term}`LuciphorAdapter` from within **TOPPAS.app** + + If you face issues while running Java based thirdparty tools from within {term}`TOPPAS.app`, run the {term}`TOPPAS.app` + from within the Terminal.app (e.g. with the `open` command) to get access to the path where Java is located. + Java is usually present in the `PATH` of the terminal. Advanced users can set this path in the `Info.plist` of/inside + the TOPPAS.app. diff --git a/docs/installations/installation-on-windows.md b/docs/installations/installation-on-windows.md index 57553e22..c9848c77 100644 --- a/docs/installations/installation-on-windows.md +++ b/docs/installations/installation-on-windows.md @@ -1,16 +1,20 @@ Installation on Windows ======================= -To Install the binary package of OpenMS & TOPP: +## Install via Windows installer + +To Install the binary package of OpenMS & {term}`TOPP`: 1. Download and execute the installer `OpenMS--Win64.exe` from the [archive](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/release/latest/) and follow its instructions. 2. Run the installer under the user account that later runs OpenMS. It is not advised to use admin account for installation. When asked for an admin authentication, please enter the credentials. -> **_NOTE:_** The windows binary version works with most versions of windows from Win7 to Win10 (older versions might +```{tip} +The windows binary version works with most versions of windows from Win7 to Win10 (older versions might still work but are untested). +``` - ## Known Issues +## Known issues 1. During installation, an error message pops up, saying "The installation of the Microsoft .NET 3.5 SP1' package failed! @@ -30,7 +34,7 @@ To Install the binary package of OpenMS & TOPP: 3. Error: "Error opening installation log file" To fix, check the system environment variables. Make sure they are apt. There should a `TMP` and a `TEMP` variable, - and both should contain ONE(!) directory only, which exists and is writable. Fix accordingly (search the internet on + and both should contain one directory only, which exists and is writable. Fix accordingly (search the internet on how to change environment variables on Windows). 4. For Win8 or later, Windows will report an error while installing `.net4` as it's mostly included. But it might occur that `.net3.5` does not get properly installed during the process. diff --git a/docs/introduction.md b/docs/introduction.md index ead62a7c..44ad262f 100644 --- a/docs/introduction.md +++ b/docs/introduction.md @@ -2,40 +2,105 @@ Introduction ============ [OpenMS](http://www.openms.org/) -is an open-source software C++ library for LC-MS data management and +is an open-source software C++ library for {term}`LC-MS` data management and analyses. It offers an infrastructure for rapid development of mass spectrometry related software. OpenMS is free software available under the three clause BSD license and runs under Windows, macOS, and Linux. -It comes with a vast variety of pre-built and ready-to-use tools for proteomics -and metabolomics data analysis (TOPPTools) as well as powerful 1D, 2D and 3D -visualization (TOPPView). +```{note} +This introduction is aimed at users new to the field of LC-MS data analysis and will introduce some basics terms +and concepts. How to handle the data analysis, available data structures, algorithms and more are covered in the various +subsections of this documentation. +``` + +# Background + +Proteomics and metabolomics are interdisciplinary research fields that study structure, function, and interaction of +proteins and metabolites. They employ large-scale experimental techniques that allow acquiring data at the level of +cellular systems to whole organisms. Mass spectrometry combined with chromatographic separation is commonly used to +identify, characterize or quantify the amount of proteins and metabolites. + +In mass spectrometry-based proteomics and metabolomics, biological samples are extracted, prepared, and separated to +reduce sample complexity. The separated analytes are ionized and measured in the mass spectrometer. Mass and abundance +of ions are stored in mass spectra and used to identify and quantify the analytes in the sample using computational +methods. The quantity and identity of analytes can then be used, for instance, in biomarker discovery, medical diagnostics, +or basic research. + +## Liquid Chromatography(LC) + +LC aims to reduce the complexity of the measured sample by separating analytes based on their physicochemical properties. +Separating analytes in time ensures that a manageable amount of analytes elute at the same time. In mass +spectrometry-based proteomics, (high-pressure) liquid chromatographic separation techniques (HPLC) are methods of choice +to achieve a high degree of separation. In HPLC, {term}`peptides` are separated on a column. Solved in a pressurized liquid +(mobile phase) they are pumped through a solid adsorbent material (stationary phase) packet into a capillary column. +Physicochemical properties of each peptide determine how strongly it interacts with the stationary phase. The most +common HPLC technique in proteomics and metabolomics uses reversed-phase chromatography (RPC) columns. RPC employs a +hydrophobic stationary phase like {term}`octadecyl (C18)`, a nonpolar carbon chain bonded to a silica base, and a polar mobile +phase. Polar molecules interact weakly with the stationary phase and elute earlier, while non-polar molecules are retained. +Interaction can be further modulated by changing the gradient of solvent concentration in the mobile phase over time. +Elution times in LC are inherently prone to variation, for example, due to fluctuations in the flow rate of the mobile +phase or change of column. Retention time shifts between runs may be compensated using computational chromatographic +retention time alignment methods. In the {term}`LC-MS` setup, the column is directly coupled to the ion source of the mass +spectrometer. + +![](images/introduction/introduction_LC.png) + +## Mass Spectrometry + +MS is an analytical technique used to determine the {term}`mass` of molecules. In order to achieve highly accurate and sensitive +mass measurements at the atomic scale, mass spectrometers manipulate charged particles using magnetic and electrostatic +fields. -OpenMS offers analyses for various quantitation protocols, including label-free -quantitation, SILAC, iTRAQ, TMT, SRM, SWATH, etc. +![](images/introduction/introduction_MS.png) -It provides built-in algorithms for de-novo identification and database search, -as well as adapters to other state-of-the art tools like X!Tandem, Mascot, -OMSSA, etc. It supports easy integration of OpenMS built tools into workflow -engines like KNIME, Galaxy, WS-Pgrade, and TOPPAS via the TOPPtools concept and -a unified parameter handling via a 'common tool description' (CTD) scheme. +In a typical mass spectrometer, three principal components can be identified: -With pyOpenMS, OpenMS offers Python bindings to a large part of the OpenMS API -to enable rapid algorithm development. OpenMS supports the Proteomics Standard -Initiative (PSI) formats for MS data. The main contributors of OpenMS are -currently the Eberhard-Karls-Universität in Tübingen, the Freie Universität -Berlin, and the ETH Zürich. +- **Ion Source**: A mass spectrometer only handles {term}`ions`. Thus, charge needs first be transferred to uncharged + particles. The component responsible for the ionization is the ion source. Different types of ion sources and ionization + techniques exist with {term}`electrospray ionization (ESI)` being currently the most widely used ionization technique + for mass spectrometry-based proteomics. -Get started with installing OpenMS on different operating systems! +- **Mass Analyzer**: Most commonly used mass analyzer in proteomics are {term}`time-of-flight (TOF)` mass analyzers, + {term}`quadrupole mass filters`, and {term}`orbitrap analyzers`. In TOF mass analyzers, the ions are accelerated in an electric field. + The flight time of an ion allows calculating the velocity which in turn is used to calculate the mass-to-charge ratio + (m/z). Varying the electric field allows filtering certain mass-to-charge ratios before they enter the detector. In + quadrupole mass filters, ions pass through an oscillating electric field created by four parallel rods. For a + particular voltage, only ions in a certain mass-to-charge range will reach the detector. The orbitrap is an ion trap + mass analyzer (and detector) that traps ions in orbital motion between a barrel-like outer electrode and a spindle-like + central electrode allowing for prolonged mass measurement. As a result of the prolonged mass measurements, a high mass + resolution can be achieved. -```{tab} Installation on GNU/Linux +- **Detector**: The last component of the mass spectrometer is the detector. It determines the abundance of ions that + passed through the mass analyzer. Ion intensities (a value that relates to its abundance) and the mass-to-charge ratio + are recorded in a mass spectrum. + +A sample is measured over the retention time of the chromatography typically resulting in tens of thousands of spectra. +The measurement of one sample is called an MS run and the set of spectra called an {term}`MS(1)` map or peak map. + +![](images/introduction/spectrum_peakmap.png) + +The left image displays spectrum with peaks (m/z and intensity values) and the right image shows spectra stacked in +retention time yielding a peak map. + +In proteomics and metabolomics, the {term}`MS(1)` intensity is often used for the quantification of an analyte. Identification +based on the {term}`MS(1)` mass-to-charge and the isotope pattern is highly ambiguous. To improve identification, tandem mass +spectrometry {term}`(MS/MS)` can be applied to assess the analyte substructure. To this end, the precursor ion is isolated and +kinetically fragmented using an inert gas (e.g., Argon). Fragments produced by {term}`collision-induced dissociation (CID)` +are stored in an {term}`MS^2` (or {term}`MS/MS`) spectrum and provide information that helps to resolve the ambiguities in identification. +Alternatively, {term}`MS/MS` spectra can be used for quantification. + +Get started with installing OpenMS using the installers available for different operating systems! + +## Installation on different platforms + +```{tab} GNU/Linux wget https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/release/latest/OpenMS-2.8.0-Debian-Linux-x86_64.deb ``` -```{tab} Installation on Windows +```{tab} Windows wget https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/release/latest/OpenMS-2.8.0-Win64.exe ``` -```{tab} Installation on macOS +```{tab} MacOS wget https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/release/latest/OpenMS-2.8.0-macOS.dmg -``` +``` \ No newline at end of file diff --git a/docs/research/whitepapers/OpenMS-2.0-kernel-whitepaper.md b/docs/research/whitepapers/OpenMS-2.0-kernel-whitepaper.md new file mode 100644 index 00000000..719e571c --- /dev/null +++ b/docs/research/whitepapers/OpenMS-2.0-kernel-whitepaper.md @@ -0,0 +1,122 @@ +OpenMS 2.0 Kernel Whitepaper +=========================== + +## Introduction + +With the constantly increasing amount of data that needs to be handled in computational mass spectrometry we start to +reach the boundary of what is possible given the current design and implementation of the OpenMS kernel. Already in 2012 +the OpenMS core team discussed and identified the need for a redesign of the OpenMS kernel with respect to it's scalability +and ease of use. So far, only partial solutions have been proposed that lack the approval of the OpenMS core team as +well as the OpenMS community. + +This document summarises the discussion of the OpenMS core team, held at the OpenMS developer retreat in Izmir, +Turkey 2014. + +## Detailed problem description + +With modern mass spectrometry instruments and measurement techniques the amounts of data that need to be analysed in +computational mass spectrometry reach magnitudes that do not fit into the main memory of neither desktop nor clusters +machines. Therefore the OpenMS kernel has to be redesigned to decouple the algorithms from the actual implementation of +the data structures. + +Here we describe the interfaces and concepts we plan to implement for OpenMS 2.0. The OpenMS team has acknowledged the +fact, that this will only be possible in an incremental way, i.e., implementing those interfaces on top of the existing +data structures and then switch the individual classes to the new design, instead of a large redesign in a separate +development line. + +## Benchmarking the changes + +Before we start to implement individual changes we intend to create a benchmark suite. This benchmark suite should +enable us to assess the changes in memory consumption and runtime due to the new data structures. + +## Spectrum/Chromatogram interface + +The first decision reached during the discussion was to abandon the `Peak` data structure. It will be replaced by a pair +of `double`s such that a spectrum/chromatogram is a combination of two `std::vector`. + +```cpp +class RawDataArray +{ + std::vector intensities_; + std::vector positions_; +}; +``` + +with invariant `intensities.size() == positions.size()`. This data structure will be augmented with a container to hold +position-wise meta data (see for instance `FloatDataArray`). + +```cpp +class RawDataArray +{ + typedef std::vector MetaDataArray; + typedef std::string MetaDataID; + + std::vector intensities_; + std::vector positions_; + std::map meta_; +}; +``` + +Given this base class `Spectrum` and `Chromatogram` can be derived from this class, redefining `position_` to either +`m/z` or `rt`. Additionally the `Spectrum` and `Chromatogram` will add relevant members (e.g., MS-level for `Spectrum`) +that should not be stored as meta data. + +``` +class Spectrum : public RawDataArray +{ + int ms_level_; + Precursor precursor_; + double rt_; + int scan_id_; +}; +``` + +The overall design is summarised in the following figure. + +![](../../images/research/whitepapers/summary.png) + +## MSRun (was MSExperiment) + +`MSExperiment` will be renamed to `MSRun`. On top of MSRun we will define an interface that hides the actual implementation +of `MSRun` from the user allowing the use of cached, on-disc, or full in-memory implementations. Direct access to +chromatograms will be hidden behind the `ISpectrumAccess` and `IChromatogramAccess` interfaces. + +```cpp +class ISpectrumAccess +{ +public: + int getNrSpectra() const; + getMetaSpectrum(int i) const; + const ISpectrum& getSpectrum(int i) const; + const ExpMeta& getExperimentMeta() const; + // iterators for spectra + SpectrumIterator spectrumBegin(); + SpectrumIterator spectrumEnd(); +}; + +class IChromatogramAccess +{ +public: + int getNrChromatograms() const; + getMetaSpectrum(int i) const; + const IChromatogram& getChromatogram(int i) const; + const ExpMeta& getExperimentMeta() const; + // iterators for chromatograms + ChromatogramIterator chromatogramBegin(); + ChromatogramIterator chromatogramEnd(); +}; +``` + +The overall design is summarised in the following figure. + +![](../../images/research/whitepapers/design-summary.png) + +## Modularisation + +We decided that the OpenMS library has to be modularised. Starting with SuperHirn as a proof of concept we will identify +other parts of the library that should be an independent module and move them into a separate library. Additionally we +will add a new library `libOpenMSChemistry` that should implement light-weight alternatives to the existing chemistry +data structures. + +To avoid the direct integration of classes associated to UTILS into OpenMS, we will additionally support multi-file UTILS +compared to the existing TOPP like utils, that should consist of only one file. diff --git a/docs/topp/adding-new-tool-to-topp.md b/docs/topp/adding-new-tool-to-topp.md new file mode 100644 index 00000000..36b9082c --- /dev/null +++ b/docs/topp/adding-new-tool-to-topp.md @@ -0,0 +1,100 @@ +Adding New Tool to The TOPP suite +===================================== + +## The OpenMS pipeline (TOPP) + +Any tool that is written with the OpenMS library can easily be made into a TOPP tool by simply using the OpenMS command +line parser which is able to parse ParamXML, a powerful XML based description of the tool. Hence most analysis algorithms +in OpenMS are available as a stand-alone tool which can be called on the command line or integrated into workflow engines +via the CTD mechanism. A current list of TOPP tools can be found in [the documentation](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/release/latest/html/TOPP_documentation.html). + +## What do I have to do to add a new TOPP tool? + +The recommended way is to inherit from the class TOPPBase as in existing TOPP tools (sources available in /src/topp/). This will add command line parsing functionality to your tool as described in the TOPP section of this page. + +- Add the code to `src/topp/` and register it in `src/topp/executables.cmake` +- Add your tool (with the correct category) to `getTOPPToolList()` in `src/openms/source/APPLICATIONS/ToolHandler.cpp`. + This creates a doxygen page with the `–help` output of the tool (using `TOPPDocumenter`). This page must be included + at the end of the doxygen documentation of your tool (see other tools for an example). +- Add it to the TOPP docu page (in `doc/doxygen/public/TOPP.doxygen`) +- Add the name to `src/topp/executables.cmake` +- Write a TOPP test (add it to `src/tests/topp/CMakeLists.txt`) + +```{warning} +Handle any kind of input files to your TOPP tool via command line flags and use the `${DATA_DIR_TOPP}` prefix. Use +ini-files to specify output-files, but not input-files. Doing otherwise will break out-of-source builds. +``` + +```{hint} +add `-test` to the call of your TOPP tool and also create the expected output that you put in `src/tests/topp` with that +flag active. The flag ensures that UniqueId's, dates etc are equal no matter where and when the tool is run. +``` + +## What do I have to do to add a new UTILS tool? + +- Add the code to `src/utils/` and register it in `src/utils/executables.cmake`. +- Add your tool to `getUtilList()` in `src/openms/source/APPLICATIONS/ToolHandler.cpp`. This creates a doxygen page with + the `–help` output of the tool (using `TOPPDocumenter`). This page must be included at the end of the doxygen + documentation of your tool (see other tools for an example). +- Add it to the UTILS docu page (in `doc/doxygen/public/UTILS.doxygen`) +- Write a test (this is optional for UTILS). See TOPP tools above and add the test to the bottom of `src/tests/topp/CMakeLists.txt`. + +## I want to implement a new file adapter. What is to be done? + +First, add a file adapter class to the `include/OpenMS/FORMAT/` and `source/FORMAT/` folders. The file adapter should +implement a default constructor, a load method and a store method. Make sure your code conforms to the OpenMS Coding +conventions. For automatic file type recognition, you need to + +- register your new file type at the Type enum in `/include/OpenMS/FORMAT/FileTypes.h`, +- flag the file type as supported in the isSupported method of `/source/FORMAT/FileHandler.C` +- register the file extension in the getTypeByFileName method of `/source/FORMAT/FileHandler.C` + +If the new file is a peak or feature file format you should also add it to loadExperiment or loadFeatures, respectively, +of the FileHandler class. To add the file format to the TOPPView open dialog, you have to modify the file +`/source/APPLICATIONS/TOPPViewBase.C`. + +- Add the file extensions to the filter_all and filter_single variables of the getFileList_ method. + +To add your format to TOPP applications: + +- add the file extension to the extensions list of the respective parameter: + ``` + e.g. setValidStrings_("in_type", StringList::create("mzData,mzXML,mzML")); in FileInfo + ``` + +## How to create an icon file for a TOPP tool under Windows? + +- Create an .ico file: first, you need some graphics program (The GIMP is recommended) think of a motive and remind + yourself that you have limited space. Create at least a 16x16, 32x32, 48x48 and 64x64 pixel version and save each of + them in a separate layer of the respective size. Do not add any larger sized layers, since Win XP will not display any + icon then. When saving the image as type `.ico` the GIMP will ask you for the color depth of each layer. As it is + recommended to have multiple color depths of each icon-size, go back to the layers and duplicate each layer twice. + That should give you 12 layers. Now, save the image as `.ico` (e.g. TOPPView.ico) file, giving each group of equal + sized layers a 32 bit (8 bit transparency), 8 bit (1 bit transparency), 4 bit (1 bit transparency) color depth. + +```{attention} +Make sure to assign the higher color depth to the upper layers as Windows will not pick the highest possible color +otherwise. +``` + +- Create a resource file: Create a text file named `.rc` (e.g. TOPPView.rc) Insert the following line: 101 ICON + "TOPPView.ico" , replacing TOPPView with your binary name. Put both files in `OpenMS/source/APPLICATIONS/TOPP/` + (similar files for other TOPP tools already present). Re-run cmake and re-link your TOPP tool. + +Voila. You should have an iconized TOPP tool. + +## Develop your Tool in an external project using OpenMS + +To include the OpenMS library in one of your projects, we recommend to have a look at a small emulated external project +in our repository. We strongly suggest to use CMake for building your project together with OpenMS to make use of the +macros and environment information generated during the build of the OpenMS library. + +## The Common Tool Description (CTD) + +The CTD is a format developed from the OpenMS team to allow the user to use TOPP tools also in other workflow engines. +Each tool can output a CTD description of itself (the XML scheme for the CTD can be found here), which can then be used +by a node generator program to generate nodes for different workflow engines. The CTD mechanism is shared by OpenMS with +other mature libraries like SeqAn and BALL. An example for a node generation program are the Generic KNIME Nodes. The +most complete description on how to generate your own Generic KNIME Nodes based on a CTD (e.g. from your freshly +developed command line tool), can be found on the SeqAn documentation. We are working on a tutorial specifically +tailored to OpenMS. diff --git a/docs/topp/ini-file-editor.md b/docs/topp/ini-file-editor.md index e5635936..541316c1 100644 --- a/docs/topp/ini-file-editor.md +++ b/docs/topp/ini-file-editor.md @@ -3,8 +3,8 @@ INIFileEditor Can be used to visually edit INI files of TOPP tools. -The values can be edited by double-clicking or pressing F2. +The values can be edited by double-clicking or pressing F2. The documentation of each value is shown in the text area on the bottom of the widget. -![](../../images/topp/INIFileEditor.png) +![INIFileEditor](../images/topp/INIFileEditor.png) diff --git a/docs/topp/swathwizard.md b/docs/topp/swathwizard.md index 0b26a647..22412fdd 100644 --- a/docs/topp/swathwizard.md +++ b/docs/topp/swathwizard.md @@ -16,9 +16,9 @@ both the intermediate files from OpenSWATH (e.g. the XIC data in `.sqMass` forma This is how the wizard looks like: -![](../../images/topp/SwathWizard.png) +![SwathWizard](../images/topp/SwathWizard.png) -Schematic of the internal data flow (all tools are called by SwathWizard in the background) can be found in the +A schematic of the internal data flow (all tools are called by SwathWizard in the background) can be found in the [TOPP Documentation: SwathWizard](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_SwathWizard.html). A recommended test data for the Wizard is the [PASS00779](https://db.systemsbiology.net/sbeams/cgi/PeptideAtlas/PASS_View?identifier=PASS00779) dataset. diff --git a/docs/topp/topp.md b/docs/topp/topp.md index adead82c..7f3c3881 100644 --- a/docs/topp/topp.md +++ b/docs/topp/topp.md @@ -10,7 +10,13 @@ experiments and OpenSWATH, peptide property prediction, cross-linking, quality-c Few of the graphical tools are explained below: -- [TOPPView](toppview.md): A viewer for mass spectrometry data. -- [TOPPAS](toppas.md): An assistant for GUI-driven TOPP workflow design. -- [INIFileEditor](ini-file-editor.md): An editor for OpenMS configuration files. -- [SwathWizard](swathwizard.md): A user-friendly step-by-step wizard for SWATH data analysis. +```{toctree} +:maxdepth: 1 + +: A viewer for mass spectrometry data. +: An assistant for GUI-driven TOPP workflow design. +: An editor for OpenMS configuration files. +: A user-friendly step-by-step wizard for SWATH data analysis. +``` + +For advanced documentation on every TOPP tool, see the [OpenMS TOPP API Reference](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_documentation.html). diff --git a/docs/topp/toppas.md b/docs/topp/toppas.md index 594855e8..cd496144 100644 --- a/docs/topp/toppas.md +++ b/docs/topp/toppas.md @@ -3,7 +3,7 @@ TOPPAS An assistant for GUI-driven TOPP workflow design. -TOPPAS allows to create, edit, open, save, and run TOPP workflows. Pipelines can be created conveniently in a GUI by +**TOPPAS** allows to create, edit, open, save, and run TOPP workflows. Pipelines can be created conveniently in a GUI by means of mouse interactions. The parameters of all involved tools can be edited within the application and are also saved as part of the pipeline definition in the `.toppas` file. Furthermore, TOPPAS interactively performs validity checks during the pipeline editing process, in order to make it more difficult to create an invalid workflow. Once set @@ -11,13 +11,13 @@ up and saved, a workflow can also be run without the GUI using the `ExecutePipel The following figure shows a simple example pipeline that has just been created and executed successfully: -![](../../images/topp/TOPPAS_simple_example.png) +![TOPPAS simple example](../images/topp/TOPPAS_simple_example.png) -More information about TOPPAS can be found in the [TOPPAS tutorial](../../tutorials/TOPP/TOPPAS-tutorial.md). +More information about TOPPAS can be found in the [TOPPAS tutorial](../tutorials/TOPP/TOPPAS-tutorial.md). -The command line parameters of this tool are: +**The command line parameters of this tool are**: -``` +```bash TOPPAS -- An assistant for GUI-driven TOPP workflow design. Usage: diff --git a/docs/topp/toppview.md b/docs/topp/toppview.md index 13601453..0b001e51 100644 --- a/docs/topp/toppview.md +++ b/docs/topp/toppview.md @@ -1,17 +1,17 @@ TOPPView ======= -TOPPView is a viewer for MS and HPLC-MS data. It can be used to inspect files in mzML, mzData, mzXML and several other +**TOPPView** is a viewer for MS and HPLC-MS data. It can be used to inspect files in mzML, mzData, mzXML and several other file formats. It also supports viewing data from an OpenMS database. The following figure shows two instances of TOPPView displaying a HPLC-MS map and a MS raw spectrum: -![](../../images/topp/TOPPView.png) +![TOPPView](../images/topp/TOPPView.png) -More information about TOPPView can be found in the [TOPP tutorial](../../tutorials/TOPP/TOPP-tutorial.md). +More information about TOPPView can be found in the [TOPP tutorial](../tutorials/TOPP/TOPP-tutorial.md). **The command line parameters of this tool are**: -``` +```bash TOPPView -- A viewer for mass spectrometry data. Usage: diff --git a/docs/tutorials/TOPP/TOPP-tutorial.md b/docs/tutorials/TOPP/TOPP-tutorial.md index cdbc3502..90aea6f2 100644 --- a/docs/tutorials/TOPP/TOPP-tutorial.md +++ b/docs/tutorials/TOPP/TOPP-tutorial.md @@ -1,53 +1,79 @@ TOPP Tutorial ============ -This tutorial is an introduction to TOPP and TOPPView. Let's start with understanding the intent and concept of TOPP -compared to OpenMS. Later, we will move to the handling of TOPPView which is our central GUI. Apart from viewing data, -TOPPView can also be used to analyze it using selected TOPP tools; how-to is explained in the third part of the tutorial. +This tutorial is an introduction to {term}`TOPP` and {term}`TOPPView`. Let's start with understanding the intent and +concept of {term}`TOPP` compared to OpenMS. Later, we will move to the handling of {term}`TOPPView` which is our central +GUI. Apart from viewing data, {term}`TOPPView` can also be used to analyze it using selected {term}`TOPP tools`; how-to +is explained in the third part of the tutorial. Finally, the tutorial lists the command line interfaces for all TOPP tools contained in the release. -## Concepts +# Concepts Let's understand the intent of [TOPP and OpenMS](topp-and-openms-introduction.md) in this part. -## TOPPView Main Interface +```{toctree} +:maxdepth: 1 -In this part of the tutorial, we will learn about the main features of TOPPView and its basic uses. +topp-and-openms-introduction +``` -- [TOPPView introduction](toppview-introduction.md) -- [Views in TOPPView](views-in-toppview.md) -- [Display modes and view options](display-modes-and-view-options.md) -- [Data analysis in TOPPView](data-analysis-in-toppview.md) -- [Data editing](data-editing-in-toppview.md) -- [Hotkeys](hotkeys-table.md) +# TOPPView Main Interface -## Calling TOPP tools from TOPPView +In this part of the tutorial, we will learn about the main features of {term}`TOPPView` and its basic uses. -This part of the tutorial illustrates how to interactively analyse proteomics data using TOPP tools from within TOPPView. +```{toctree} +:maxdepth: 1 -- [Smoothing raw data](smoothing-raw-data.md) -- [Subtracting a baseline from a spectrum](subtracting-a-baseline-from-a-spectrum.md) -- [Picking peaks](picking-peaks.md) -- [Feature detection on centroided data](feature-detection-on-centroided-data.md) +toppview-introduction +views-in-toppview +display-modes-and-view-options +data-analysis-in-toppview +data-editing-in-toppview +hotkeys-table +``` -## Advanced Users: Tips & Tricks +# Calling TOPP tools from TOPPView + +This part of the tutorial illustrates how to interactively analyse {term}`proteomics` data using {term}`TOPP tools` from +within {term}`TOPPView`. + +```{toctree} +:maxdepth: 1 + +smoothing-raw-data +subtracting-a-baseline-from-a-spectrum +picking-peaks +feature-detection-on-centroided-data +``` + +# Advanced Users: Tips & Tricks Read [TOPP for Advanced Users](topp-for-advanced-users.md) to know more about how to use advanced functionalities, increasing productivity. -## Scripting with TOPP - -This part of the tutorial gives a detailed overview of the most important TOPP tools. First, some basics needed for -every TOPP tool are explained, then several example pipelines are shown. - -- [General introduction](general-introduction.md) -- [File Handling](file-handling.md) -- [Profile data processing](profile-data-processing.md) -- [Calibration](calibration.md) -- [Map alignment](map-alignment.md) -- [Feature detection](feature-detection.md) -- [Feature grouping](feature-grouping.md) -- [Consensus peptide identification](consensus-peptide-identification.md) -- [Peptide property prediction](peptide-property-prediction.md) -- [Quality control](quality-control.md) -- [Conversion between OpenMS XML formats and text formats](conversion-between-openms-xml-formats-and-text-formats.md) +```{toctree} +:maxdepth: 1 + +topp-for-advanced-users +``` + +# Scripting with TOPP + +This part of the tutorial gives a detailed overview of the most important {term}`TOPP tools`. First, some basics needed +for every {term}`TOPP tool` are explained, then several example pipelines are shown. + +```{toctree} +:maxdepth: 1 + +general-introduction +file-handling +profile-data-processing +calibration +map-alignment +feature-detection +feature-grouping +consensus-peptide-identification +peptide-property-prediction +quality-control +conversion-between-openms-xml-formats-and-text-formats +``` diff --git a/docs/tutorials/TOPP/calibration.md b/docs/tutorials/TOPP/calibration.md index 4262049b..fa65e17b 100644 --- a/docs/tutorials/TOPP/calibration.md +++ b/docs/tutorials/TOPP/calibration.md @@ -3,7 +3,7 @@ Calibration OpenMS offer two calibration methods: an internal and an external calibration. Both can handle peak data as well as profile data. To calibrate profile data, a peak picking step is necessary, the important parameters can be set via the -ini-file. If you have already picked data, don't forget the `-peak\_data` flag. +ini-file. If you have already picked data, don't forget the `-peak_data` flag. The external calibration (**TOFCalibration**) is used to convert flight times into m/z values with the help of external calibrant spectra containing e.g. a polymer like polylysine. For the calibrant spectra, the calibration constants the @@ -15,7 +15,7 @@ using a linear function. In a typical setting one would first pick the TOF-data, then perform the TOFCalibration and then the InternalCalibration: -``` +```bash PeakPickerWavelet -in raw_tof.mzML -out picked_tof.mzML -ini pp.ini TOFCalibration -in picked_tof.mzML -out picked.mzML -ext_calibrants ext_cal.mzML -ref_masses ext_cal_masses diff --git a/docs/tutorials/TOPP/consensus-peptide-identification.md b/docs/tutorials/TOPP/consensus-peptide-identification.md index bd132405..74658911 100644 --- a/docs/tutorials/TOPP/consensus-peptide-identification.md +++ b/docs/tutorials/TOPP/consensus-peptide-identification.md @@ -11,11 +11,12 @@ In order to improve the identification accuracy, several identification engines identification can be calculated from the results. The image below shows an example where Mascot and OMSSA results are fed to the **ConsensusID** tool (ConsensusID is currently usable for Mascot, OMSSA and XTandem). -![](../../images/tutorials/topp/TOPP_consensus_id.png) +![TOPP Consensus ID](../../images/tutorials/topp/TOPP_consensus_id.png) To combine quantitation and identification results: Protein/peptide identifications can be annotated to quantitation results (featureXML, consensusXML) by the **IDMapper** tool. The combined results can then be exported by the **TextExporter** tool: + [Conversion between OpenMS XML formats and text formats](conversion-between-openms-xml-formats-and-text-formats.md). diff --git a/docs/tutorials/TOPP/conversion-between-openms-xml-formats-and-text-formats.md b/docs/tutorials/TOPP/conversion-between-openms-xml-formats-and-text-formats.md index 6b0fb09e..57d2af30 100644 --- a/docs/tutorials/TOPP/conversion-between-openms-xml-formats-and-text-formats.md +++ b/docs/tutorials/TOPP/conversion-between-openms-xml-formats-and-text-formats.md @@ -13,25 +13,25 @@ It converts the the following OpenMS XML formats to text files: - idXML - consensusXML -The use of the **TextExporter** is is very simple: +The use of the `TextExporter` is is very simple: `TextExporter -in infile.idXML -out outfile.txt` ## Import of feature data to OpenMS OpenMS offers a lot of visualization and analysis functionality for feature data. -Feature data in text format, e.g. from other analysis tools, can be imported using the **TextImporter**. The default +Feature data in text format, e.g. from other analysis tools, can be imported using the `TextImporter`. The default mode accepts comma separated values containing the following columns: RT, m/z, intensity. Additionally meta data columns may follow. If meta data is used, meta data column names have to be specified in a header line. Without headers: -``` +```bash 1201 503.123 1435000 1201 1006.246 1235200 ``` Or with headers: -``` +```bash RT m/z Int isHeavy myMeta 1201 503.123 1435000 true 2 1201 1006.246 1235200 maybe 1 @@ -41,7 +41,7 @@ Example invocation: `TextImporter -in infile.txt -out outfile.featureXML` -The tool also supports data from msInspect,SpecArray and Kroenik(Hardkloer sibling), just specify the -mode option +The tool also supports data from msInspect,SpecArray and Kroenik(Hardkloer sibling), just specify the `-mode` option accordingly. ## Import of protein/peptide identification data to OpenMS diff --git a/docs/tutorials/TOPP/data-analysis-in-toppview.md b/docs/tutorials/TOPP/data-analysis-in-toppview.md index 4b30b5e8..f7732c08 100644 --- a/docs/tutorials/TOPP/data-analysis-in-toppview.md +++ b/docs/tutorials/TOPP/data-analysis-in-toppview.md @@ -1,21 +1,21 @@ Data Analysis in TOPPView ========================= -TOPPView also offers limited data analysis capabilities for single layers, which will be illustrated in the following +{term}`TOPPView` also offers limited data analysis capabilities for single layers, which will be illustrated in the following sections. The functionality presented here can be found in the **Tools** menu: -![](../../images/tutorials/topp/TOPPView_tools_menu.png) +![TOPPView Tools Menu](../../images/tutorials/topp/TOPPView_tools_menu.png) ## TOPP Tools -Single TOPP tools can be applied to the data of the currently selected layer or to the visible data of the current layer. +Single {term}`TOPP tools` can be applied to the data of the currently selected layer or to the visible data of the current layer. The following example image shows the TOPP tools dialog: -![](../../images/tutorials/topp/TOPPView_tools.png) +![TOPPView Tools](../../images/tutorials/topp/TOPPView_tools.png) -To apply a TOPP tool, follow the instructions below: +To apply a {term}`TOPP tool`, follow the instructions below: -1. Select a TOPP tool and if necessary a type. +1. Select a {term}`TOPP tool` and if necessary a type. 2. Specify the command line option of the tool, that takes the input file name. 3. Specify the command line option of the tool, that takes the output file name. 4. Set the algorithm parameters manually or load them from an INI file. @@ -25,14 +25,16 @@ To apply a TOPP tool, follow the instructions below: One can access the metadata, the layer is annotated with. This data comprises e.g. contact person, instrument description and sample description. -![](../../images/tutorials/topp/MetaDataBrowser.png) +![Meta Data Browser](../../images/tutorials/topp/MetaDataBrowser.png) -> **_NOTE:_** Identification data, e.g. from a Mascot run, can be annotated to the spectra or features, too. After +```{tip} +Identification data, e.g. from a {term}`Mascot` run, can be annotated to the spectra or features, too. After annotation, this data is listed in the metadata. +``` ## Statistics Statistics about peak/feature intensities and peak meta information can be displayed. For intensities, it is possible to display an additional histogram view. -![](../../images/tutorials/topp/TOPPView_statistics.png) +![TOPPView Statistics](../../images/tutorials/topp/TOPPView_statistics.png) diff --git a/docs/tutorials/TOPP/data-editing-in-toppview.md b/docs/tutorials/TOPP/data-editing-in-toppview.md index 1c6fe80a..85666f5e 100644 --- a/docs/tutorials/TOPP/data-editing-in-toppview.md +++ b/docs/tutorials/TOPP/data-editing-in-toppview.md @@ -1,11 +1,11 @@ Data Editing in TOPPView ======================== -TOPPView offers editing functionality for feature layers. +{term}`TOPPView` offers editing functionality for feature layers. After enabling the feature editing mode in the context menu of the feature layer, the following actions can be performed: -- Features can be dragged with the mouse in order to change the m/z and RT position. +- Features can be dragged with the mouse in order to change the {term}`m/z` and {term}`RT` position. - The position, intensity and charge of a feature can be edited by double-clicking a feature. - Features can be created by double-clicking the layer background. -- Features can be removed by selecting them and pressing the DEL key. +- Features can be removed by selecting them and pressing the DEL key. diff --git a/docs/tutorials/TOPP/display-modes-and-view-options.md b/docs/tutorials/TOPP/display-modes-and-view-options.md index ab9f41ca..503d3c41 100644 --- a/docs/tutorials/TOPP/display-modes-and-view-options.md +++ b/docs/tutorials/TOPP/display-modes-and-view-options.md @@ -4,7 +4,7 @@ Display Modes and View Options in TOPPView All of the views support several display modes and view options. Display modes determine how intensities are displayed. View options configure the view. -![](../../images/tutorials/topp/TOPPView_icons.png) +![TOPPView Icons](../../images/tutorials/topp/TOPPView_icons.png) # Display Modes @@ -35,7 +35,7 @@ Switching between raw data and peak mode. ### 2D (Peaks) `MS/MS` precursor peaks can be highlighted. -Projections to `m/z` and `RT` axis can be shown. +Projections to **m/z** and **RT** axis can be shown. ### 2D (Features) diff --git a/docs/tutorials/TOPP/feature-detection-on-centroided-data.md b/docs/tutorials/TOPP/feature-detection-on-centroided-data.md index 9ffc0347..b837bd6c 100644 --- a/docs/tutorials/TOPP/feature-detection-on-centroided-data.md +++ b/docs/tutorials/TOPP/feature-detection-on-centroided-data.md @@ -28,4 +28,4 @@ traces, a tolerated mass deviation between isotopic peaks has to be set (`isotop The image shows the centroided peak data and the found peptide features. The used parameters can be found in the TOPP tools dialog. -![](../../images/tutorials/topp/TOPPView_tools_ff_centroided.png) +![TOPPView Tools FF\_Centrioided](../../images/tutorials/topp/TOPPView_tools_ff_centroided.png) diff --git a/docs/tutorials/TOPP/feature-detection.md b/docs/tutorials/TOPP/feature-detection.md index bc76b81a..da9369bf 100644 --- a/docs/tutorials/TOPP/feature-detection.md +++ b/docs/tutorials/TOPP/feature-detection.md @@ -14,7 +14,8 @@ wavelet**, which has been tailored to the detection of isotopic patterns followi information about the theory behind this technique, please refer to Hussong et al.: "Efficient Analysis of Mass Spectrometry Data Using the Isotope Wavelet" (2007). -Please note that this algorithm features no "modelling stage", since the structure of the isotopic pattern is explicitly +```{attention} +This algorithm features no "modelling stage", since the structure of the isotopic pattern is explicitly coded by the wavelet itself. The algorithm also works for 2D maps (in combination with the so-called *sweep-line* technique (Schulz-Trieglaff et al.: "A Fast and Accurate Algorithm for the Quantification of Peptides from Mass Spectrometry Data" (2007))). The algorithm could originally be executed on (several) high-speed CUDA graphics cards. @@ -22,6 +23,7 @@ Tests on real-world data sets revealed potential speedups beyond factors of 200 Support for CUDA was removed in [OpenMS]() due to maintenance overhead. Please refer to Hussong et al.: "Highly accelerated feature detection in proteomics data sets using modern graphics processing units" (2009) for more details on the implementation. +``` ### Seeding diff --git a/docs/tutorials/TOPP/feature-grouping.md b/docs/tutorials/TOPP/feature-grouping.md index 678d734c..9d8ac124 100644 --- a/docs/tutorials/TOPP/feature-grouping.md +++ b/docs/tutorials/TOPP/feature-grouping.md @@ -21,13 +21,15 @@ To differentially quantify the features of an isotope-labeled HPLC-MS map, follo - feature quality of feature 2 - quality measure for the shift (how near is it to the optimal shift) - ![](../../images/tutorials/topp/TOPP_labeled_quant.png) + ![TOPP labeled quant](../../images/tutorials/topp/TOPP_labeled_quant.png) ## Label-free quantitation To differentially quantify the features of two or more label-free HPLC-MS map. -![](../../images/tutorials/topp/TOPP_labelfree_quant.png) +![TOPP labelfree quant](../../images/tutorials/topp/TOPP_labelfree_quant.png) -> **_NOTE:_** This algorithm assumes that the retention time axes of all input maps are very similar. To correct for +```{tip} +This algorithm assumes that the retention time axes of all input maps are very similar. To correct for retention time distortions, please have a look at [Map alignment](map-alignment.md). +``` diff --git a/docs/tutorials/TOPP/file-handling.md b/docs/tutorials/TOPP/file-handling.md index 70733249..9394d040 100644 --- a/docs/tutorials/TOPP/file-handling.md +++ b/docs/tutorials/TOPP/file-handling.md @@ -19,7 +19,7 @@ If you are experiencing problems while processing an XML file, check if the file Validation is available for several file formats including mzML, mzData, mzXML, featureXML and idXML. -Another frequently-occurring problem is corrupt data. You can check for corrupt data in peak files with FileInfo as well: +Another frequently-occurring problem is corrupt data. You can check for corrupt data in peak files with `FileInfo` as well: `FileInfo -c -in infile.mzML` @@ -35,24 +35,24 @@ formats of the input and output file can be given explicitly. ## Converting between DTA and mzML -Sequest DTA files can be extracted from a mzML file using the **DTAExtractor**: +Sequest DTA files can be extracted from a mzML file using the `DTAExtractor`: `DTAExtractor -in infile.mzML -out outfile` The retention time of a scan, the precursor mass-to-charge ratio (for MS/MS scans) and the file extension are appended to the output file name. -To combine several files (e.g. DTA files) to an mzML file use the **FileMerger**: +To combine several files (e.g. DTA files) to an `mzML` file use the `FileMerger`: `FileMerger -in infile_list.txt -out outfile.mzML` -The retention times of the scans can be generated, taken from the *infile_list.txt* or can be extracted from the DTA +The retention times of the scans can be generated, taken from the `infile_list.txt` or can be extracted from the DTA file names. See the FileMerger documentation for details. ## Extracting part of the data from a file -To extract part of the data from an mzML file, use the **FileFilter** tool. It allows filtering for RT, `m/z` and -intensity range or for MS level. To extract the MS/MS scans between retention time 100 and 1500, use the following +To extract part of the data from an `mzML` file, use the `FileFilter` tool. It allows filtering for RT, `m/z` and +intensity range or for MS level. To extract the MS/MS scans between retention time `100` and `1500`, use the following command: `FileFilter -in infile.mzML -levels 2 -rt 100:1500 -out outfile.mzML` diff --git a/docs/tutorials/TOPP/general-introduction.md b/docs/tutorials/TOPP/general-introduction.md index 39a7687d..12266c7d 100644 --- a/docs/tutorials/TOPP/general-introduction.md +++ b/docs/tutorials/TOPP/general-introduction.md @@ -1,7 +1,7 @@ General Introduction ==================== -This tutorial will gives a brief overview of the most important TOPP tools. First, some basics that are required for +This tutorial will gives a brief overview of the most important **TOPP** tools. First, some basics that are required for every TOPP tool, then there are several example pipelines explained. ## File formats @@ -61,10 +61,11 @@ can contain the configuration of one or several TOPP tools. The following examples will give an overview of how TOPP tools can be chained in order to create analysis pipelines. INI files are the recommended way to store all settings of such a pipeline in a single place. -Note that the issue of finding suitable parameters for the tools is not addressed here. For problems during the +```{attention} +The issue of finding suitable parameters for the tools is not addressed here. For problems during the execution of the example pipelines on the data, adapt the parameters. Have a look at the documentation of the corresponding TOPP tool in that case. - +``` ### Parameter documentation @@ -73,7 +74,7 @@ command line flag `-help`. Some TOPP tools also have subsections of parameters t algorithm. The documentation of these subsections is not displayed with `–help`. It is however displayed in **INIFileEditor**. -![](../../images/tutorials/topp/INIFileEditor.png) +![INIFileEditor](../../images/tutorials/topp/INIFileEditor.png) In the [INIFileEditor](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_INIFileEditor.html), click on the respective parameter, the documentation of the parameters is displayed in the window at the bottom. @@ -83,7 +84,7 @@ For an old INI file which does not work for a newer [OpenMS]() version (due to r can rescue parameters whose name did not change into the new version by using [INIUpdater](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/UTILS_INIUpdater.html) tool by calling it with (a list of) outdated INI and/or TOPPAS files. See the INIUpdater tool description for details. This will remove invalid parameters and add new parameters (if available) while retaining values for unchanged parameters. As an alternative to the INIUpdater, use -the commandline by calling the TOPP tool from which the ini originated and combining `-write_ini` and `-ini`, e.g., +the command line by calling the TOPP tool from which the ini originated and combining `-write_ini` and `-ini`, e.g., ``` FileInfo -ini old_fi.ini -write_ini fi.ini @@ -94,7 +95,7 @@ This will transfer all values of parameters from `old_fi.ini` which are still va ### General structure of an INI file An INI file is always enclosed by the `` tag. Inside this tag, a tree-like hierarchy is created with `` -tags that represent sections and `` tags, each of which stores one of the parameters. The first two level of the +tags that represent *sections* and `` tags, each of which stores one of the parameters. The first two level of the hierarchy have a special meaning. **Example**: Below is the content of an INI file for **FileFilter**. @@ -110,10 +111,10 @@ contain nested subsections in order to group related parameters. - If a parameter is not found in the instance section, the *tool-specific common section* is considered. - Finally, we look if the *general common section* contains a value for the parameter. -As an example, let's call the **FileFilter** tool with the INI file given below and instance number `2`. The FileFilter +As an example, let's call the `FileFilter` tool with the INI file given below and instance number `2`. The `FileFilter` parameters `rt` and `mz` are looked up by the tool. *mz* can be found in section **FileFilter** - `2`. `rt` is not specified in this section, thus the `common` **FileFilter** section is checked first, where it is found in our example. -When looking up the *debug* parameter, the tool would search the instance section and tool-specific common section +When looking up the `debug` parameter, the tool would search the instance section and tool-specific common section without finding a value. Finally, the *general common section* would be checked, where the debug level is specified. ```xml diff --git a/docs/tutorials/TOPP/hotkeys-table.md b/docs/tutorials/TOPP/hotkeys-table.md index 6c0096ff..d19fc609 100644 --- a/docs/tutorials/TOPP/hotkeys-table.md +++ b/docs/tutorials/TOPP/hotkeys-table.md @@ -3,6 +3,7 @@ TOPPView Hotkeys ## File handling + | Hotkey | Function | |-------------------------------------------------------------------------|-----------------------------------------------------| | CTRL + O | Open file | @@ -14,62 +15,65 @@ TOPPView Hotkeys ## Navigation in the data | Hotkey | Function | -|----------------------|-----------------------------------------------------| -| `CTRL` | Activate zoom mode | -| `SHIFT` | Activate measurement mode | -| `Arrow keys` | Translate currently shown data (1D View: Shift + Left/Right moves to the next peak in sparse data)| -| `CTRL++`, `CTRL+- | Move up and down in zoom history | +|------------------------------------|-----------------------------------------------------| +| CTRL | Activate zoom mode | +| SHIFT | Activate measurement mode | +| Arrow keys | Translate currently shown data (1D View: Shift + Left/Right moves to the next peak in sparse data)| +|CTRL + +,
CTRL + - | Move up and down in zoom history | | Mouse wheel | Move up and down in zoom history | -| `CTRL+G` | Goto dialog | -| `Backspace` | Reset zoom | -| `PageUp` | Select previous layer | -| `PageDown` | Select next layer | +| CTRL + G | Go to dialog | +| Backspace | Reset zoom | +| PageUp | Select previous layer | +| PageDown | Select next layer | + -## Visalization options +## Visualization options | Hotkey | Function | |----------------------|-----------------------------------------------------| -| `CTRL+R` | Show/hide grid lines | -| `CTRL+L` | Show/hide axis legends | -| `N` | Intensity mode: Normal | -| `P` | Intensity mode: Perecentage | -| `S` | Intensity mode: Snam-to-maximum | -| `I` | 1D draw mode: peaks | -| `R` | 1D draw mode: raw data | -| `CTRL`+`ALT`+`Home` | 2D draw mode: increase minimum canvas coverage threshold (for raw peak scaling)| -| `CTRL`+`ALT`+`End` | 2D draw mode: decrease minimum canvas coverage threshold (for raw peak scaling) | -| `CTRL`+`ALT`+`+` | 2D draw mode: increase maximum point size (for raw peak scaling) | -| `CTRL`+`ALT`+`-` | 2D draw mode: decrease maximum point size (for raw peak scaling) | - -> **_NOTE:_** `Home` on macOS keyboards is also `Fn`+`ArrowLeft`. `End` on macOSX keyboards is also `Fn`+`ArrowRight`. +| CTRL + R | Show/hide grid lines | +| CTRL + L | Show/hide axis legends | +| N | Intensity mode: Normal | +| P | Intensity mode: Percentage | +| S | Intensity mode: Snam-to-maximum | +| I | 1D draw mode: peaks | +| R | 1D draw mode: raw data | +| CTRL + ALT + Home | 2D draw mode: increase minimum canvas coverage threshold (for raw peak scaling)| +| CTRL + ALT + End | 2D draw mode: decrease minimum canvas coverage threshold (for raw peak scaling) | +| CTRL + ALT + + | 2D draw mode: increase maximum point size (for raw peak scaling) | +| CTRL + ALT + - | 2D draw mode: decrease maximum point size (for raw peak scaling) | + +```{tip} +kbd>Home on macOS keyboards is also Fn + ArrowLeft. End on macOSX keyboards is also Fn + ArrowRight. +``` ## Annotations in 1D view -| Hotkey | Function | -|----------------------|-----------------------------------------------------| -| `CTRL`+`A` | Select all annotations of the current layer | -| `DEL` | Delete all currently selected annotations | +| Hotkey | Function | +|------------------------------------------|-----------------------------------------------------| +| CTRL + A | Select all annotations of the current layer | +| DEL | Delete all currently selected annotations | + ## Advanced -| Hotkey | Function | -|----------------------|-----------------------------------------------------| -| `CTRL`+`T` | Apply TOPP tool to the current layer | -| `CTRL`+`SHIFT`+`T` | Apply TOPP tool to the visible data of the current layer| -| `F4` | Re-run TOPP tool | -| `CTRL` + `M` | Show layer meta information | -| `CTRL` + `I` | Annotate with identification results | -| `1` | Show precursor peaks (2D peak layer) | -| `2` | Show projections (2D peak layer) | -| `5` | Show overall convex hull (2D feature layer) | -| `6` | Show all convex hulls (2D feature layer) | -| `7` | Show numbers and labels (2D feature layer) | -| `9` | Show consensus elements (2D consensus layer) | +| Hotkey | Function | +|-----------------------------------------------------|---------------------------------------------------------| +| CTRL + T | Apply TOPP tool to the current layer | +| CTRL + SHIFT + T | Apply TOPP tool to the visible data of the current layer| +| F4 | Re-run TOPP tool | +| CTRL + M | Show layer meta information | +| CTRL + I | Annotate with identification results | +| 1 | Show precursor peaks (2D peak layer) | +| 2 | Show projections (2D peak layer) | +| 5 | Show overall convex hull (2D feature layer) | +| 6 | Show all convex hulls (2D feature layer) | +| 7 | Show numbers and labels (2D feature layer) | +| 9 | Show consensus elements (2D consensus layer) | ## Help - -| Hotkey | Function | -|----------------------|-----------------------------------------------------| -| `F1` | Show TOPPView online tutorial | -| `SHIFT` + `F1` | Activate *What's this?* mode | +| Hotkey | Function | +|----------------------------------------|-----------------------------------------------------| +| F1 | Show TOPPView online tutorial | +| SHIFT + F1 | Activate *What's this?* mode | diff --git a/docs/tutorials/TOPP/map-alignment.md b/docs/tutorials/TOPP/map-alignment.md index 89021719..c88dc781 100644 --- a/docs/tutorials/TOPP/map-alignment.md +++ b/docs/tutorials/TOPP/map-alignment.md @@ -9,13 +9,13 @@ corrects for shifted and scaled retention times, which may result from changes o The different **MapAligner** tools take `n` input maps, de-warp them and store the `n` de-warped maps. The following image shows the general procedure: -![](../../images/tutorials/topp/TOPP_alignment.png) +![TOPP Alignment](../../images/tutorials/topp/TOPP_alignment.png) There are different map alignment tools available. The following table gives a rough overview of them: -| Application | Applicable To | Description | +| Tool | Applicable To | Description | |-------------|---------------|-------------| -| MapAlignerPoseClustering | feature maps, peak maps | This algorithm does a star-wise alignment of the input data. The center of the star is the map with most data points. All other maps are then aligned to the center map by estimating a linear transformation (shift and scaling) of retention times. The transformation is estimated using a pose clustering approach as described in doi:10.1093/bioinformatics/btm209 | -| MapAlignerIdentification | feature maps, consensus maps, identifications | This algorithm utilizes peptide identifications, and is thus applicable to files containing peptide IDs (idXML, annotated featureXML/consensusXML). It finds peptide sequences that different input files have in common and uses them as points of correspondence. From the retention times of these peptides, transformations are computed that convert each file to a consensus time scale. | -| MapAlignerSpectrum | peak maps | This *experimental* algorithm uses a dynamic-programming approach based on spectrum similarity for the alignment. The resulting retention time mapping of dynamic-programming is then smoothed by fitting a spline to the retention time pairs. | -| MapRTTransformer | peak maps, feature maps, consensus maps, identifications | This algorithm merely *applies* a set of transformations that are read from files (in TransformationXML format). These transformations might have been generated by a previous invocation of a MapAligner tool. For example, compute a transformation based on identifications and then apply it to the features or raw data. The transformation file format is not very complicated, so it is relatively easy to write (or generate) the transformation files | +| `MapAlignerPoseClustering` | feature maps, peak maps | This algorithm does a star-wise alignment of the input data. The center of the star is the map with most data points. All other maps are then aligned to the center map by estimating a linear transformation (shift and scaling) of retention times. The transformation is estimated using a pose clustering approach as described in doi:10.1093/bioinformatics/btm209 | +| `MapAlignerIdentification` | feature maps, consensus maps, identifications | This algorithm utilizes peptide identifications, and is thus applicable to files containing peptide IDs (idXML, annotated featureXML/consensusXML). It finds peptide sequences that different input files have in common and uses them as points of correspondence. From the retention times of these peptides, transformations are computed that convert each file to a consensus time scale. | +| `MapAlignerSpectrum` | peak maps | This *experimental* algorithm uses a dynamic-programming approach based on spectrum similarity for the alignment. The resulting retention time mapping of dynamic-programming is then smoothed by fitting a spline to the retention time pairs. | +| `MapRTTransformer` | peak maps, feature maps, consensus maps, identifications | This algorithm merely *applies* a set of transformations that are read from files (in TransformationXML format). These transformations might have been generated by a previous invocation of a MapAligner tool. For example, compute a transformation based on identifications and then apply it to the features or raw data. The transformation file format is not very complicated, so it is relatively easy to write (or generate) the transformation files | diff --git a/docs/tutorials/TOPP/peptide-property-prediction.md b/docs/tutorials/TOPP/peptide-property-prediction.md index 271a7845..1406e81f 100644 --- a/docs/tutorials/TOPP/peptide-property-prediction.md +++ b/docs/tutorials/TOPP/peptide-property-prediction.md @@ -9,20 +9,20 @@ kernel-based approach for computational proteomics. BMC Bioinformatics 2007, 8:4 Christian G. Huber and Oliver Kohlbacher Improving Peptide Identification in Proteome Analysis by a Two-Dimensional Retention Time Filtering Approach J. Proteome Res. 2009, 8(8):4109-15. -The predicted retention time can be used in IDFilter to filter out false identifications. For data from several +The predicted retention time can be used in `IDFilter` to filter out false identifications. For data from several identification runs: -1. first align the data using MapAligner. -2. Then use the various identification wrappers like MascotAdapter, OMSSAAdapter, ... to get the identifications. -3. To train a model using RTModel use IDFilter for one of the runs to get the high scoring identifications (40 to 200 +1. Align the data using MapAligner. +2. Use the various identification wrappers like `MascotAdapter`, `OMSSAAdapter`, ... to get the identifications. +3. To train a model using `RTModel` use `IDFilter` for one of the runs to get the high scoring identifications (40 to 200 distinct peptides should be enough). -4. Then use RTModel as described in the documentation to train a model for these spectra. With this model, use RTPredict +4. Use `RTModel` as described in the documentation to train a model for these spectra. With this model, use `RTPredict` to predict the retention times for the remaining runs. The predicted retention times are stored in the idXML files. - These predicted retention times can then be used to filter out false identifications using the IDFilter tool. + These predicted retention times can then be used to filter out false identifications using the `IDFilter` tool. A typical sequence of TOPP tools would look like this: -``` +```bash MapAligner -in Run1.mzML,...,Run4.mzML -out Run1_aligned.mzML,...,Run4_aligned.mzML MascotAdapter -in Run1_aligned.mzML -out Run1_aligned.idXML -ini Mascot.ini MascotAdapter -in Run2_aligned.mzML -out Run2_aligned.idXML -ini Mascot.ini @@ -40,26 +40,26 @@ IDFilter -in Run4_predicted.mzML -out Run4_filtered.mzML -rt_filtering For a file with certainly identified peptides used to train a model for RT prediction, use the IDs. Therefore, the file has to have one peptide sequence together with the RT per line (separated by one tab or space). This can then be loaded -by RTModel using the `-textfile_input` flag: +by `RTModel` using the `-textfile_input` flag: -``` +```bash RTModel -in IDs_with_RTs.txt -out IDs_with_RTs.model -ini RT.ini -textfile_input ``` -The likelihood of a peptide to be proteotypic can be predicted using PTModel and PTPredict. +The likelihood of a peptide to be proteotypic can be predicted using `PTModel` and `PTPredict`. -Assume we have a file PT.idXML which contains all proteotypic peptides of a set of proteins. Lets also assume, we have -a fasta file containing the amino acid sequences of these proteins called mixture.fasta. To be able to train PTPredict, +Assume we have a file `PT.idXML` which contains all proteotypic peptides of a set of proteins. Lets also assume, we have +a fasta file containing the amino acid sequences of these proteins called `mixture.fasta`. To be able to train `PTPredict`, negative peptides (peptides, which are not proteotypic) are required. Therefore, one can use the Digestor, which is -located in the APPLICATIONS/UTILS/ folder together with the IDFilter: +located in the `APPLICATIONS/UTILS/` folder together with the `IDFilter`: -``` +```bash Digestor -in mixture.fasta -out all.idXML IDFilter -in all.idXML -out NonPT.idXML -exclusion_peptides_file PT.idXML ``` In this example the proteins are digested in silico and the non proteotypic peptides set is created by subtracting all -proteotypic peptides from the set of all possible peptides. Then, train PTModel: +proteotypic peptides from the set of all possible peptides. Then, train `PTModel`: `PTModel -in_positive PT.idXML -in_negative NonPT.idXML -out PT.model -ini PT.ini` diff --git a/docs/tutorials/TOPP/picking-peaks.md b/docs/tutorials/TOPP/picking-peaks.md index a2bb2979..b31845b4 100644 --- a/docs/tutorials/TOPP/picking-peaks.md +++ b/docs/tutorials/TOPP/picking-peaks.md @@ -4,13 +4,13 @@ Picking Peaks For low resolution data, consider to smooth the data first ([Smoothing raw data](smoothing-raw-data.md)) and subtract the baseline ([Subtracting a baseline from a spectrum](subtracting-a-baseline-from-a-spectrum.md)) before peak picking. -There are two types of PeakPickers, the PeakPickerWavelet and one especially suited for high resolution data -(PeakPickerHiRes). This tutorial explains the PeakPickerWavelet. Use the file `peakpicker_tutorial_2.mzML` from the +There are two types of PeakPickers: the **PeakPickerWavelet** and one especially suited for high resolution data +(**PeakPickerHiRes**). This tutorial explains the PeakPickerWavelet. Use the file `peakpicker_tutorial_2.mzML` from the examples data (select **File** > **Open example data**). The main parameters are the peak width and the minimal signal to noise ratio for a peak to be picked. If you don't know -the approximate `fwhm` of peaks, use the estimation included in the PeakPickerWavelet, set the flag `estimate\_peak\_width` -to true. After applying the PeakPickerWavelet, observe which peak width was estimated and used for peak picking in the +the approximate `fwhm` of peaks, use the estimation included in the PeakPickerWavelet, set the flag `estimate_peak_width` +to `true`. After applying the PeakPickerWavelet, observe which peak width was estimated and used for peak picking in the log window. To estimate the peak width, use the measuring tool ([Action Modes and Their Uses](views-in-toppview.md##action-modes-and-their-uses)) to determine @@ -18,9 +18,9 @@ the fwhm of one or several representative peaks. If the peak picker delivers only a few peaks even though the `peak_with` and `signal_to_noise` parameters are set to good values, consider changing the advanced parameter `fwhm_lower_bound_factor` to a lower value. All peaks with a lower -`fwhm` than `fwhm_lower_bound_factor` \* `peak\_width` are discarded. +`fwhm` than `fwhm_lower_bound_factor` \* `peak_width` are discarded. -The following image shows a part of the spectrum with the picked peaks shown in green, the estimated peak width in the +The following image shows a part of the spectrum with the picked peaks shown in green, the estimated peak width in the log window and the measured peak width. -![](../../images/tutorials/topp/TOPPView_tools_pp_picked.png) +![TOPPView tools pp\_picked](../../images/tutorials/topp/TOPPView_tools_pp_picked.png) diff --git a/docs/tutorials/TOPP/profile-data-processing.md b/docs/tutorials/TOPP/profile-data-processing.md index 96bdc653..55340593 100644 --- a/docs/tutorials/TOPP/profile-data-processing.md +++ b/docs/tutorials/TOPP/profile-data-processing.md @@ -9,7 +9,7 @@ To find all peaks in the profile data: 2. The now smoothed profile data can be further processed by subtracting the baseline with the **BaselineFilter**. 3. Then use one of the **PeakPickers** to find all peaks in the baseline-reduced profile data. -![](../../images/tutorials/topp/TOPP_raw_data.png) +![TOPP raw data](../../images/tutorials/topp/TOPP_raw_data.png) There are two different smoothing filters: NoiseFilterGaussian and NoiseFilterSGolay. To use the Savitzky Golay filter, or the **BaselineFilter** with non equally spaced profile data, e.g. TOF data, you have to generate equally spaced data @@ -23,7 +23,7 @@ available, PeakPickerWavelet and PeakPickerHiRes. ### PeakPickerWavelet This peak picking algorithm uses the continuous wavelet transform of a raw data signal to detect mass peaks. Afterwards -a given asymmetric peak function is fitted to the raw data and important peak parameters (e.g. fwhm) are extracted. In +a given asymmetric peak function is fitted to the raw data and important peak parameters (e.g. `fwhm`) are extracted. In an optional step these parameters can be optimized using a non-linear optimization method. The algorithm is described in detail in Lange et al. (2006) Proc. PSB-06. @@ -32,7 +32,7 @@ The algorithm is described in detail in Lange et al. (2006) Proc. PSB-06. - **Application**: This algorithm was designed for low and medium resolution data. It can also be applied to high-resolution data, but can be slow on large datasets. -See the PeakPickerCWT class documentation for a parameter list. +See the `PeakPickerCWT` class documentation for a parameter list. ### PeakPickerHiRes @@ -49,7 +49,7 @@ Please notice that this method is still **experimental** since it has not been t These properties facilitate a fast computation of picked peaks so that even large data sets can be processed very quickly. - See the PeakPickerHiRes class documentation for a parameter list. + See the `PeakPickerHiRes` class documentation for a parameter list. ## Finding the right parameters for the @@ -62,4 +62,4 @@ good parameters, following this procedure: 2. Extract a single scan from the middle of the HPLC gradient (Right click on **scan**). 3. Experiment with the parameters until you have found the proper settings -You can find the **NoiseFilters**, the **BaselineFilter**, and the **PeakPickers** in **TOPPView** in the menu **Layer** > **Apply TOPP tool**. +Find the **NoiseFilters**, the **BaselineFilter**, and the **PeakPickers** in **TOPPView** in the menu **Layer** > **Apply TOPP tool**. diff --git a/docs/tutorials/TOPP/quality-control.md b/docs/tutorials/TOPP/quality-control.md index 7799bf95..6f7060fd 100644 --- a/docs/tutorials/TOPP/quality-control.md +++ b/docs/tutorials/TOPP/quality-control.md @@ -1,21 +1,21 @@ Quality Control =============== -To check the quality of the data (supports label-free workflows and [TOPP Documentation:IsobaricAnalyzer](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_IsobaricAnalyzer.html) output): +To check the quality of the data (supports label-free workflows and [IsobaricAnalyzer](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_IsobaricAnalyzer.html) output): -The [TOPP Documentation:QualityControl](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_QualityControl.html) TOPP tool computes and collects data which allows to compute QC metrics to check the quality of -LC-MS data. Depending on the given input data this tool collects data for metrics (see section `Metrics` below). New +The [QualityControl](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_QualityControl.html) TOPP tool computes and collects data which allows to compute QC metrics to check the quality of +LC-MS data. Depending on the given input data, this tool collects data for metrics (see section [Metrics](quality-control.md#Metrics)). New metavalues will be added to existing data and the information will be written out in mzTab format. This mzTab file can -then be processed using custom scripts or via the R package (see [github:cbielow:PTXQC](https://github.com/cbielow/PTXQC/). +then be processed using custom scripts or via the R package (see [PTXQC](https://github.com/cbielow/PTXQC/). ## Workflow ![](../../images/tutorials/topp/TOPP_qualitycontrol.png) -An example workflow can be found in `OpenMS/share/OpenMS/examples/TOPPAS/QualityControl.toppas`. +Find an example workflow in `OpenMS/share/OpenMS/examples/TOPPAS/QualityControl.toppas`. -For data from [TOPP Documentation:IsobaricAnalyzer](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_IsobaricAnalyzer.html), just provide the consensusXML as input to -[TOPP Documentation:QualityControl](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_QualityControl.html). No FeatureXMLs or TrafoXMLs are required. The mzML raw file can be added as input though. +For data from [IsobaricAnalyzer](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_IsobaricAnalyzer.html), just provide the consensusXML as input to +[QualityControl](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_QualityControl.html). No FeatureXMLs or TrafoXMLs are required. The mzML raw file can be added as input though. ## Metrics @@ -58,7 +58,7 @@ Contaminants Fasta file, PostFDR FeatureXML ### FragmentMassError -The FragmentMassError metric computes a list of fragment mass errors for each annotated MS2 spectrum in ppm and Da. +The **FragmentMassError** metric computes a list of fragment mass errors for each annotated MS2 spectrum in ppm and Da. Afterwards it calculates the mass delta between observed and theoretical peaks. #### Required input data @@ -80,7 +80,7 @@ PostFDR FeatureXML, raw mzML file ### MissedCleavages -This MissedCleavages metric counts the number of MissedCleavages per PeptideIdentification given a FeatureMap and returns +The **MissedCleavages** metric counts the number of MissedCleavages per PeptideIdentification given a FeatureMap and returns an agglomeration statistic (observed counts). Additionally the first PeptideHit of each PeptideIdentification in the FeatureMap is augmented with metavalues. @@ -102,7 +102,7 @@ PostFDR FeatureXML ### MS2IdentificationRate -The MS2IdentificationRate metric calculates the Rate of the MS2 identification as follows: The number of all +The **MS2IdentificationRate** metric calculates the Rate of the MS2 identification as follows: The number of all PeptideIdentifications are counted and that number is divided by the total number of MS2 spectra. #### Required input data @@ -122,10 +122,10 @@ PostFDR FeatureXML, raw mzML file. ### MzCalibration -This metric adds new metavalues to the first (best) hit of each PeptideIdentification. For this metric it is also possible -to use this without an MzML File, but then only uncalibrated m/z error (ppm) will be reported. However for full -functionality a PeakMap/MSExperiment with original m/z-values before m/z calibration generated by InternalCalibration -has to be given. +The **MzCalibration** metric adds new metavalues to the first (best) hit of each PeptideIdentification. For this metric +it is also possible to use this without an MzML File, but then only uncalibrated m/z error (ppm) will be reported. +However, for full functionality a PeakMap/MSExperiment with original m/z-values before m/z calibration generated by +InternalCalibration has to be given. #### Required input data @@ -145,8 +145,8 @@ PostFDR FeatureXML ### RTAlignment -The RTAlignment metric checks what the retention time was before the alignment and how it is after the alignment. These -two values are added to the metavalues in the PeptideIdentification. +The **RTAlignment** metric checks what the retention time was before the alignment and how it is after the alignment. +These two values are added to the metavalues in the PeptideIdentification. #### Required input data @@ -164,7 +164,7 @@ PostFDR FeatureXML, trafoXML file ### TIC -This TIC metric calculates the total ion count of an MSExperiment if a bin size in RT seconds greater than 0 is given. +The **TIC** metric calculates the total ion count of an MSExperiment if a bin size in RT seconds greater than 0 is given. All MS1 abundances within a bin are summed up. #### Required input data @@ -182,7 +182,7 @@ raw mzML file ### TopNoverRT -The TopNoverRT metric calculates the ScanEventNumber (number of the MS2 scans after the MS1 scan) and adds them as the +The **TopNoverRT** metric calculates the ScanEventNumber (number of the MS2 scans after the MS1 scan) and adds them as the new metavalue `ScanEventNumber` to the PeptideIdentifications. It finds all unidentified MS2-Spectra and adds corresponding `empty` PeptideIdentifications without sequence as placeholders to the unassigned PeptideIdentification list. Furthermore, it adds the metavalue `identified` to the PeptideIdentification. diff --git a/docs/tutorials/TOPP/smoothing-raw-data.md b/docs/tutorials/TOPP/smoothing-raw-data.md index 2e8ebae8..32876a08 100644 --- a/docs/tutorials/TOPP/smoothing-raw-data.md +++ b/docs/tutorials/TOPP/smoothing-raw-data.md @@ -1,9 +1,8 @@ Smoothing Raw Data ================== -To smooth raw data, call one of the available NoiseFilters via the Tools-menu, (select **Tools** > **Apply TOPP tool**), then select **NoiseFilterSGolay** or **NoiseFilterGaussian** as TOPPtool (green rectangle). The parameters for the filter type can be -adapted (blue rectangle). For the `Savitzky-Golay` set the **frame_length** and the **polynomial_order** fitted. -For the Gaussian filter the gaussian width and the ppm tolerance for a flexible gaussian width depending on the `m/z` +To smooth raw data, call one of the available NoiseFilters via the Tools-menu, (select **Tools** > **Apply TOPP tool**), then select **NoiseFilterSGolay** or **NoiseFilterGaussian** as TOPP tool (green rectangle). The parameters for the filter type can be adapted (blue rectangle). For the `Savitzky-Golay` filter, set the **frame_length** and the **polynomial_order** fitted. +For the Gaussian filter, the gaussian width and the ppm tolerance for a flexible gaussian width depending on the `m/z` value can be adapted. Press **Ok** to run the selected `NoiseFilter`. ![](../../images/tutorials/topp/TOPPView_tools_noisefilter.png) diff --git a/docs/tutorials/TOPP/subtracting-a-baseline-from-a-spectrum.md b/docs/tutorials/TOPP/subtracting-a-baseline-from-a-spectrum.md index f329e8e7..f0f9e7c1 100644 --- a/docs/tutorials/TOPP/subtracting-a-baseline-from-a-spectrum.md +++ b/docs/tutorials/TOPP/subtracting-a-baseline-from-a-spectrum.md @@ -2,15 +2,16 @@ Subtracting a Baseline from a Spectrum ===================================== First, load the spectrum to be analyzed in TOPPView. To use the described tools, open the tutorial data via the -File-menu (**File** > **Open example file**, then select `peakpicker\_tutorial\_1.mzML`). The BaselineFilter can be called via +File-menu (**File** > **Open example file**, then select `peakpicker_tutorial_1.mzML`). The BaselineFilter can be called via the Tools-menu (**Tools** > **Apply TOPP tool**), then select **BaselineFilter** as TOPPtool (red rectangle). You can choose, between different types of filters (green rectangle), the one mainly used is TopHat. The other important parameter is the length of the structuring element (blue rectangle). The default value is `3` Thomson. Press **Ok** to start the baseline subtraction. -![](../../images/tutorials/topp/TOPPView_tools_baseline.png) +![TOPPView Tools Baseline](../../images/tutorials/topp/TOPPView_tools_baseline.png) -The following image shows a part of the spectrum after baseline filtering as green line, the original raw data is shown -by the blue line. +The following image shows: +- A part of the spectrum after baseline filtering as a green line. +- The original raw data as a blue line. -![](../../images/tutorials/topp/TOPPView_tools_baseline_filtered.png) +![TOPPView Tools Baseline Filtered](../../images/tutorials/topp/TOPPView_tools_baseline_filtered.png) diff --git a/docs/tutorials/TOPP/topp-and-openms-introduction.md b/docs/tutorials/TOPP/topp-and-openms-introduction.md index 504d2bf7..bbd018e2 100644 --- a/docs/tutorials/TOPP/topp-and-openms-introduction.md +++ b/docs/tutorials/TOPP/topp-and-openms-introduction.md @@ -1,16 +1,16 @@ TOPP and OpenMS ============== -TOPP, the OpenMS Proteomics Pipeline provides a set of computational tools that can be easily combined into analysis +{term}`TOPP`, the OpenMS Proteomics Pipeline provides a set of computational tools that can be easily combined into analysis pipelines even by non-experts and then be used in proteomics workflows. These applications range from useful utilities -file format conversion, peak picking over wrapper applications for known applications (e.g. Mascot) to completely new -algorithmic techniques for data reduction and data analysis. TOPP is based on the OpenMS library and as more -functionality is added to new OpenMS releases, TOPP will naturally contain new or updated tools. +file format conversion, peak picking over wrapper applications for known applications (e.g.{term}`Mascot`) to completely new +algorithmic techniques for data reduction and data analysis. {term}`TOPP` is based on the OpenMS library and as more +functionality is added to new OpenMS releases, {term}`TOPP` will naturally contain new or updated tools. -In addition to offering a toolbox, TOPP contains TOPPView - the central graphical user interface - which allows the user -to view, inspect and manipulate proteomics data. TOPPView reads standard formats and allows the user not only to view -the data, but also to interactively call TOPP tools and display the results. As such it is a powerful tool for -interactive data manipulation. +In addition to offering a toolbox, {term}`TOPP` contains {term}`TOPPView` - the central graphical user interface - which +allows the user to view, inspect and manipulate {term}`proteomics` data. {term}`TOPPView` reads standard formats and +allows the user not only to view the data, but also to interactively call {term}`TOPP tools` and display the results. +As such it is a powerful tool for interactive data manipulation. -In this tutorial, let's understand in detail the capabilities of TOPPView and the TOPP tools using real data sets and -standard analysis tasks. +In this tutorial, let's understand in detail the capabilities of {term}`TOPPView` and the {term}`TOPP tools` using real +data sets and standard analysis tasks. diff --git a/docs/tutorials/TOPP/topp-for-advanced-users.md b/docs/tutorials/TOPP/topp-for-advanced-users.md index 1effd425..770ec5f6 100644 --- a/docs/tutorials/TOPP/topp-for-advanced-users.md +++ b/docs/tutorials/TOPP/topp-for-advanced-users.md @@ -6,26 +6,30 @@ This tutorials has some advanced concepts of TOPP, which will increase productiv ## Global database for search engine adapters In your `$HOME` directory find an `OpenMS.ini` in the `.OpenMS` subfolder. This INI file contains global parameters -which are used by many/all TOPP tools, depending on what the parameters refer to. The `id\_db\_dir` parameter allows to +which are used by many/all TOPP tools, depending on what the parameters refer to. The `id_db_dir` parameter allows to specify one or more directories where FASTA files (or related, e.g., `.psq` files) are placed. Specified filename (without path) in an ID engine adapter, will be looked up in the current working directory. If its not found, the directories specified in `id_db_dir` will be searched. This allows to build scripts and/or TOPPAS pipelines which are portable across several computers, just adapt the OpenMS.ini once on each machine. -> **_NOTE:_** When using TOPPAS: Use an "input file" node to specify the FASTA file for several engines simultaneously. +```{tip} +When using TOPPAS: Use an "input file" node to specify the FASTA file for several engines simultaneously. However, when selecting the file, TOPPAS will use the absolute pathname and the dialog will not allow to name a non-existing file. After selecting the file you can however edit the filename and remove the path (this will issue a warning which can be ignored). +``` -> **_NOTE:_** This approach does not work for OpenMS MascotAdapters, as each Mascot instance has its own database +```{note} +This approach does not work for OpenMS MascotAdapters, as each Mascot instance has its own database managed internally. Make sure that the database is present on all mascot servers you're going to use, thus making the INI settings portable. +``` ## Using external tools in workflows -OpenMS supports the wrapping of external tools (like msconvert from ProteoWizard), thus allowing to build scripts and/or +OpenMS supports the wrapping of external tools (like msconvert from {term}`ProteoWizard`), thus allowing to build scripts and/or TOPPAS pipelines containing external tools. -**See also** - - `share/OpenMS/TOOLS/EXTERNAL/ReadMe.txt` in your local installation for details. +```{seealso} +`share/OpenMS/TOOLS/EXTERNAL/ReadMe.txt` in your local installation for details. +``` diff --git a/docs/tutorials/TOPP/toppview-introduction.md b/docs/tutorials/TOPP/toppview-introduction.md index 000c7b24..7ac5e984 100644 --- a/docs/tutorials/TOPP/toppview-introduction.md +++ b/docs/tutorials/TOPP/toppview-introduction.md @@ -1,13 +1,13 @@ TOPPView Introduction ===================== -TOPPView is a viewer for MS and HPLC-MS data. It can be used to inspect files in mzML, mzData, mzXML and several other -text-based file formats. +{term}`TOPPView` is a viewer for {term}`MS` and {term}`HPLC-MS` data. It can be used to inspect files in {term}`mzML`, +{term}`mzData`, {term}`mzXML` and several other text-based file formats. In each view, several datasets can be displayed using the layer concept. This allows visual comparison of several datasets as well as displaying input data and output data of an algorithm together. -TOPPView is intended for visual inspection of the data by experimentalists as well as for analysis software by +{term}`TOPPView` is intended for visual inspection of the data by experimentalists as well as for analysis software by developers. -![](../../images/tutorials/topp/TOPPView_parts.png) +![TOPPView Parts](../../images/tutorials/topp/TOPPView_parts.png) diff --git a/docs/tutorials/TOPP/views-in-toppview.md b/docs/tutorials/TOPP/views-in-toppview.md index 585624fd..4153b0c4 100644 --- a/docs/tutorials/TOPP/views-in-toppview.md +++ b/docs/tutorials/TOPP/views-in-toppview.md @@ -1,8 +1,8 @@ Views in TOPPView ================= -TOPPView offers three types of views – a 1D view for spectra, a 2D view for peak maps and feature maps, and a 3D view -for peak maps. All three views can be freely configured to suit the individual needs of the user. +{term}`TOPPView` offers three types of views – a 1D view for {term}`spectra`, a 2D view for peak maps and feature maps, +and a 3D view for peak maps. All three views can be freely configured to suit the individual needs of the user. ## Action Modes and Their Uses @@ -13,19 +13,19 @@ and one for measuring: - It is activated by default - Move the mouse while holding the mouse button down to translate the current view - Arrow keys can be used to translate the view without entering translate mode (in 1D-View you can additionally - use `Shift`-key to jump to the next peak) + use Shift to jump to the next peak) - Zoom mode - - All previous zoom levels are stored in a zoom history. The zoom history can be traversed using `CTRL+`/`CTRL-` or the + - All previous zoom levels are stored in a zoom history. The zoom history can be traversed using CTRL + +/CTRL + - or the mouse wheel (scroll up and down) - Zooming into the data: - - Mark an area in the current view with your mouse, while holding the left mouse button plus the `CTRL` key to zoom + - Mark an area in the current view with your mouse, while holding the left mouse button plus the CTRL key to zoom to this area. - You can also use your mouse wheel to traverse the zoom history. - - If you have reached the end of the history, keep on pressing `CTRL+` or scroll up, the current area will be + - If you have reached the end of the history, keep on pressing CTRL + + or scroll up, the current area will be enlarged by a factor of `1.25`. - - Pressing the `Backspace` key resets the zoom and zoom history. + - Pressing Backspace resets the zoom and zoom history. - Measure mode - - It is activated using the `SHIFT` key + - It is activated using SHIFT. - Press the left mouse button down while a peak is selected and drag the mouse to another peak to measure the distance between peaks. - This mode is implemented in the 1D and 2D mode. @@ -49,9 +49,9 @@ between abundant peaks have been measured and subsequently replaced by their cor This is done by right-clicking a distance annotation and selecting **Edit** from the context menu. Additionally, peak annotations and text labels have been added by right-clicking peaks and selecting **Add peak** annotation or by right clicking anywhere and selecting **Add Label**, respectively. Multiple annotations can be selected by holding down the -`CTRL` key while clicking them. They can be moved around by dragging the mouse and deleted by pressing `DEL`. +CTRL key while clicking them. They can be moved around by dragging the mouse and deleted by pressing DEL. -![](../../images/tutorials/topp/TOPPView_1D.png) +![TOPPView 1D](../../images/tutorials/topp/TOPPView_1D.png) Through the **context menu**: of the 1D view you can: @@ -70,7 +70,7 @@ layer individually. The following example image shows a small section of a peak map and the detected features in a second layer. -![](../../images/tutorials/topp/Plot2DWidget.png) +![Plot 2D Widget](../../images/tutorials/topp/Plot2DWidget.png) In addition to the normal top-down view, the 2D view can display the projections of the data to the `m/z` and `RT` axis. This feature is mainly used to assess the quality of a feature without opening the data region in 3D view. @@ -93,7 +93,7 @@ layer individually. The following example image shows a small region of a peak map: -![](../../images/tutorials/topp/Plot3DWidget.png) +![Plot 3D Widget](../../images/tutorials/topp/Plot3DWidget.png) Through the **context menu**: of the 3D view you can: diff --git a/docs/tutorials/TOPPAS/TOPPAS-tutorial.md b/docs/tutorials/TOPPAS/TOPPAS-tutorial.md index 14569599..f44f47a7 100644 --- a/docs/tutorials/TOPPAS/TOPPAS-tutorial.md +++ b/docs/tutorials/TOPPAS/TOPPAS-tutorial.md @@ -3,8 +3,12 @@ TOPPAS Tutorial **TOPPAS** allows to create, edit, open, save, and run TOPP workflows. Pipelines can be created conveniently in a GUI. The parameters of all involved tools can be edited within TOPPAS and are also saved as part of the pipeline definition - in the .toppas file. + in the `.toppas` file. - - [General introduction](general-introduction.md) - - [User interface](user-interface.md) - - [Examples](examples.md) +```{toctree} +:maxdepth: 1 + +general-introduction +user-interface +examples +``` diff --git a/docs/tutorials/TOPPAS/examples.md b/docs/tutorials/TOPPAS/examples.md index 68e2b864..9d42c732 100644 --- a/docs/tutorials/TOPPAS/examples.md +++ b/docs/tutorials/TOPPAS/examples.md @@ -2,8 +2,7 @@ Examples ======== The following sections explain the example pipelines TOPPAS comes with. Open them by selecting **File** > **Open example file**. -All input files and parameters are already specified, so you can just hit **Pipeline** > **Run** (or press `F5`) and see what -happens. +All input files and parameters are already specified, so you can just hit **Pipeline** > **Run** (or press F5) and see what happens. ## Profile data processing @@ -13,7 +12,7 @@ noise-filtered and baseline-reduced profile data. This workflow is also describe [Profile data processing](../TOPP/profile-data-processing.md). The individual steps are explained in more detail in the TOPPView tutorial: [Smoothing raw data](../TOPP/smoothing-raw-data.md), [Subtracting a baseline from a spectrum](../TOPP/subtracting-a-baseline-from-a-spectrum.md), and [Picking peaks](../TOPP/picking-peaks.md). -![](../../images/tutorials/toppas/TOPPAS_example_profile_data_processing.png) +![TOPPAS Example Profile Data Processing](../../images/tutorials/toppas/TOPPAS_example_profile_data_processing.png) ## Identification of E. coli peptides @@ -32,43 +31,44 @@ path to the OMSSA executable (omssacl) must be set in the parameters of the OMSS - FalseDiscoveryRate computes q-values for the IDs. - Finally, IDFilter selects only those IDs with a q-value of less than 1%. -![](../../images/tutorials/toppas/TOPPAS_Ecoli_Identification.png) +![TOPPAS Ecoli Identification](../../images/tutorials/toppas/TOPPAS_Ecoli_Identification.png) Extensions to this pipeline would be to do the annotation of the spectra with multiple search engines and combine the -results afterwards, using the ConsensusID TOPP tool. +results afterwards, using the **ConsensusID** TOPP tool. -The results may be exported using the TextExporter tool, for further analysis with different tools. +The results may be exported using the **TextExporter** tool, for further analysis with different tools. ## Quantitation of BSA runs The simple pipeline described in this section (`BSA_Quantitation.toppas`) can be used to quantify peptides that occur -on different runs. The example dataset contains three different bovine serum albumin (BSA) runs. First, FeatureFinderCentroided -is called since the dataset is centroided. The results of the feature finding are then annotated with (existing) -identification results. For convenience, we provide these search results from OMSSA with peptides with an FDR of 5% in -the BSA directory. +on different runs. The example dataset contains three different bovine serum albumin (BSA) runs. First, **FeatureFinderCentroided** is called since the dataset is centroided. The results of the feature finding are then annotated with (existing) identification results. For convenience, we provide these search results from OMSSA with peptides with an FDR of 5% in the BSA directory. -![](../../images/tutorials/toppas/TOPPAS_BSA_Quantitation.png) +![TOPPAS BSA Quantitation](../../images/tutorials/toppas/TOPPAS_BSA_Quantitation.png) -Identifications are mapped to features by the IDMapper. The last step is performed by FeatureLinkerUnlabeled which links +Identifications are mapped to features by the **IDMapper**. The last step is performed by **FeatureLinkerUnlabeled** which links corresponding features. The results can be used to calculate ratios, for example. The data could also be exported to a text based format using the TextExporter for further processing (e.g., in Microsoft Excel). The results can be opened in TOPPView. The next figures show the results in 2D and 3D view, together with the feature -intermediate results. One can see that the intensities and retention times are slightly different between the runs. To -correct for retention times shift, a map alignment could be done, either on the spectral data or on the feature data. +intermediate results. -![](../../images/tutorials/toppas/TOPPAS_BSA_results_2d.png) +```{tip} +The intensities and retention times are slightly different between the runs. To correct for retention +times shift, a map alignment could be done, either on the spectral data or on the feature data. +``` -![](../../images/tutorials/toppas/TOPPAS_BSA_results_3d.png) +![TOPPAS BSA Results 2D](../../images/tutorials/toppas/TOPPAS_BSA_results_2d.png) + +![TOPPAS BSA Results 3D](../../images/tutorials/toppas/TOPPAS_BSA_results_3d.png) ## Merger and Collect nodes -The following example is actually not a useful workflow but is supposed to demonstrate how merger and collector nodes -can be used in a pipeline. Have a look at `merger_tutorial.toppas`: +The following example is trivial but demonstrates how merger and collector nodes can be used in a pipeline. Have a look +at `merger_tutorial.toppas`: -![](../../images/tutorials/toppas/TOPPAS_example_merger.png) +![TOPPAS Example Merger](../../images/tutorials/toppas/TOPPAS_example_merger.png) -As its name suggests, a merger merges its incoming file lists, i.e., files of all incoming edges are appended into new +A Merger merges its incoming file lists, i.e., files of all incoming edges are appended into new lists (which have as many elements as the merger has incoming connections). All tools this merger has outgoing connections to are called with these merged lists as input files. All incoming connections should pass the same number of files (unless the corresponding preceding tool is in recycling mode). diff --git a/docs/tutorials/TOPPAS/general-introduction.md b/docs/tutorials/TOPPAS/general-introduction.md index 78eb4b11..afd02cf2 100644 --- a/docs/tutorials/TOPPAS/general-introduction.md +++ b/docs/tutorials/TOPPAS/general-introduction.md @@ -9,12 +9,12 @@ invalid workflows. Once set up and saved, a workflow can also be run without the The following figure shows a simple example pipeline that has just been created and executed successfully: -![](../../images/tutorials/toppas/TOPPAS_simple_example.png) +![TOPPAS Simple Example](../../images/tutorials/toppas/TOPPAS_simple_example.png) To create a new TOPPAS file, do any of the following: -- open TOPPAS without providing any existing workflow - an empty workflow will be opened automatically. -- in a running TOPPAS program choose: **File** > **New** -- create an empty file in your file browser (explorer) with the suffix `.toppas` and double-click it (on Windows systems +- Open TOPPAS without providing any existing workflow - an empty workflow will be opened automatically. +- In a running TOPPAS program, choose: **File** > **New**. +- Create an empty file in your file browser (explorer) with the suffix `.toppas` and double-click it (on Windows systems all `.toppas` files are associated with TOPPAS automatically during installation of OpenMS, on Linux, and macOS you might need to manually associate the extension). diff --git a/docs/tutorials/TOPPAS/user-interface.md b/docs/tutorials/TOPPAS/user-interface.md index a5420377..d7956d03 100644 --- a/docs/tutorials/TOPPAS/user-interface.md +++ b/docs/tutorials/TOPPAS/user-interface.md @@ -19,7 +19,7 @@ workflow, or if it just would not make sense, e.g. if its target is an input fil file node (#1) and the OMSSAAdapter (#5) tool is painted yellow which indicates it is not ready yet, because no input files have been specified. -![](../../images/tutorials/TOPPAS_edges.png) +![TOPPAS Edges](../../images/tutorials/toppas/TOPPAS_edges.png) The input/output mapping of connections can be changed at any time during the editing process by double-clicking an connections or by selecting **Edit I/O mapping** from the context menu which appears when a connection is right-clicked. @@ -30,30 +30,31 @@ The following figure shows a possible next step: the user has double-clicked one its parameters. By default, the standard parameters are used for each tool. Again, this can also be done by selecting **Edit parameters** from the context menu of the tool. -![](../../images/tutorials/TOPPAS_parameters.png) +![TOPPAS Parameters](../../images/tutorials/toppas/TOPPAS_parameters.png) -Once the pipeline has been set up, the input files have to be specified before the pipeline can be executed. This is -done by double-clicking an input node and selecting the desired files in the dialog that appears. Input nodes have a -special mode named **recycling mode**, i.e., if the input node has fewer files than the following node has rounds -(as it might have two incoming connections) then the files are recycled until all rounds are satisfied. This might be -useful if one input node specifies a single database file (for a Search-Adapter like Mascot) and another input node has -the actual MS2 experiments (which is usually more than one). Then the database input node would be set to `recycle` the -database file, i.e. use it for every run of the MascotAdapter node. The input list can be recycled an arbitrary number -of times, but the recycling has to be `complete`, i.e. the number of rounds of the downstream node have to be a multiple -of the number of input files. Recycling mode can be activated by right-clicking the input node and selecting the according -entry from the context menu. Finally, if you have input and output nodes at every end of your pipeline and all -connections are green, select **Pipeline** > **Run** in the menu bar or just press `F5`. +Once the pipeline has been set up, specify the input files before executing the pipeline. This is done by double-clicking +an input node and selecting the desired files in the dialog that appears. Input nodes have a special mode named +**recycling mode**, i.e., if the input node has fewer files than the following node has rounds (as it might have two incoming +connections) then the files are recycled until all rounds are satisfied. This might be useful if one input node specifies a +single database file (for a Search-Adapter like Mascot) and another input node has the actual MS2 experiments (which is +usually more than one). Then the database input node would be set to `recycle` the database file, i.e. use it for every run of +the MascotAdapter node. The input list can be recycled an arbitrary number of times, but the recycling has to be `complete`, +i.e. the number of rounds of the downstream node have to be a multiple of the number of input files. Recycling mode can be +activated by right-clicking the input node and selecting the according entry from the context menu. Finally, if you have input +and output nodes at every end of your pipeline and all connections are green, select **Pipeline** > **Run** in the menu bar or +just press F5. -![](../../images/tutorials/TOPPAS_run_options.png) +![TOPPAS Run Options](../../images/tutorials/toppas/TOPPAS_run_options.png) -When asked for an output file directory where a sub-directory, `TOPPAS_out`, will be created. This directory will -contain the output files. Specify the number of jobs TOPPAS is allowed to run in parallel, if a number greater than 1 is +When asked for an output file directory `TOPPAS_out` as sub-directory, will be created. This directory will +contain the output files. Specify the number of jobs TOPPAS is allowed to run in parallel. If a number greater than 1 is selected, TOPPAS will parallelize the pipeline execution in the following scenarios: - A tool has to process more than one file. In this case, multiple files can be processed in parallel. - The pipeline contains multiple branches that are independent of each other. In this case, multiple tools can run in parallel. +```{caution} Be careful with this setting, however, as some of the TOPP tools require large amounts of RAM (depending on the size of the dataset). Running too many parallel jobs on a machine with not enough memory will cause problems. Also, do not confuse this setting with the `threads` parameter of the individual TOPP tools: every TOPP tool has this parameter @@ -61,6 +62,7 @@ specifying the maximum number of threads the tool is allowed to use (although on of this parameter, since there are tasks that cannot be computed in parallel). Be especially careful with combinations of both parameters! For a pipeline containing the `FeatureFinderCentroided`, as an example, and set its `threads` parameter to 8, and additionally set the number of parallel jobs in **TOPPAS** to 8, then you end up using 64 threads. +``` In addition to `TOPPAS_out`, a `TOPPAS_tmp` directory will be created in the OpenMS temp path (call the `OpenMSInfo` tool to see where exactly). It will contain all temporary files that are passed from tool to tool within the pipeline. @@ -76,8 +78,8 @@ Using the mouse: - drag and drop tools from the TOPP tool list onto the workflow window (you can also double-click them instead) - select items (by clicking) -- select multiple items (by holding down `CTRL` while clicking) -- select multiple items (by holding down `CTRL` and dragging the mouse in order to "catch" items with a selection +- select multiple items (by holding down CTRL while clicking) +- select multiple items (by holding down CTRL and dragging the mouse in order to "catch" items with a selection rectangle) - move all selected items (by dragging one of them) - draw a new connection from one node to another (by dragging; source must be deselected first) @@ -90,15 +92,15 @@ Using the mouse: Using the keyboard: -- delete all selected items (`DEL` or `BACKSPACE`) -- zoom in and out (`+`/`-`) -- run the pipeline (`F5`) -- open this tutorial (`F1`) +- delete all selected items (DEL or BACKSPACE) +- zoom in and out (+ / -) +- run the pipeline (F5) +- open this tutorial (F1) -Using the mouse` + `keyboard: +Using the mouse + keyboard: - copy a node's parameters to another node (only parameters with identical names will be copied, e.g., - `fixed_modifications`) (`CTRL` while creating an edge) The edge will be colored as dark magenta to indicate parameter + `fixed_modifications`) (CTRL while creating an edge) The edge will be colored as dark magenta to indicate parameter copying. ## Menus diff --git a/openms-wiki/about.md b/openms-wiki/about.md new file mode 100644 index 00000000..79d4233b --- /dev/null +++ b/openms-wiki/about.md @@ -0,0 +1,15 @@ + +About +===== + +OpenMS is an open-source C++ library for Mass Spectrometry (MS) data management and analyses. It offers an infrastructure for the rapid development of mass spectrometry-related software. OpenMS is free software available under the [three-clause BSD license](https://github.com/OpenMS/OpenMS-docs/blob/main/LICENSE). It is available for Windows, macOS and Linux. + +OpenMS comes with a vast variety of pre-built and ready-to-use tools for proteomics and metabolomics data analysis (TOPP tools), and powerful 2D and 3D visualisation (TOPPView). It supports analyses for various quantification protocols, including label-free shotgun proteomics, SILAC, iTRAQ, SRM, and SWATH. It also provides built-in algorithms for peptide/protein identification, both de-novo and via database searching, as well as adapters to state-of-the-art tools like X! Tandem, Mascot, OMSSA, and others. + +Through the TOPP tools concept and unified parameter handling (CTD), OpenMS supports easy integration into workflow engines like TOPPAS (included), KNIME, Galaxy, and WS-PGRADE. + +With pyOpenMS, OpenMS offers Python bindings to a large part of the API to enable rapid computational mass spectrometry development. + +OpenMS supports the Proteomics Standard Initiative (PSI) file formats for MS data. + +The main contributors of OpenMS are currently Eberhard-Karls Universität in Tübingen, Freie Universität Berlin and ETH Zurich. diff --git a/openms-wiki/contributors-quickstart-guide.md b/openms-wiki/contributors-quickstart-guide.md new file mode 100644 index 00000000..a387a9ac --- /dev/null +++ b/openms-wiki/contributors-quickstart-guide.md @@ -0,0 +1,71 @@ +Contribute to OpenMS +==================== + +If you would like to contribute to OpenMS: + +* Familiarise yourself with our [online documentation](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/release/latest/html/index.html). + +* Learn how to [build OpenMS](build-openms-from-source.md). + +* Check out the [OpenMS tutorial for developers](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/release/latest/html/OpenMS_tutorial.html). + +For any questions, please [contact us](contact-us.md) at [open-ms-general]((https://sourceforge.net/projects/open-ms/lists/open-ms-general) mailing list. + +## Technical Documentation +Untested installers and containers are known as the nightly snapshot, are released every night. They generally pass automated continuous integration tests but no manual tests. + +View the documentation for the nightly snapshot of [OpenMS develop branch](https://github.com/OpenMS/OpenMS/tree/develop) at the [build archive](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/index.html). + +View the [doxygen log](https://abibuilder.informatik.uni-tuebingen.de/jenkins/job/openms/job/ntly/job/TstPkg/compiler=appleclang-11.0.0,os_label=catalina/lastBuild/artifact/build/doc/doxygen/doxygen-error.log). + +See the documentation for the [latest release](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/release/latest/html/index.html). + +View the [doxygen log](https://abibuilder.informatik.uni-tuebingen.de/jenkins/job/openms_release_packaging/lastBuild/compiler=appleclang-7.3.0,os_label=elcapitan/artifact/build/doc/doxygen/doxygen-error.log). + +## Contribution Guidelines + +Before contributing to OpenMS, read information on the development model and conventions followed to maintain a coherent code base. + +### Development model + +OpenMS follows the [Gitflow development workflow](http://nvie.com/posts/a-successful-git-branching-model/). + +Every contributor is encouraged to create their own fork (even if they are eligible to push directly to OpenMS). To create a fork: +1. Follow the documentation on [forking](https://help.github.com/articles/fork-a-repo). +2. Keep your fork [up-to-date](https://help.github.com/articles/syncing-a-fork). +3. Create a [pull request](https://help.github.com/articles/using-pull-requests). Before opening the pull request, please view the [checklist](pull-request-checklist.md). + +### Coding conventions + +See the manual for coding style recommended by OpenMS: [Coding conventions](https://github.com/OpenMS/OpenMS/wiki/Coding-conventions). +also see: [C++ Guide](https://github.com/OpenMS/OpenMS/wiki/Cpp-Guide). + +View the [manual]() for creating a new build unit (to be completed). + +OpenMS automatically tests for common coding convention violations using a modified version of `cpplint`. +Style testing can be enabled using `cmake` options. We also provide a configuration file for `Uncrustify` for automated style corrections (see `tools/uncrustify.cfg`). + +### Commit messages + +View the guidelines for commit messages: [How to write commit messages](https://github.com/OpenMS/OpenMS/wiki/HowTo---Write-Commit-Messages). + +### Automated unit tests + +Nightly tests run on different platforms. It is recommended to test on different platforms. This will save you time and surprises during continuous integration tests. + +Nightly tests: [CDASH](http://cdash.openms.de/index.php?project=OpenMS) + +## Further Contributor Resources + +You may want to consider the following resources: +* **Guidelines for adding new dependency libraries** + + View the guidelines for [adding new dependency libraries](). +* **Experimental installers** + + We automatically build installers for different platforms. These usually contain unstable or partially untested code. + + The nightly (unstable) installers are available at the [build archive](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/nightly/). +* **Developer FAQ** + + Visit the [Developer FAQ](developer-faq.md) to get answers to frequently asked questions. diff --git a/openms-wiki/developer-faq.md b/openms-wiki/developer-faq.md new file mode 100644 index 00000000..960387d1 --- /dev/null +++ b/openms-wiki/developer-faq.md @@ -0,0 +1,491 @@ +Developer FAQ +============= + +The following contains answers to typical questions from developers about OpenMS. + +## General + +The following section provides general information to new contributors. + +### I am new to OpenMS. What should I do first? + +* Check out the development version of OpenMS (see website). +* Try and build OpenMS according to the installation instructions. +* Read the [OpenMS Coding Conventions](https://github.com/OpenMS/OpenMS/wiki/Coding-conventions). +* Read the OpenMS Tutorial. +* Create a GitHub account. +* Subscribe to the [open-ms-general](https://sourceforge.net/projects/open-ms/lists/open-ms-general) and [open-ms-developers]() mailing lists. + > ***_NOTE:_*** You can see the developers list only if you are logged in to Sourceforge and if you are a OpenMS developer. + +### What is the difference between an OpenMS tool and util? + +A tool starts its lifecycle in `UTILS` and may exist without being thoroughly tested. Tools may be promoted from `UTILS` to `TOOLS` if they are stable enough, are fully tested, fully documented, and a test workflow exists. + +### I have written a class for OpenMS. What should I do? + +Follow the [OpenMS coding conventions](coding-conventions.md). + +Coding style (brackets, variable names, etc.) must conform to the conventions. + +* The class and all the members should be properly documented. +* Check your code with the tool `tools/checker.php`. Call `php tools/checker.php` for detailed instructions. + +Please open a pull request and follow the [pull request guidelines](pull-request-checklist.md). + +### How do I update the [openms.de](https://www.openms.de)? + +Login to the [wordpress admin area](www.openms.de/wp-admin) with your username and password assigned by the current Homepage maintainers. + +### Can I use QT designer to create GUI widgets? + +Yes. Create a class called `Widget: Create .ui-File` with `QT designer` and store it as `Widget.ui.`, add the class to `sources.cmake`. +From the .ui-File the file `include/OpenMS/VISUAL/UIC/ClassTemplate.h` is generated by the build system. + +> **_NOTE:_** Do not check in this file, as it is generated automatically when needed. + +Derive the class `Widget` from `WidgetTemplate`. For further details, see the `Widget.h` and `Widget.cpp` files. + +### Can the START_SECTION-macro not handle template methods that have two or more arguments? + +Insert round brackets around the method declaration. + +### Where can I find the binary installers created? + +View the binary installers at the [build archive](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/nightly/). +Please verify the creation date of the individual installers, as there may have been an error while creating the installer. + +## Troubleshooting + +The following section provides information about how to troubleshoot common OpenMS issues. + +### OpenMS complains about boost not being found but I'm sure its there + +`CMake` got confused. Set up a new build directory and try again. If you build from source (not recommended), deleting the `CMakeCache.txt` and `cmake` directory might help. + +## Build System + +The following questions are related to the build system. + +### What is CMake? + +`CMake` builds BuildSystems for different platforms, e.g. VisualStudio Solutions on Windows, Makefiles on Linux etc. +This allows us to define in one central location (namely `CMakeLists.txt`) how OpenMS is build and have the platform specific stuff handled by `CMake`. + +View the [cmake website](http://www.cmake.org) for more information. + +### How do I use CMake? + +See Installation instructions for your platform. +In general, call `CMake(.exe)` with some parameters to create the native build-system. + +> **_NOTE:_** whenever `ccmake` is mentioned in this document, substitute this by `CMake-GUI` if your OS is Windows. Edit the `CMakeCache.txt` file directly. + +### How do I generate a build-system for Eclipse, KDevelop, CodeBlocks etc? + +Type `cmake` into a console. This will list the available code generators available on your platform; use them with `CMake` using the `-G` option. + +### What are user definable CMake cache variables? + +They allow the user to pass options to `CMake` which will influence the build system. The most important option which should be given when calling `CMake.exe` is: + +`CMAKE_FIND_ROOT_PATH`, which is where `CMake` will search for additional libraries if they are not found in the default system paths. By default we add `OpenMS/contrib`. + +If you have installed all libraries on your system already, there is no need to change `CMAKE_FIND_ROOT_PATH`. If you need the `contrib` libraries, set the variable `CMAKE_FIND_ROOT_PATH`. +On Windows, you always need the `contrib` folder, as there are no system developer packages. To pass this variable to `CMake` use the `-D` switch e.g. `cmake -D CMAKE_FIND_ROOT_PATH:PATH="D:\\somepath\\contrib"`. +Everything else can be edited using `ccmake` afterwards. + +The following options are of interest: + +* `CMAKE_BUILD_TYPE` Define if you want to build Debug or Release version of OpenMS. Release is the default. + +* `CMAKE_FIND_ROOT_PATH` The path to the `contrib` libraries. + > **_NOTE:_** that you can also provide more then one value here (e.g., `-D CMAKE_FIND_ROOT_PATH="/path/to/contrib;/usr/"` will search in your `contrib` path and in `/usr` for the required libraries) + +* `STL_DEBUG` Enables STL debug mode. + +* `DB_TEST` (deprecated) Enables database testing. + +* `QT_DB_PLUGIN` (deprecated) Defines the db plugin used by Qt. + +* `MT_CUDA_BUILD_TYPE` ... + +View the description for each option by calling `ccmake`. + +### Can I use another solver other than GLPK? + +Other solvers can be used, but by default, the build system only links to GLPK (this is how OpenMS binary packages must be built). +To to use another solver, use `cmake ... -D USE_COINOR=1 ....` and refer to the documentation of the `LPWrapper` class. + +### How do I switch to debug or release configuration? + +For Makefile generators (typically on Linux), you can set the `CMAKE_BUILD_TYPE` variable to either Debug or Release by calling `ccmake`. +For Visual Studio, this is not necessary as all configurations are generated and you can choose the one you like within the IDE itself. +The 'Debug' configuration enabled debug information. The 'Release' configuration disables debug information and enables optimisation. + +### I changed the `contrib` path, but re-running `CMake` won't change the library paths? + +Once a library is found and its location is stored in a cache variable, it will only be searched again if the corresponding entry in the cache file is set to false. +You can delete the `CMakeCache.txt`, but all other custom settings will be lost. + +### What are the most useful `make` targets? + +In Visual Studio you can see all targets on the left. For Makefiles type make help. However, this list is quite long. +The most useful targets will be shown to you by calling the targets target, i.e. make targets. + +### `CMake` can't seem to find a `Qt` library (usually `QtCore`). What now? + +`CMake` finds `QT` by looking for `qmake` in your PATH or for the Environment Variable `QTDIR`. Set these accordingly. +If the problem still persists: do you have a second installation of Qt (especially the MinGW version)? This might lead ``CMake`` to the wrong path (it's searching for the ``Qt*.lib`` files). +You should only move or delete the offending `Qt` version if you know what you are doing! +A save workaround is to edit the `CMakeCache` file (e.g. via `ccmake`) and set all paths relating to `QT` (e.g. `QT_LIBRARY_DIR`) manually. + +### (Windows) What version of Visual Studio should I use? + +Use the latest if possible. Get the latest `CMake`, as its generator needs to support your VS. If your VS is too new and there is no `CMake` for that yet, you're gonna be faced with a lot of conversion issues. +This happens whenever the Build-System calls `CMake` (which can be quite often, e.g., after changes to `CMakeLists.txt`). + +### How do I add a new class to the build system? + +1. Create the new class in the corresponding sub-folder of the sub-project. The header has to be created in `src//include/OpenMS` and the cpp file in `src//source`, e.g., `src/openms/include/OpenMS/FORMAT/NewFileFormat.h` and `src/openms/source/FORMAT/NewFileFormat.cpp`. +2. Add both to the respective sources.cmake file in the same directory (e.g., `src/openms/source/FORMAT/` and `src/openms/include/OpenMS/FORMAT/`). +3. Add the corresponding class test to src/tests/class_tests// (e.g., `src/tests/class_tests/openms/source/NewFileFormat_test.cpp`). +4. Add the test to the `executables.cmake` file in the test folder (e.g., `src/tests/class_tests/openms/executables.cmake`). +5. Add them to git by using the command `git add`. + +### How do I add a new directory to the build system? + +1. Create two new `sources.cmake` files (one for `src//include/OpenMS/MYDIR`, one for `src//source/MYDIR`), using existing `sources.cmake` files as template. +2. Add the new `sources.cmake` files to `src//includes.cmake` +3. If you created a new directory directly under `src/openms/source`, then have a look at `src/tests/class_tests/openms/executables.cmake`. +4. Add a new section that makes the unit testing system aware of the new (upcoming) tests. +5. Look at the very bottom and augment `TEST_executables`. +6. Add a new group target to `src/tests/class_tests/openms/CMakeLists.txt`. + +### Class/Unit tests and TOPP/Tool tests + +Class or unit tests are built as standalone, additional executables that include the class to be tested and the testing utility classes to test outcomes of single functions of the class in question. + +Only add tests for functions added outside of your additional mode. + +Tool tests are using the tool executable that the user would also receive. We use those executables to run the full algorithm on a small test dataset, to ensure that from version to version the results stay the same and are meaningful. + +Each tool test consists of: + +* An executable call on a test dataset (by using either fixed command line parameters or an ini file). + +* A `FuzzyDiff` call that compares the temporary output file of the last call and a reference test output that you have to provide. + +* A line to add a dependency of the FuzzyDiff call on the actual executable call (so they get executed after each other). + +Use e.g., `ctest -V -R IDMapper` to only test tests that include the regex `IDMapper` (-V is just verbose). Make sure to build the `IDMapper` and `IDMapper_test` (if edited) executable first everytime. +`ctest` does not have any automatic dependency on the timestamps of the executables. + +### How do I add a test for a new class? + +You should always add a test alongside every new class added to OpenMS. + +To add a new class test: + +1. Add the class test to `src/tests/class_tests//` (e.g., `src/tests/class_tests/openms/source/NewFileFormat_test.cpp`). +2. Add the test to the `executables.cmake` file in the test folder. +3. Add them to git using the `git add` command. + +A test template for your specific class can be generated by the `create_test.php` script found in `tools/`. + +To generate a test template for your class: + +1. Make sure your generated XML files containing the class information make doc_xml. +2. Call: +```bash +php tools/create_test.php /BUILD_DIRECTORY/ / PATH_TO_HEADER/MyClass.h \ "FIRSTNAME LASTNAME" > ./src/tests/class_tests/openms/source/MyClass_test.cpp +``` + +### How do I add a new GUI test (for QT Gui classes) for the class `MyClass`? +To add a new GUI test: + +1. Create the `MyClass_test.cpp` in `src/tests/class_tests/openms_gui/source`. +2. Add it to `src/tests/class_tests/openms_gui/CMakeLists.txt` in the GUI section. +3. Have a look at existing GUI tests, as they use the `QT TestLib` framework and not the OpenMS macros. + +### (Linux) All tests fail when you execute `make test` + +Check the `LD_LIBRARY_PATH` environment variable: + +You can print the `LD_LIBRARY_PATH` with `echo $LD_LIBRARY_PATH`. If your `/lib/` folder is included, check that `libOpenMS.so` is present. +With the `ldd` command, you can show the libraries used by an executable, e.g. `ldd /bin/ClassTest_test`. + +### How can I speed up the compile process of OpenMS? + +To speed up the compile process of OpenMS, use several threads. If you have several processors/cores, you can build OpenMS classes/tests and `TOPP` tools in several threads. On Linux, use the `make option -j: make -j8 OpenMS TOPP test_build`. + +On Windows, Visual Studio solution files are automatically build with the `/MP` flag, such that Visual Studio uses all available cores of the machine. + +## Release + +View [Preparation of a new OpenMS release](https://github.com/OpenMS/OpenMS/wiki/Preparation-of-a-new-OpenMS-release#release_developer) to learn more about contributing to releases. + +## Continuous integration + +### How does travis work? + +[Travis](https://travis-ci.org/) is an automated system for continuous integration. Each new commit and pull request is automatically run through the travis build system. +This is controlled by a `.travis.yaml` file in the source tree. + +### What can we do if travis times out? + +Try to restart travis, it sometimes hangs and since it builds on shared infrastructure, the next build may work better. This needs to be done by a OpenMS core developer. + +Since we use extensive caching, the build may take much longer when many files are touched and may never complete in that case (running into the travis time limit). In that case we can rebuild the cache using the following approach: +```bash + $ git cherry-pick 89c5cd7f2d9d343b3d63fc6bab18e08dcd969c05 + + $ git push origin develop + + # Now wait for the build to complete + + $ git revert 89c5cd7f2d9d343b3d63fc6bab18e08dcd969c05 + + $ git push origin develop +``` + +## Working in Integrated Development Environments (IDEs) + +### Why are there no `source/TEST` and `source/APPLICATIONS/TOPP|UTILS` folder? + +All source files added to an IDE are associated with their targets. You can find the source files for each test within its own subproject. The same is true for the `TOPP` and `UTILS` classes. + +### How can I easily update a lot of test files (e.g., after a small format change in result files)? + +Using `grep` one can simply extract the lines starting with `diff FILENAME1 FILENAME2` and replace the `diff` by `copy`. + +### I'm getting the error "Error C2471: cannot update program database" + +This is a bug in Visual Studio and there is a [bug fix] (http://code.msdn.microsoft.com/KB946040) Only apply it if you encounter the error. The bug fix might have unwanted side effects! + +### Visual Studio can't read the clang-format file. + +Depending on the Visual Studio version you're using you might get an error like `Error while formating with ClangFormat`. This is because Visual Studio is using an outdated version of clang-format. Unfortunately there is no easy way to update this using Visual Studio itself. +There is a plugin provided by LLVM designed to fix this problem, but the plugin doesn't work with every Visual Studio version. However, you can update clang-format by hand using the pre-build clang-format binary. Both the binary and a link to the plugin can be found [here](https://llvm.org/builds/). +To update clang-format by hand just download the binary and exchange it with the clang-format binary in your Visual Studio folder. +For Visual Studio 17 and 19 it should be located at: `C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\Llvm\bin`. + +### The indexer gets stuck at some file which ``#includes seqan`` + +It seems that SeqAn code is just too confusing for older eclipse C++ indexers. You should upgrade to eclipse galileo (CDT 6.0.x). Also, increase the available memory limit in eclipse.ini, e.g. -Xmx1024m for one gig. + +### The parser is confused after OPENMS_DLLAPI and does not recognize standard C++ headers + +Go to ``Project -> Properties -> C/C++ Include Paths and Preprocessor Symbols -> Add Preprocessor symbol -> "OPENMS_DLLAPI="``. This tells eclipse that the macro is defined empty. In the same dialog you can also add an external include path to e.g. ``/usr/include/c++/4.3.3/``, etc. The issue with C++ headers was fixed in the latest galileo release. + +Hints to resolve the OPENMS_DLLAPI issue using the ``cmake`` generator are welcome! + +## Debugging + +The following section provides information about how to debug your code. + +### How do I run a single test? + +You can can execute an OpenMS class test using the CTest regular expressions: + +```bash + +ctest -V -R "^_test" + +# To build a class test, you simply call the respective make target in ./source/TEST: + +make _test +``` +To run a TOPP test, you can use: + +```bash + +ctest -V -R "TOPP_" +``` + +To build the tool, use: + +```bash +make +``` +### How do I debug uncaught exceptions? + +You can dump a core if an uncaught exception occurs, by setting the environment variable `OPENMS_DUMP_CORE`. + +Each time an uncaught exception occurs, the `OPENMS_DUMP_CORE` variable is checked and a segmentation fault is caused, if it is set. + +### (Linux) Why is no core dumped, although a fatal error occured? + +Try the `ulimit -c` unlimited command. It sets the maximum size of a core to unlimited. + +> **_NOTE:_** We observed that, on some systems, no core is dumped even if the size of the core file is set to unlimited. We are not sure what causes this problem + +### (Linux) How can I set breakpoints in gdb to debug OpenMS? + +Imagine you want to debug the TOPPView application and you want it to stop at line 341 of SpectrumMDIWindow.C. + +1. Enter the following in your terminal: + + ```bash + Run gdb: + shell> gdb TOPPView +``` + +2. Start the application (and close it): + + ```bash + gdb> run [arguments] +``` +3. Set the breakpoint: + ```bash + gdb> break SpectrumMDIWindow.C:341 +``` +4. Start the application again (with the same arguments): + + ```bash + gdb> run + ``` + +### How can I find out which shared libraries are used by an application? + +Linux: Use `ldd`. + +Windows (Visual studio console): Try [Dependency Walker](http://www.dependencywalker.com/) (use x86 for 32bit builds and the x64 version for 64bit builds. Using the wrong version of depends.exe will give the wrong results) or ``dumpbin /DEPENDENTS OpenMS.dll``. + +### How can I get a list of the symbols defined in a (shared) library or object file? + +Linux: Use `nm `. + +Use `nm -C` to switch on demangling of low-level symbols into their C++-equivalent names. `nm` also accepts .a and .o files. + +Windows (Visual studio console): Use ``dumpbin /ALL ``. + +You can use dumpbin on object files (.o) or (shared) library files (.lib) or the DLL itself e.g. `dumpbin /EXPORTS OpenMS.dll`. + +## Cross-platform thoughts + +OpenMS runs on three major platforms.. Here are the most prominent causes of "it runs on Platform A, but not on B. What now?" + +### Reading or writing binary files +Reading or writing binary files causes different behaviour. Usually Linux does not make a difference between text-mode and binary-mode when reading files. This is quite different on Windows as some bytes are interpreted as `EOF`, which lead might to a premature end of the reading process. + +If reading binary files, make sure that you explicitly state that the file is binary when opening it. + +During writing in text-mode on Windows a line-break (`\n`) is expanded to (`\r\n`). Keep this in mind or use the `eol-style` property of subversion to ensure that line endings are correctly checked out on non-Windows systems. + +### `UInt` vs `Size` +Both `unsigned int` vs `size_t` `UInt` and `Size` have the same size on Linux GCC (32bit on 32bit systems, 64bit on 64 bit systems), however on Windows this only holds for 32bit. On a 64bit Windows, the `UInt` type is still 32bit, while the `Size` type is 64bit. This might lead to warnings (at best) or overflows and other drawbacks. +Therefore, do not assume that `UInt` is equal to `Size`. + +### Paths and system functions + +Avoid hardcoding e.g.`String tmp_dir = "/tmp";`. This will fail on Windows. Use Qt's `QDir` to get a path to the systems temporary directory if required. + +Avoid names like uname which are only available on Linux. + +When working with files or directories, it is usually safe to use "/" on all platforms. Take care of spaces in directory names though. You should always quote paths if they are used in a system call to ensure that the subsequent interpreter takes the spaced path as a single entity. + + +## PyOpenMS - Troubleshooting + +The following section contains answers to common issues experienced in pyOpenMS. + +### How can I wrap my new method with PyOpenMS? + +You will have to add an entry to `src/pyOpenMS/pxds/CLASS_NAME.pxd` with the signature of your new method(s). + +### How can I wrap my new class with PyOpenMS? + +You will have to create a new file `src/pyOpenMS/pxds/CLASS_NAME.pxd` that is explained in the [pyOpenMS documentation](https://pyopenms.readthedocs.io/en/latest/wrap_classes.html?highlight=Wrapping%20classes#wrapping-workflow-and-wrapping-new-classes). + +### Can I use output parameters for methods with multiple outputs? + +Python does not support passing primitive types (`int`, `double`, etc.) by reference. Therefore, `void calculate(double &)` will not work. + +## Doxygen Documentation + +### Where can I find the definition of the main page? + +You can find a definition of the main page [here](https://github.com/OpenMS/OpenMS/edit/develop/doc/doxygen/public/Main.doxygen). + +### Where can I add a new module? + +You can add a new module [here](https://github.com/OpenMS/OpenMS/edit/develop/doc/doxygen/public/Modules.doxygen). + +### How is the parameter documentation for classes derived from DefaultParamHandler created? + +You have to add your class to the program ``OpenMS/doc/doxygen/parameters/DefaultParamHandlerDocumenter.cpp``. This program generates a html table with the parameters. This table can then be included in the class documentation using the following `doxygen` command:`@htmlinclude OpenMS_.parameters`. + +> **_NOTE:_** parameter documentation is automatically generated for `TOPP/UTILS` included in the static `ToolHandler.cpp` tools list. + +To include TOPP/UTILS parameter documentation use following `doxygen` command: + + +`@htmlinclude TOPP_.parameters` + +or + +`@htmlinclude UTILS_.parameters` + +You can test if everything worked by calling `make doc_param_internal`. The parameters documentation is written to `OpenMS/doc/doxygen/parameters/output/`. + +### How is the command line documentation for TOPP/UTILS tools created? + +The program `OpenMS/doc/doxygen/parameters/TOPPDocumenter.cpp` creates the command line documentation for all classes that are included in the static `ToolHandler.cpp` tools list. It can be included in the documentation using the following `doxygen` command: + +`@verbinclude TOPP_.cli` + +You can test if everything worked by calling `make doc_param_internal`. The command line documentation is written to `OpenMS/doc/doxygen/parameters/output/`. + +### What are the important files for adding a new tutorial section? + +View the following OpenMS tutorials: + +* `OpenMS/doc/OpenMS_tutorial/refman_overwrite.tex.in` (for PDF tutorials) +* `OpenMS/doc/doxygen/public/OpenMS_Tutorial_html.doxygen~` (for html tutorials) + +For TOPP and TOPPView tutorials, view: + +* `OpenMS/doc/TOPP_tutorial/refman_overwrite.tex.in` (for PDF tutorials) +* `OpenMS/doc/doxygen/public/TOPP_Tutorial_html.doxygen` (for html tutorials) + +## Bug Fixes + +### How to contribute a bug fix? + +To contribute to a bug fix: +1. Submit the bug as a GitHub issue. +2. Create a feature branch (e.g. `feature/fix_missing_filename_issue_615`) from your (up-to-date) develop branch in your fork of OpenMS. +3. Fix the bug and add a test. +4. Create a pull request for your branch. +5. After approval and merge make sure the issue is closed. + +### How can I profile my code? + +Try IBM's profiler, available for all platforms (and free for academic use): Purify(Plus) and/or Quantify. + +Windows: this is directly supported by Visual Studio (Depending on the edition: Team and above). Follow their documentation. + +Linux: + +1. Build OpenMS in debug mode (set `CMAKE_BUILD_TYPE` to `Debug`). +2. Call the executable with valgrind: `valgrind –tool=callgrind`. + > **_NOTE:_** other processes running on the same machine can influence the profiling. Make sure your application gets enough resources (memory, CPU time). + +3. You can start and stop the profiling while the executable is running e.g. to skip initialization steps: +4. Start valgrind with the option `–instr-atstart=no`. +5. Call `callgrind -i [on|off]` to start/stop the profiling. +6. The output can be viewed with `kcachegrind callgrind.out`. + +### (Linux) How do I check my code for memory leaks? + +* Build OpenMS in debug mode (set ``CMAKE_BUILD_TYPE`` to ``Debug``). +* Call the executable with ``valgrind: valgrind --suppressions=OpenMS/tools/valgrind/openms_external.supp –leak-check=full ``. + +Common errors are: + +* ``'Invalid write/read ...'`` - Violation of container boundaries. +* ``'... depends on uninitialized variable'`` - Uninitialized variables: +* ``'... definitely lost'`` - Memory leak that has to be fixed +* ``'... possibly lost'`` - Possible memory leak, so have a look at the code + +For more information see the [`valgrind` documentation](http://valgrind.org/docs/manual/) . diff --git a/openms-wiki/pull-request-checklist.md b/openms-wiki/pull-request-checklist.md new file mode 100644 index 00000000..1cd0bba7 --- /dev/null +++ b/openms-wiki/pull-request-checklist.md @@ -0,0 +1,48 @@ +Pull Request Checklist +====================== + +Before you open a pull request, make sure you check the following: + +* **Does the code build?** + + Execute `make` (or your build system's equivalent, e.g., `cmake --build . --target ALL_BUILD --config Release` on Windows). +* **Do all tests pass?** + + To check if all tests have passed, execute `ctest`. + If a test that is unrelated to your changes fails, check the [nightly builds](http://cdash.openms.de/index.php?project=OpenMS) to see if the error is also in `develop`. If the error is in develop, [create a github issue](write-and-label-github-issues.md). +* **Is the code documented?** + + Document all new classes, including their methods and parameters. + It is also recommended that you also document non-public members and methods. +* **Does the code introduce changes to the API?** + + If the code introduces changes to the API, make sure that the documentation is up-to-date and that the Python bindings (pyOpenMS) still work. For each change in the C++ API, make a change in the Python API wrapper via the `pyOpenMS/pxds/` files. + +* **Have you completed regression testing?** + + Make sure that you include a test in the test suite for: + + * Public methods of a class + * TOPP tools + * Bug fixes + +Make sure to: +* **Rebase before you open a pull request.** + + To include all recent changes, rebase your branch on `develop` before you open a pull request. + + If you pushed your branch to `origin` before rebasing, git will most likely tell you after the rebase that your local branch and the remote branch have diverged. + + If you are sure that the remote branch does not contain any local commits in the rebased version, you can safely push using `git push -f origin ` to enforce overwrite. If not, contact your local git expert on how to get the changes into your local branch. + +* **Capture similar changes in a single commit** + + Each commit should represent one logical unit. Consolidate multiple commits if they belong together or split single commits if they are unrelated. For example, committing code formatting together with a one-line fix makes it very hard to figure out what the fix was and which changes were inconsequential. + +* **Create a pull request for a single feature or bug** + + If you have multiple features or fixes in a pull request, you might get asked to split your request and open multiple pull requests instead. + +* **Describe what you have changed in your pull request.** + + When opening the pull request, give a detailed overview of what has changed and why. Include a clear rationale for the changes and add benchmark data if available. See [this request](https://github.com/bitly/dablooms/pull/19) for an example. diff --git a/openms-wiki/reporting-bugs-and-issues.md b/openms-wiki/reporting-bugs-and-issues.md new file mode 100644 index 00000000..7d1db3b6 --- /dev/null +++ b/openms-wiki/reporting-bugs-and-issues.md @@ -0,0 +1,15 @@ +Reporting Bugs and Issues +========================= + +A list of known issues in the current OpenMS release can be found [here](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/nightly/html/known_dev_bugs.html). Please check if your OpenMS version matches the current version and if the bug has already been reported. + +In order to report a new bug, please use either our [GitHub issues system](write-and-label-github-issues.md) or [contact us](contact-us.md). + +Include the following information in your bug report: +* the command line (i.e. call) including the TOPP tool and the arguments you used, or the steps you followed in a GUI tool (e.g. TOPPView) - e.g. `FeatureFinderCentroided -in myfile.mzML -out myfile.featureXML` +* the output of OpenMS/TOPP (or a screenshot in case of a GUI problem) +* operating system (e.g. "Windows XP 32bit", "Win 7 64bit", "Fedora 8 32bit", "macOS 10.6 64bit") +* OpenMS version (e.g. "OpenMS 1.11.1", "Revision 63082 from the SVN repository") +* OpenMS architecture ("32 bit" or "64 bit") + +Please provide files that we need to reproduce the bug (e.g. `TOPP INI` files, data files — usually mzML) via a download link, via the mailing list or by directly contacting one of the developers. diff --git a/openms-wiki/users-quickstart-guide.md b/openms-wiki/users-quickstart-guide.md new file mode 100644 index 00000000..a57ae4ed --- /dev/null +++ b/openms-wiki/users-quickstart-guide.md @@ -0,0 +1,10 @@ +Users Quick Start Guide +======================= + +Download the current stable version of OpenMS from the [OpenMS download site](http://open-ms.de/downloads) or via the [OpenMS docker containers](). + +Familiarise yourself with OpenMS by reading [Röst et al, OpenMS: a flexible open-source software platform for mass spectrometry data analysis. Nat Methods. 2016](https://www.ncbi.nlm.nih.gov/pubmed/27575624) as well as [Getting Started](http://www.openms.de/getting-started) on openms.de. + +Read the OpenMS documentation (especially for TOPP) available from the [OpenMS documentation site](http://www.openms.de/current_doxygen) or in the folder `doc/index.html` of stable releases. + +Download some example workflows with optimised parameter settings from the [repository](). To understand which tools are available, read the [TOPP Tool documentation page](http://www.openms.de/current_doxygen/html/TOPP_documentation.html) as well as the [UTILS Tool documentation page](http://www.openms.de/current_doxygen/html/UTILS_documentation.html). diff --git a/openms-wiki/write-and-label-github-issues.md b/openms-wiki/write-and-label-github-issues.md new file mode 100644 index 00000000..d788ca76 --- /dev/null +++ b/openms-wiki/write-and-label-github-issues.md @@ -0,0 +1,40 @@ +Write and Label GitHub Issues +============================= + +## Create an Issue + +To create an issue: + +1. Go to the [OpenMS codebase](https://github.com/OpenMS/OpenMS). +2. Click **Issues** in the menu at the top of the screen. +3. Click **New issue**. +4. Provide a title and a description to the issue. +5. Label the issue. +6. Click **Submit new issue**. + +The issue will be listed under **Issues**. + +## Label an Issue + +To label an issue: +1. On the right of the screen, select the cog icon under **Labels**. +2. Choose a label from the list. Normally, an issue can have one or more of the following labels: + * **defect** + + A defect refers to a bug in OpenMS. This is a high priority issue. + + * **enhancement** + + An enhancement refers to a feature idea to enhance the current OpenMS code. This is a medium priority issue. + + * **task** + + A task refers to a single piece of work that a developer can undertake. This is a medium priority issue. + + * **refactoring** + + A refactoring issue refers to a suggestion to streamline the code without changing how the code function. + + * **question** + + A question could trigger to a discussion about tools, parameters and scientific tasks. diff --git a/requirements.txt b/requirements.txt index 888f76a6..5046ed20 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5,6 +5,5 @@ sphinx-copybutton myst-parser sphinx-notfound-page sphinxcontrib-images -linkify-it-py sphinx-inline-tabs -pygments +pygments \ No newline at end of file