DetailView

Tree species classification for 3D4EcoTec by Zoe Schindler and Julian Frey (Chair of Forest Growth and Dendroecology, University of Freiburg)

The project is based on pytorch. Point clouds of single trees are augmented and projected to grids (4 side views, 1 top view, 1 bottom view, 1 detail view 1 - 1.5 m height).
The dataset is available on Zenodo.

The training weights are available on FreiData.

Classification is done with three densenet201 instances, one for the side views, one for the top and bottom views and the last one for the detail view. These classifications are merged and the tree height is also added. After that, two more linear layers with a relu layer in between are used as classifiers.

The final predictions are made by adding the probabilities of 50 augmentations of each test tree and selecting the most probable class.

Any scientific publication using the data should cite the following paper:

Puliti, S., Lines, E., Müllerová, J., Frey, J., Schindler, Z., Straker, A., Allen, M.J., Winiwarter, L., Rehush, N., Hristova, H., Murray, B., Calders, K., Terryn, L., Coops, N., Höfle, B., Krůček, M., Krokm, G., Král, K., Luck, L., Levick, S.R., Missarov, A., Mokroš, M., Owen, H., Stereńczak, K., Pitkänen, T.P., Puletti, N., Saarinen, N., Hopkinson, C., Torresan, C., Tomelleri, E., Weiser, H., Junttila, S., and Astrup, R. (2024) Benchmarking tree species classification from proximally-sensed laser scanning data: introducing the FOR-species20K dataset. ArXiv; available here

The development of the model was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Project FR 4404/1-1, “Making the direct link between light regime and forest biodiversity – a 3D spatially explicit modelling approach based on TLS, CNNs and ray tracing”.

Code Overview

Main Code

1. downsample.py: downsample data & remove point clouds with < 100 points
2. balancing.py: balancing the data set
3. parallel_densenet.py: setting up neural network & data loader
4. training.py: training & validating
5. predict.py: predicting to test data The model weights trained in the FOR-species20K paper can be downloaded here.

Utilities

• augmentation.py: augment point clouds
• read_las.py: read las files to numpy arrays
• sideview.py: create sideviews of point clouds

Results

• loss_202305171452.csv: history of the loss & validation loss
• test_predictions.csv: predicted labels for the test data

How to use the model to perform predictions

Docker is used to run the model in a container. You can use the provided Dockerfile to build a Docker image that contains all necessary dependencies to run the model.

docker build -t detailview .
docker run --gpus all -it --rm -v /path/to/your/data:/data -v /path/to/output/folder:/output detailview --prediction_data /data/your_las_file.las --tree_id_col TreeID --model_path /data/weights

If you dont want to use docker, you can use the model in a lokal environment to perform predictions on your own data. Setup the python environment with the required packages.

conda create -n detailview python=3.12
conda activate detailview
pip3 install numpy pandas scikit-learn laspy matplotlib requests lazrs[all] tqdm
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

You will need a single las/laz file with a column with single tree labels (e.g. TreeID, can be specified with the argument tree_id_col) or one file for every tree and a csv file with the columns filename, species_id, tree_H. filename gives the ralative path to the path_las argument, species_id can be blank or -999, tree_H is the height of the tree in meters. You can then use the predict.py script to perform predictions on your data.

python3 predict.py --prediction_data /path/to/your/las/file \
  --tree_id_col TreeID --output_dir /path/to/output/folder \
  --model_path /path/to/your/model/weights \
  --projection_backend torch --output_type both

The argument projection_backend can be set to "torch" or "numpy". The torch backend is faster but the original publication was done with the numpy version. The output_type can be set to "csv", "las" or "both". Depending on the choice the output will be a csv file with the predictions, a las file with the predictions as an additional column or both. By default the model performs multiple augmentations (10) and averages the predictions. This can be changed with the argument n_aug.