wigum::website - Deep Learning

SilvaScenes

2025-10-13T00:00:00+00:00

Co-authored paper: SilvaScenes: Tree Segmentation and Species Classification from Under-Canopy Images in Natural Forests

Interest in robotics for forest management is growing, but perception in complex, natural environments remains a significant hurdle. Conditions such as heavy occlusion, variable lighting, and dense vegetation pose challenges to automated systems, which are essential for precision forestry, biodiversity monitoring, and the automation of forestry equipment. These tasks rely on advanced perceptual capabilities, such as detection and fine-grained species classification of individual trees. Yet, existing datasets are inadequate to develop such perception systems, as they often focus on urban settings or a limited number of species. To address this, we present SilvaScenes, a new dataset for instance segmentation of tree species from under-canopy images. Collected across five bioclimatic domains in Quebec, Canada, SilvaScenes features 1476 trees from 24 species with annotations from forestry experts. We demonstrate the relevance and challenging nature of our dataset by benchmarking modern deep learning approaches for instance segmentation. Our results show that, while tree segmentation is easy, with a top mean average precision (mAP) of 67.65%, species classification remains a significant challenge with an mAP of only 35.69%. Our dataset and source code will be available at this https URL.

Using Citizen Science Data for UAV Image Analysis

2025-04-09T00:00:00+00:00

Co-authored paper in MDPI: Using Citizen Science Data as Pre-Training for Semantic Segmentation of High-Resolution UAV Images for Natural Forests Post-Disturbance Assessment

The ability to monitor forest areas after disturbances is key to ensure their regrowth. Problematic situations that are detected can then be addressed with targeted regeneration efforts. However, achieving this with automated photo interpretation is problematic, as training such systems requires large amounts of labeled data. To this effect, we leverage citizen science data (iNaturalist) to alleviate this issue. More precisely, we seek to generate pre-training data from a classifier trained on selected exemplars. This is accomplished by using a moving-window approach on carefully gathered low-altitude images with an Unmanned Aerial Vehicle (UAV), WilDReF-Q (Wild Drone Regrowth Forest—Quebec) dataset, to generate high-quality pseudo-labels. To generate accurate pseudo-labels, the predictions of our classifier for each window are integrated using a majority voting approach. Our results indicate that pre-training a semantic segmentation network on over 140,000 auto-labeled images yields an $F1$ score of 43.74% over 24 different classes, on a separate ground truth dataset. In comparison, using only labeled images yields a score of 32.45%, while fine-tuning the pre-trained network only yields marginal improvements (46.76%). Importantly, we demonstrate that our approach is able to benefit from more unlabeled images, opening the door for learning at scale. We also optimized the hyperparameters for pseudo-labeling, including the number of predictions assigned to each pixel in the majority voting process. Overall, this demonstrates that an auto-labeling approach can greatly reduce the development cost of plant identification in regeneration regions, based on UAV imagery.

Terrain Classification for Boreal Forests

2024-09-10T00:00:00+00:00

Co-authored paper at IROS 2024: Proprioception Is All You Need: Terrain Classification for Boreal Forests

Recent works in field robotics highlighted the importance of resiliency against different types of terrains. Boreal forests, in particular, are home to many mobility-impeding terrains that should be considered for off-road autonomous navigation. Also, being one of the largest land biomes on Earth, boreal forests are an area where autonomous vehicles are expected to become increasingly common. In this paper, we address the issue of classifying boreal terrains by introducing BorealTC, a publicly available dataset for proprioceptive-based terrain classification (TC). Recorded with a Husky A200, our dataset contains 116 min of Inertial Measurement Unit (IMU), motor current, and wheel odometry data, focusing on typical boreal forest terrains, notably snow, ice, and silty loam. Combining our dataset with another dataset from the literature, we evaluate both a Convolutional Neural Network (CNN) and the novel state space model (SSM)-based Mamba architecture on a TC task. We show that while CNN outperforms Mamba on each separate dataset, Mamba achieves greater accuracy when trained on a combination of both. In addition, we demonstrate that Mamba’s learning capacity is greater than a CNN for increasing amounts of data. We show that the combination of two TC datasets yields a latent space that can be interpreted with the properties of the terrains. We also discuss the implications of merging datasets on classification. Our source code and dataset are publicly available online.

Benchmarking of Real-Time Object Detection Models

2024-06-05T00:00:00+00:00

Published a paper to ArXiv: Replication Study and Benchmarking of Real-Time Object Detection Models.

This work examines the reproducibility and benchmarking of state-of-the-art real-time object detection models. As object detection models are often used in real-world contexts, such as robotics, where inference time is paramount, simply measuring models’ accuracy is not enough to compare them. We thus compare a large variety of object detection models’ accuracy and inference speed on multiple graphics cards. In addition to this large benchmarking attempt, we also reproduce the following models from scratch using PyTorch on the MS COCO 2017 dataset: DETR, RTMDet, ViTDet and YOLOv7. More importantly, we propose a unified training and evaluation pipeline, based on MMDetection’s features, to better compare models. Our implementation of DETR and ViTDet could not achieve accuracy or speed performances comparable to what is declared in the original papers. On the other hand, reproduced RTMDet and YOLOv7 could match such performances. Studied papers are also found to be generally lacking for reproducibility purposes. As for MMDetection pretrained models, speed performances are severely reduced with limited computing resources (larger, more accurate models even more so). Moreover, results exhibit a strong trade-off between accuracy and speed, prevailed by anchor-free models - notably RTMDet or YOLOx models. The code used is this paper and all the experiments is available in the repository at this https URL.

RandAugment

2024-04-03T00:00:00+00:00

Presentation made for the course GIF-7010: Avancées en apprentissage automatique at Université Laval on RandAugment.

Here are the slides:

Segment Anything

2024-02-21T00:00:00+00:00

Presentation made for the course GIF-7010: Avancées en apprentissage automatique at Université Laval on Segment Anything.

Here are the slides:

MaskBEV

2023-07-11T00:00:00+00:00

Accepted paper at IROS 2023: MaskBEV: Joint Object Detection and Footprint Completion for Bird’s-eye View 3D Point Clouds

Recent works in object detection in LiDAR point clouds mostly focus on predicting bounding boxes around objects. This prediction is commonly achieved using anchor-based or anchor-free detectors that predict bounding boxes, requiring significant explicit prior knowledge about the objects to work properly. To remedy these limitations, we propose MaskBEV, a bird’s-eye view (BEV) mask-based object detector neural architecture. MaskBEV predicts a set of BEV instance masks that represent the footprints of detected objects. Moreover, our approach allows object detection and footprint completion in a single pass. MaskBEV also reformulates the detection problem purely in terms of classification, doing away with regression usually done to predict bounding boxes. We evaluate the performance of MaskBEV on both SemanticKITTI and KITTI datasets while analyzing the architecture advantages and limitations.

Learn Transformers

2023-03-23T00:00:00+00:00

Here’s a small library I made to explain the Transformer architecture in PyTorch: willGuimont/transformers. The code is heavily commented and should be easy to follow.

This repository also contains some popular transformer-based models, such as Vision Transformer.

This was made while developping new practical exercises for GLO-7030 at Université Laval.

torch_waymo

2023-02-02T00:00:00+00:00

Here’s a small library I made to use the Waymo Open Dataset in PyTorch, without any dependency to Tensorflow: willGuimont/torch_waymo.

Here’s the corresponding PyPI package: torch-waymo.

Apprentissage profonds sur les nuages de points 3D

2022-09-12T00:00:00+00:00

Papier réalisé pour le cours IFT-6001 @ Université Laval sur l’apprentissage par réseaux de neurones profonds sur les nuages de points 3D: Apprentissage par réseaux de neurones profonds sur les nuages de points 3D

Point-BERT IFT-6001

2022-04-28T00:00:00+00:00

Voici une présentation que j’ai donnée le 28 avril 2022 pour le cours IFT-6001 Introduction à la recherche en informatique : communication et méthodologie à l’Université Laval sur Point-BERT.

Voici les diapositives: Point-BERT

Voici l’enregistrement de la présentation:

Compute Canada Quickstart

2022-01-28T00:00:00+00:00

Here’s a small guide on how to use Compute Canada clusters to train deep learning models.

In this guide, we’ll do the necessary setup to get you up and running on a cluster. This guide is written to work with Narval, but it should be easy to adapt it to other clusters by changing the URLs. You can see available clusters at this URL.

We recommend Narval for deep learning as of now (2021-2022) because it has 4 NVIDIA A100 (40 Gb memory) per node.

This guide was written by William Guimont-Martin, 2021

Also available on Norlab Wiki.

Account creation

First of all, you’ll need a Compute Canada account to access the clusters. To do so, please see Tips for new students. You’ll see the URL to apply for an account and Philippe Giguère’s sponsor code.

You’ll need your username and your password for the following steps.

SSH and `ssh-copy-id`

To access Compute Canada’s clusters, you’ll need to ssh into them. You can get the login node URL by checking the cluster’s wiki page. For Narval, the URL is narval.computecanada.ca.

To connect to the cluster:

# replace <username> with your actual Compute Canada username
ssh <username>@narval.computecanada.ca

You’ll be asked to enter your password. If everything goes right, you’ll be greeted by the cluster’s welcome message.

To make things easier later, we recommend to set up an SSH key to connect to the cluster.

To do so, you’ll need to generate an SSH key if you don’t already have one you’d like to use.

Please see Generating a new SSH key and adding it to the ssh-agent from GitHub to generate a new SSH key.

Once you have added your SSH key and added it to the ssh-agent, copy your key to the cluster with

# replace <username> with your actual Compute Canada username
ssh-copy-id <username>@narval.computecanada.ca

If everything went right, you should now be able to ssh into your cluster without being asked your password.

Clone your code

Now that you are logged in the cluster, you should have a bash shell. From that shell, you can use most Linux commands you’re used to like ls, cd, etc..

We’ll now clone your project on the cluster.

From the cluster, you can create another SSH key and add it to GitHub.

This will allow you to git clone your repository into your home directory.

Setup your Python venv

It is recommended to create your Python venv from inside a SLURM job, if your installation script is quite complex, it will be easier to set up before starting any job.

To create a Python venv:

load python/3.8.10  # loads python, you can change the version, you can see available versions using "module avail python"
python -m venv venv  # creates a virtual environment in the folder ./venv/

Once created, you can activate it using:

source venv/bin/activate

You should now see (venv) at the beginning of your shell prompt.

You can now pip install all of your dependencies.

Please note that when installing mmdetection and its dependencies, you might need to also run:

pip install -U numpy
pip install -U scikit-learn

If you want to create your venv from inside a job, note the commands you did to setup your environment and copy them into your job script.

Putting your data on the cluster

To put your data on the cluster, we recommend using sftp.

First of all, take the time to familiarize yourself with the different types of storage available on the clusters.

From that, you’ll have to upload your dataset in ~/projects/def-philg/<username>. Note that this space is shared across the people in the lab. Please keep it clean!

From you local computer:

cd path/to/your/dataset/on/your/computer
# replace <username> with your actual Compute Canada username
sftp <username>@narval.computecanada.ca
sftp> cd ~/projects/def-philg/<username>
# use "put file" to upload a file, use "get file" to download a file
# you can use * to wildcard select multiple files

If you are using mmdetection, you might want to add a symlink to your data in code/data. To do so, run the following commands:

cd to/your/code/folder  # TODO change this path
mkdir data
cd data
ln -s path/to/your/dataset <dataset-name>  # TODO change path and dataset name

SLURM jobs

We can now create jobs. To do so, create a .sh file and paste the following inside:

#!/bin/bash
#SBATCH --gres=gpu:4
#SBATCH --cpus-per-task=8
#SBATCH --mem=40000M
#SBATCH --time=1-00:00
#SBATCH --job-name=01_baseline_pointpillars
#SBATCH --output=%x-%j.out

# Variables
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
export MAKEFLAGS="-j$(nproc)"

# Modules
module load cuda/11.0
module load python/3.8.10
module load qt/5.12.8
module load geos
module load llvm/8.0.1

# Setup Python
source ~/venv/bin/activate

# Start task
# TODO change that part to how you run your python script, this example shows how to do multi GPU training using mmdetection3d
cd to/your/code/folder  # TODO change this path
CONFIG_FILE=configs/01_baseline_pointpillars.py
WORK_DIR=path/to/work/dir  # TODO change this path too
GPUS=4
./dist_train.sh ${CONFIG_FILE} ${GPUS} --work-dir ${WORK_DIR} --cfg-options cudnn_benchmark=True evaluation.gpu_collect=True  # gpu_collect=True is essential to do multi GPU training

# Cleaning up
deactivate

This script is organized in five main parts.

Configure job

This part is used to configure the resources needed for your job.

#!/bin/bash
#SBATCH --gres=gpu:4                         # Ask for 4 GPU
#SBATCH --cpus-per-task=8                    # Ask for 8 CPU
#SBATCH --mem=40000M                         # Ask for 40 000M of RAM
#SBATCH --time=1-00:00                       # Run for DD-HH:MM
#SBATCH --job-name=01_baseline_pointpillars  # Job name
#SBATCH --output=%x-%j.out                   # Log will be written to f'{job_name}_{job_id}'

Setup installation

This part sets environment variables for multithreading and loads needed modules. You might need to change the loaded versions.

# Variables
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
export MAKEFLAGS="-j$(nproc)"

# Modules
module load cuda/11.0
module load python/3.8.10
module load qt/5.12.8
module load geos
module load llvm/8.0.1

Loading the venv

This will load the venv we created earlier

# Setup Python
source ~/venv/bin/activate

Running your actual code

You will need to change this part to run what you want to run. This scripts shows how to use multi-GPU with mmdetection3d.

# Start task
# TODO change that part to how you run your python script, this example shows how to do multi GPU training using mmdetection3d
cd to/your/code/folder  # TODO change this path
CONFIG_FILE=configs/01_baseline_pointpillars.py
WORK_DIR=path/to/work/dir  # TODO change this path too
GPUS=4
./dist_train.sh ${CONFIG_FILE} ${GPUS} --work-dir ${WORK_DIR} --cfg-options cudnn_benchmark=True evaluation.gpu_collect=True  # gpu_collect=True is essential to do multi GPU training

Cleaning up

Not strictly needed, but it is always a good thing to clean up after yourself.

# Cleaning up
deactivate

Run the job

To run a job, you first have to queue it to the job scheduler. To do so, run the following command:

sbatch script.sh

You can then verify the job is queued by running:

sq

More information is available here

Useful tools

sbatch: queue a job.

sq: view your queued jobs

scancel <id>: cancel job with id

salloc --account=def-philg --gres=gpu:2 --cpus-per-task=4 --mem=32000M --time=5:00:00: start an interactive job, which will allow you to test your scripts before queuing jobs

sftp: useful tool to transfer data from and to the cluster

diskusage_report: see used disk space

Transformers in Computer Vision

2021-09-02T00:00:00+00:00

On September 2nd 2021, I gave a talk on transformer-based neural network architectures for computer vision. I gave the in University Laval’s Norlab.

Here are the slides: Transformers in Computer Vision

And here is the recording of the event:

TossingBot

2020-04-29T00:00:00+00:00

This video presents TossingBot, a robotic arm that learns to throw arbitrary objects with residual physics.

This video was made for the class GLO-7030: Deep Learning at Université Laval.

On Learning

2020-01-23T00:00:00+00:00

This is just some rambles on learning…

I’m currently taking a class on deep neural networks. In the first class, we discussed multiple ways to have artificial intelligence systems.

One way would be to code hand-designed programs to accomplish the task. If something happens, then the system does that, else if this other thing happens, then do this, else if… We can achieve great things using rule-based programs, but on more complex tasks we would need a lot of cases to handle a lot of different cases.

Another way would be to make the computer learn from the data. So we analyze the data, find features for the system to learn from, then use classical machine learning algorithms such as SVM. Again, for complex tasks, we may miss some useful feature to correctly learn from the data.

The next step is to make the algorithm find features by itself. This is what neural networks do. Each layer of a deep neural network extract features for the higher layers to “reason” about. The classical example is a neural network that detects faces. The first layers might detect very simple patterns like edges. Higher up, a layer might detect corners and contours. Layers at the end of the network might detect complex object parts used by the final layer to classify the object. This is what is called representation learning, the artificial intelligence system finds a way of representing the data so that it is easy to solve the task.^[1].

A good way of visualizing the different takes on artificial intelligence is from the book Deep Learning^[1].

Dreyfus

When I learned about representation learning, it got me thinking about how we learn and the Dreyfus model^[2].

I think we can draw links between the Novice stage and rule-based systems. In this first stage of the Dreyfus model, the learner relies on rules to determine the action.

Perhaps, there are other links between machine learning and the Dreyfus model.

Mental representations

In The Sciences of the artificial^[3] from Herbert A. Simon, the author discusses the experience of A. de Groot and al. on chess perception. They showed chessboard with pieces in positions of real games for 5 seconds, then removed the pieces. Then they asked the subjects to put the pieces back in their position.

As you might guess, somebody not accustomed to chess would have a pretty hard time to reconstruct the positions. For grandmasters and masters though, they were able to reconstruct the positions with almost no error!

They then retried the experiment with random piece positions. In that case, even grandmasters could not reconstruct the board.

This shows that a grandmaster or a master built a good mental representation of the game. Instead of memorizing all pieces, they memorized relations between pieces, which allow them to memorize them more easily.

So, when I learned about representation learning, this reminded me of that particular example. Under the hood, a neural network tries to do exactly what the grandmaster did, build a good representation of the problem to solve. Once the representation is built, a simple linear layer can solve the problem!

I think this is what is really going on when humans learn. All the studying isn’t about memorizing facts or formulas, it’s about building a good mental representation that can be used to solve many problems. Once you understand basic algebra, you can use that framework to solve many problems you weren’t trained on.

In a neural network, we are usually not concerned about memorizing all the input data, instead, we want to generalize to unseen examples.

Perhaps we should tell students that they aren’t in school to memorize facts, but instead to build a toolbox of mental representations to help them in life. Our brain is a neural network that needs to be trained with great care!

Sources

Goodfellow, Ian & Bengio, Yoshua & Courville, Aaron. Deep Learning. 2016. ↩ ↩2
Dreyfus, Stuart E. The Five-Stage Model of Adult Skill Acquisition. 2004. ↩
Simon, Herbert A. The Sciences of the Artificial, 1969. ↩