Urchin Population across Seasons at Campbell River
Catherine Tang
Scientific Context
Sea urchins are important in coastal ecosystems and are impacting the kelp forest structure. Seasonal changes in temperature, and other factors might affect their abundance and activity.
However, collecting consistent observations in underwater environments is difficult due to depth, visibility, and accessibility constraints. Fixed underwater cameras are able to provide continuous monitoring, allowing us to conduct quantitative comparison across seasons.
A computer vision model enables continuous, non-invasive observation across different times of day and seasons, providing insights into how environmental conditions influence urchin presence. This study tested whether sea urchin encounter rates differ between summer and winter at the Campbell River Station by a trained Computer Vision model, which made the data collecting process autonomous and controllable.
Dataset Description
The dataset consists of underwater images collected from fixed camera systems at the Campbell River Station, monitored by Ocean Networks Canada. Images are selected at two time points per day (noon and night) across multiple months (August and December), allowing analysis of both seasonal and diurnal variation.
Each image is annotated with bounding boxes representing individual sea urchins, where each annotation corresponds to one detected organism. The dataset includes metadata extracted from filenames, including location, month, day of a month, and time of a day.
The dataset contains one primary class: Urchin.
The number of annotations per image provides a direct estimate of urchin population density. There are ~100 images researched, with ~2800 annotations in total. Images vary in lighting, clarity, and background complexity (sediment, rocks, etc.), reflecting realistic underwater conditions.
To deal with low-quality data, we only used images where sea urchins were clearly visible. Images where the view was blocked or unclear were not included.
For example, the image below is considered as low-quality data and has been ignored:
Model Selection
I selected a YOLO-based instance segmentation model for this project.
Since sea urchin spines often overlap, this model goes beyond traditional object detection. In addition to identifying the location of sea urchins with bounding boxes, it also generates a pixel-level mask for each individual organism.
This choice is important for underwater scenes because sea urchins often appear in dense clusters and irregular shapes, making simple detection insufficient for accurate detecting. Instance segmentation allows the model to distinguish overlaps and closely packed individuals, which improves counting accuracy compared to other standard detection methods.
YOLO-based models are also well-suited for this project since they have efficient and accurate performance on real world images, even though the environment has various lighting and color conditions. This is particularly important for underwater environments, where imagery is more likely to be blurry, containing high-level noise, having low contrast levels, and so on.
At the same time, to make the model more robust, we used augmentations like slight shear, brightness variation, and blur to simulate different underwater environments.
The model is trained using annotated images where each sea urchin is labeled as a separate instance. During inference, the model predicts segmentation masks for each detected urchin, and the total number of predicted instances is used to estimate population counts. The model training process started with 2 annotated images, and the dataset was gradually expanded to 12, 22, 31, 62, and ~100 images. Along with the increasing size of the dataset, the model behavior was also boosted each time.
To improve generalization, the model is trained on images from multiple conditions (August vs December, noon vs night), allowing it to learn features that are robust to seasonal and lighting variations.
Model Assessment
Curve Assessment
Model performance is assessed using metrics appropriate for instance segmentation, including F1–Confidence Curve, Precision–Confidence Curve, Precision–Recall Curve, Recall–Confidence Curve, Confusion Matrix (Normalized).
The F1 score increases rapidly at low confidence thresholds and peaks around a confidence value of approximately ~0.35,
indicating a balance between precision and recall. After that, F1 decreases while higher confidence thresholds reduce recall,
showing that filtering too strictly has harmed the overall performance of this model.
The precision–recall curve maintains high precision across a wide range of recall values,
with an mAP @ 0.5 of approximately 0.91, indicating that the overall detection performance was strong.
The sharp drop at very high recall suggests the model tended to include more false positives when trying to detect more instances.
Precision steadily increases as the confidence threshold rises, approaching near-perfect values at high confidence levels.
This showed that higher-confidence predictions are highly reliable, although they might be missing some true detections.
Recall value reached the highest level at low confidence thresholds and decreases as it increases,
showing that stricter confidence filtering caused the model to miss more true detections.
This showed that there is a tradeoff between capturing all instances and maintaining prediction certainty.
From the normalized confusion matrix, we can see that the model correctly identifies approximately 86% of true urchins,
indicating strong detection performance for the target class.
However, about 14% of the urchins were misclassified as background, showing that it might be hard to detect urchins accurately at conditions such as blur or low contrast.
Visual Assessment
Below are the two images as examples showing the model performance during December and August:
By visualling analyzing the accuracy of the labels, it can be easily seen that during Summer, the plants underwater were mislabelled as urchins.
Although plants and urchins appear visually distinct to humans, the model may be relying more on local textures, shapes, or lighting patterns rather than overall color.
These errors highlighted a limitation in the model’s ability to distinguish between objects in more complex environments.
Model Comparisons
Since I trained six versions of the model, I was able to observe some subtle differences in their performance.
Here is an image showing the change of performance:
Due to the small-sized dataset in the first version, the model was first trained to have a relatively inaccurate performance compared to later versions.
While the model performance kept being improved when more imagery data was involved in the training, I noticed that version 4-6 appeared to be relatively worse
than the 3rd version.
This might be indicating that with additional data and training, the model may have started to fit too closely to the training set,
reducing its ability to generalize well to new images, which resulted in model overfitting.
Model Use Case
This model can be used to study how sea urchin abundance changes across seasons and times of day using fixed underwater camera imagery, and performs the best at the Campbell River Underwater Camera Station. It can answer the research question of whether sea urchin encounter rates differ between summer and winter at the Campbell River Station.
Hypothesis: Sea urchin encounter rates will vary seasonally due to environmental differences between the periods.
To answer the research question, this model provided reliable urchin population data since instance segmentation can separate individual urchins in dense benthic scenes, making it possible to generate counts from complex underwater images. The curves in the model assessment section also suggested a somewhat accurate performance of the model.
Using labels produced by this model and some further data processing, the following data visualizations can be made:
From these two plots, we were able to see the population distribution of urchins during noons and nights in December and August.
And I made two other graphs showing the average urchin population comparison between August and December and average urchin population by month and time:
From these two plots, we can see that:
Urchin populations are significantly higher in December compared to August across both time periods, indicating strong seasonal variation and suggesting that urchins are more active in colder conditions. Additionally, the difference between noon and night reverses across seasons: December shows higher counts at night while August shows slightly higher counts at noon, implying that both environmental factors and time of day influence observed urchin presence.
However, there were also limitations to this study. As mentioned in previous sections, there are still uncertainties in this model that might have produced error in detecting the urchins, which means it was more likely for an overcounting to exist in the number computation of objects during summer. Therefore, it remains unknown if urchins react to different times in a day.
Future Improvements
According to the behavior of the model and the data it has produced, this model can still be expanded to multiple sites. After training the model with a variety of imagery data from multiple sites, this model can be applied to more sites and species of urchins, which also improves the accuracy of its detections and predictions.
Additionally, more classes can also be added to the model, so that it can distinguish special classes that it might get confused with (e.g. rock, sediments).