PhD position, Automated Monitoring Using an Underwater Multicamera System for Real-Time Object Detection, Classification and Tracking

Summary

The main objective of this project is to develop, test and implement a real-time multicamera system including computer vision algorithms for the detection, classification and tracking of underwater objects. This work will include both living and nonliving biological entities (e.g. fish, woody debris) as well as non-biological objects (e.g. refuse) in order to automate environmental monitoring in shallow (< 30 m) freshwater, brackish and saltwater environments. Specifically, this project addresses three research questions, where the first two questions address technology and knowledge gaps and the third question is designed to guide and inform future works: RQ1) What multicamera configurations are most suitable for computer vision pipelines considering real-time detection, classification and tracking tasks? RQ2) What are the most effective combinations of hardware and algorithms for enabling real-time freshwater fish counting and coastal pollution monitoring, considering power efficiency, size constraints, and computational performance? RQ3) What are the hardware (high performance desktop vs. embedded), software, environmental, human resource and financial requirements to develop automated real-time monitoring systems based on the best available technologies?

Description

Main supervisor: Jeffrey Andrew Tuhtan

Co-supervisor: Shujaat Khan (King Fahd University of Petroleum and Minerals (KFUPM))

Background on the need for this research

Underwater cameras are used to manually monitor fish biodiversity and migration because they provide non-invasive, continuous, and when water conditions are suitable, highly detailed observations of both individuals and groups. Recent advances in machine learning applied to computer vision systems over the last decade now allow for new and improved possibilities to replace the use of human experts to detect fish, identify their species, size and classify their up- and downstream migration behaviour. In addition to fish biodiversity, there is a growing demand for underwater cameras capable of monitoring of critical infrastructure such as water supply intakes, underwater cables and aqueducts to detect and track of debris, plastic waste and other potentially undesirable foreign objects. The main objective of this work is to develop, test and implement updated camera hardware with real-time computer vision algorithms to address two major gaps which hinder the widespread implementation of automated monitoring systems capable of replacing human experts.

Gap 1: Multicamera systems optimized for underwater computer vision monitoring are unavailable

Currently, the largest gap in the European context is that commercially-available underwater camera systems are designed for human expert assessments of imagery and video. Commercially available systems are typically designed for marine videography applications and thus tend to be costly and over-engineered for shallower riverine and coastal environments. Furthermore, as these systems are designed for human assessments, they typically record imagery using a single camera at low frame rates suitable for humans (e.g. 5-30 fps) and at high image resolution (e.g. 3840 x 2160 px). Although excellent videos can be produced, the technical needs for robust computer vision systems are quite different than those of humans. For example, computer vision systems with multiple lower-resolution, higher speed (e.g. 60 fps) cameras are more suitable for detection and tracking tasks, whereas a single higher resolution camera operating at low speed (e.g. 5 fps) may be best-suited for classification tasks. The lack of multicamera systems optimized specifically for underwater computer vision applications is therefore a substantial bottleneck for researchers, who need to evaluate computational performance and model accuracy, for commercial developers, who would benefit from optimized camera hardware, and for regulators who need robust and maximally reliable computer vision methods which are automated to the greatest extent possible to decrease monitoring costs.

Gap 2: Lack of benchmark datasets and training, testing and validation guidelines for underwater camera systems which use computer vision for underwater monitoring

Although there has been a large body of academic and commercial research on the topic of underwater camera-based monitoring, there are only a few datasets with multiple fish species and non-ideal environmental conditions including low-light, overexposure, biofouling, air bubbles and turbidity. Many small-bodied fish such as dace, goby and gudgeon are missing from openly available datasets, and invasive species such as pumpkinseed or Asian carp are also missing. In the last decade, the presence of plastics and other harmful debris in waterways has grown as a topic of research interest as well as becoming subject to local, regional, national and international regulations. Therefore, new benchmark datasets covering a wider range of European species and non-biological objects are urgently needed, as identifying small endemic fish as well as invasive species and plastics are of high interest to regulators due to their roles and potential negative impacts on local ecosystems. The significance and impact of addressing this gap is expected to be large in the European context, as existing openly available datasets are taken using legacy camera systems developed for human experts, are several years old, and accordingly their frame rates, image resolution and the overall image quality are lower than the multicamera system developed in this project.

 This PhD project aims to bridge these gaps and advance the state-of-the-art by developing, testing, and validating an optimized multicamera system for real-time computer vision applications. This PhD project will have high impact on the European technical landscape, and provide a scientifically-tested and commercially-viable system for the automated object detection, classification and tracking of biological and non-biological objects in shallow river and coastal environments. Specifically, this project will provide the first real-time underwater camera monitoring solution capable of reliably replacing human experts at fish biodiversity monitoring as well as detecting unwanted foreign objects near critical water infrastructure.

Responsibilities and main tasks of this PhD position

• Perform a literature review of existing camera hardware and computer vision methods for fish detection, species classification and size estimation, with a focus on riverine and coastal environments.
• Test and evaluate new and state-of-the-art camera configurations and computer vision models for fish detection, species, size and counting individual fish with different swimming behaviour. Data will be collected at test sites in Europe within the first two years of the research project.
• Prepare and publish an open access journal paper focused on comparing camera designs and computer vision models for underwater object detection, classification and tracking of fish and debris.
• Collaborating with junior and senior researchers to learn how to prepare news articles, press releases and open house events disseminating the research outcomes from this PhD.
• Aid in the preparation of research proposals related to automated camera-based underwater monitoring for fish and critical water infrastructure.
• Participate in workshops, conferences and public events as a representative of the Tallinn University of Technology to share, improve and grow your knowledge on the topic of computer vision methods for underwater camera-based monitoring.

Applicants should fulfil the following requirements:

• a master’s degree in computer science, with an emphasis on computer vision
• a clear and related interest in the topic of the position based on their previous experience
• excellent command of English
• strong and demonstrable software development, writing and analytical skills
• capacity to work both as an independent researcher and as part of an international team
• capacity and willingness to aid in organizational tasks relevant to the project
• be able to travel within Europe for up to two weeks per year for collaboration and training events

The following experience is beneficial:

• Programming in Python
• Experience in training, testing and validation of machine learning models
• Solid foundation in frequentist statistics
• Working knowledge of video and image annotation processes for classification
• Demonstrated skill in convolutional neural networks applied to image classification tasks
• Code documentation and the use of shared repositories
• Basic understanding of aquatic ecosystems and/or hydraulics in rivers and coastal regions

To be considered for evaluation, a candidate must submit a two-page research plan for the topic, including the overall research strategy. Here it is strongly encouraged to elaborate on their choices of suitable computer vision methods, based on those available in the literature. The format of the research plan is up to the candidate, but should include references to relevant books, journal and conference publications to highlight the candidate’s ability to independently source references.

We offer:

• 4-year PhD position in the leading environmental sensing research center in Estonia with a large portfolio of ongoing pan-European, regional, national and local projects.
• The chance to do high-level research in a leading international group on underwater sensing
• Opportunities for conference, research stays and networking with globally leading universities and researchers in the fields of computer vision, underwater sensing and critical infrastructure monitoring.

About the research group

The Centre for Environmental Sensing and Intelligence is an internationally-recognized and highly interdisciplinary research group at the Tallinn University of Technology focusing on environmentally relevant and future-oriented research and teaching topics:

• Data-driven modelling of large-scale environmental sensing networks
• Computer vision applications for fish monitoring in freshwater environments
• Development of rugged and robust underwater sensors for extreme physical environments
• Human kinematic measurement systems for underwater and microgravity environments
• Environmental technology innovation with small to medium enterprises
• Teaching large-scale environmental sensor development (MSc) and academic writing (PhD)

Additional information

For further information, please contact:

Assoc. Prof. Jeffrey A. Tuhtan

Email: [email protected]

To get more information or to apply online, visit https://taltech.glowbase.com/positions/954 or scan the code on the left with your smartphone.