Abstract
In an endeavor to study natural systems at multiple spatial and taxonomic resolutions, there is an urgent need for automated, high-throughput frameworks that can handle plethora of information. The coalescence of remote-sensing, computer-vision, and deep-learning elicits a new era in ecological research. However, in complex systems, such as marine-benthic habitats, key ecological processes still remain enigmatic due to the lack of cross-scale automated approaches (mms to kms) for community structure analysis. We address this gap by working towards scalable and comprehensive photogrammetric surveys, tackling the profound challenges of full semantic segmentation and 3D grid definition. Full semantic segmentation (where every pixel is classified) is extremely labour-intensive and difficult to achieve using manual labeling. We propose using label-augmentation, i.e., propagation of sparse manual labels, to accelerate the task of full segmentation of photomosaics. Photomosaics are synthetic images generated from a projected point-of-view of a 3D model. In the lack of navigation sensors (e.g., a diver-held camera), it is difficult to repeatably determine the slope-angle of a 3D map. We show this is especially important in complex topographical settings, prevalent in coral-reefs. Specifically, we evaluate our approach on benthic habitats, in three different environments in the challenging underwater domain. Our approach for label-augmentation shows human-level accuracy in full segmentation of photomosaics using labeling as sparse as 0.1%, evaluated on several ecological measures. Moreover, we found that grid definition using a leveler improves the consistency in community-metrics obtained due to occlusions and topology (angle and distance between objects), and that we were able to standardise the 3D transformation with two percent error in size measurements. By significantly easing the annotation process for full segmentation and standardizing the 3D grid definition we present a semantic mapping methodology enabling change-detection, which is practical, swift, and cost-effective. Our workflow enables repeatable surveys without permanent markers and specialized mapping gear, useful for research and monitoring, and our code is available online. Additionally, we release the Benthos data-set, fully manually labeled photomosaics from three oceanic environments with over 4500 segmented objects useful for research in computer-vision and marine ecology.
Original language | English |
---|---|
Article number | 659 |
Pages (from-to) | 1-19 |
Number of pages | 19 |
Journal | Remote Sensing |
Volume | 13 |
Issue number | 4 |
DOIs | |
State | Published - 2 Feb 2021 |
Bibliographical note
Publisher Copyright:© 2021 by the authors. Licensee MDPI, Basel, Switzerland.
Funding
Funding: TT was supported by the The Leona M. and Harry B. Helmsley Charitable Trust, The Maurice Hatter Foundation, the Israel Ministry of National Infrastructures, Energy and Water Resources Grant 218-17-008, the Israel Ministry of Science, Technology and Space grant 3-12487, and the Technion Ollendorff Minerva Center for Vision and Image Sciences. IA and ACM were supported by project PGC2018-098817-A-I00 MCIU/AEI/FEDER, UE. MY was supported by the PADI Foundation (application #32618), the Murray Foundation for student research, ASSEMBLE+ European Horizon 2020 (transnational access #216), and Microsoft AI for Earth; AI for Coral Reef Mapping. YL was funded by the Israel Science Foundation (ISF) grant No. 1191/16. GE was supported by the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement #796025.
Funders | Funder number |
---|---|
Microsoft AI | |
Murray Foundation | 216 |
PADI Foundation | 32618 |
Leona M. and Harry B. Helmsley Charitable Trust | |
Horizon 2020 Framework Programme | 796025 |
Ministry of Science, Technology and Space | 3-12487 |
Israel Science Foundation | 1191/16 |
Ministry of National Infrastructure, Energy and Water Resources | 218-17-008 |
Ollendorff Minerva Center for Vision and Image Sciences | PGC2018-098817-A-I00 MCIU/AEI/FEDER |
Keywords
- Benthic mapping
- Change-detection
- Community ecology
- Computer-vision
- Coral-reefs
- Label-augmentation
- Multi-level superpixels
- Orthorectification
- Photogrammetry