A photo that might be a typical one taken by tourists viewing penguin colonies at Devil Island, Antarctica.
Lead investigator Heather Lynch, PhD, the Institute for Advanced Computational Science (IACS) Endowed Professor of Ecology & Evolution at Stony Brook University, has studied penguins in Antarctica for years. The changing patterns of penguin abundance has become an important gauge with which scientists can track how climate change and other threats are impacting the Antarctic environment. The Lynch Lab for Quantitative Ecology is dedicated to advancing this work.
“There are far more tourists in Antarctica than scientists, and virtually everybody has a camera in their pocket and many take photos of penguins,” says Lynch. “With many thousands of photos of Antarctica potentially available on the web, the question is how can we use the data collected by all those photos to track penguin populations? Our challenge was to figure out how to extract information about precisely where the penguins were in a photograph even if no additional information was available.”
The research took data from satellite imagery of Antarctica and employed 3D computerization methods to identify where the camera was when it took the photo in question. This enables the researchers to estimate both the location and orientation or direction a camera is facing during a photo. Then data from this process are combined with a separate tool to delineate the boundary of a penguin colony in the photograph. By doing this, the investigators could then figure out precisely where the colony boundary was when the photo was taken.
Lynch explains that the first problem for the team was identifying the edges of the colony in the photograph. This task, called segmentation in computer science, is challenging for penguin colonies because they often have gradual boundaries with penguins nesting at lower densities near the edges. For this paper, the team had success with a new AI model called the Segment Anything Model, in which they were able to successfully automate the delineation of the penguin colony boundaries in the photographs they were using.
The second problem, called georeferencing, requires identifying where the penguins in the photograph were located in terms of their actual geographic coordinates. This process is particularly challenging in Antarctica because the continent lacks the sharp edges or static features (such as buildings) common to more human-dominated environments that are often used to identify how multiple images of a scene match up to one another – a requirement for a process called structure-from-motion that computer scientists use to create a 3D model from a series of 2D images.
The researchers created a 3D model of the islands occupied by penguins by draping a satellite image over a digital elevation model of the area. Then, they were able to determine the camera location as it took the picture in that 3D representation. Finally, the researchers took the boundary supplied by the Segment Anything model and draped it on the 3D model to extract the exact location of the penguin colony’s boundary.
“In theory, this information gathered by the computational technique can be compared to other similarly processed images of the Antarctic to see how penguin colonies are changing over time,” says Lynch.
While satellite images and aerial photos (via planes and drones) are routinely used to track changes on the Antarctic landscape over time, there are many applications in which neither type of data is readily available for analysis, the authors point out. Therefore, identifying the location of tourist photos of penguins could greatly expand the volume of data available for long-term environmental monitoring.
The researchers report that the technique demonstrated promising performance, yet challenges persist due to variations in image quality and the dynamic aspects of the natural Antarctic landscapes. They believe the method to be “a straightforward and effective tool for the georegistration of ad-hoc photos in natural landscapes, with additional applications such as monitoring glacial retreat.”
This research involves interdisciplinary collaboration involving ecologists, computer scientists, mathematicians and geologists. The Lynch Lab worked closely with Dimitris Samaras’ lab in the Department of Computer Science at Stony Brook University and with the IACS to develop the technique.
###
RSS