Earth Observation technologies are becoming increasingly valuable for understanding our changing world. They are also critical for formulating policies to protect ecosystems and support local communities. The availability of imagery taken from satellites, airplanes, and drones offers opportunities to detect deforestation, glacier melting, crop productivity, hurricane alerts, ocean temperatures, and much more. These applications have become wide-spread and are commonly featured in the news and on social media; an informed public can now be acutely aware of the significant effects that humans have on the landscape.
However, other processes of land use change are too complex to be detected and understood solely from space. For example, detecting rich minerals, tracking illicit land activities (e.g., coca farming), and monitoring water salinity. For these and other complex land changes, the imagery itself might not be enough to understand the processes shaping these changes in the landscape. New ways of addressing these are needed.
About 26 years ago, Jacqueline Geoghegan and her co authors wrote a chapter in the book People and Pixels: Linking Remote Sensing and Social Science, titled “Socializing the Pixel and Pixelizing the Social in Land-Use and Land-Cover Change.” This chapter was a pioneering step forward in extracting more information from images and linking them with the social processes behind them. It helped me during my PhD studies and subsequent research to develop a strategy for connecting quantitative data from pixels with social mechanisms that impact land change. But how do these two approaches work together?
The full Interdialogue with Paulo Murillo.
The role of classification in traditional land use science relies on how effectively we can classify elements on the land surface. Classification is the most relevant task in remote sensing. We want to classify or differentiate between forests and agricultural fields or among diverse types of crops using images taken from satellites or drones. Classification is the core of remote sensing at different scales, and scientists are constantly creating better algorithms to improve classification.
In the context of detecting broad patterns of land change on our planet, classification at national or global scales is important for creating policies to safeguard the environment and optimize resources. For example, near real-time deforestation alerts can inform local agencies of land-clearing activities in order to enact quick control actions against illegal actors on the ground to safeguard pristine forests. Large scale agriculture, as another example, might enhance yield given a better understanding of the vegetative conditions of crops using aerial or satellite imagery. Governments can therefore make informed decisions about the natural capital their countries possess through remote sensing data and classification.
Currently, the way digital image processing is applied focuses on the direct classification of phenomena over large regions using well-known training data, something we land use scientists call “Pixelizing the Social.”
Pixelizing the Social
In the Pixelizing the Social approach, we collect hundreds of samples when we “know” (or suspect to know) what the land cover type or the process displayed in a given pixel, represents. This approach allows us to directly label pixels within specific categories, such as deforestation, flooding, infrastructure damage and other processes of land change.
In this approach, we move from the process to the pattern. For example, natural or anthropogenic fires are a common factor of land change. They can be small or large and can drastically affect ecosystems. The fire pattern on the landscape is irregular, and we can see the area is mostly black. Fires are common throughout the Amazon and I have observed them frequently in my research focused on the Colombian Amazon, and after deforestation, locals burn dry vegetation to clear land. Training samples allow us to classify the fire pattern and its spatial extent. In other words, we know what happened and can now classify it. The Andes-Amazon region is a critical transitional zone that connects the Andean biome with the vast lowland Amazon rainforest. It is a biodiversity corridor with untold numbers of endemic and migratory terrestrial and aquatic species that also supports a diverse tapestry of peasant and Indigenous communities. In my research into post-conflict deforestation in the Andes-Amazon region of Colombia, I was able to identify the rapid expansion of pastures for cattle ranching as the main driver of deforestation.
The full Interdialogue with Paulo Murillo.
During the study a large amount of training data was collected through satellite imagery, and patterns of new deforestation and the subsequent establishment of farms became evident. Large new deforested patches surrounding previous cattle farms gave us the data necessary to infer that more farms were established after the peace agreement between the Colombian Government and the FARC (Fuerzas Armadas Revolucionarias de Colombia) guerillas (Figure 1).
Another example of Pixelizing the Social is the expansion of agricultural systems. In Colombia, sugarcane crops are located in the Cauca River Valley, which has one of the highest crop productivity per hectare worldwide. Sugarcane has also expanded into other regions in Colombia. The plots are generally very regular, and we know the area is suitable for agriculture. The process is linked to increased crop productivity and the outcome on the landscape is, in this case, a pattern that is regular or very rectangular.
Pixelizing the Social is the common way of teaching about satellite imagery processing in Universities. It moves from classifying an image to creating a thematic landscape representation of the processes presented. In this approach, we can classify imagery because we know exactly what is in each pixel. But what happens when we do not know precisely what is in a pixel, or sufficient training data does not exist to verify the process driving the land change?
Socializing the Pixel
The second and less conventional approach that can be taken by land use scientists is that of “Socializing the Pixel.” Socializing the Pixel means that human actions can be uncovered in a pixel analysis. Another way of thinking about this idea is moving from an observable pattern (Pixelizing the Social) to inferring the process driving it (Socializing the Pixel). Satellite imagery shows patterns we suspect are linked to a process. In other words, we move from the pattern to the process.
The full Interdialogue with Paulo Murillo.
The Great Wall of China, for example, is the only human-constructed element visible from the moon. If we can see that large, linear pattern from an image, we can seriously “suspect” that an organized, long-term or even “well-organized” civilization built it. So the Great Wall of China pattern allows us to create a hypothesis: the hypothesis that a well-organized human population could create this giant wall. While this ancient example helps to establish a preliminary understanding of what Socializing the Pixel can provide to researchers, more contemporary examples shed light on how we can apply it.
For instance, deforestation in Brazil, especially in the Amazon, is progressive. Satellite imagery detects a deforestation pattern called the “fishbone.” This pattern is described by straight lines (i.e., roads) that branch off of a main road. This forms a network of smaller sections of new roads at almost perpendicular axes. While the pattern is common and identifiable, at least in the Brazilian Amazon, what is behind it?
Field verification indicates that once a fishbone pattern is determined, it is linked to urbanization – meaning urban expansion in previously forested areas. New houses are built surrounding the preliminary roads. Therefore, a town full of new people will be settled.
My own research in Colombia has used historical satellite imagery from 1984 to 2023 that includes periods of heavy armed conflict, de-escalation (negotiations) and a final peace agreement. The changes we observed in the landscape suggest patterns of a new land use regimen.
During times of war in the Andes-Amazon Colombia region, land changes were slow and the patches were irregular and small. After the signing of the PeaceAccords in 2016 and the demobilization, disarmament and exodus of the FARC from formerly controlled forested territories, these patterns changed to more rapid, regular and larger patches.
Throughout the more than 50-years long civil war, the patterns were messy, with no specific trends or patterns found. In other words, the pattern was a non-pattern. However, during the years after the Peace Accords, something interesting happened. The regular fishbone pattern found frequently in the Brazilian Amazon began being detected in the Colombian Andes-Amazon region. This particular pattern indicates that more population and new urban areas are being settled in places where before only pristine forests that were controlled by the FARC, were located.
These social processes linked to armed conflict are complex, context-dependent and divergent. It is not as simple as saying that deforestation increased after the FARC left in all regions. Recent research that I co-authored conducted in the Sumapaz paramo – a type of high-altitude moorland ecosystem rich in endemic species and vital to providing fresh water to tens of millions of people in the neotropics – showed a very different pattern.
Sumapaz is an important water source for over 15 million people in Colombia and this region was also a strategic corridor for the FARC to move combatants, weapons, and money from the center of the country to southeastern Colombia. Once the Colombian army started constructing military bases in the paramo, war became very intense there. Whilst the Colombian army expanded military bases into the paramo, wood resources were quickly degraded for uses in construction, protection, and certain related livelihoods. So during intense periods of war, more landscape disturbances were found.
After the Peace Accords in 2016, however, deforestation nearly disappeared. Once the FARC left the Sumapaz paramo, land disturbances decreased and deforestation stopped. The role of local communities, integrated land management and the lack of forest extraction, also currently contributes to keeping the paramo landscape stable.
The full Interdialogue with Paulo Murillo.
Another complex land use and deforestation pattern that I have helped to uncover with my remote sensing studies, is associated with illicit land use activities in the Andes-Amazon region of Colombia. Coca and cattle farms in the Colombian Amazon are persistent, and mapping them is difficult because both look very similar in satellite imagery taken from space. We can hypothesize that coca plots are relatively small and far away from the main roads, but are surrounded by large cattle farms (Figure 2). If we can see a pattern that shows large patches (>30ha) and small (1-2ha) patches that are far away from the large ones we might suspect two different changes linked to different processes. If we observe that particular pattern we can identify the process. Small areas are used for coca production whereas large areas are linked to pasture to grow cattle.
As these above examples help demonstrate, Socializing the Pixel should be a more common strategy used by land use scientists and taught to aspiring land use science students. It offers an interesting and multifaceted methodology for using satellite imagery to create hypotheses about social drivers of land change patterns and to ask novel questions. This approach is useful when we do not have enough training data or collecting this data is challenging and time-consuming.
The impacts of disruptive data-driven technologies on land use science
The value of remote sensing is not only to create intriguing and sophisticated maps of diverse land cover and land use types but also to contribute to understanding the interconnected sociopolitical, economic, and environmental processes that drive them.
With advances in Deep Learning and Artificial intelligence (AI), advanced algorithms can now classify satellite imagery much better than previous algorithms and are projected to drastically improve over the coming years and decades. However, these algorithms require vast training data to classify the images. If we do have prodigious amounts of training data, Pixelizing the Social should be sufficient for detecting the underlying patterns, so in theory, we would not need the Socializing the Pixel approach in these scenarios. A good current example of this is the Segment Anything Meta-AI. This platform has been trained with millions of images to detect and classify objects automatically. You can upload any image and the platform segments each object within it.
AI has also been applied to satellite imagery to create real time land cover maps using Sentinel-2 imagery. The application, Dynamic World, classifies each satellite image within nine land cover classes.
The full Interdialogue with Paulo Murillo.
Furthermore, mapping tree height globally is now available at 1m pixel size using high-resolution imagery. This spatial resolution surpasses previous remote sensing technologies that only achieved 30m pixel size. This increase in tree height detail allows us to estimate biomass and carbon with higher certainty, which is important for understanding carbon storage capacities and developing policies that protect densely forested areas.
One land cover type that stores enormous amounts of carbon is mangroves forests. Mangroves provide many benefits for people and ecosystems beyond storing carbon, such as regulating water quality, protecting from flooding and coastal erosion, and serving as a nursery for aquatic species. Better understanding tree height using remote sensing and deep learning architectures is crucial to identifying the distribution of mangroves and other forests worldwide within rural, coastal, and urban landscapes.
The full Interdialogue with Ernesto Mancera.
I have had the opportunity to map mangroves along the Pacific coast of Colombia. While we detected important land changes through satellite imagery, truly understanding the drivers behind these changes came from local communities, who explained to us the real causes of deforestation and mangrove degradation that they have observed first-hand.
AI technology is evidently an important strategic ally for the Pixelizing the Social approach to land use science whenever the training data is available to map whatever process exists. However, when there is a lack of training data, developing hypotheses and testing new theories that inform new questions would still be part of the less utilized and equally important Socializing the Pixel approach.
The collection of sufficient training data for all of the complex and interrelated processes that impact our living planet is an enormous and practically infeasible task to accomplish in the near future. We continue to need new and innovative approaches to understanding and preventing the most destructive processes that are putting Earth’s ecosystems under increasing stress, and Socializing the Pixel is an important framework for fostering multi-scalar understanding and action. Detecting the patterns and asking about the driving process will remain part of research and development in land use science. Our curiosity for monitoring and understanding the changes on Earth from space requires not only a perfect delineation of the patterns but also uncovering the interconnected social linkages that embrace the complexity of land changes that are driven by a natural and anthropogenic mixture.
Artificial intelligence is going to play an increasingly powerful role in the next decade. Advanced ways of classifying elements over terrestrial surfaces, the cryosphere, oceans and the atmosphere will help us to understand more about the complex processes and changes taking place on our planet, and to make specific informed policy decisions. Tracking interrelated illegal land use activities, rapidly melting ice sheets and glaciers and the rise of ocean temperatures, carbon dioxide and other greenhouse gasses in the atmosphere, will provide innovative ways to address climate change scenarios. Near real-time alert applications will become more common, and integrating them with social knowledge will provide key strategies for halting complex land use issues such as deforestation and the degradation of key biodiversity corridors. Socializing the Pixel and Pixelizing the Social will continue to be critical aspects of remote sensing science. Perhaps in the future they will take on other names or will be without labels, but the fundamental approaches of process to pattern and pattern to process should be explained to local communities and stakeholders, taught in depth to enthusiastic future land use scientists at universities, and implemented by researchers who work in the field and in the sky.
Banner image: Choropleth map of land use disturbances in the Andes-Amazon region of Colombia in wartime, negotiation and post-agreement. Credit: P. Murilllo.
