As the Compute Gardener Scheduler continues to evolve, I've been fortunate enough to engage in some fascinating conversations around a fundamental question in carbon-aware computing: should we optimize for average or marginal carbon intensity when scheduling workloads? The question may seem nuanced and technical, but it could have meaningful implications for how we build truly effective carbon-aware systems.
The Signal Question
If, like me, you're new to the concepts:
Average carbon intensity tells us the emissions associated with the overall electricity mix on the grid at a given moment — essentially, what's the carbon footprint of a kWH (kilowatt-hour) of electricity right now, considering all generation sources currently active.
Marginal carbon intensity attempts to predict which generation sources would respond to a new load being added to the grid — in other words, what's the carbon footprint of the next kWh if we were to consume it right now.
The difference initially seems subtle, but it can be substantial. In many grids, the marginal generator (the one that ramps up to meet new demand) might be a natural gas plant, even when solar and wind make up a significant portion of the current mix.
The Community Conversation
This topic has sparked lively debates in the climate tech community. I first became aware of it through a conversation including Tammy Sukprasert, a researcher at UMass Amherst, on the Environment Variables podcast. In that episode, she highlighted the theoretical differences between average and marginal emissions signals and mentioned that organizations like WattTime have been exploring the use of marginal signals. She was kind enough, a few weeks after the episode, to talk with me and help me begin to grok the distinction.
During the recent San Francisco Climate Week, I had the opportunity to briefly discuss this with Olivier Corradi, CEO of Electricity Maps API, and a few of his colleagues. Their perspective is that average signals are more reliable, accessible and ultimately more actionable for most use cases.
WattTime has been a prominent voice in the space examining the potential benefits of marginal emissions. Their argument is that marginal emissions might more accurately reflect the true impact of new electrical loads on the grid. For what it's worth, I haven't had the chance to talk nearly as much with their team or others who follow this school of thought.
The Case for Marginal Signals
WattTime presents a compelling economic and operational case for marginal intensity signals. They argue that while average emissions tell us about the overall grid composition, they don't necessarily reflect what happens when we decide to consume electricity at a specific moment. Grid operators dispatch generation sources in order of cost (known as merit order dispatch), which means the generators responding to new demand often differ from the average mix. From an economic perspective, this is analogous to the difference between average cost and marginal cost - what matters for decision-making is the impact of your next action, less so the previous state of the system.
For carbon-aware computing specifically, this distinction becomes meaningful because our goal isn't just to measure or attribute emissions, but to actively reduce them through timing decisions. If a data center chooses to run a batch workload now versus in three hours, the real-world emissions impact of that decision is determined by which generators respond to that load - the marginal sources - not the average grid mix. Studies by researchers at Rocky Mountain Institute (Dyson & Mandel et al., 2017) and the University of Michigan (Li et al., 2017) have validated that timing decisions based on marginal emissions can lead to greater carbon reductions than those based on average emissions, particularly in grids with diverse generation sources.
The Complexity of Marginal Signals
The theory behind marginal signals is compelling. However, several practical challenges have informed our current approach with Compute Gardener:
-
Hyperlocal Dependencies: Marginal signals can vary significantly based on precise grid connection points. Without extremely granular location data (beyond what most cloud providers expose), the signal's accuracy degrades.
-
Load Scale Assumptions: Many marginal intensity models assume relatively large loads that can actually influence dispatch decisions. For individual computing workloads, even substantial clusters, this assumption may not hold.
-
Coordination Challenges: While marginal signals aim to reduce emissions by capturing the impact of new loads, research has identified scenarios where they could backfire. Research on load synchronization issues has shown that when many distributed actors independently respond to the same marginal emissions signal without coordination, they can create new demand peaks that could potentially trigger high-carbon peaker plants (Radovanovic et al., 2021).
-
Data Availability and Reliability: Average intensity data is more widely available, standardized and accessible across different regions, making it easier to implement globally consistent policies. Further, it is likely better understood and considered more reliable for the purposes of analytical emissions mitigation validation.
Olivier Corradi of Electricity Maps API has emphasized the coordination challenge in conversations and in their blog series "Marginal signals explained", noting that the theoretical benefits of marginal signals can be undermined when applied at scale without proper grid coordination. This synchronization effect is particularly concerning in grids with high renewable penetration that already face steep evening ramps when solar generation decreases, such as California's famous "duck curve" scenario, where rapid ramping requirements in the evening can necessitate firing up more carbon-intensive peaker plants.
Our Implementation Choice
For Compute Gardener's initial release, we've chosen to focus on average carbon intensity signals, specifically through the Electricity Maps API. This decision wasn't made lightly and attempts to strike a pragmatic balance between building useful tools now and engaging with the debate as it rages on and we accumulate more data.
Our focus has been on validating the core hypothesis: we can meaningfully reduce emissions by temporally shifting computing workloads based on carbon intensity signals. Starting with average intensity allows us to:
-
Implement widely: The scheduler can work across diverse geographic regions with consistent signal quality.
-
Validate quickly: We can measure and demonstrate emissions reductions in real-world scenarios.
-
Iterate responsibly: As our understanding evolves, we can incorporate more sophisticated signals without disrupting existing installations.
-
Avoid overcomplicated abstractions: By starting with a clearer signal, we build interfaces and patterns that make sense to users and avoid premature optimization.
Looking Forward
While we've started with average intensity signals, Compute Gardener's architecture is designed to accommodate future signal sources. The modular implementation of our carbon-aware scheduling allows for:
-
Multiple signal sources: Organizations with access to high-quality, locationally relevant marginal signals could plug them into the scheduler.
-
Hybrid approaches: Combining average and marginal signals based on workload characteristics and regional grid dynamics.
-
Spatial shifting: As we expand beyond temporal optimization to consider moving workloads between regions, we can incorporate both signal types for more sophisticated decision-making.
The Bigger Context
This discussion exists within a broader question of how distributed systems should respond to grid carbon signals. Should every individual system independently optimize for carbon, or should there be coordination? Do we need carbon-aware load balancing at the grid level? How might pricing mechanisms better align individual incentives with grid-level outcomes?
These are the questions that keep us climate-focused nerds up at night and they don't have simple or one-size-fits-all answers.
Your Thoughts?
I'm curious what you think about this debate. Have you implemented carbon-aware systems using intensity signals? What has your experience been? For those with particular expertise in data center and grid operations, what critical perspectives might I be missing?
This is an exciting and ever evolving conversation and one that will shape how computing adapts to our always constrained world. I look forward to continuing it with you.
Want to take action now?
Check out our Kubernetes scheduler if you're interested in learning more or are looking for a way to take some real steps forward in your sustainable compute game today!