Welcome to the Assume Project

Agent-Based Electricity Markets Simulation Toolbox

Introduction

The transformation of electricity markets associated with the transition towards high shares of renewable power generation results in the constant development of market mechanisms, increasing sector coupling, and creating new market platforms. Introducing a new market or changing the current market design does, however, affect all other markets and their participants because of their strong interrelation in not necessarily foreseeable ways, as the last changes in the German reserve markets demonstrated. This raises the need for tools and simulation models to investigate and understand such complex interplay of markets and predict possible adverse effects and misuse of market power.

With ASSUME a team of researchers aims to develop a highly modular and easy-to-use energy market simulation toolbox with integrated reinforcement learning methods. Different reinforcement learning algorithms in multi-agent simulations of electricity markets were already tested and have created promising results. This toolbox will enable an agile analysis of market designs and bidding strategies of new actors and emerging market dynamics in our fast changing energy system.

The Software Package

The ASSUME project aims to develop an open-source, agent-based simulation toolbox for electricity markets using deep reinforcement learning (DRL) algorithms.
We are very happy about and constructive feedback and contributions to the Git-repositories.

Here you can visit the docs.

What?At what stage?Description
Markets
Markets modularityStage 1
Parent classes providing basic structure, functionality, and syntax, that are adapted by specific markets to allow interchangeability, start with a combination of IDM and DAM
Market couplingStage 1
Spatially interconnected electricity markets (ntc) and different commodity markets (gas, hydrogen)
Interchangeable market clearing algorithmsStage 1
Clearing is treated as one specific interchangeable part that can be adapted easily in each market representation
Redispatch MarketStage 2 Market after the day-ahead electricity market that handels redispatch considering the grid constraints
Sector couplingplannedHow to have more sector coupling technologies
Different Market representation (separate DA and ID) plannedHow electricity markets are modeled, at the first stage only merge of DA and ID, which might be redefined later on
Agents
Assets modularityStage 1
Parent classes providing basic structure, functionality, and syntax, that are adapted by specific agents to allow interchangeability
Learning agentsStage 1
Usage of Deep Reinforcement Learning for the definition of bidding policies 
Exchangeable bidding policiesStage 1
Naive policies and policies that are learned by the RL agents
Flexible Demand Stage 2In stage 1, fixed demand and further we integrate different demand side agents that have an individual collection of technologies themself, such as heatpumps and PV
Different bid typesStage 2Incorporation of different bids such as block bids etc.
Portfolio OptimizationplannedOption is accounted for in architecture, the actual implementation to be discussed
General
NetworkStage 1
The idea is to represent the markets as a network using graph theory, this network does not represent the power grid, rather defines relations and the degree of relations to other agents/components
Communication layerStage 1
E.g. Order-books, or mango agents tbd
Interoperable IO formatsStage 1
Standard data formats adapted from other open source tools and general conventions
Scalability and parallel executionStage 2Might be solved with predefined packages like Mango agents or repast4py, tbd
Deep Reinforcement Learning
Strategies for multiple unit typesStage 1including power plants and different storages
Strategy for Multi Market Biddingplanned
To handel the variety of markets configureable we must adjust the DRL Algorithmic setting to handel muitple markets at once. 
Eplainability featuresplanned
As the developed staregy of DRL follow an rather explorative nature, we need to develope ways of assessing the learned startegies with the help of xRL. 
Different Algorithmsplanned
Implementing a PPO in addtioin to the MATD3, incorperating LSTM plocies and importance sampling buffers for the MATD3

Related Research

Nick Harder, Ramiz Qussous, and Anke Weidlich
Fit for purpose: Modeling wholesale electricity markets realistically with multi-agent deep reinforcement learning
Energy and AI, Volume 14, 2023
https://doi.org/10.1016/j.egyai.2023.100295

Florian Maurer, Kim K. Miskiw, Rebeca Ramirez Acosta, Nick Harder, Volker Sander & Sebastian Lehnhoff
Market Abstraction of Energy Markets and Policies – Application in an Agent-Based Modeling Toolbox
Jørgensen, B.N., da Silva, L.C.P., Ma, Z. (eds) Energy Informatics. EI.A 2023. Lecture Notes in Computer Science, vol 14468.
http://dx.doi.org/10.1007/978-3-031-48652-4_10

Kim K. Miskiw, Nick Harder and Philipp Staudt
Multi Power-Market Bidding: Stochastic Programming and Reinforcement Learning
Proceedings of the 57th Hawaii International Conference on System Sciences | 2024.
https://scholarspace.manoa.hawaii.edu/bitstreams/ab278af7-2dfe-4c36-a538-eaccb8be1262/download

Nick Harder, Anke Weidlich, and Philipp Staudt
Modeling Participation of Storage Units in Electricity Markets using Multi-Agent Deep Reinforcement Learning.
In Proceedings of the 14th ACM International Conference on Future Energy Systems (e-Energy ’23). Association for Computing Machinery, New York, NY, USA, 439–445
https://doi.org/10.1145/3575813.3597351

—————-

Upcoming workshop at DACH 2023: https://www.energy-informatics2023.org/workshops/

Newsletter

Subscribe to our newsletter

* indicates required

Intuit Mailchimp

People behind the project

Nick Harder is a research associate and a doctoral student at the Institute for Sustainable Systems Engineering (INATECH) at the University of Freiburg. He completed his master’s studies in “Sustainable Systems Engineering” at the same university. His research primarily focuses on utilizing deep reinforcement learning methods to model electricity markets and understand the behavior of market participants. Additionally, he holds the role of project coordinator for the ASSUME project, which focuses on developing a modeling toolbox for analyzing electricity markets and market designs through the use of deep reinforcement learning techniques. Overall, Nick’s expertise lies in the intersection of sustainable systems engineering and advanced machine learning approaches applied to energy markets.

Kim K. Miskiw is a research associate and a doctoral student at the Chair for Information and Market Engineering (IISM) within the Faculty of Economics and Business Engineering at the Karlsruhe Institute of Technology (KIT). Her research interests revolve around deep reinforcement learning in electricity market simulations, agent-based electricity market modeling, energy market engineering, and stochastic optimization. Previously, she held the position of Junior-Project Associate at the Institute for Industrial Production, Chair of Energy Economics (KIT). Kim completed her Master’s degree in Industrial Engineering at KIT, focusing her master’s thesis on stochastically optimized bidding strategies in sequential electricity markets and examining their benefits in relation to risk preferences and portfolio setups.

Manish Khanra is a research associate at the Competence Center, Energy Technologies and Energy Systems (CC-E) at Fraunhofer ISI. He is also a doctoral student at the Institute for Industrial Production (IIP) at Karlsruhe Institute of Technology (KIT). With specialisation in integrating hydrogen and efuels for decarbonising Hard-to-Abate sectors, Manish investigates their impact on the power system.
Holding an MSc in Renewable Energy and Energy Efficiency for the Middle East and North Africa, his research has focused on the transformation paths in the heat sector in Germany. Currently, his work encompasses developing electricity market and technology diffusion models, analysing policy aspects for sectors such as steel, cement, chemicals, aviation, and maritime, and conducting applied research in the hydrogen economy.

Florian Maurer is a research associate and doctoral student at the University of Applied Sciences Aachen in cooperation with the University of Oldenburg. After completing a dual study program in software development, he obtained his Master’s degree in “Applied Mathematics and Computer Science” at FH Aachen, where he developed charging solutions for e-mobility. Florian is involved in research projects related to energy measurements and prosumer market integration. His research interests include open-source development, wireless communication and energy market design. Currently, he is researching agent-based modeling of energy markets to provide a simulation framework that covers the comparison of different market designs and policies.