Welcome to the Assume Project
Agent-Based Electricity Markets Simulation Toolbox
Introduction
The transformation of electricity markets associated with the transition towards high shares of renewable power generation results in the constant development of market mechanisms, increasing sector coupling, and creating new market platforms. Introducing a new market or changing the current market design does, however, affect all other markets and their participants because of their strong interrelation in not necessarily foreseeable ways, as the last changes in the German reserve markets demonstrated. This raises the need for tools and simulation models to investigate and understand such complex interplay of markets and predict possible adverse effects and misuse of market power.
With ASSUME a team of researchers aims to develop a highly modular and easy-to-use energy market simulation toolbox with integrated reinforcement learning methods. Different reinforcement learning algorithms in multi-agent simulations of electricity markets have already been tested creating promising results. This toolbox will enable an agile analysis of market designs and bidding strategies of new actors and emerging market dynamics in our fast-changing energy system.
The Software Package
The ASSUME project aims to develop an open-source, agent-based simulation toolbox for electricity markets using deep reinforcement learning (DRL) algorithms.
- We are very happy about and constructive feedback and contributions to the Git-Repository.
- For a deeper dive into the software, check out the readthedocs page.
- To help you get started, we also provide multiple online tutorials using the Jupyter Collab interface.
| What? | Status | Description |
|---|---|---|
| Markets | ||
| Markets modularity | Implemented | Parent classes providing basic structure, functionality, and syntax, that are adapted by specific markets to allow interchangeability, start with a combination of IDM and DAM |
| Market coupling | Implemented | Spatially interconnected electricity markets (NTC) and different commodity markets (gas, hydrogen) |
| Interchangeable market clearing algorithms | Implemented | Clearing is treated as one specific interchangeable part that can be adapted easily in each market representation |
| Redispatch Market | Implemented | Market after the day-ahead electricity market that handles redispatch considering the grid constraints |
| Sector coupling | Implemented | How to model sector coupling technologies in detail |
| Agents | ||
| Assets modularity | Implemented | Parent classes provide basic structure, functionality, and syntax, that are adapted by specific agents to allow interchangeability |
| Learning agents | Implemented | Usage of Deep Reinforcement Learning for the definition of bidding policies |
| Exchangeable bidding policies | Implemented | Naive policies and policies that are learned by the RL agents |
| Demand Side Management Units | Implemented | In Stage 1, fixed demand is used, and later we integrate different demand-side agents that have an individual collection of technologies themselves, such as heat pumps and PV |
| Different bid types | Implemented | Incorporation of different bids such as block bids, etc. |
| Portfolio Optimization | Planned | The option is accounted for in architecture; the actual implementation to be discussed |
| General | ||
| Database support for storing outputs | Implemented | Users can use traditional CSV formats for outputs or utilize the provided timescaleDB for storing simulation results |
| Graphical interface for analyzing simulation results | Implemented | Users can utilize the provided Grafana Dashboards for analyzing individual simulation results, compare several simulations to each other, and track learning progress of DRL agents |
| Network | Implemented | Adaptable network representation that is also able to read and solve PyPSA examples in Assume |
| Communication layer | Implemented | e.g. Order-books used between agents and markets, there we rely on mango agents |
| Interoperable IO formats | Implemented | Standard data formats adapted from other open-source tools and general conventions; enable scenario reading from Amiris and PyPSA |
| Scalability and parallel execution | Implemented | Enables running Assume in different Docker containers |
| Deep Reinforcement Learning | ||
| Strategies for multiple unit types | Implemented | Including power plants and different storages |
| Strategy for Multi-Market Bidding | Work in progress | To handle the variety of configurable markets, we must adjust the DRL algorithmic settings to manage multiple markets at once. |
| Explainability features | Work in progress | As the developed strategy of DRL follows an rather explorative nature, we need to develope ways of assessing the learned startegies with the help of explainable RL. |
| Different Algorithms | Planned | Implementing a PPO in addition to the MATD3, incorporating LSTM policies and importance sampling buffers for the MATD3 |
Upcoming Workshops
Final Workshop – Agenda & Session Abstracts
- Date: Wednesday, 9 July 2025
- Format: Online via Zoom
- Registration: via this link
Introduction
The transition to high shares of renewable power, coupled with emerging actors and rapidly evolving market rules, calls for tools that can simulate, stress‑test, and help design tomorrow’s electricity markets. ASSUME (Agent‑Based Electricity Markets Simulation Toolbox) combines agent‑based modelling with deep‑reinforcement learning to explore adaptive bidding behaviour, sector‑coupling, and system‑level effects across multiple inter‑linked markets.
This final workshop showcases the toolbox, underlying science, and practical workflows. Sessions are modular—participants may join whichever topics interest them—but we strongly recommend attending the Opening Session for essential context and quick overviews of the upcoming sessions.
Session Format
Each technical session follows a common structure:
- 20 min — Scientific presentation (conference‑style deep dive)
- 70 min — Hands‑on notebook walk‑through using Google Colab to reproduce key results and experiment with the model
Agenda Overview
| Time | Session | Speaker(s) | Duration |
| 09:00 – 09:30 | Opening Session – Importance of simulation tools, project overview, session previews | Prof. Dr. Anke Weidlich, ASSUME Team | 30 min |
| 09:30 – 11:00 | Session 1 – Adaptive Behavior in Zero‑Marginal‑Cost Systems | Kim Miskiw | 90 min |
| 13:00 – 14:30 | Session 2 – Demand‑Side Management Modeling | Manish Khanra | 90 min |
| 15:00 – 16:30 | Session 3 – Redispatch Modelling & Network Integration | Parag Patil | 90 min |
Session Abstracts
Session 1 – Adaptive Behavior and Market Dynamics in Zero‑Marginal‑Cost Energy Systems
Lead: Kim Miskiw (KIT)
Scientific presentation (20 min)
The presentation explains why electricity systems with near‑zero marginal costs require new analytical approaches to predict bidding behavior and price formation. It outlines the multi‑agent deep‑reinforcement‑learning framework implemented in ASSUME and highlights key modelling challenges—partial observability, non‑stationarity, and convergence of competing strategies. Example architectures, including the centralised‑critic/decoupled‑actor setup and MATD3, demonstrate scalable solutions. Case‑study results illustrate how storage and renewable agents learn profitable bids and how their interaction shapes market prices and system stability.
Hands‑on workshop (70 min)
Participants split into small teams to build and test bidding strategies for storage and renewable agents in a simplified zero‑marginal‑cost market. Each team defines the agents’ observation and action spaces, implements them in an interactive Google Colab notebook, and runs multi‑agent simulations. Interim results are shared in short peer presentations, followed by instructor feedback. The exercise wraps up with convergence testing and visual analysis of price‑duration curves to assess how learned strategies shape market outcomes and stability.
Session 2 – Industrial Demand‑Side Management in ASSUME
Lead: Manish Khanra (Fraunhofer ISI)
Scientific presentation (20 min)
This talk shows how ASSUME couples investment planning with operational market participation for energy‑intensive industries. Using a paper‑production plant, it demonstrates an investment layer that reflects heterogeneous risk profiles and evaluates retrofit options under uncertainties in CO₂ prices, natural‑gas prices, renewable availability, and policy incentives. Participants will see how Flex‑Bid strategies in ancillary‑service markets and participation in Redispatch 3.0 convert inherent flexibility into new revenue streams while reducing CO₂ emissions and redispatch costs.
Hands‑on workshop (70 min)
Attendees configure Demand‑Side Units for low‑temperature heat processes, integrate heat pumps and thermal storage, and link them to multiple markets inside ASSUME. They experiment with Flex‑Bid parameters, simulate Redispatch 3.0 participation, and compare risk‑adjusted revenues, CO₂ savings, and redispatch reductions across various retrofit scenarios.
Session 3 – Redispatch Modelling and Network Integration
Lead: Parag Patil (Fraunhofer IEG)
Scientific presentation (20 min)
The presentation examines Germany’s growing grid‑congestion challenge amid rapid renewable expansion. It details methods to locate congestion points, outlines the current redispatch process, and quantifies how escalating renewable penetration drives redispatch volumes and costs. Industrial flexibilities—such as steel and pulp & paper plants—are highlighted as virtual power plants capable of supplying demand‑side relief.
Hands‑on workshop (70 min)
Participants work through a sequence of three‑node case studies: (1) baseline redispatch; (2) inclusion of Demand-Side Units; (3) addition of a large industrial flexibility provider. They measure congestion and cost metrics at each stage before scaling to a Germany‑wide network to evaluate regional redispatch quantities and the system‑wide impact of industrial flexibility.
Practical Information & Next Steps
- A single Zoom link will be provided after registration; break‑out rooms will be opened for the hands‑on segments.
- Prerequisites: Google account (for Colab) and basic Python familiarity for the hands-on sessions.
- For questions, contact nick.harder@inatech.uni-freiburg.de
Related Research
Nick Harder, Ramiz Qussous, and Anke Weidlich
Fit for purpose: Modeling wholesale electricity markets realistically with multi-agent deep reinforcement learning
Energy and AI, Volume 14, 2023
https://doi.org/10.1016/j.egyai.2023.100295
Nick Harder, Anke Weidlich, and Philipp Staudt
Finding individual strategies for storage units in electricity market models using deep reinforcement learning
Energy Inform 6 (Suppl 1), 41, 2023
https://doi.org/10.1186/s42162-023-00293-0
Florian Maurer, Kim K. Miskiw, Rebeca Ramirez Acosta, Nick Harder, Volker Sander & Sebastian Lehnhoff
Market Abstraction of Energy Markets and Policies – Application in an Agent-Based Modeling Toolbox
Jørgensen, B.N., da Silva, L.C.P., Ma, Z. (eds) Energy Informatics. EI.A 2023. Lecture Notes in Computer Science, vol 14468.
http://dx.doi.org/10.1007/978-3-031-48652-4_10
Kim K. Miskiw, Nick Harder and Philipp Staudt
Multi Power-Market Bidding: Stochastic Programming and Reinforcement Learning
Proceedings of the 57th Hawaii International Conference on System Sciences, 2024.
https://scholarspace.manoa.hawaii.edu/bitstreams/ab278af7-2dfe-4c36-a538-eaccb8be1262/download
Nick Harder, Anke Weidlich, and Philipp Staudt
Modeling Participation of Storage Units in Electricity Markets using Multi-Agent Deep Reinforcement Learning.
In Proceedings of the 14th ACM International Conference on Future Energy Systems (e-Energy ’23). Association for Computing Machinery, New York, NY, USA, 439–445
https://doi.org/10.1145/3575813.3597351
Manish Khanra, Parag Patil, Marian Klobasa, and Daniel Scholz
Economic Evaluation of Electricity and Hydrogen-Based Steel Production Pathways: Leveraging Market Dynamics and Grid Congestion Mitigation through Demand Side Flexibility.
In Proceedings of the 20th International Conference on European Energy Market (EEM24). IEEE, Istanbul, Turkey, 2024
https://doi.org/10.1109/EEM60825.2024.10608890
Florian Maurer, Felix Nitsch, Johannes Kochems, Christoph Schimeczek, Volker Sander, and Sebastian Lehnhoff
Know Your Tools – A Comparison of Open-Source Energy Market Simulation Models.
In Proceedings of the 20th International Conference on European Energy Market (EEM24). IEEE, Istanbul, Turkey, 2024
https://doi.org/10.1109/EEM60825.2024.10609021
Johanna Adams, Nick Harder, and Anke Weidlich
Do Block Orders Matter? Impact of Regular Block and Linked Orders on Electricity Market Simulation Outcomes.
In Proceedings of the 20th International Conference on European Energy Market (EEM24). IEEE, Istanbul, Turkey, 2024
https://doi.org/10.1109/EEM60825.2024.10608956
Kim K. Miskiw and Philipp Staudt
Explainable Deep Reinforcement Learning for Multi-Agent Electricity Market Simulations.
In Proceedings of the 20th International Conference on European Energy Market (EEM24). IEEE, Istanbul, Turkey, 2024
https://doi.org/10.1109/EEM60825.2024.10608907
People behind the project
Nick Harder is a research associate and a doctoral student at the Institute for Sustainable Systems Engineering (INATECH) at the University of Freiburg. He completed his master’s studies in “Sustainable Systems Engineering” at the same university. His research primarily focuses on utilizing deep reinforcement learning methods to model electricity markets and understand the behavior of market participants. Additionally, he holds the role of project coordinator for the ASSUME project, which focuses on developing a modeling toolbox for analyzing electricity markets and market designs through the use of deep reinforcement learning techniques. Overall, Nick’s expertise lies in the intersection of sustainable systems engineering and advanced machine learning approaches applied to energy markets.
Kim K. Miskiw is a research associate and a doctoral student at the Chair for Information and Market Engineering (IISM) within the Faculty of Economics and Business Engineering at the Karlsruhe Institute of Technology (KIT). Her research interests revolve around deep reinforcement learning in electricity market simulations, agent-based electricity market modeling, energy market engineering, and stochastic optimization. Previously, she held the position of Junior-Project Associate at the Institute for Industrial Production, Chair of Energy Economics (KIT). Kim completed her Master’s degree in Industrial Engineering at KIT, focusing her master’s thesis on stochastically optimized bidding strategies in sequential electricity markets and examining their benefits in relation to risk preferences and portfolio setups.
Manish Khanra is a research associate at the Competence Center, Energy Technologies and Energy Systems (CC-E) at Fraunhofer ISI. He is also a doctoral student at the Institute for Industrial Production (IIP) at Karlsruhe Institute of Technology (KIT). With specialisation in integrating hydrogen and efuels for decarbonising Hard-to-Abate sectors, Manish investigates their impact on the power system.
Holding an MSc in Renewable Energy and Energy Efficiency for the Middle East and North Africa, his research has focused on the transformation paths in the heat sector in Germany. Currently, his work encompasses developing electricity market and technology diffusion models, analysing policy aspects for sectors such as steel, cement, chemicals, aviation, and maritime, and conducting applied research in the hydrogen economy.
Florian Maurer is a research associate and doctoral student at the University of Applied Sciences Aachen in cooperation with the University of Oldenburg. After completing a dual study program in software development, he obtained his Master’s degree in “Applied Mathematics and Computer Science” at FH Aachen, where he developed charging solutions for e-mobility. Florian is involved in research projects related to energy measurements and prosumer market integration. His research interests include open-source development, wireless communication and energy market design. Currently, he is researching agent-based modeling of energy markets to provide a simulation framework that covers the comparison of different market designs and policies.