Case study · AI & digital twin

Deep reinforcement learning for pump pressure optimisation in a regional transmission network

A continuous-control deep reinforcement learning (DRL) agent, trained on a calibrated EPANET digital twin, sets the discharge pressures of three interconnected pumping stations in real time. It keeps every DMA above minimum pressure while cutting excess pressure and pumping energy. The research has been published in the ASCE Journal of Water Resources Planning and Management.

Network
Regional transmission network, UAE

Scope
Real-time pump pressure set points

Tools
EPANET + EPyT, OpenAI Gym, Stable-Baselines3

Publication
ASCE JWRPM (2026)

≈ 14%

Estimated pumping energy saving over a historical diurnal pattern

197 km

DN600–1600 ductile iron mains, about 972 km² served

3 stations

10,000–15,000 m³/h each, variable-speed pumps

35 DMAs

Each with a PRV and flow meter, 15-minute SCADA data

The challenge

Fixed pressures waste energy

Demand changes through the day, but pumping stations are often run at pressures set for the peak. At night and in low-demand hours this over-pressurises the network, wastes energy and increases leakage and pipe stress. With three interconnected stations, finding the lowest pressures that still keep every DMA in service is a continuous, nonlinear control problem.

Transmission network schematic with three pumping stations and DMAs — Network schematic: three pumping stations supplying 35 DMAs

Our approach

Digital twin and DRL agent

Calibrated EPANET model of the transmission network used as the digital twin, linked to Python through EPyT
Custom OpenAI Gym environment wrapping the hydraulic model
Continuous action space: the agent sets the pressure change at all three pumping stations, matching real variable-speed operation
46-dimensional state covering the 35 DMA demands, current and previous set points, and pressure deviation metrics
Reward that favours lower maximum pressure and on-target pressures, and heavily penalises any breach of minimum pressure
SAC, TD3 and DDPG agents trained with Stable-Baselines3, then evaluated on historical demand patterns

Reinforcement learning control loop with a calibrated EPANET environment — RL control loop: agent, digital twin and reward

Results

Key results

Pumping energy, DRL tool vs conventional method — Pumping energy over a day: DRL vs conventional operation

Maximum network pressure, DRL tool vs conventional — Maximum network pressure: DRL vs conventional

Minimum network pressure, DRL tool vs conventional — Minimum pressure held at the service level

Key findings

The DRL agent learns a stable policy: the training reward rises and levels off as it masters the pressure trade-off.
Over a historical diurnal pattern, the agent holds minimum network pressure at the service level, while conventional operation keeps it much higher in off-peak hours.
Maximum pressure across the network falls throughout the day, reducing leakage and pipe stress.
Pumping energy falls by an estimated 14% compared with conventional operation.
A web-based tool and a supervisory digital-twin loop were developed so the trained policy can propose live set points from real DMA demands.

Deployment

From research to operation

In deployment, the trained policy runs in a supervisory loop. Real DMA demands feed the digital twin, the agent proposes pump discharge set points, and operators or the control system apply them in the field. The work is continuing towards multi-objective control of pressure and water quality, transfer between network topologies, and safe online learning.

Digital twin supervisory control loop for deployment — Deployment: supervisory digital twin control loop

Interested in a pilot for your network?

If you have a calibrated model and SCADA history, we can assess the savings potential and scope a digital twin pilot.

Discuss a pilot →

AI optimisation services