arXiv:2610.37853v1 Announce Type: new Abstract: We study the optimal control of a stylized finite-horizon heat-storage system operated by a prosumer who can buy heat from and sell heat to a district heating network. Heat production and consumption are uncertain, and charging and discharging decisions have to be taken before the subsequent heat balance is observed. The resulting problem is formulated as a Markov decision process with continuous state and action spaces. We first establish a structural characterization of the optimal policy as a two-parameter threshold policy. Under additional regularity assumptions, we prove differentiability and Lipschitz properties of the value functions. Exploiting the threshold structure, we parameterize the policy by the time-dependent lower and upper thresholds and derive a pathwise stochastic-gradient estimator. We then propose a projected stochastic policy-gradient algorithm, establish a constrained Kurdyka-\L ojasiewicz inequality and derive global finite-sample bounds for both the objective optimality gap and the distance to the optimal threshold vector. Finally, numerical experiments for a non-stationary heat-balance process illustrate the evolution of the threshold parameters.
Continue to arXiv Mathematics →You are being forwarded in 6s — stay on this page