\section{Introduction and Motivation}


Due to the enormous increasing in the demand for power supply, in the last decades the scientific and industrial communities have put a huge effort
in research and development of efficient techniques to produce, store and distribute energy. Due to the shift from a fully centralized scenario, in
which there are a few large power plants which supply in a fixed way residential and industrial areas, to the so called \emph{smart grids}, a growing
interest is put in energy market control algorithms (see also \emph{1st IEEE International Conference on Smart Grid Communications (October 2010,
Gaithersburg, MD)}).

Such a type of algorithms, in which each producer and consumer is thought as an autonomous agent able to take decisions, aims to solve several
problems which are typical of the centralized policy. A first goal is to reduce the annual summer demand peak, which is mainly due to the large use
for air conditioning. The huge demand sets a maximum required capacity for the system, and entails the maintenance of expensive and polluting mega
power plants, which are used just a few days or hours per year.\\
A second rationale for a dynamic control of the energy market is related to the increasing amount of energy produced on the basis of renewable
resources, such as wind--power and solar energy. These are affected by a high degree of uncontrolled and sometimes hardly predictable stochastic
uncertainty, since weather forecast could be erroneous in timing as well as in \virg{quantity} of wind or sun, and thus on the produced energy.
Moreover, the market has been recently flowed by relatively cheap small solar plants for buildings' roofs, so that we have a plethora of small
producers whose role cannot be controlled in a centralized way. Rather, again, a dynamic control algorithm allows all the agents to autonomously and
actively participate in the energy market, providing the system operators a mechanism for matching supply and demand by choosing the price of the
energy. Such a mechanism must, however, by cleverly designed in order to guarantee the stability and reliability of the network despite stochastic
uncertainties on the amount of supply or demand, and, more generally, to steer the amount of consumed energy to a value which is \virg{optimal}.
Optimality, in this research field, is usually defined as the maximization of the sum of the benefit for all the users. Clearly, in a realistic
scenario, the agents could be weighted differently for some reasons, but, as it will be clear in what follows, the model we use does not allow to go
in such a detail in the distribution of the energy among the users, so that we won't address this problem.

Theoretical consequences of dynamic pricing have been investigated by several authors. In Borenstein at al. \cite{Borenstein} a comparison is done
among different dynamic pricing strategies. The authors conclude in favour of real--time pricing, which is characterized by electric rates that
reflect the actual wholesale market prices to the customers. In other words, the idea is that of minimizing the gap between energy production cost
for the suppliers and retail prices of the market, in order to get as close as possible to the energy consumption that
maximize the total benefit for the users.

The dynamic pricing has already been tested on the market by several companies, such as ComEd (which is an energy delivery subsidiary of Exelon
Corporation); in most commercial applications, the energy price changes every hour, and the electric rate is calculated by applying different prices
depending on the actual cost of the energy at every time step. It has been proved \cite{Sanjoy} that if the real--time retail prices are communicated
to the customers, who will then myopically adjust their consumption of energy in function of their own advantage, the direct linking between the
consumer prices and the wholesale market prices may cause an unstable close--loop feedback situation. In a later paper \cite{Sanjoy2}, a stabilizing
pricing algorithm has been presented for a simple market model, under opportune assumptions on the behavior of the market
participants. Such a procedure steers the price to a value which is optimal in the sense presented above. 

This last paper is the basis of our analysis. We reduced the algorithm the authors present, which holds for a network of agents, performing a
well--known consumer and producer aggregation in order to deal with a simple network of only three agents, an aggregate consumer, an aggregate
producer, and a system operator, the ISO, charged of modifying the prices according to some pre--designed policy. Once this is done, we provided a
bound on the algorithm parameter which guarantees local stability, offering some simulation to show that the algorithm is also globally stable. Given
such bound, we improved the algorithm by introducing a clever choice of the parameter which ensures the maximization of the convergence rate to the
optimal value. The choice is done according to different interpolation curves which estimate the real characteristics of the consumers, and which are
computed using the past available data on the consumers' behavior.\\
A further problem which has been addressed is to balance the budget of the system operator which provides the control algorithm to the market
participants. This is, in fact, a non--for--profit agent which should not overall gain or loose money for its service. Unfortunately, this is
something intrinsically unavoidable during the transient of the algorithm, so that it develops a total income, or loss. We proposed a sort of Integral
control algorithm to gradually control to zero this income or loss, and we provided a sufficient condition, based on Lyapunov Linearization criterion,
on the parameters of the algorithm to ensure stability. \\
The last problem faced has been introducing a dynamic evolution of the customer's behavior, which should reflect, for the particular case
considered, the different activity during a day. This was done in order to test the performance of our algorithm while chasing a
time--variant optimal equilibrium point. Numerical simulation are also given for the case in which the consumers are supposed not to know exactly
their value function. This situation has been modeled by adding a stochastic disturb to the available data from the market model.

\subsection{Summary}

The paper is organized as follows:
\begin{itemize}
  \item In \textbf{Section \ref{sec:2}} we present the model, the market participants and the assumptions on their behavior. We introduce the market dynamics
  and we mention some theoretical result from \cite{Sanjoy2}, which will be used in the following section.
  \item In \textbf{Section \ref{sec:3}} we analyze the stability bounds for the algorithm proposed in \cite{Sanjoy2}, and provide the proof of local
  stability (in both linear and non--linear expression of the customers' value function). Moreover, we use several functions to interpolate the
  available data, in order to maximize the convergence rate. Numerical results are also shown.
  \item In \textbf{Section \ref{sec:4}} we address the problem of balancing the total revenues of the system operator, and propose a control algorithm,
  giving also theoretical bounds to the parameters.
  \item In \textbf{Section \ref{sec:5}} we add a stochastic disturb to the customer's value function in order to simulate the uncertainties that may
  affect the consumers' behavior.
  \item In \textbf{Section \ref{sec:6}} we propose a simplified model for the dynamic evolution of the system. We suppose the customers' value
  functions to change four times a day, and show the simulative results of the algorithm, trying to chase the equilibrium.
  \item In \textbf{Section \ref{sec:7}}  we sum up the main results of the article and show possible directions for future work.
\end{itemize}
