We address the problem of distributed unconstrained convex optimization under separability assumptions, i.e., the framework where a network of agents, each endowed with local private multidimensional convex cost and subject to communication constraints, wants to collaborate to compute the minimizer of the sum of the local costs. We propose a design methodology that combines average consensus algorithms and separation of time-scales ideas. This strategy is proven, under suitable hypotheses, to be globally convergent to the true minimizer. Intuitively, the procedure lets the agents distributedly compute and sequentially update an approximated Newton-Raphson direction by means of suitable average consensus ratios. We show with numerical simulations that the speed of convergence of this strategy is comparable with alternative optimization strategies such as the Alternating Direction Method of Multipliers. Finally, we propose some alternative strategies which trade-off communication and computational requirements with convergence speed.

The knowledge of the size of a network, i.e.\ of the number of nodes composing it, is important for maintenance and organization purposes. In networks where the identity of the nodes or is not unique or cannot be disclosed for privacy reasons, the size-estimation problem is particularly challenging since the exchanged messages cannot be uniquely associated with a specific node. In this work, we propose a totally distributed anonymous strategy based on statistical inference concepts. In our approach, each node starts generating a vector of independent random numbers from a known distribution. Then nodes compute a common function via some distributed consensus algorithms, and finally they compute the Maximum Likelihood (ML) estimate of the network size exploiting opportune statistical inferences. In this work we study the performance that can be obtained following this computational scheme when the consensus strategy is either the maximum or the average. In the max-consensus scenario, when data come from absolutely continuous distributions, we provide a complete characterization of the ML estimator. In particular, we show that the squared estimation error decreases as $1/M$, where $M$ is the amount of random numbers locally generated by each node, independently of the chosen probability distribution. Differently, in the average-consensus scenario, we show that if the locally generated data are independent Bernoulli trials, then the probability for the ML estimator to return a wrong answer decreases exponentially in $M$. Finally, we provide a discussion as how the numerical errors may affect the estimators performance under different scenarios.

This paper considers a variation of the 17$^{\text{th}}$ century problem commonly known as the Newton-Pepys problem, or the John Smith's problem. We provide its solution, interpreting the result in terms of maximum likelihood estimation and Ockham's razor. In addition, we illustrate the practical relevance of these findings for solving size-estimation problems, and in particular for determining the number of agents in a wireless sensor network.

The paper considers the classification problem using Support Vector Machines, and investigates how to maximally reduce the size of the training set without losing information. Under linearly separable dataset assumptions, we derive the exact conditions stating which observations can be discarded without diminishing the overall information content. For this purpose, we introduce the concept of Potential Support Vectors, i.e., those data that can become Support Vectors when future data become available. Complementary, we also characterize the set of Discardable Vectors, i.e., those data that, given the current dataset, can never become Support Vectors. These vectors are thus useless for future training purposes, and can eventually be removed without loss of information. We then provide an efficient algorithm based on linear programming which returns the potential and discardable vectors by constructing a simplex tableau. Finally we compare it with alternative algorithms available in the literature on some synthetic data as well as on datasets from standard repositories.

In this paper we focus on collaborative wireless sensor networks, where agents are randomly distributed over a region of interest and collaborate to achieve a common estimation goal. In particular, we introduce two consensus-based distributed linear estimators. The first one is designed for a Bayesian scenario, where an unknown common finite-dimensional parameter vector has to be reconstructed,  while the second one  regards the nonparametric reconstruction of an unknown function sampled at different locations by the sensors. Both of the algorithms are characterized in terms of the trade-off between estimation performance, communication, computation and memory complexity. In the finite-dimensional setting, we derive mild sufficient conditions which ensure that distributed estimator performs better than the local optimal ones in terms of estimation error variance. In the nonparametric setting, we introduce an on-line algorithm that allows the agents both to compute the function estimate with small computational, communication and data storage efforts, and to quantify its distance from the centralized estimate given by a Regularization Network, one of the most powerful regularized kernel methods. These results are obtained by deriving bounds on the estimation error that provide insights on how the uncertainty inherent in a sensor network, such as imperfect knowledge on the number of agents and the measurement models used by the sensors, can degrade the performance of the estimation process. Numerical experiments are included to support the theoretical findings.

A new Bayesian approach to linear system identification
has been proposed in a series of recent papers. The
main idea is to frame linear system identification as predictor
estimation in an infinite dimensional space, with the aid of
regularization/Bayesian techniques. This approach guarantees
the identification of stable predictors based on the prediction
error minimization. Unluckily, the stability of the predictors
does not guarantee the stability of the impulse response of
the system. In this paper we propose and compare various
techniques to guarantee that the final model identified following
this Bayesian approach is stable. First, we consider the socalled
“LMI - constraint” approach and adapt it to constrain
the eigenvalues of the estimated model within the unit circle.
A second possibility which is being considered is to add to the
“classic” Stable-Spline algorithm a penalty term, depending on
the maximum absolute value of the eigenvalue of the system.
This last technique has the advantage of being integrated
directly inside the pre-existing optimization problem and not to
simply post-process the estimated model to guarantee stability.
Finally, we considered a Monte Carlo Markov Chain approach
sampling in both the space of hyper-parameters and of impulse
responses. Simulations results comparing these techniques will
be provided.

We consider the distributed unconstrained minimization of separable convex costfunctions, where the global cost is given by the sum of several local and private costs, eachassociated to a specific agent of a given communication network. We specifically address anasynchronous distributed optimization technique called Newton-Raphson consensus. Besidehaving low computational complexity, low communication requirements and being interpretableas a distributed Newton-Raphson algorithm, the technique has also the beneficial properties ofrequiring very little coordination and naturally support time-varying topologies. In this workwe analytically prove that under some assumptions it shows local convergence properties, andcorroborate this result by means of numerical simulations.

The paper considers the problem of reconstructing a probability density function from a finite set of samples independently drawn from it. We cast the problem in a Bayesian setting where the unknown density is modeled via a nonlinear transformation of a Bayesian prior placed on a Reproducing Kernel Hilbert Space. The learning of the unknown density function is then formulated as a minimum variance estimation problem. Since this requires the solution of analytically intractable integrals, we solve this problem by proposing a novel algorithm based on the Markov chain Monte Carlo framework. Simulations are used to corroborate the goodness of the new approach.

To maintain and organize distributed systems it is necessary to have a certain degree of knowledge of their status like the number of cooperating agents. The estimation of this number, usually referred as the network size, can pose challenging questions when agents' identification information cannot be disclosed, since the exchanged information cannot be associated to who originated it. In this paper we propose a totally distributed network size estimation strategy based on statistical inference concepts that can be applied under anonymity constraints. The scheme is based on the following paradigm: agents locally generate some Bernoulli trials, then distributedly compute averages of these generated data, finally locally compute the Maximum Likelihood estimate of the network size exploiting its probabilistic dependencies with the previously computed averages. In this work we study the statistical properties of this estimation strategy, and show how the probability of returning a wrong evaluation decreases exponentially in the number of locally generated trials. Finally, we discuss how practical implementation issues may affect the estimator, and show that there exists a neat phase transition between insensitivity to numerical errors and uselessness of the results.

In this work we consider a multidimensional distributed optimization technique that is suitable for multiagents systems subject to limited communication connectivity. In particular, we consider a convex unconstrained additive problem, i.e. a case where the global convex unconstrained multidimensional cost function is given by the sum of local cost functions available only to the specific owning agents. We show how, by exploiting the separation of time-scales principle,the multidimensional consensus-based strategy approximates a Newton-Raphson descent algorithm. We propose two alternative optimization strategies corresponding to approximations of the main procedure. These approximations introduce tradeoffs between the required communication bandwidth and the convergence speed/accuracy of the results. We provide analytical proofs of convergence and numerical simulations supporting the intuitions developed through the paper.

We consider the convergence rates of two peculiar2 convex optimization strategies in the context of multi agent3 systems, namely the Newton-Raphson consensus optimization4 and a distributed Gradient-Descent opportunely derived from5 the first. To allow analytical derivations, the convergence6 analyses are performed under the simplificative assumption of7 quadratic local cost functions. In this framework we derive8 sufficient conditions which guarantee the convergence of the9 algorithms. From these conditions we then obtain closed form10 expressions that can be used to tune the parameters for11 maximizing the rate of convergence. Despite these formulae12 have been derived under quadratic local cost functions13 assumptions, they can be used as rules-of-thumb for tuning14 the parameters of the algorithms in general situations.

In this work we study the problem of unconstrained distributed optimization in the context of multi-agents systems subject to limited communication connectivity. In particular we focus on the minimization of a sum of convex cost functions, where each component of the global function is available only to a specific agent and can thus be seen as a private local cost. The agents need to cooperate to compute the minimizer of the sum of all costs. We propose a consensus-like strategy to estimate a Newton-Raphson descending update for the local estimates of the global minimizer at each agent. In particular, the algorithm is based on the separation of time-scales principle and it is proved to converge to the global minimizer if a specific parameter that tunes the rate of convergence is chosen sufficiently small. We also provide numerical simulations and compare them with alternative distributed optimization strategies like the Alternating Direction Method of Multipliers and the Distributed Subgradient Method.

We analyze the problem of data sets reduction

for support vector classification. The work is also motivated

by distributed problems, where sensors collect binary mea-

surements at different locations moving inside an environment

that needs to be divided into a collection of regions labeled in

two different ways. The scope is to let each agent retain and

exchange only those measurements that are mostly informative

for the collective reconstruction of the decision boundary. For

the case of separable classes, we provide the exact conditions

and an efficient algorithm to determine if an element in the

training set can become a support vector when new data arrive.

The analysis is then extended to the non-separable case deriving

a sufficient discardability condition and a general data selection

scheme for classification. Numerical experiments relative to the

distributed problem show that the proposed procedure allows

the agents to exchange a small amount of the collected data to

obtain a highly predictive decision boundary.

The distributed estimation of the number of active sensors in a network can be important for estimation and organization purposes. We propose a design methodology based on the following paradigm: some locally randomly generated values are exchanged among the various sensors and thus modified by known consensus-based strategies. Statistical analysis of the a-consensus values allows estimation of the number of participant sensors. The main features of this approach are: algorithms are completely distributed, since they do not require leader election steps; sensors are not requested to transmit authenticative information (for example identificative numbers or similar data), and thus the strategy can be implemented whenever privacy problems arise. After a rigorous formulation of the paradigma we analyze some practical examples, fully characterize them from a statistical point of view, and finally we provide some general theoretical results and asymptotic analyses.

In this paper we analyze the problem of estimating a function from different noisy data sets collected by spatially distributed sensors and subject to unknown temporal shifts. We propose a novel approach based on non-parametric Gaussian regression and reproducing kernel Hilbert space theory that exploit compact and accurate representations of function estimates. As a first result, suitable minimization of inner products in reproducing kernel Hilbert spaces is used to obtain a novel time delay estimation technique when attention is restricted only to two signals. Then, we derive both a centralized and a distributed estimator to simultaneously identify the unknown function and delays for a generic number of networked sensors subject to a restricted communication graph. Numerical simulations are used to test the effectiveness of the proposed approaches.