Luca Schenato

We address the problem of distributed unconstrained convex optimization under separability assumptions, i.e., the framework where a network of agents, each endowed with local private multidimensional convex cost and subject to communication constraints, wants to collaborate to compute the minimizer of the sum of the local costs. We propose a design methodology that combines average consensus algorithms and separation of time-scales ideas. This strategy is proven, under suitable hypotheses, to be globally convergent to the true minimizer. Intuitively, the procedure lets the agents distributedly compute and sequentially update an approximated Newton-Raphson direction by means of suitable average consensus ratios. We show with numerical simulations that the speed of convergence of this strategy is comparable with alternative optimization strategies such as the Alternating Direction Method of Multipliers. Finally, we propose some alternative strategies which trade-off communication and computational requirements with convergence speed.

The knowledge of the size of a network, i.e.\ of the number of nodes composing it, is important for maintenance and organization purposes. In networks where the identity of the nodes or is not unique or cannot be disclosed for privacy reasons, the size-estimation problem is particularly challenging since the exchanged messages cannot be uniquely associated with a specific node. In this work, we propose a totally distributed anonymous strategy based on statistical inference concepts. In our approach, each node starts generating a vector of independent random numbers from a known distribution. Then nodes compute a common function via some distributed consensus algorithms, and finally they compute the Maximum Likelihood (ML) estimate of the network size exploiting opportune statistical inferences. In this work we study the performance that can be obtained following this computational scheme when the consensus strategy is either the maximum or the average. In the max-consensus scenario, when data come from absolutely continuous distributions, we provide a complete characterization of the ML estimator. In particular, we show that the squared estimation error decreases as $1/M$, where $M$ is the amount of random numbers locally generated by each node, independently of the chosen probability distribution. Differently, in the average-consensus scenario, we show that if the locally generated data are independent Bernoulli trials, then the probability for the ML estimator to return a wrong answer decreases exponentially in $M$. Finally, we provide a discussion as how the numerical errors may affect the estimators performance under different scenarios.

This paper considers a variation of the 17$^{\text{th}}$ century problem commonly known as the Newton-Pepys problem, or the John Smith's problem. We provide its solution, interpreting the result in terms of maximum likelihood estimation and Ockham's razor. In addition, we illustrate the practical relevance of these findings for solving size-estimation problems, and in particular for determining the number of agents in a wireless sensor network.

The paper considers the classification problem using Support Vector Machines, and investigates how to maximally reduce the size of the training set without losing information. Under linearly separable dataset assumptions, we derive the exact conditions stating which observations can be discarded without diminishing the overall information content. For this purpose, we introduce the concept of Potential Support Vectors, i.e., those data that can become Support Vectors when future data become available. Complementary, we also characterize the set of Discardable Vectors, i.e., those data that, given the current dataset, can never become Support Vectors. These vectors are thus useless for future training purposes, and can eventually be removed without loss of information. We then provide an efficient algorithm based on linear programming which returns the potential and discardable vectors by constructing a simplex tableau. Finally we compare it with alternative algorithms available in the literature on some synthetic data as well as on datasets from standard repositories.