Abstract: In this paper is developed an accelerated version of the steepest descent method by a two-step iteration. The new algorithm uses information with delay to define the iterations. Specifically, in the first step, a prediction of the new test point is calculated by using the gradient method with the exact minimal gradient steplength and then, a correction is computed by a weighted sum between the prediction and the predecessor iterate of the current point. A convergence result is provided. In order to compare the efficiency and effectiveness of the proposal, with similar methods existing in the literature, numerical experiments are performed. The numerical comparison of the new algorithm with the classical conjugate gradient method shows that our method is a good alternative to solve large-scale problems. PubDate: 2019-12-01

Abstract: The shortest path tree problem is one of the most studied problems in network optimization. Given a directed weighted graph, the aim is to find a shortest path from a given origin node to any other node of the graph. When any change occurs (i.e., the origin node is changed, some nodes/arcs are added/removed to/from the graph, the cost of a subset of arcs is increased/decreased), in order to determine a (still) optimal solution, two different strategies can be followed: a re-optimization algorithm is applied starting from the current optimal solution or a new optimal solution is built from scratch. Generally speaking, the Re-optimization Shortest Path Tree Problem (R-SPTP) consists in solving a sequence of shortest path problems, where the kth problem differs only slightly from the \((k-1){th}\) one, by exploiting the useful information available after each shortest path tree computation. In this paper, we propose an exact algorithm for the R-SPTP, in the case of origin node change. The proposed strategy is based on a dual approach, which adopts a strongly polynomial auction algorithm to extend the solution under construction. The approach is evaluated on a large set of test problems. The computational results underline that it is very promising and outperforms or at least is not worse than the solution approaches reported in the literature. PubDate: 2019-12-01

Abstract: The vertex bisection problem (VBP) is an NP-hard combinatorial optimization problem with important practical applications in the context of network communications. The problem consists in finding a partition of the set of vertices of a generic undirected graph into two subsets (A and B) of approximately the same cardinality in such a way that the number of vertices in A with at least one adjacent vertex in B is minimized. In this article, we propose two new integer linear programming (ILP) formulations for VBP. Our first formulation (ILPVBP) is based on the redefinition of the objective function of VBP. The redefinition consists in computing the objective value from the vertices in B rather than from the vertices in A. As far as we are aware, this is the first time that this representation is used to solve VBP. The second formulation (MILP) reformulates ILPVBP in such a way that the number of variables and constraints is reduced. In order to assess the performance of our formulations, we conducted a computational experiment and compare the results with the best ILP formulation available in the literature (ILPLIT). The experimental results clearly indicate that our formulations outperform ILPLIT in (i) average objective value, (ii) average computing time and (iii) number of optimal solutions found. We statistically validate the results of the experiment through the well-known Wilcoxon rank sum test for a confidence level of 99.9%. Additionally, we provide 404 new optimal solutions and 73 new upper and lower bounds for 477 instances from 13 different groups of graphs. PubDate: 2019-12-01

Abstract: Despite the vast literature on DRS and ADMM, there has been very little work analyzing their behavior under pathologies. Most analyses assume a primal solution exists, a dual solution exists, and strong duality holds. When these assumptions are not met, i.e., under pathologies, the theory often breaks down and the empirical performance may degrade significantly. In this paper, we establish that DRS only requires strong duality to work, in the sense that asymptotically iterates are approximately feasible and approximately optimal. PubDate: 2019-12-01

Abstract: Recently a second directional derivative-based Hessian updating formula was used for Hessian approximation in mesh adaptive direct search (MADS). The approach combined with a quadratic program solver significantly improves the performance of MADS. Unfortunately it imposes some strict requirements on the position of points and the order in which they are evaluated. The subject of this paper is the introduction of a Hessian update formula that utilizes the points from the neighborhood of the incumbent solution without imposing such strict restrictions. The obtained approximate Hessian can then be used for constructing a quadratic model of the objective and the constraints. The proposed algorithm was compared to the reference implementation of MADS (NOMAD) on four sets of test problems. On all but one of them it outperformed NOMAD. The biggest performance difference was observed on constrained problems. To validate the algorithm further the approach was tested on several real-world optimization problems arising from yield approximation and worst case analysis in integrated circuit design. On all tested problems the proposed approach outperformed NOMAD. PubDate: 2019-12-01

Abstract: In this paper, the problem of computing the projection, and therefore the minimum distance, from a point onto a Minkowski sum of general convex sets is studied. Our approach is based on Nirenberg’s minimum norm duality theorem and Nesterov’s smoothing techniques. It is shown that the projection onto a Minkowski sum of sets can be represented as the sum of points on constituent sets so that, at these points, all of the sets share the same normal vector which is the negative of the dual solution. For numerically solving the problem, the most suitable algorithm is the one suggested by Gilbert (SIAM J Control 4:61–80, 1966). This algorithm has been widely used in collision detection and path planning in robotics. However, a main drawback of this method is that in some cases, it turns to be very slow as it approaches the solution. In this paper we proposed NESMINO whose \(O\left( \frac{1}{\sqrt{\epsilon }}\ln (\frac{1}{\epsilon })\right) \) complexity bound is better than the worst-case complexity bound of \(O(\frac{1}{\epsilon })\) of Gilbert’s algorithm. PubDate: 2019-12-01

Abstract: Since the initial proposal in the late 80s, spectral gradient methods continue to receive significant attention, especially due to their excellent numerical performance on various large scale applications. However, to date, they have not been sufficiently explored in the context of distributed optimization. In this paper, we consider unconstrained distributed optimization problems where n nodes constitute an arbitrary connected network and collaboratively minimize the sum of their local convex cost functions. In this setting, building from existing exact distributed gradient methods, we propose a novel exact distributed gradient method wherein nodes’ step-sizes are designed according to the novel rules akin to those in spectral gradient methods. We refer to the proposed method as Distributed Spectral Gradient method. The method exhibits R-linear convergence under standard assumptions for the nodes’ local costs and safeguarding on the algorithm step-sizes. We illustrate the method’s performance through simulation examples. PubDate: 2019-12-01

Abstract: We propose two limited-memory BFGS (L-BFGS) trust-region methods for large-scale optimization with linear equality constraints. The methods are intended for problems where the number of equality constraints is small. By exploiting the structure of the quasi-Newton compact representation, both proposed methods solve the trust-region subproblems nearly exactly, even for large problems. We derive theoretical global convergence results of the proposed algorithms, and compare their numerical effectiveness and performance on a variety of large-scale problems. PubDate: 2019-12-01

Abstract: In this paper we consider two relevant optimization problems: the problem of selecting the best sparse linear regression model and the problem of optimally identifying the parameters of auto-regressive models based on time series data. Usually these problems, which although different are indeed related, are solved through a sequence of separate steps, alternating between choosing a subset of features and then finding a best fit regression. In this paper we propose to model both problems as mixed integer non linear optimization ones and propose numerical procedures based on state of the art optimization tools in order to solve both of them. The proposed approach has the advantage of considering both model selection as well as parameter estimation as a single optimization problem. Numerical experiments performed on widely available datasets as well as on synthetic ones confirm the high quality of our approach, both in terms of the quality of the resulting models and in terms of CPU time. PubDate: 2019-12-01

Abstract: Many numerical methods for conic problems use the homogenous primal–dual embedding, which yields a primal–dual solution or a certificate establishing primal or dual infeasibility. Following Themelis and Patrinos (IEEE Trans Autom Control, 2019), we express the embedding as the problem of finding a zero of a mapping containing a skew-symmetric linear function and projections onto cones and their duals. We focus on the special case when this mapping is regular, i.e., differentiable with nonsingular derivative matrix, at a solution point. While this is not always the case, it is a very common occurrence in practice. In this paper we do not aim for new theorerical results. We rather propose a simple method that uses LSQR, a variant of conjugate gradients for least squares problems, and the derivative of the residual mapping to refine an approximate solution, i.e., to increase its accuracy. LSQR is a matrix-free method, i.e., requires only the evaluation of the derivative mapping and its adjoint, and so avoids forming or storing large matrices, which makes it efficient even for cone problems in which the data matrices are given and dense, and also allows the method to extend to cone programs in which the data are given as abstract linear operators. Numerical examples show that the method improves an approximate solution of a conic program, and often dramatically, at a computational cost that is typically small compared to the cost of obtaining the original approximate solution. For completeness we describe methods for computing the derivative of the projection onto the cones commonly used in practice: nonnegative, second-order, semidefinite, and exponential cones. The paper is accompanied by an open source implementation. PubDate: 2019-12-01

Abstract: We consider the stochastic variational inequality problem in which the map is expectation-valued in a component-wise sense. Much of the available convergence theory and rate statements for stochastic approximation schemes are limited to monotone maps. However, non-monotone stochastic variational inequality problems are not uncommon and are seen to arise from product pricing, fractional optimization problems, and subclasses of economic equilibrium problems. Motivated by the need to address a broader class of maps, we make the following contributions: (1) we present an extragradient-based stochastic approximation scheme and prove that the iterates converge to a solution of the original problem under either pseudomonotonicity requirements or a suitably defined acute angle condition. Such statements are shown to be generalizable to the stochastic mirror-prox framework; (2) under strong pseudomonotonicity, we show that the mean-squared error in the solution iterates produced by the extragradient SA scheme converges at the optimal rate of \({{\mathcal {O}}}\left( \frac{1}{{K}}\right) \), statements that were hitherto unavailable in this regime. Notably, we optimize the initial steplength by obtaining an \(\epsilon \)-infimum of a discontinuous nonconvex function. Similar statements are derived for mirror-prox generalizations and can accommodate monotone SVIs under a weak-sharpness requirement. Finally, both the asymptotics and the empirical rates of the schemes are studied on a set of variational problems where it is seen that the theoretically specified initial steplength leads to significant performance benefits. PubDate: 2019-12-01

Abstract: The alternating direction method of multipliers (ADMM) is being widely used in a variety of areas; its different variants tailored for different application scenarios have also been deeply researched in the literature. Among them, the linearized ADMM has received particularly wide attention in many areas because of its efficiency and easy implementation. To theoretically guarantee convergence of the linearized ADMM, the step size for the linearized subproblems, or the reciprocal of the linearization parameter, should be sufficiently small. On the other hand, small step sizes decelerate the convergence numerically. Hence, it is interesting to probe the optimal (largest) value of the step size that guarantees convergence of the linearized ADMM. This analysis is lacked in the literature. In this paper, we provide a rigorous mathematical analysis for finding this optimal step size of the linearized ADMM and accordingly set up the optimal version of the linearized ADMM in the convex programming context. The global convergence and worst-case convergence rate measured by the iteration complexity of the optimal version of linearized ADMM are proved as well. PubDate: 2019-11-15

Abstract: The elementary Euclidean concept of circumcenter has recently been employed to improve two aspects of the classical Douglas–Rachford method for projecting onto the intersection of affine subspaces. The so-called circumcentered–reflection method is able to both accelerate the average reflection scheme by the Douglas–Rachford method and cope with the intersection of more than two affine subspaces. We now introduce the technique of circumcentering in blocks, which, more than just an option over the basic algorithm of circumcenters, turns out to be an elegant manner of generalizing the method of alternating projections. Linear convergence for this novel block-wise circumcenter framework is derived and illustrated numerically. Furthermore, we prove that the original circumcentered–reflection method essentially finds the best approximation solution in one single step if the given affine subspaces are hyperplanes. PubDate: 2019-11-15

Abstract: We consider a cutting-plane algorithm for solving mixed-integer semidefinite optimization (MISDO) problems. In this algorithm, the positive semidefinite (psd) constraint is relaxed, and the resultant mixed-integer linear optimization problem is solved repeatedly, imposing at each iteration a valid inequality for the psd constraint. We prove the convergence properties of the algorithm. Moreover, to speed up the computation, we devise a branch-and-cut algorithm, in which valid inequalities are dynamically added during a branch-and-bound procedure. We test the computational performance of our cutting-plane and branch-and-cut algorithms for three types of MISDO problem: random instances, computing restricted isometry constants, and robust truss topology design. Our experimental results demonstrate that, for many problem instances, our branch-and-cut algorithm delivered superior performance compared with general-purpose MISDO solvers in terms of computational efficiency and stability. PubDate: 2019-11-14

Abstract: The multi-objective spanning tree (MoST) is an extension of the minimum spanning tree problem (MST) that, as well as its single-objective counterpart, arises in several practical applications. However, unlike the MST, for which there are polynomial-time algorithms that solve it, the MoST is NP-hard. Several researchers proposed techniques to solve the MoST, each of those methods with specific potentialities and limitations. In this study, we examine those methods and divide them into two categories regarding their outcomes: Pareto optimal sets and Pareto optimal fronts. To compare the techniques from the two groups, we investigated their behavior on 2, 3 and 4-objective instances from different classes. We report the results of a computational experiment on 8100 complete and grid graphs in which we analyze specific features of each algorithm as well as the computational effort required to solve the instances. PubDate: 2019-11-13

Abstract: This paper concerns saddle points of rational functions, under general constraints. Based on optimality conditions, we propose an algorithm for computing saddle points. It uses Lasserre’s hierarchy of semidefinite relaxation. The algorithm can get a saddle point if it exists, or it can detect its nonexistence if it does not. Numerical experiments show that the algorithm is efficient for computing saddle points of rational functions. PubDate: 2019-11-13

Abstract: The weighted Euclidean one-center (WEOC) problem is one of the classic problems in facility location theory, which is also a generalization of the minimum enclosing ball (MEB) problem. Given m points in \({\mathbb {R}}^{n}\), the WEOC problem computes a center point \(c\in {\mathbb {R}}^{n}\) that minimizes the maximum weighted Euclidean distance to m given points. The rank-two update algorithm is an effective method for solving the minimum volume enclosing ellipsoid (MVEE) problem. It updates only two components of the solution at each iteration, which was previously proposed in Cong et al. (Comput Optim Appl 51(1):241–257, 2012). In this paper, we further develop and analyze the rank-two update algorithm for solving the WEOC problem. At each iteration, the calculation of the optimal step-size for the WEOC problem needs to distinguish four different cases, which is a challenge in comparison with the MVEE problem. We establish the theoretical results of the complexity and the core set size of the rank-two update algorithm for the WEOC problem, which are the generalizations of the currently best-known results for the MEB problem. In addition, by constructing an important inequality for the WEOC problem, we establish the linear convergence of this rank-two update algorithm. Numerical experiments show that the rank-two update algorithm is comparable to the Frank–Wolfe algorithm with away steps for the WEOC problem. In particular, the rank-two update algorithm is more efficient than the Frank–Wolfe algorithm with away steps for problem instances with \(m\gg n\) under high precision. PubDate: 2019-11-07

Abstract: Many real-world applications can usually be modeled as convex quadratic problems. In the present paper, we want to tackle a specific class of quadratic programs having a dense Hessian matrix and a structured feasible set. We hence carefully analyze a simplicial decomposition like algorithmic framework that handles those problems in an effective way. We introduce a new master solver, called Adaptive Conjugate Direction Method, and embed it in our framework. We also analyze the interaction of some techniques for speeding up the solution of the pricing problem. We report extensive numerical experiments based on a benchmark of almost 1400 instances from specific and generic quadratic problems. We show the efficiency and robustness of the method when compared to a commercial solver (Cplex). PubDate: 2019-11-05