Authors:Donald Goldfarb; Cun Mu; John Wright; Chaoxu Zhou Pages: 479 - 502 Abstract: Minimization methods that search along a curvilinear path composed of a non-ascent negative curvature direction in addition to the direction of steepest descent, dating back to the late 1970s, have been an effective approach to finding a stationary point of a function at which its Hessian is positive semidefinite. For constrained nonlinear programs arising from recent applications, the primary goal is to find a stationary point that satisfies the second-order necessary optimality conditions. Motivated by this, we generalize the approach of using negative curvature directions from unconstrained optimization to equality constrained problems and prove that our proposed negative curvature method is guaranteed to converge to a stationary point satisfying second-order necessary conditions. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9925-6 Issue No:Vol. 68, No. 3 (2017)

Authors:Chengjing Wang; Peipei Tang Pages: 503 - 532 Abstract: In this paper, we propose a primal majorized semismooth Newton-CG augmented Lagrangian method for large-scale linearly constrained convex programming problems, especially for some difficult problems. The basic idea of this method is to apply the majorized semismooth Newton-CG augmented Lagrangian method to the primal convex problem. And we take two special nonlinear semidefinite programming problems as examples to illustrate the algorithm. Furthermore, we establish the global convergence and the iteration complexity of the algorithm. Numerical experiments demonstrate that our method works very well for the testing problems, especially for many ill-conditioned ones. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9930-9 Issue No:Vol. 68, No. 3 (2017)

Authors:E. Bergou; Y. Diouane; S. Gratton Pages: 533 - 554 Abstract: We consider solving unconstrained optimization problems by means of two popular globalization techniques: trust-region (TR) algorithms and adaptive regularized framework using cubics (ARC). Both techniques require the solution of a so-called “subproblem” in which a trial step is computed by solving an optimization problem involving an approximation of the objective function, called “the model”. The latter is supposed to be adequate in a neighborhood of the current iterate. In this paper, we address an important practical question related with the choice of the norm for defining the neighborhood. More precisely, assuming here that the Hessian B of the model is symmetric positive definite, we propose the use of the so-called “energy norm”—defined by \(\Vert x\Vert _B= \sqrt{x^TBx}\) for all \(x \in \mathbb {R}^n\) —in both TR and ARC techniques. We show that the use of this norm induces remarkable relations between the trial step of both methods that can be used to obtain efficient practical algorithms. We furthermore consider the use of truncated Krylov subspace methods to obtain an approximate trial step for large scale optimization. Within the energy norm, we obtain line search algorithms along the Newton direction, with a special backtracking strategy and an acceptability condition in the spirit of TR/ARC methods. The new line search algorithm, derived by ARC, enjoys a worst-case iteration complexity of \(\mathcal {O}(\epsilon ^{-3/2})\) . We show the good potential of the energy norm on a set of numerical experiments. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9929-2 Issue No:Vol. 68, No. 3 (2017)

Authors:Geovani N. Grapiglia; Ekkehard W. Sachs Pages: 555 - 577 Abstract: A general class of non-monotone line search algorithms has been proposed by Sachs and Sachs (Control Cybern 40:1059–1075, 2011) for smooth unconstrained optimization, generalizing various non-monotone step size rules such as the modified Armijo rule of Zhang and Hager (SIAM J Optim 14:1043–1056, 2004). In this paper, the worst-case complexity of this class of non-monotone algorithms is studied. The analysis is carried out in the context of non-convex, convex and strongly convex objectives with Lipschitz continuous gradients. Despite de nonmonotonicity in the decrease of function values, the complexity bounds obtained agree in order with the bounds already established for monotone algorithms. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9928-3 Issue No:Vol. 68, No. 3 (2017)

Authors:Xiao Wang; Shuxiong Wang; Hongchao Zhang Pages: 579 - 618 Abstract: We study an inexact proximal stochastic gradient (IPSG) method for convex composite optimization, whose objective function is a summation of an average of a large number of smooth convex functions and a convex, but possibly nonsmooth, function. Variance reduction techniques are incorporated in the method to reduce the stochastic gradient variance. The main feature of this IPSG algorithm is to allow solving the proximal subproblems inexactly while still keeping the global convergence with desirable complexity bounds. Different subproblem stopping criteria are proposed. Global convergence and the component gradient complexity bounds are derived for the both cases when the objective function is strongly convex or just generally convex. Preliminary numerical experiment shows the overall efficiency of the IPSG algorithm. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9932-7 Issue No:Vol. 68, No. 3 (2017)

Authors:Zhaosong Lu; Yong Zhang; Jian Lu Pages: 619 - 642 Abstract: In this paper we study the \(\ell _p\) (or Schatten-p quasi-norm) regularized low-rank approximation problems. In particular, we introduce a class of first-order stationary points for them and show that any local minimizer of these problems must be a first-order stationary point. In addition, we derive lower bounds for the nonzero singular values of the first-order stationary points and hence also of the local minimizers of these problems. The iterative reweighted singular value minimization (IRSVM) methods are then proposed to solve these problems, whose subproblems are shown to have a closed-form solution. Compared to the analogous methods for the \(\ell _p\) regularized vector minimization problems, the convergence analysis of these methods is significantly more challenging. We develop a novel approach to establishing the convergence of the IRSVM methods, which makes use of the expression of a specific solution of their subproblems and avoids the intricate issue of finding the explicit expression for the Clarke subdifferential of the objective of their subproblems. In particular, we show that any accumulation point of the sequence generated by the IRSVM methods is a first-order stationary point of the problems. Our computational results demonstrate that the IRSVM methods generally outperform the recently developed iterative reweighted least squares methods in terms of solution quality and/or speed. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9933-6 Issue No:Vol. 68, No. 3 (2017)

Authors:Mingqiang Li; Congying Han; Ruxin Wang; Tiande Guo Pages: 643 - 660 Abstract: Total variation regularization introduced by Rudin, Osher, and Fatemi (ROF) is widely used in image denoising problems for its capability to preserve repetitive textures and details of images. Many efforts have been devoted to obtain efficient gradient descent schemes for dual minimization of ROF model, such as Chambolle’s algorithm or gradient projection (GP) algorithm. In this paper, we propose a general gradient descent algorithm with a shrinking factor. Both Chambolle’s and GP algorithm can be regarded as the special cases of the proposed methods with special parameters. Global convergence analysis of the new algorithms with various step lengths and shrinking factors are present. Numerical results demonstrate their competitiveness in computational efficiency and reconstruction quality with some existing classic algorithms on a set of gray scale images. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9931-8 Issue No:Vol. 68, No. 3 (2017)

Authors:André Berger; Alexander Grigoriev; Andrej Winokurow Pages: 661 - 669 Abstract: The single facility location problem with demand regions seeks for a facility location minimizing the sum of the distances from n demand regions to the facility. The demand regions represent sales markets where the transportation costs are negligible. In this paper, we assume that all demand regions are disks of the same radius, and the distances are measured by a rectilinear norm, e.g. \(\ell _1\) or \(\ell _\infty \) . We develop an exact combinatorial algorithm running in time \(O(n\log ^c n)\) for some c dependent only on the space dimension. The algorithm is generalizable to the other polyhedral norms. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9935-4 Issue No:Vol. 68, No. 3 (2017)

Authors:Peter A. Graf; Stephen Billups Pages: 671 - 687 Abstract: Computational materials design has suffered from a lack of algorithms formulated in terms of experimentally accessible variables. Here we formulate the problem of (ternary) alloy optimization at the level of choice of atoms and their composition that is normal for synthesists. Mathematically, this is a mixed integer problem where a candidate solution consists of a choice of three elements, and how much of each of them to use. This space has the natural structure of a set of equilateral triangles. We solve this problem by introducing a novel version of the DIRECT algorithm that (1) operates on equilateral triangles instead of rectangles and (2) works across multiple triangles. We demonstrate on a test case that the algorithm is both robust and efficient. Finally, we offer an explanation of the efficacy of DIRECT—specifically, its balance of global and local search—by showing that “potentially optimal rectangles” of the original algorithm are akin to the Pareto front of the “multi-component optimization” of global and local search. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9922-9 Issue No:Vol. 68, No. 3 (2017)

Authors:Simone Sagratella Pages: 689 - 717 Abstract: We consider generalized potential games, that constitute a fundamental subclass of generalized Nash equilibrium problems. We propose different methods to compute solutions of generalized potential games with mixed-integer variables, i.e., games in which some variables are continuous while the others are discrete. We investigate which types of equilibria of the game can be computed by minimizing a potential function over the common feasible set. In particular, for a wide class of generalized potential games, we characterize those equilibria that can be computed by minimizing potential functions as Pareto solutions of a particular multi-objective problem, and we show how different potential functions can be used to select equilibria. We propose a new Gauss–Southwell algorithm to compute approximate equilibria of any generalized potential game with mixed-integer variables. We show that this method converges in a finite number of steps and we also give an upper bound on this number of steps. Moreover, we make a thorough analysis on the behaviour of approximate equilibria with respect to exact ones. Finally, we make many numerical experiments to show the viability of the proposed approaches. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9927-4 Issue No:Vol. 68, No. 3 (2017)

Authors:C. Yalçın Kaya Pages: 719 - 747 Abstract: Markov–Dubins path is the shortest planar curve joining two points with prescribed tangents, with a specified bound on its curvature. Its structure, as proved by Dubins in 1957, nearly 70 years after Markov posed the problem of finding it, is elegantly simple: a selection of at most three arcs are concatenated, each of which is either a circular arc of maximum (prescribed) curvature or a straight line. The Markov–Dubins problem and its variants have since been extensively studied in practical and theoretical settings. A reformulation of the Markov–Dubins problem as an optimal control problem was subsequently studied by various researchers using the Pontryagin maximum principle and additional techniques, to reproduce Dubins’ result. In the present paper, we study the same reformulation, and apply the maximum principle, with new insights, to derive Dubins’ result again. We prove that abnormal control solutions do exist. We characterize these solutions, which were not studied adequately in the literature previously, as a concatenation of at most two circular arcs and show that they are also solutions of the normal problem. Moreover, we prove that any feasible path of the types mentioned in Dubins’ result is a stationary solution, i.e., that it satisfies the Pontryagin maximum principle. We propose a numerical method for computing Markov–Dubins path. We illustrate the theory and the numerical approach by three qualitatively different examples. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9923-8 Issue No:Vol. 68, No. 3 (2017)

Authors:Mercedes Landete; Alfredo Marín; José Luis Sainz-Pardo Pages: 749 - 773 Abstract: Decomposition methods for optimal spanning trees on graphs are explored in this work. The attention is focused on optimization problems where the objective function depends only on the degrees of the nodes of the tree. In particular, we deal with the Minimum Leaves problem, the Minimum Branch Vertices problem and the Minimum Degree Sum problem. The decomposition is carried out by identifying the articulation vertices of the graph and then its blocks, solving certain subproblems on the blocks and then bringing together the optimal sub-solutions following adequate procedures. Computational results obtained using similar Integer Programming formulations for both the original and the decomposed problems show the advantage of the proposed methods on decomposable graphs. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9924-7 Issue No:Vol. 68, No. 3 (2017)

Authors:Jesús Sánchez-Oro; Anna Martínez-Gavara; Manuel Laguna; Rafael Martí; Abraham Duarte Pages: 775 - 797 Abstract: Automated graph-drawing systems utilize procedures to place vertices and arcs in order to produce graphs with desired properties. Incremental or dynamic procedures are those that preserve key characteristics when updating an existing drawing. These methods are particularly useful in areas such as planning and logistics, where updates are frequent. We propose a procedure based on the scatter search methodology that is adapted to the incremental drawing problem in hierarchical graphs. These drawings can be used to represent any acyclic graph. Comprehensive computational experiments are used to test the efficiency and effectiveness of the proposed procedure. PubDate: 2017-12-01 DOI: 10.1007/s10589-017-9926-5 Issue No:Vol. 68, No. 3 (2017)

Authors:Jianchao Bai; Jicheng Li; Fengmin Xu; Hongchao Zhang Abstract: The alternating direction method of multipliers (ADMM) has been proved to be effective for solving separable convex optimization subject to linear constraints. In this paper, we propose a generalized symmetric ADMM (GS-ADMM), which updates the Lagrange multiplier twice with suitable stepsizes, to solve the multi-block separable convex programming. This GS-ADMM partitions the data into two group variables so that one group consists of p block variables while the other has q block variables, where \(p \ge 1\) and \(q \ge 1\) are two integers. The two grouped variables are updated in a Gauss–Seidel scheme, while the variables within each group are updated in a Jacobi scheme, which would make it very attractive for a big data setting. By adding proper proximal terms to the subproblems, we specify the domain of the stepsizes to guarantee that GS-ADMM is globally convergent with a worst-case \({\mathcal {O}}(1/t)\) ergodic convergence rate. It turns out that our convergence domain of the stepsizes is significantly larger than other convergence domains in the literature. Hence, the GS-ADMM is more flexible and attractive on choosing and using larger stepsizes of the dual variable. Besides, two special cases of GS-ADMM, which allows using zero penalty terms, are also discussed and analyzed. Compared with several state-of-the-art methods, preliminary numerical experiments on solving a sparse matrix minimization problem in the statistical learning show that our proposed method is effective and promising. PubDate: 2017-11-24 DOI: 10.1007/s10589-017-9971-0

Authors:Hiva Ghanbari; Katya Scheinberg Abstract: A general, inexact, efficient proximal quasi-Newton algorithm for composite optimization problems has been proposed by Scheinberg and Tang (Math Program 160:495–529, 2016) and a sublinear global convergence rate has been established. In this paper, we analyze the global convergence rate of this method, in the both exact and inexact settings, in the case when the objective function is strongly convex. We also investigate a practical variant of this method by establishing a simple stopping criterion for the subproblem optimization. Furthermore, we consider an accelerated variant, based on FISTA of Beck and Teboulle (SIAM 2:183–202, 2009), to the proximal quasi-Newton algorithm. Jiang et al. (SIAM 22:1042–1064, 2012) considered a similar accelerated method, where the convergence rate analysis relies on very strong impractical assumptions on Hessian estimates. We present a modified analysis while relaxing these assumptions and perform a numerical comparison of the accelerated proximal quasi-Newton algorithm and the regular one. Our analysis and computational results show that acceleration may not bring any benefit in the quasi-Newton setting. PubDate: 2017-11-20 DOI: 10.1007/s10589-017-9964-z

Authors:Markus Leitner; Ivana Ljubić; Martin Luipersbeck; Markus Sinnl Abstract: A new algorithmic approach for solving the stochastic Steiner tree problem based on three procedures for computing lower bounds (dual ascent, Lagrangian relaxation, Benders decomposition) is introduced. Our method is derived from a new integer linear programming formulation, which is shown to be strongest among all known formulations. The resulting method, which relies on an interplay of the dual information retrieved from the respective dual procedures, computes upper and lower bounds and combines them with several rules for fixing variables in order to decrease the size of problem instances. The effectiveness of our method is compared in an extensive computational study with the state-of-the-art exact approach, which employs a Benders decomposition based on two-stage branch-and-cut, and a genetic algorithm introduced during the DIMACS implementation challenge on Steiner trees. Our results indicate that the presented method significantly outperforms existing ones, both on benchmark instances from literature, as well as on large-scale telecommunication networks. PubDate: 2017-11-20 DOI: 10.1007/s10589-017-9966-x

Authors:Jamie Fairbrother; Adam N. Letchford; Keith Briggs Abstract: In the k -partition problem (k-PP), one is given an edge-weighted undirected graph, and one must partition the node set into at most k subsets, in order to minimise (or maximise) the total weight of the edges that have their end-nodes in the same subset. Various hierarchical variants of this problem have been studied in the context of data mining. We consider a ‘two-level’ variant that arises in mobile wireless communications. We show that an exact algorithm based on intelligent preprocessing, cutting planes and symmetry-breaking is capable of solving small- and medium-size instances to proven optimality, and providing strong lower bounds for larger instances. PubDate: 2017-11-18 DOI: 10.1007/s10589-017-9967-9

Authors:José F. S. Bravo Ferreira; Yuehaw Khoo; Amit Singer Abstract: The matching problem between two adjacency matrices can be formulated as the NP-hard quadratic assignment problem (QAP). Previous work on semidefinite programming (SDP) relaxations to the QAP have produced solutions that are often tight in practice, but such SDPs typically scale badly, involving matrix variables of dimension \(n^2\) where n is the number of nodes. To achieve a speed up, we propose a further relaxation of the SDP involving a number of positive semidefinite matrices of dimension \(\mathcal {O}(n)\) no greater than the number of edges in one of the graphs. The relaxation can be further strengthened by considering cliques in the graph, instead of edges. The dual problem of this novel relaxation has a natural three-block structure that can be solved via a convergent Alternating Direction Method of Multipliers in a distributed manner, where the most expensive step per iteration is computing the eigendecomposition of matrices of dimension \(\mathcal {O}(n)\) . The new SDP relaxation produces strong bounds on quadratic assignment problems where one of the graphs is sparse with reduced computational complexity and running times, and can be used in the context of nuclear magnetic resonance spectroscopy to tackle the assignment problem. PubDate: 2017-11-17 DOI: 10.1007/s10589-017-9968-8

Authors:Veronika Karl; Daniel Wachsmuth Abstract: In the present work we apply an augmented Lagrange method to solve pointwise state constrained elliptic optimal control problems. We prove strong convergence of the primal variables as well as weak convergence of the adjoint states and weak-* convergence of the multipliers associated to the state constraint. In addition, we show that the sequence of generated penalty parameters is bounded only in exceptional situations, which is different from classical results in finite-dimensional optimization. In addition, numerical results are presented. PubDate: 2017-11-16 DOI: 10.1007/s10589-017-9965-y