Authors:Lorenzo Stella; Andreas Themelis; Panagiotis Patrinos Pages: 443 - 487 Abstract: The forward–backward splitting method (FBS) for minimizing a nonsmooth composite function can be interpreted as a (variable-metric) gradient method over a continuously differentiable function which we call forward–backward envelope (FBE). This allows to extend algorithms for smooth unconstrained optimization and apply them to nonsmooth (possibly constrained) problems. Since the FBE can be computed by simply evaluating forward–backward steps, the resulting methods rely on a similar black-box oracle as FBS. We propose an algorithmic scheme that enjoys the same global convergence properties of FBS when the problem is convex, or when the objective function possesses the Kurdyka–Łojasiewicz property at its critical points. Moreover, when using quasi-Newton directions the proposed method achieves superlinear convergence provided that usual second-order sufficiency conditions on the FBE hold at the limit point of the generated sequence. Such conditions translate into milder requirements on the original function involving generalized second-order differentiability. We show that BFGS fits our framework and that the limited-memory variant L-BFGS is well suited for large-scale problems, greatly outperforming FBS or its accelerated version in practice, as well as ADMM and other problem-specific solvers. The analysis of superlinear convergence is based on an extension of the Dennis and Moré theorem for the proposed algorithmic scheme. PubDate: 2017-07-01 DOI: 10.1007/s10589-017-9912-y Issue No:Vol. 67, No. 3 (2017)

Authors:Tianxiang Liu; Ting Kei Pong Pages: 489 - 520 Abstract: In this paper, we further study the forward–backward envelope first introduced in Patrinos and Bemporad (Proceedings of the IEEE Conference on Decision and Control, pp 2358–2363, 2013) and Stella et al. (Comput Optim Appl, doi:10.1007/s10589-017-9912-y, 2017) for problems whose objective is the sum of a proper closed convex function and a twice continuously differentiable possibly nonconvex function with Lipschitz continuous gradient. We derive sufficient conditions on the original problem for the corresponding forward–backward envelope to be a level-bounded and Kurdyka–Łojasiewicz function with an exponent of \(\frac{1}{2}\) ; these results are important for the efficient minimization of the forward–backward envelope by classical optimization algorithms. In addition, we demonstrate how to minimize some difference-of-convex regularized least squares problems by minimizing a suitably constructed forward–backward envelope. Our preliminary numerical results on randomly generated instances of large-scale \(\ell _{1-2}\) regularized least squares problems (Yin et al. in SIAM J Sci Comput 37:A536–A563, 2015) illustrate that an implementation of this approach with a limited-memory BFGS scheme usually outperforms standard first-order methods such as the nonmonotone proximal gradient method in Wright et al. (IEEE Trans Signal Process 57:2479–2493, 2009). PubDate: 2017-07-01 DOI: 10.1007/s10589-017-9900-2 Issue No:Vol. 67, No. 3 (2017)

Authors:Daniel O’Connor; Lieven Vandenberghe Pages: 521 - 541 Abstract: Image deblurring techniques based on convex optimization formulations, such as total-variation deblurring, often use specialized first-order methods for large-scale nondifferentiable optimization. A key property exploited in these methods is spatial invariance of the blurring operator, which makes it possible to use the fast Fourier transform (FFT) when solving linear equations involving the operator. In this paper we extend this approach to two popular models for space-varying blurring operators, the Nagy–O’Leary model and the efficient filter flow model. We show how splitting methods derived from the Douglas–Rachford algorithm can be implemented with a low complexity per iteration, dominated by a small number of FFTs. PubDate: 2017-07-01 DOI: 10.1007/s10589-017-9901-1 Issue No:Vol. 67, No. 3 (2017)

Authors:Dingtao Peng; Naihua Xiu; Jian Yu Pages: 543 - 569 Abstract: The affine rank minimization problem is to minimize the rank of a matrix under linear constraints. It has many applications in various areas such as statistics, control, system identification and machine learning. Unlike the literatures which use the nuclear norm or the general Schatten \(q~ (0<q<1)\) quasi-norm to approximate the rank of a matrix, in this paper we use the Schatten 1 / 2 quasi-norm approximation which is a better approximation than the nuclear norm but leads to a nonconvex, nonsmooth and non-Lipschitz optimization problem. It is important that we give a global necessary optimality condition for the \(S_{1/2}\) regularization problem by virtue of the special objective function. This is very different from the local optimality conditions usually used for the general \(S_q\) regularization problems. Explicitly, the global necessary optimality condition for the \(S_{1/2}\) regularization problem is a fixed point inclusion associated with the singular value half thresholding operator. Naturally, we propose a fixed point iterative scheme for the problem. We also provide the convergence analysis of this iteration. By discussing the location and setting of the optimal regularization parameter as well as using an approximate singular value decomposition procedure, we get a very efficient algorithm, half norm fixed point algorithm with an approximate SVD (HFPA algorithm), for the \(S_{1/2}\) regularization problem. Numerical experiments on randomly generated and real matrix completion problems are presented to demonstrate the effectiveness of the proposed algorithm. PubDate: 2017-07-01 DOI: 10.1007/s10589-017-9898-5 Issue No:Vol. 67, No. 3 (2017)

Authors:Dang Van Hieu Pages: 571 - 594 Abstract: In this paper, three parallel hybrid subgradient extragradient algorithms are proposed for finding a common solution of a finite family of equilibrium problems in Hilbert spaces. The proposed algorithms originate from previously known results for variational inequalities and can be considered as modifications of extragradient methods for equilibrium problems. Theorems of strong convergence are established under the standard assumptions imposed on bifunctions. Some numerical experiments are given to illustrate the convergence of the new algorithms and to compare their behavior with other algorithms. PubDate: 2017-07-01 DOI: 10.1007/s10589-017-9899-4 Issue No:Vol. 67, No. 3 (2017)

Authors:Christian Kanzow; Yekini Shehu Pages: 595 - 620 Abstract: The Krasnoselskii–Mann iteration plays an important role in the approximation of fixed points of nonexpansive operators; it is known to be weakly convergent in the infinite dimensional setting. In this present paper, we provide a new inexact Krasnoselskii–Mann iteration and prove weak convergence under certain accuracy criteria on the error resulting from the inexactness. We also show strong convergence for a modified inexact Krasnoselskii–Mann iteration under suitable assumptions. The convergence results generalize existing ones from the literature. Applications are given to the Douglas–Rachford splitting method, the Fermat–Weber location problem as well as the alternating projection method by John von Neumann. PubDate: 2017-07-01 DOI: 10.1007/s10589-017-9902-0 Issue No:Vol. 67, No. 3 (2017)

Authors:J. C. De Los Reyes; E. Loayza; P. Merino Pages: 225 - 258 Abstract: We present a second order algorithm, based on orthantwise directions, for solving optimization problems involving the sparsity enhancing \(\ell _1\) -norm. The main idea of our method consists in modifying the descent orthantwise directions by using second order information both of the regular term and (in weak sense) of the \(\ell _1\) -norm. The weak second order information behind the \(\ell _1\) -term is incorporated via a partial Huber regularization. One of the main features of our algorithm consists in a faster identification of the active set. We also prove that a reduced version of our method is equivalent to a semismooth Newton algorithm applied to the optimality condition, under a specific choice of the algorithm parameters. We present several computational experiments to show the efficiency of our approach compared to other state-of-the-art algorithms. PubDate: 2017-06-01 DOI: 10.1007/s10589-017-9891-z Issue No:Vol. 67, No. 2 (2017)

Authors:Patrick R. Johnstone; Pierre Moulin Pages: 259 - 292 Abstract: This paper is concerned with convex composite minimization problems in a Hilbert space. In these problems, the objective is the sum of two closed, proper, and convex functions where one is smooth and the other admits a computationally inexpensive proximal operator. We analyze a family of generalized inertial proximal splitting algorithms (GIPSA) for solving such problems. We establish weak convergence of the generated sequence when the minimum is attained. Our analysis unifies and extends several previous results. We then focus on \(\ell _1\) -regularized optimization, which is the ubiquitous special case where the nonsmooth term is the \(\ell _1\) -norm. For certain parameter choices, GIPSA is amenable to a local analysis for this problem. For these choices we show that GIPSA achieves finite “active manifold identification”, i.e. convergence in a finite number of iterations to the optimal support and sign, after which GIPSA reduces to minimizing a local smooth function. We prove local linear convergence under either restricted strong convexity or a strict complementarity condition. We determine the rate in terms of the inertia, stepsize, and local curvature. Our local analysis is applicable to certain recent variants of the Fast Iterative Shrinkage–Thresholding Algorithm (FISTA), for which we establish active manifold identification and local linear convergence. Based on our analysis we propose a momentum restart scheme in these FISTA variants to obtain the optimal local linear convergence rate while maintaining desirable global properties. PubDate: 2017-06-01 DOI: 10.1007/s10589-017-9896-7 Issue No:Vol. 67, No. 2 (2017)

Authors:Teobaldo Bulhões; Anand Subramanian; Gilberto F. Sousa Filho; Lucídio dos Anjos F. Cabral Pages: 293 - 316 Abstract: Given an input graph, the p-cluster editing problem consists of minimizing the number of editions, i.e., additions and/or deletions of edges, so as to create p vertex-disjoint cliques (clusters). In order to solve this \({\mathscr {NP}}\) -hard problem, we propose a branch-and-price algorithm over a set partitioning based formulation with exponential number of variables. We show that this formulation theoretically dominates the best known formulation for the problem. Moreover, we compare the performance of three mathematical formulations for the pricing subproblem, which is strongly \({\mathscr {NP}}\) -hard. A heuristic algorithm is also proposed to speedup the column generation procedure. We report improved bounds for benchmark instances available in the literature. PubDate: 2017-06-01 DOI: 10.1007/s10589-017-9893-x Issue No:Vol. 67, No. 2 (2017)

Authors:Frank E. Curtis; Arvind U. Raghunathan Pages: 317 - 360 Abstract: An algorithm for solving nearly-separable quadratic optimization problems (QPs) is presented. The approach is based on applying a semismooth Newton method to solve the implicit complementarity problem arising as the first-order stationarity conditions of such a QP. An important feature of the approach is that, as in dual decomposition methods, separability of the dual function of the QP can be exploited in the search direction computation. Global convergence of the method is promoted by enforcing decrease in component(s) of a Fischer–Burmeister formulation of the complementarity conditions, either via a merit function or through a filter mechanism. The results of numerical experiments when solving convex and nonconvex instances are provided to illustrate the efficacy of the method. PubDate: 2017-06-01 DOI: 10.1007/s10589-017-9895-8 Issue No:Vol. 67, No. 2 (2017)

Authors:Matúš Benko; Helmut Gfrerer Pages: 361 - 399 Abstract: We propose an SQP algorithm for mathematical programs with vanishing constraints which solves at each iteration a quadratic program with linear vanishing constraints. The algorithm is based on the newly developed concept of \({\mathcal {Q}}\) -stationarity (Benko and Gfrerer in Optimization 66(1):61–92, 2017). We demonstrate how \({\mathcal {Q}}_M\) -stationary solutions of the quadratic program can be obtained. We show that all limit points of the sequence of iterates generated by the basic SQP method are at least M-stationary and by some extension of the method we also guarantee the stronger property of \({\mathcal {Q}}_M\) -stationarity of the limit points. PubDate: 2017-06-01 DOI: 10.1007/s10589-017-9894-9 Issue No:Vol. 67, No. 2 (2017)

Authors:Richard C. Barnard; Christian Clason Pages: 401 - 419 Abstract: This work is concerned with a class of PDE-constrained optimization problems that are motivated by an application in radiotherapy treatment planning. Here the primary design objective is to minimize the volume where a functional of the state violates a prescribed level, but prescribing these levels in the form of pointwise state constraints leads to infeasible problems. We therefore propose an alternative approach based on \(L^1\) penalization of the violation that is also applicable when state constraints are infeasible. We establish well-posedness of the corresponding optimal control problem, derive first-order optimality conditions, discuss convergence of minimizers as the penalty parameter tends to infinity, and present a semismooth Newton method for their efficient numerical solution. The performance of this method for a model problem is illustrated and contrasted with an alternative approach based on (regularized) state constraints. PubDate: 2017-06-01 DOI: 10.1007/s10589-017-9897-6 Issue No:Vol. 67, No. 2 (2017)

Authors:Anuj Bajaj; Warren Hare; Yves Lucet Pages: 421 - 442 Abstract: Computing explicitly the \(\varepsilon \) -subdifferential of a proper function amounts to computing the level set of a convex function namely the conjugate minus a linear function. The resulting theoretical algorithm is applied to the the class of (convex univariate) piecewise linear–quadratic functions for which existing numerical libraries allow practical computations. We visualize the results in a primal, dual, and subdifferential views through several numerical examples. We also provide a visualization of the Brøndsted–Rockafellar theorem. PubDate: 2017-06-01 DOI: 10.1007/s10589-017-9892-y Issue No:Vol. 67, No. 2 (2017)

Authors:B. Jadamba; A. Khan; M. Sama Pages: 39 - 71 Abstract: In this paper, we study new aspects of the integral contraint regularization of state-constrained elliptic control problems (Jadamba et al. in Syst Control Lett 61(6):707–713, 2012). Besides giving new results on the regularity and the convergence of the regularized controls and associated Lagrange multipliers, the main objective of this paper is to give abstract error estimates for the regularization error. We also consider a discretization of the regularized problems and derive numerical estimates which are uniform with respect to the regularization parameter and the discretization parameter. As an application of these results, we prove that this discretization is indeed a full discretization of the original problem defined in terms of a problem with finitely many integral constraints. Detailed numerical results justifying the theoretical findings as well as a comparison of our work with the existing literature is also given. PubDate: 2017-05-01 DOI: 10.1007/s10589-016-9885-2 Issue No:Vol. 67, No. 1 (2017)

Authors:Porfirio Suñagua; Aurelio R. L. Oliveira Pages: 111 - 127 Abstract: The class of splitting preconditioners for the iterative solution of linear systems arising from Mehrotra’s predictor-corrector method for large scale linear programming problems needs to find a basis through a sophisticated process based on the application of a rectangular LU factorization. This class of splitting preconditioners works better near a solution of the linear programming problem when the matrices are highly ill-conditioned. In this study, we develop and implement a new approach to find a basis for the splitting preconditioner, based on standard rectangular LU factorization with partial permutation of the scaled transpose linear programming constraint matrix. In most cases, this basis is better conditioned than the existing one. In addition, we include a penalty parameter in Mehrotra’s predictor-corrector method in order to reduce ill-conditioning of the normal equations matrix. Computational experiments show a reduction in the average number of iterations of the preconditioned conjugate gradient method. Also, the increased efficiency and robustness of the new approach become evident by the performance profile. PubDate: 2017-05-01 DOI: 10.1007/s10589-016-9887-0 Issue No:Vol. 67, No. 1 (2017)

Authors:Xin-He Miao; Yu-Lin Chang; Jein-Shan Chen Pages: 155 - 173 Abstract: Merit function approach is a popular method to deal with complementarity problems, in which the complementarity problem is recast as an unconstrained minimization via merit function or complementarity function. In this paper, for the complementarity problem associated with p-order cone, which is a type of nonsymmetric cone complementarity problem, we show the readers how to construct merit functions for solving p-order cone complementarity problem. In addition, we study the conditions under which the level sets of the corresponding merit functions are bounded, and we also assert that these merit functions provide an error bound for the p-order cone complementarity problem. These results build up a theoretical basis for the merit method for solving p-order cone complementarity problem. PubDate: 2017-05-01 DOI: 10.1007/s10589-016-9889-y Issue No:Vol. 67, No. 1 (2017)

Authors:Biao Qu; Changyu Wang; Naihua Xiu Pages: 175 - 199 Abstract: In this paper, based on a merit function of the split feasibility problem (SFP), we present a Newton projection method for solving it and analyze the convergence properties of the method. The merit function is differentiable and convex. But its gradient is a linear composite function of the projection operator, so it is nonsmooth in general. We prove that the sequence of iterates converges globally to a solution of the SFP as long as the regularization parameter matrix in the algorithm is chosen properly. Especially, under some local assumptions which are necessary for the case where the projection operator is nonsmooth, we prove that the sequence of iterates generated by the algorithm superlinearly converges to a regular solution of the SFP. Finally, some numerical results are presented. PubDate: 2017-05-01 DOI: 10.1007/s10589-016-9884-3 Issue No:Vol. 67, No. 1 (2017)

Authors:Yong Zhang; Wanzhou Ye; Jianjun Zhang Abstract: In this paper, we propose an iterative algorithm for solving the generalized elastic net regularization problem with smoothed \(\ell _{q} (0<q \le 1)\) penalty for recovering sparse vectors. We prove the convergence result of the algorithm based on the algebraic method. Under certain conditions, we show that the iterative solutions converge to a local minimizer of the generalized elastic net regularization problem and we also present an error bound. Theoretical analysis and numerical results show that the proposed algorithm is promising. PubDate: 2017-05-18 DOI: 10.1007/s10589-017-9916-7

Authors:Guoyin Li; Tianxiang Liu; Ting Kei Pong Abstract: We study the applicability of the Peaceman–Rachford (PR) splitting method for solving nonconvex optimization problems. When applied to minimizing the sum of a strongly convex Lipschitz differentiable function and a proper closed function, we show that if the strongly convex function has a large enough strong convexity modulus and the step-size parameter is chosen below a threshold that is computable, then any cluster point of the sequence generated, if exists, will give a stationary point of the optimization problem. We also give sufficient conditions guaranteeing boundedness of the sequence generated. We then discuss one way to split the objective so that the proposed method can be suitably applied to solving optimization problems with a coercive objective that is the sum of a (not necessarily strongly) convex Lipschitz differentiable function and a proper closed function; this setting covers a large class of nonconvex feasibility problems and constrained least squares problems. Finally, we illustrate the proposed algorithm numerically. PubDate: 2017-05-13 DOI: 10.1007/s10589-017-9915-8

Authors:Adam N. Letchford; Qiang Ni; Zhaoyu Zhong Abstract: We consider a challenging resource allocation problem arising in mobile wireless communications. The goal is to allocate the available channels and power in a so-called OFDMA system, in order to maximise the transmission rate, subject to quality of service constraints. Standard MINLP software struggled to solve even small instances of this problem. Using outer approximation, perspective cuts and several implementation “tricks”, we are able to solve realistic instances in about one minute. A novel ingredient of our algorithm is what we call pre-emptive cut generation: the generation of cutting planes that are not violated in the current iteration, but are likely to be violated in subsequent iterations. PubDate: 2017-05-05 DOI: 10.1007/s10589-017-9914-9