Authors:Wei Wan; Lorenz T. Biegler Pages: 401 - 424 Abstract: Barrier methods have led to several nonlinear programming (NLP) solvers (e.g. IPOPT, KNITRO, LOQO). However, certain regularity conditions are required for convergence of these methods. These conditions are violated for optimization models with dependent constraints, thus leading to method failure. These shortcomings can be identified by checking the inertia of the KKT matrix, and current solvers either add regularizing terms to correct the inertia of the KKT matrix or revert to more expensive trust region methods to solve the barrier problem. This study improves on these approaches with a new structured regularization strategy; within the Newton step it identifies an independent subset of equality constraints and removes the remaining constraints without modifying the KKT matrix structure. This approach leads to more accurate Newton steps and faster convergence, while maintaining global convergence properties. Implemented in IPOPT with linear solvers HSL_MA57, HSL_MA97 and MUMPS, we present numerical experiments on hundreds of examples from the CUTEr test set, modified for dependency. These results show an average reduction in iterations of more than 50 % over the current version of IPOPT. In addition, several nonlinear blending problems are solved with the proposed algorithm, and improvements over existing regularization strategies are further demonstrated. PubDate: 2017-04-01 DOI: 10.1007/s10589-016-9880-7 Issue No:Vol. 66, No. 3 (2017)

Authors:Quoc Tran-Dinh Pages: 425 - 451 Abstract: We propose an adaptive smoothing algorithm based on Nesterov’s smoothing technique in Nesterov (Math Prog 103(1):127–152, 2005) for solving “fully” nonsmooth composite convex optimization problems. Our method combines both Nesterov’s accelerated proximal gradient scheme and a new homotopy strategy for smoothness parameter. By an appropriate choice of smoothing functions, we develop a new algorithm that has the \(\mathcal {O}\left( \frac{1}{\varepsilon }\right) \) -worst-case iteration-complexity while preserves the same complexity-per-iteration as in Nesterov’s method and allows one to automatically update the smoothness parameter at each iteration. Then, we customize our algorithm to solve four special cases that cover various applications. We also specify our algorithm to solve constrained convex optimization problems and show its convergence guarantee on a primal sequence of iterates. We demonstrate our algorithm through three numerical examples and compare it with other related algorithms. PubDate: 2017-04-01 DOI: 10.1007/s10589-016-9873-6 Issue No:Vol. 66, No. 3 (2017)

Authors:Naohiko Arima; Sunyoung Kim; Masakazu Kojima; Kim-Chuan Toh Pages: 453 - 479 Abstract: The Lagrangian-doubly nonnegative (DNN) relaxation has recently been shown to provide effective lower bounds for a large class of nonconvex quadratic optimization problems (QAPs) using the bisection method combined with first-order methods by Kim et al. (Math Program 156:161–187, 2016). While the bisection method has demonstrated the computational efficiency, determining the validity of a computed lower bound for the QOP depends on a prescribed parameter \(\epsilon > 0\) . To improve the performance of the bisection method for the Lagrangian-DNN relaxation, we propose a new technique that guarantees the validity of the computed lower bound at each iteration of the bisection method for any choice of \(\epsilon > 0\) . It also accelerates the bisection method. Moreover, we present a method to retrieve a primal-dual pair of optimal solutions of the Lagrangian-DNN relaxation using the primal-dual interior-point method. As a result, the method provides a better lower bound and substantially increases the robustness as well as the effectiveness of the bisection method. Computational results on binary QOPs, multiple knapsack problems, maximal stable set problems, and quadratic assignment problems illustrate the robustness of the proposed method. In particular, a tight bound for QAPs with size \(n=50\) could be obtained. PubDate: 2017-04-01 DOI: 10.1007/s10589-016-9879-0 Issue No:Vol. 66, No. 3 (2017)

Authors:Hui-Ping Cao; Dong-Hui Li Pages: 481 - 505 Abstract: In this paper, we present two partitioned quasi-Newton methods for solving partially separable nonlinear equations. When the Jacobian is not available, we propose a partitioned Broyden’s rank one method and show that the full step partitioned Broyden’s rank one method is locally and superlinearly convergent. By using a well-defined derivative-free line search, we globalize the method and establish its global and superlinear convergence. In the case where the Jacobian is available, we propose a partitioned adjoint Broyden method and show its global and superlinear convergence. We also present some preliminary numerical results. The results show that the two partitioned quasi-Newton methods are effective and competitive for solving large-scale partially separable nonlinear equations. PubDate: 2017-04-01 DOI: 10.1007/s10589-016-9878-1 Issue No:Vol. 66, No. 3 (2017)

Authors:Sanja Rapajić; Zoltan Papp Pages: 507 - 532 Abstract: In this paper we propose Jacobian smoothing inexact Newton method for nonlinear complementarity problems (NCP) with derivative-free nonmonotone line search. This nonmonotone line search technique ensures globalization and is a combination of Grippo-Lampariello-Lucidi (GLL) and Li-Fukushima (LF) strategies, with the aim to take into account their advantages. The method is based on very well known Fischer-Burmeister reformulation of NCP and its smoothing Kanzow’s approximation. The mixed Newton equation, which combines the semismooth function with the Jacobian of its smooth operator, is solved approximately in every iteration, so the method belongs to the class of Jacobian smoothing inexact Newton methods. The inexact search direction is not in general a descent direction and this is the reason why nonmonotone scheme is used for globalization. Global convergence and local superlinear convergence of method are proved. Numerical performances are also analyzed and point out that high level of nonmonotonicity of this line search rule enables robust and efficient method. PubDate: 2017-04-01 DOI: 10.1007/s10589-016-9881-6 Issue No:Vol. 66, No. 3 (2017)

Authors:K. Lakshmanan; Shalabh Bhatnagar Pages: 533 - 556 Abstract: We propose a multi-time scale quasi-Newton based smoothed functional (QN-SF) algorithm for stochastic optimization both with and without inequality constraints. The algorithm combines the smoothed functional (SF) scheme for estimating the gradient with the quasi-Newton method to solve the optimization problem. Newton algorithms typically update the Hessian at each instant and subsequently (a) project them to the space of positive definite and symmetric matrices, and (b) invert the projected Hessian. The latter operation is computationally expensive. In order to save computational effort, we propose in this paper a quasi-Newton SF (QN-SF) algorithm based on the Broyden-Fletcher-Goldfarb-Shanno (BFGS) update rule. In Bhatnagar (ACM TModel Comput S. 18(1): 27–62, 2007), a Jacobi variant of Newton SF (JN-SF) was proposed and implemented to save computational effort. We compare our QN-SF algorithm with gradient SF (G-SF) and JN-SF algorithms on two different problems – first on a simple stochastic function minimization problem and the other on a problem of optimal routing in a queueing network. We observe from the experiments that the QN-SF algorithm performs significantly better than both G-SF and JN-SF algorithms on both the problem settings. Next we extend the QN-SF algorithm to the case of constrained optimization. In this case too, the QN-SF algorithm performs much better than the JN-SF algorithm. Finally we present the proof of convergence for the QN-SF algorithm in both unconstrained and constrained settings. PubDate: 2017-04-01 DOI: 10.1007/s10589-016-9875-4 Issue No:Vol. 66, No. 3 (2017)

Authors:Zheng-Hai Huang; Liqun Qi Pages: 557 - 576 Abstract: In this paper, we consider a class of n-person noncooperative games, where the utility function of every player is given by a homogeneous polynomial defined by the payoff tensor of that player, which is a natural extension of the bimatrix game where the utility function of every player is given by a quadratic form defined by the payoff matrix of that player. We will call such a problem the multilinear game. We reformulate the multilinear game as a tensor complementarity problem, a generalization of the linear complementarity problem; and show that finding a Nash equilibrium point of the multilinear game is equivalent to finding a solution of the resulted tensor complementarity problem. Especially, we present an explicit relationship between the solutions of the multilinear game and the tensor complementarity problem, which builds a bridge between these two classes of problems. We also apply a smoothing-type algorithm to solve the resulted tensor complementarity problem and give some preliminary numerical results for solving the multilinear games. PubDate: 2017-04-01 DOI: 10.1007/s10589-016-9872-7 Issue No:Vol. 66, No. 3 (2017)

Authors:Ali Mohammad-Nezhad; Tamás Terlaky Pages: 577 - 600 Abstract: The primal-dual Dikin-type affine scaling method was originally proposed for linear optimization and then extended to semidefinite optimization. Here, the method is generalized to symmetric conic optimization using the notion of Euclidean Jordan algebras. The method starts with an interior feasible but not necessarily centered primal-dual solution, and it features both centering and reducing the duality gap simultaneously. The method’s iteration complexity bound is analogous to the semidefinite optimization case. Numerical experiments demonstrate that the method is viable and robust when compared to SeDuMi, MOSEK and SDPT3. PubDate: 2017-04-01 DOI: 10.1007/s10589-016-9874-5 Issue No:Vol. 66, No. 3 (2017)

Authors:Maziar Salahi; Akram Taati; Henry Wolkowicz Pages: 223 - 244 Abstract: We study large-scale extended trust-region subproblems (eTRS) i.e., the minimization of a general quadratic function subject to a norm constraint, known as the trust-region subproblem (TRS) but with an additional linear inequality constraint. It is well known that strong duality holds for the TRS and that there are efficient algorithms for solving large-scale TRS problems. It is also known that there can exist at most one local non-global minimizer (LNGM) for TRS. We combine this with known characterizations for strong duality for eTRS and, in particular, connect this with the so-called hard case for TRS. We begin with a recent characterization of the minimum for the TRS via a generalized eigenvalue problem and extend this result to the LNGM. We then use this to derive an efficient algorithm that finds the global minimum for eTRS by solving at most three generalized eigenvalue problems. PubDate: 2017-03-01 DOI: 10.1007/s10589-016-9867-4 Issue No:Vol. 66, No. 2 (2017)

Authors:Johannes Brust; Jennifer B. Erway; Roummel F. Marcia Pages: 245 - 266 Abstract: In this article, we consider solvers for large-scale trust-region subproblems when the quadratic model is defined by a limited-memory symmetric rank-one (L-SR1) quasi-Newton matrix. We propose a solver that exploits the compact representation of L-SR1 matrices. Our approach makes use of both an orthonormal basis for the eigenspace of the L-SR1 matrix and the Sherman–Morrison–Woodbury formula to compute global solutions to trust-region subproblems. To compute the optimal Lagrange multiplier for the trust-region constraint, we use Newton’s method with a judicious initial guess that does not require safeguarding. A crucial property of this solver is that it is able to compute high-accuracy solutions even in the so-called hard case. Additionally, the optimal solution is determined directly by formula, not iteratively. Numerical experiments demonstrate the effectiveness of this solver. PubDate: 2017-03-01 DOI: 10.1007/s10589-016-9868-3 Issue No:Vol. 66, No. 2 (2017)

Authors:Jinyan Fan; Anwa Zhou Pages: 267 - 283 Abstract: A symmetric tensor, which has a symmetric nonnegative decomposition, is called a completely positive tensor. In this paper, we characterize the completely positive tensor as a truncated moment sequence, and transform the problem of checking whether a tensor is completely positive to checking whether its corresponding truncated moment sequence admits a representing measure, then present a semidefinite algorithm to solve it. If a tensor is not completely positive, a certificate for it can be obtained; if it is completely positive, a nonnegative decomposition can be obtained. PubDate: 2017-03-01 DOI: 10.1007/s10589-016-9870-9 Issue No:Vol. 66, No. 2 (2017)

Authors:Na Zhao; Qingzhi Yang; Yajun Liu Pages: 285 - 307 Abstract: Tensor is a hot topic in the past decade and eigenvalue problems of higher order tensors become more and more important in the numerical multilinear algebra. Several methods for finding the Z-eigenvalues and generalized eigenvalues of symmetric tensors have been given. However, the convergence of these methods when the tensor is not symmetric but weakly symmetric is not assured. In this paper, we give two convergent gradient projection methods for computing some generalized eigenvalues of weakly symmetric tensors. The gradient projection method with Armijo step-size rule (AGP) can be viewed as a modification of the GEAP method. The spectral gradient projection method which is born from the combination of the BB method with the gradient projection method is superior to the GEAP, AG and AGP methods. We also make comparisons among the four methods. Some competitive numerical results are reported at the end of this paper. PubDate: 2017-03-01 DOI: 10.1007/s10589-016-9865-6 Issue No:Vol. 66, No. 2 (2017)

Authors:Meijiao Liu; Yong-Jin Liu Pages: 309 - 326 Abstract: This paper focuses on a singly linearly constrained class of convex quadratic programs with box-like constraints. We propose a new fast algorithm based on parametric approach and secant approximation method to solve this class of quadratic problems. We design efficient implementations for our proposed algorithm and compare its performance with two state-of-the-art standard solvers called Gurobi and Mosek. Numerical results on a variety of test problems demonstrate that our algorithm is able to efficiently solve the large-scale problems with the dimension up to fifty million and it substantially outperforms Gurobi and Mosek in terms of the running time. PubDate: 2017-03-01 DOI: 10.1007/s10589-016-9863-8 Issue No:Vol. 66, No. 2 (2017)

Authors:Liang Chen; Defeng Sun; Kim-Chuan Toh Pages: 327 - 343 Abstract: This note serves two purposes. Firstly, we construct a counterexample to show that the statement on the convergence of the alternating direction method of multipliers (ADMM) for solving linearly constrained convex optimization problems in a highly influential paper by Boyd et al. (Found Trends Mach Learn 3(1):1–122, 2011) can be false if no prior condition on the existence of solutions to all the subproblems involved is assumed to hold. Secondly, we present fairly mild conditions to guarantee the existence of solutions to all the subproblems of the ADMM and provide a rigorous convergence analysis on the ADMM with a computationally more attractive large step-length that can even exceed the practically much preferred golden ratio of \((1+\sqrt{5})/2\) . PubDate: 2017-03-01 DOI: 10.1007/s10589-016-9864-7 Issue No:Vol. 66, No. 2 (2017)

Authors:Reza Eghbali; Maryam Fazel Pages: 345 - 381 Abstract: We study the convergence rate of the proximal-gradient homotopy algorithm applied to norm-regularized linear least squares problems, for a general class of norms. The homotopy algorithm reduces the regularization parameter in a series of steps, and uses a proximal-gradient algorithm to solve the problem at each step. Proximal-gradient algorithm has a linear rate of convergence given that the objective function is strongly convex, and the gradient of the smooth component of the objective function is Lipschitz continuous. In many applications, the objective function in this type of problem is not strongly convex, especially when the problem is high-dimensional and regularizers are chosen that induce sparsity or low-dimensionality. We show that if the linear sampling matrix satisfies certain assumptions and the regularizing norm is decomposable, proximal-gradient homotopy algorithm converges with a linear rate even though the objective function is not strongly convex. Our result generalizes results on the linear convergence of homotopy algorithm for \(\ell _1\) -regularized least squares problems. Numerical experiments are presented that support the theoretical convergence rate analysis. PubDate: 2017-03-01 DOI: 10.1007/s10589-016-9871-8 Issue No:Vol. 66, No. 2 (2017)

Authors:Elvira Antal; Tamás Vinkó Pages: 383 - 400 Abstract: This paper gives an exact mathematical programming model and algorithm of the max–min fairness bandwidth allocation problem in multi-swarm peer-to-peer content sharing community. The proposed iterative method involves solution of LP and MILP problems of large scale. Based on real-world data traces, numerical experiments demonstrate that the new algorithm is computationally faster than an earlier developed one for larger problem sizes, and it provides better numerical stability. Moreover, even if its execution is stopped after some initial steps it still grants feasible solution with good approximation to max–min fairness. PubDate: 2017-03-01 DOI: 10.1007/s10589-016-9866-5 Issue No:Vol. 66, No. 2 (2017)

Authors:M. Sayadi Shahraki; H. Mansouri; M. Zangiabadi Abstract: In this paper, we present two primal–dual interior-point algorithms for symmetric cone optimization problems. The algorithms produce a sequence of iterates in the wide neighborhood \(\mathcal {N}(\tau ,\,\beta )\) of the central path. The convergence is shown for a commutative class of search directions, which includes the Nesterov–Todd direction and the xs and sx directions. We derive that these two path-following algorithms have $$\begin{aligned} \text{ O }\left( \sqrt{r\text{ cond }(G)}\log \varepsilon ^{-1}\right) , \text{ O }\left( \sqrt{r}\left( \text{ cond }(G)\right) ^{1/4}\log \varepsilon ^{-1}\right) \end{aligned}$$ iteration complexity bounds, respectively. The obtained complexity bounds are the best result in regard to the iteration complexity bound in the context of the path-following methods for symmetric cone optimization. Numerical results show that the algorithms are efficient for this kind of problems. PubDate: 2017-03-09 DOI: 10.1007/s10589-017-9905-x

Authors:Ramsharan Rangarajan Abstract: We analyze a class of discrete, univariate, and strictly quasiconcave max–min problems. A distinctive feature of max–min-type optimization problems is the nonsmoothness of the objective being maximized. Here we exploit strict quasiconcavity of the given set of functions to prove existence and uniqueness of the optimizer, and to provide computable bounds for it. The analysis inspires an efficient algorithm for computing the optimizer without having to resort to any regularization or heuristics. We prove correctness of the proposed algorithm and briefly discuss the effect of tolerances and approximate computation. Our study finds direct application in the context of certain mesh deformation methods, wherein the optimal perturbation for a vertex is computed as the solution of a max–min problem of the type we consider here. We include examples demonstrating improvement of simplicial meshes while adopting the proposed algorithm for resolution of the optimization problems involved, and use these numerical experiments to examine the performance of the algorithm. PubDate: 2017-03-06 DOI: 10.1007/s10589-017-9903-z

Authors:Christian Kanzow; Yekini Shehu Abstract: The Krasnoselskii–Mann iteration plays an important role in the approximation of fixed points of nonexpansive operators; it is known to be weakly convergent in the infinite dimensional setting. In this present paper, we provide a new inexact Krasnoselskii–Mann iteration and prove weak convergence under certain accuracy criteria on the error resulting from the inexactness. We also show strong convergence for a modified inexact Krasnoselskii–Mann iteration under suitable assumptions. The convergence results generalize existing ones from the literature. Applications are given to the Douglas–Rachford splitting method, the Fermat–Weber location problem as well as the alternating projection method by John von Neumann. PubDate: 2017-02-21 DOI: 10.1007/s10589-017-9902-0

Authors:Daniel O’Connor; Lieven Vandenberghe Abstract: Image deblurring techniques based on convex optimization formulations, such as total-variation deblurring, often use specialized first-order methods for large-scale nondifferentiable optimization. A key property exploited in these methods is spatial invariance of the blurring operator, which makes it possible to use the fast Fourier transform (FFT) when solving linear equations involving the operator. In this paper we extend this approach to two popular models for space-varying blurring operators, the Nagy–O’Leary model and the efficient filter flow model. We show how splitting methods derived from the Douglas–Rachford algorithm can be implemented with a low complexity per iteration, dominated by a small number of FFTs. PubDate: 2017-02-18 DOI: 10.1007/s10589-017-9901-1