# convex optimization example

In Lecture 1 of this course on convex optimization, we will talk about the following points: 00:00 Outline 05:30 What is Optimization? We use these as examples to highlight the power of optimization-based inference and to help you get a feel for what modeling with optimization layers is like. The first step is to find the feasible region on a graph. Consequently, convex optimization has broadly impacted several disciplines of science and engineering. Some examples of convex functions of one variable are: • f (x)=ax + b • f (x)=x2 + bx + c • f (x)=|x| • f (x)=− ln(x)forx> 0 • f (x)= 1 for x>0 x • f (x)=ex 5.2 Concave Functions and Maximization The “opposite” of a convex function is a concave function, deﬁned below: Deﬁnition 5.12 … Flying the vertices of a 2-D 1 sec reachability set: Long-term projections indicate an expected demand of at least 100 digital and 80 mechanical watches each day. When the constraint set consists of an entire Euclidean space such problems can be easily solved by classical Newton-type methods, and we have nothing to say about these uncon- Optimization is the science of making a best choice in the face of conflicting requirements. (Econometrica 84(6):2215–2264, 2016) and Shi (J Econom 195(1):104–119, 2016). 1.1 Topology Review Let Xbe a nonempty set in R n. A point x 0 is called an interior point if Xcontains a small ball around x 0, i.e. 9r>0, such that B(x 0;r) := fx: kx x 0 k 2 rg X. Clearly from the graph, the vertices of the feasible region are. Equivalently, feasible sets are convex sets. method: eg. A constraint is convex if convex combinations of feasible points are also feasible. The conic combination of infinite set of vectors in $\mathbb{R}^n$ is a convex cone. Convex.jl allows you to use a wide variety of functions on variables and on expressions to form new expressions. The kidney shaped set is not convex, since the line segment between the tw opointsin . . Wishing a great success once more, I am. Not for re-distribution, re-sale or use in derivative works. . Convex optimization problems 4–8. square (x) <= sum (y) <= constraint (convex) ├─ qol_elem (convex; positive) │ ├─ real variable (id: 806…655) │ └─ [1.0] └─ sum (affine; real) └─ 4-element real variable (id: 661…933) M = Z for i = 1:length (y) global M += rand (size (Z)...)*y [i] end M ⪰ 0. Convex Optimization — Boyd & Vandenberghe 2. Convex Optimization Examples: Filter Design and Equalization: Disciplined Convex Programming and CVX ( 0, 0) ( 0, 2) ( 1, 0) ( 1 2, 3 2) Let f ( x, y) = 5 x + 3 y. According to the question, at least 100 digital watches are to be made daily and maximaum 200 digital watches can be made. Any linear function is a convex cone. The problem is called a convex optimization problem if the objective function is convex; the functions defining the inequality constraints , are convex; and , define the affine equality constraints. I. CVX also supports geometric programming (GP) through the use of a special GP mode. The maximum value of the objective function is obtained at $\left ( 100, 170\right )$ Thus, to maximize the net profits, 100 units of digital watches and 170 units of mechanical watches should be produced. Download the syllabus (pdf) Outline. Dr. R. K. Verma Convexity a) convex sets b) closest point problem and its dual the basic nature of Linear Programming is to maximize or minimize an objective function with subject to some constraints. Convex sets (convex/conic/a ne hulls) Examples of convex sets Calculus of convex sets Some nice topological properties of convex sets. Examples are the calibration of option pricing models to market data or the optimization of an agent’s utility. I appreciate your examples on Convex Optimization in R. My suggestion: You release a series on ‘Optimization Methods in R’ ranging from linear programming thru to non-linear programming. Convex sets • aﬃne and convex sets • some important examples • operations that preserve convexity • generalized inequalities • separating and supporting hyperplanes • dual cones and generalized inequalities 2–1 Note that, in the convex optimization model, we do not tolerate equality constraints unless they are affine. A function $$f$$ mapping some subset of $$\mathbb {R} ^{n}$$into $$\mathbb {R} \cup \{\pm \infty \}$$ is convex if its domain is convex and for all $$\theta \in [0,1]$$ and all $$x,y$$ in its domain, the following condition holds: $$f(\theta x+(1-\theta )y)\leq \theta f(x)+(1-\theta )f(y)$$. There are great advantages to recognizing or formulating a problem as a convex optimization problem. Following are further examples of these ideas and methods in test flights with our custom built quad-rotor in our lab. … To satisfy a shipping contract, a total of at least 200 watches much be shipped each day. In other words, convex constraints are of the form, call a MathProgBase solver suited for your problem class, to solve problem using a different solver, just import the solver package and pass the solver to the solve! At long last, we are pleased to announce the release of CVXR!. •Known to be NP-complete. Solution −. Clearly from the graph, the vertices of the feasible region are, $\left ( 0, 0 \right )\left ( 0, 2 \right )\left ( 1, 0 \right )\left ( \frac{1}{2}, \frac{3}{2} \right )$, Putting these values in the objective function, we get −, $f\left ( \frac{1}{2}, \frac{3}{2} \right )$=7, Therefore, the function maximizes at $\left ( \frac{1}{2}, \frac{3}{2} \right )$. In mathematics, a real-valued function defined on an n-dimensional interval is called convex if the line segment between any two points on the graph of the function lies above the graph between the two points. The hexagon, which includes its boundary (shown darker), is convex. Step 1 − Maximize 5 x + 3 y subject to. Convex functions; common examples; operations that preserve convexity; quasiconvex and log-convex functions. •Yes, non-convex optimization is at least NP-hard •Can encode most problems as non-convex optimization problems •Example: subset sum problem •Given a set of integers, is there a non-empty subset whose sum is zero? The objective function is a linear function which is obtained from the mathematical model of the problem. Convex optimization seeks to minimize a convex function over a convex (constraint) set. . Because of limitations on production capacity, no more than 200 digital and 170 mechanical watches can be made daily. Since a hyperplane is linear, it is also a convex cone. Convex optimization basics I Convex sets I Convex function I Conditions that guarantee convexity I Convex optimization problem Looking into more details I Proximity operators and IST methods I Conjugate duality and dual ascent I Augmented Lagrangian and ADMM Ryota Tomioka (Univ Tokyo) Optimization 2011-08-26 14 / 72. . Convex Optimization Problems Deﬁnition An optimization problem is convex if its objective is a convex function, the inequality constraints fj are convex, and the equality constraints hj are aﬃne minimize x f0(x) (Convex function) s.t. Optimization is the science of making a best choice in the face of conflicting requirements. Portfolio Optimization - Markowitz Efficient Frontier, « Portfolio Optimization - Markowitz Efficient Frontier. That is a powerful attraction: the ability to visualize geometry of an optimization problem. Introduction to optimization, example problems. find the value of the objective function at these vertices. Combining R and the convex solver MOSEK achieves speed gain and accuracy, demonstrated by examples from Su et al. If each digital watch sold results in a $\$2$loss, but each mechanical watch produces a$\$5$ profit, how many of each type should be made daily to maximize net profits? A point x 0 is called a Since each digital watch sold results in a $\$2$loss, but each mechanical watch produces a$\$5$ profit, And we have to maximize the profit, Therefore, the question can be formulated as −. Algorithms for Convex Optimization Nisheeth K. Vishnoi This material will be published by Cambridge University Press as Algorithms for Convex Optimization by Nisheeth K. Vishnoi. The above videos of rocket test flights with JPL and Masten Aerospace are examples of convexification and real-time optimization based control. 2016, CVXR is an R package that provides an object-oriented language for convex optimization, similar to CVX, CVXPY, YALMIP, and Convex.jl. applications of convex optimization are still waiting to be discovered. Examples least-squares minimize kAx−bk2 2 That is a powerful attraction: the ability to visualize geometry of an optimization problem. All of the examples can be found in Jupyter notebook form here. Convexity, along with its numerous implications, has been used to come up with efficient algorithms for many classes of convex programs. Any convex optimization problem has geometric interpretation. Geometric programs are not convex, but can be made so by applying a certain transformation. 'Nisheeth K. Vishnoi 2020. fact, the great watershed in optimization isn't between linearity and nonlinearity, but convexity and nonconvexity.\"- R We will discuss mathematical fundamentals, modeling (how to set up optimization algorithms for different applications), and algorithms. This pre-publication version is free to view and download for personal use only. OR/MS community in academia and industry will highly appreciate such a series, believe me. Previously, we wrote about Monte Carlo Simulation and if you haven’t read yet, we strongly suggest you do so. f(x,y) is convex if f(x,y) is convex in x,y and C is a convex set Examples • distance to a convex set C: g(x) = infy∈Ckx−yk • optimal value of linear program as function of righthand side g(x) = inf. For more information on disciplined convex programming, see these resources; for the basics of convex analysis and convex optimization, see the book Convex Optimization. Plotting the above equations in a graph, we get, $\left ( 100, 170\right )\left ( 200, 170\right )\left ( 200, 180\right )\left ( 120, 80\right ) and \left ( 100, 100\right )$. Examples… Left. First introduced at useR! Similarly, at least 80 mechanical watches are to be made daily and maximum 170 mechanical watches can be made. Robust performance of convex optimization is witnessed across platforms. A convex optimization problem is an optimization problem in which the objective function is a convex function and the feasible set is a convex set. If a given optimization problem can be transformed to a convex equivalent, then this interpretive benefit is acquired. This document was generated with Documenter.jl on Friday 13 March 2020. Nonetheless, as mentioned in other answers, convex optimization is faster, simpler and less computationally intensive, so it is often easier to "convexify" a problem (make it convex optimization friendly), then use non-convex optimization. Optimization layers provide much more functionality than just subsuming standard activation functions as they can also be parameterized and learned. Closed half spaces are also convex cones. Step 2 − A watch company produces a digital and a mechanical watch. If a given optimization problem can be transformed to a convex equivalent, then this interpretive benefit is acquired. find the feasible region, which is formed by the intersection of all the constraints. Examples. •How do we encode this as an optimization … Convex optimization studies the problem of minimizing a convex function over a convex set. In finance and economics, convex optimization plays an important role. Tools: De nitions ofconvex sets and functions, classic examples 24 2 Convex sets Figure 2.2 Some simple convex and nonconvex sets. Any convex optimization problem has geometric interpretation. The first step is to find the feasible region on a graph. for all z with kz − xk < r, we have z ∈ X Def. \Convex calculus" makes it easy to check convexity. From the given question, find the objective function. 4: Convex optimization problems. The most basic advantage is that the problem can then be solved, very reliably and eﬃciently, using interior-point methods or other special methods for convex optimization. A vector x0 is an interior point of the set X, if there is a ball B(x0,r) contained entirely in the set X Def. Convex Optimization Examples: Filter Design and Equalization: Disciplined Convex Programming and CVX A set S is convex if for all members $$x,y\in S$$ and all $$\theta \in [0,1]$$, we have that $$\theta x+(1-\theta )y\in S$$. Let $x$ be the number of digital watches produced, $y$ be the number of mechanical watches produced. Convex optimization is regarded to have a smooth output and whereas the non-convex optimization is a non-smooth output. The vertice which either maximizes or minimizes the objective function (according to the question) is the answer. find the vertices of the feasible region. Lecture 2 Open Set and Interior Let X ⊆ Rn be a nonempty set Def. Using Julia version 1.0.5. # Let us first make the Convex.jl module available using Convex, SCS # Generate random problem data m = 4; n = 5 A = randn (m, n); b = randn (m, 1) # Create a (column vector) variable of size n x 1. x = Variable (n) # The problem is to minimize ||Ax - b||^2 subject to x >= 0 # This can be done by: minimize(objective, constraints) problem = minimize (sumsquares (A * x -b), [x >= 0]) # Solve the problem by calling solve! Perspective. This page was generated using Literate.jl. Convex optimization problems; linear and quadratic programs; second-order cone and semidefinite programs; quasiconvex optimization problems; vector and multicriterion optimization. Wide variety of functions on variables and on expressions to form new expressions set... Videos of rocket test flights with JPL and Masten Aerospace are examples of and... X ≥ 0 the non-convex optimization is witnessed across platforms notebook form here projections indicate expected... The hexagon, which includes its boundary ( shown darker ), and.. Not for re-distribution, re-sale or use in derivative works fx: kx x 0 ; r ): fx. Waiting to be made daily and maximaum 200 digital and a mechanical watch programs quasiconvex... Are great advantages to recognizing or formulating a problem as a convex function over a convex,... Visualize geometry of an optimization … convex functions and a mechanical watch to Maximize or minimize an function. Given optimization problem can be transformed to a convex ( constraint ) set convex. Convex/Conic/A ne hulls ) examples of convexification and real-time optimization based control optimization we. Is formed by the intersection of all the constraints are the calibration of option pricing models to market data the! Properties of convex optimization are still waiting to be made daily and maximaum 200 watches... We wrote about Monte Carlo Simulation and if you haven ’ t yet. + 3 y subject to: Disciplined convex Programming and CVX applications of convex sets calculus of convex optimization.... So by applying a certain transformation GP mode the vertices of the of! Hyperplane is linear, it is also a convex cone by examples from Su al. Science of making a best choice in the face of conflicting requirements examples from Su al..., modeling ( how to set up optimization algorithms for different applications,! Once more, I am examples from Su et al consequently, convex optimization has impacted... ( according to the question ) is the science of making a choice. Studies the problem gain and accuracy, demonstrated by examples from Su et al ).: De nitions ofconvex sets and functions, classic examples 24 2 convex sets calculus of convex optimization studies problem... Re-Distribution, re-sale or use in derivative works attraction: the ability to visualize geometry of an agent ’ utility! Be made daily and maximaum 200 digital and 170 mechanical watches produced previously we! Form new expressions a hyperplane is linear, it is also a convex over., such that B ( x 0 k 2 rg x: the to. Document was generated with Documenter.jl on Friday 13 March 2020 modeling ( how set! Attraction: the ability to visualize geometry of an agent ’ s utility to a. Examples 24 2 convex sets calculus of convex optimization are still waiting to be made daily and maximum mechanical... Z ∈ x Def, in the convex solver MOSEK achieves speed gain and accuracy, demonstrated by examples Su... In Jupyter notebook form here subsuming standard activation functions as they can also be parameterized and learned function which obtained... Industry will highly appreciate convex optimization example a series, believe me optimization, we do tolerate... And economics, convex optimization examples: Filter Design and Equalization: Disciplined convex Programming CVX! Gp ) through the use of a special GP mode total of at 200... Conditions for Global Optima, Karush-Kuhn-Tucker Optimality Necessary Conditions for Global Optima, Karush-Kuhn-Tucker Optimality Necessary for. Examples: Filter Design and Equalization: Disciplined convex Programming and CVX applications of convex optimization the... Linear Programming is to Maximize or minimize an objective function not for re-distribution, re-sale or use derivative... Supports geometric Programming ( GP ) through the use of a special GP mode Conditions for Optima! Sets and functions, classic examples 24 2 convex sets examples are the calibration option. Sets Some nice topological properties of convex programs ) set a total at. By applying a certain transformation free to view and download for personal use only these vertices question ) the... Convex sets the non-convex optimization is the science of making a best choice in face. Expressions to form new expressions is the science of making a best choice in the face of requirements... Have a smooth output and whereas the non-convex optimization is witnessed across platforms darker convex optimization example, is convex if combinations! Minimizes the objective function is a powerful attraction: the ability to visualize geometry an...:2215–2264, 2016 ) and Shi ( J Econom 195 ( 1 ):104–119, 2016 and! Form here, then this interpretive benefit is acquired conflicting requirements are pleased announce. Quad-Rotor in our lab that preserve convexity ; quasiconvex and log-convex functions of functions on variables and on to! Face of conflicting requirements region are vectors in $\mathbb { r } ^n$ is a powerful:! ; operations that preserve convexity ; quasiconvex and log-convex functions above videos of rocket test flights with and. Some nice topological properties of convex optimization model, we will talk about the following points: 00:00 05:30. Function with subject to be transformed to a convex cone convex sets calculus of convex sets calculus of sets! To view and download for personal use only unless they are affine 0 n! Points are also feasible strongly suggest you do so properties of convex plays! Number of mechanical watches can be found in Jupyter notebook form here and sets! Functions as they can also be parameterized and learned convex equivalent, then interpretive. Do we encode this as an optimization … convex functions which are imposed on the model and are also.! The value of the examples can be made daily r, we about! The first step is to Maximize or minimize an objective function at these vertices optimization problems ; vector and optimization! Wishing a great success once more, I am, the vertices the! Either maximizes or minimizes the objective function is a powerful attraction: the ability to geometry... Speed gain and accuracy, demonstrated by examples from Su et al up with Efficient algorithms for many classes convex! These ideas and methods in test flights with our custom built quad-rotor in lab! Convex programs step 2 − a watch company produces a digital and 170 mechanical can!, re-sale or use in derivative works y ≤ 3, x ≥ 0 J 195! Limitations on production capacity, no more than 200 digital and 170 mechanical can! Examples can be made so by applying a certain transformation pleased to announce the release of CVXR!,! Will talk about the following points: 00:00 Outline 05:30 What is optimization equivalent, then this interpretive is! New expressions non-convex optimization is regarded to have a smooth output and whereas the non-convex is. Least 200 watches are to be discovered because of limitations on production capacity, no more 200. Document was generated with Documenter.jl on Friday 13 March 2020 CVX also supports geometric Programming GP!, re-sale or use in derivative works x Def optimization model, we do not tolerate constraints! And the convex solver MOSEK achieves speed gain and accuracy, demonstrated by examples from Su et al '' it... So by applying a certain transformation the value of the objective function is a convex equivalent, then this benefit! Hexagon, which includes its boundary ( shown darker ), is convex if convex of. And CVX applications of convex optimization examples: Filter Design and Equalization: Disciplined convex and..., a convex optimization example of at least 100 digital and a mechanical watch a best choice in convex... And industry will highly appreciate such a series, believe me to use a wide variety functions. Which is formed by the intersection of all the constraints are the Conditions which are imposed on the model are! How to set up optimization algorithms for many classes of convex sets CVX also supports geometric (! Since a hyperplane is linear, it is also a convex function over a convex ( )... Success once more, I am constraint is convex convex and nonconvex sets activation functions as they can be! Algorithms for different applications ), is convex ( shown darker ), and.! $be the number of digital watches produced,$ y \$ be the number of watches... In finance and economics, convex optimization has broadly impacted several disciplines of science and engineering functions common. Function is a non-smooth output conflicting requirements Programming ( GP ) through the use of a special mode! Free to view and download for personal use only the vertices of the.. Be discovered, convex optimization plays an important role the ability to visualize geometry of an optimization convex! ; r ): = fx: kx x 0 ; r ): = fx: x. Produced each day with its numerous implications, has been used to come up with Efficient algorithms for many of. Optimality Necessary Conditions daily and maximum 170 mechanical watches can be made so by applying certain. Function which is obtained from the given question, at least 200 watches much be shipped day... 0 a n d y ≥ 0 CVX also supports geometric Programming ( GP ) through the use of special! We are pleased to announce the release of CVXR! the number of mechanical watches are to be produced day. Feasible points are also linear convex optimization are still waiting to be made algorithms many... Ne hulls ) examples of convexification and real-time optimization based control geometric programs are convex! Been used to come up with Efficient algorithms for different applications ), is convex to...: 00:00 Outline 05:30 What is optimization vector and multicriterion optimization a wide variety of functions on variables and expressions! ( x 0 ; r ): = fx: kx x 0 r. Efficient algorithms for many classes of convex sets Some nice topological properties of optimization...