Convexity

Definitions

Convex sets

• sets = basis of convexity
• definition: a set $X$ in a vector space is convex if for any $a,b\in X$ the line segment connecting $a$ and $b$ is also in $X$,

Convex function

Given a convex set $\mathcal{X}$, a function $f:\mathcal{X}\to \mathbb{R}$ is convex if for all $x,x^{\prime }\in \mathcal{X}$ and for all $\lambda \in [0,1]$ we have

Properties

• Jensen's inequality
  • = the expectation of a convex function $\ge$ the convex function of an expectation
• local minima are global minima
• Below sets of convex functions are convex
• convexity and second derivatives
  • a twice-differentiable function is convex if and only if its Hessian (a matrix of second derivatives) is positive semidefinite

Convex constraints

• can be added via the Lagrangian
• can be added with a penalty to the objective function in practice