Lagrange multipliers

Lagrange multipliers in higher dimensions #

written and developed by Thomas Pietraho

The goal of this lab is to extend the method of Lagrange multipliers from class to somewhat more general situations. As mentioned in the course, the method of Lagrange multipliers for functions of two variables is often not necessary; if one can solve for one of the variables within the constraint equation, then the optimization problem can be reduced to a single variable endeavor. Such a shortcut is not possible when we consider functions of more than two variables. Furthermore, an additional complication in the higher-dimensional setting may arise with the introduction of more than one constraint! You should use the following questions to guide your understanding, but in your write up, address the questions posed in the final section.

Part 1: Functions of three variables #

Suppose that the function that we are trying to maximize depends on three variables, so it has the form \(f(x,y,x)\). Also suppose that our constraint is given by the equation \(g(x,y,z)=c\). The question that we need to answer is how to extend the method from class to this situation. Keep the following problem in mind as you develop the method by following the outline below, and then use the theory you developed to solve it.

Problem: Maximize the Cobb-Douglas production function of the form \(f(x,y,z)= x^{\frac{1}{4}} y^{\frac{1}{6}} z^{\frac{7}{12}}\) subject to the budget constraint given by the following equation: \[g(x,y,z)= 2x + 3y +z = 6.\]

Complete the following outline as you work through the lab. Take notes on your results. You will need them later on in the course. This lab has a companion Mathematica notebook with (presumably) helpful commands. Use it freely as you complete the outline below.

Exercise: Examine the level surfaces of \(f(x,y,z)\) in relation to the graph of the budget constraint. What happens at the point at which the function \(f(x,y,z)\) is maximized? How can this be stated in terms of gradient vectors \(\nabla f\) and \(\nabla g\)? Use pictures to support your answer.

Exercise: Turn your geometric observation into a system of equations and solve the above production function problem using the method that you have just developed.

Exercise: Finally, could any of this be made to work for functions of four variables? \(n\) variables?

Part 2: Multiple constraints #

Sometimes, several functions can serve as constraints in optimization problem. For instance, examine the following problem from geometry.

Problem: Find the point on the line of intersection of the planes \(x+y+z = 1 \) and \(3x+2y+z=6\) that lies closest to the origin.

So, what is the objective function, that is the function we are trying to miminize, and what are the constraints? Since we are interested in minimizing the distance between a point \((x,y,z)\) and the origin \((0,0,0)\), the distance formula comes into play. Thus, our goal should be to minimize the function \(f(x,y,z) = \sqrt{x^2+y^2+z^2}\). But we can be a little more clever and make our life a little simpler.

Observation: The function \(f(x,y,z)\) defined above is slightly nasty to work with: it involves a square root that will make calculus complicated. Before we plunge into the problem headfirst, note that minimizing \(\sqrt{x^2+y^2+z^2}\) is equivalent to minimizing \(x^2+y^2+z^2.\) That is, the point \((x,y,z)\) at which the first is minimized will also be the point where the second function is minimized. So let us try to minimize \(f(x,y,z) = {x^2+y^2+z^2}\) instead!

The constraints are a little simpler. There are two functions that limit which values of \((x,y,z)\) we can consider:

\[ g(x,y,z) = x+y+z = 1 \; \; \; \; \; \; h(x,y,z) = 3x+2y+z = 6 \]

Work through the following exercises as you solve this optimization problem with multiple constraints.

Exercise:

Draw the picture of this scenario. What happens at the point at which \(f(x,y,z)\), the distance function, is minimized? More precisely, what is the geometric relationship between the three vectors \(\nabla f\), \(\nabla g\), and \(\nabla h \), at this point? Draw a picture to support your conclusions.

Hint: Focus on the relationship between the vectors \(\nabla f\), \(\nabla g\), and \(\nabla h \), and the line of intersection \(\ell\) of the two planes.

Solution

We know that gradient vectors are perpendicular to contour surfaces. So for instance, \(\nabla g\) is perpendicular to the plane \(g(x,y,z) = x+y+z = 1\). Since the line of intersection \(\ell \) is part of this plane, \(\nabla g\) is perpendicular to \(\ell\) as well! By a similar argument, \(\nabla h\) is also perpendicular to \(\ell\).

Now let’s deal with \(\nabla f\). It is perpendicular to every contour of \(f(x,y,z)\). At the point \(x,y,z\) which minimizes \(f\), its contour surface will be tangent to \(\ell\). So again, \(\nabla f\) will be perpendicular to \(\ell\).

In the end, we have three vectors \(\nabla f\), \(\nabla g\), and \(\nabla h \), each perpendicular to a single line.

Exercise:

While the geometrical observation you made above is interesting, it is not yet useful. So, convert your geometrical observation to a mathematical equation that will help us solve the problem.

Hint: Three vectors that all share the same plane look like the following picture:

Is there an equation you can derive that relates all three?

Solution

Stretch \(\vec{v}\) and \(\vec{w}\) by appropriate amounts and then combine to make the vector \(\vec{u}\). Now apply this to our gradient vectors.

Exercise:

Solve the distance-minimization problem using the method you have just developed. Please use the companion Mathematica notebook to do the heavy lifting; the system of equations is a little complicated.

Once you are done, summarize the technique you developed and add it to your class notes. How would you modify it if there more than two constraints?