\section{Alternative Optimization Methods}


In Problem 1, to find the ``best'' fitting line of the form $y=mx$ we
found the value of $m$ that minimized the function 
$$
V_2(m)=\sqrt{(1.2-m)^2+(3.8-2m)^2+
            (4.2-3m)^2+(6.3-4m)^2+
            (7.2-5m)^2+(7.3-6m)^2}
$$
It was claimed that $V_2(m)$ represented the total distance between
the data points and the model (the $y=mx$ line), so the lower this
distance the better the fit. There are, however, alternative ways to
measure the distance between a data set and a model.
\subsection{Alternative Residuals}
 A residual is a
distance from a single data point and the curve of the model. The
total distance function is a combination of all the residuals. When
deriving the function $V_2$, we used `vertical' residuals (i.e.,
residuals which are parallel to the $y$-axis). We could, however, use
a horizontal residual (i.e., one parallel to the $x$-axis), or a
perpendicular residual (i.e., one at right-angles to the model
line). These three alternatives are illustrated below:
\begin{center}
\includegraphics[width=\textwidth]{Problem1/figures.3}
\end{center}
To calculate the total distance functions from these three residuals,
we can square each residual distance, sum all these, and find the
square root of this sum. The values of the seven residuals for these three possibilities are given below:
\begin{center}
\begin{tabular}{|c|c|c|c|}\hline
Data Point & Vertical & Horizontal & Perpendicular\\\hline
$(0,0  )$ &$0$ & $0$ & $0$\\
$(1,1.2)$ &$1.2-m $      & $1-\frac{1.2}{m}$ & $\frac{1.2-m }{\sqrt{m^2+1}}$\rule[-2pt]{0pt}{18pt} \\
$(2,3.8)$ &$3.8-2m$      & $2-\frac{3.8}{m}$ & $\frac{3.8-2m}{\sqrt{m^2+1}}$\rule[-2pt]{0pt}{18pt} \\
$(3,4.2)$ &$4.2-3m$      & $3-\frac{4.2}{m}$ & $\frac{4.2-3m}{\sqrt{m^2+1}}$\rule[-2pt]{0pt}{18pt} \\
$(4,6.3)$ &$6.3-4m$      & $4-\frac{6.3}{m}$ & $\frac{6.3-4m}{\sqrt{m^2+1}}$\rule[-2pt]{0pt}{18pt} \\
$(5,7.2)$ &$7.2-5m$      & $5-\frac{7.2}{m}$ & $\frac{7.2-5m}{\sqrt{m^2+1}}$\rule[-2pt]{0pt}{18pt} \\
$(6,7.3)$ &$7.3-6m$      & $6-\frac{7.3}{m}$ & $\frac{7.3-6m}{\sqrt{m^2+1}}$\rule[-2pt]{0pt}{18pt} \\\hline
\end{tabular}
\end{center}
From this table we can easily write down the different total distance
functions:
\begin{align*}
V_2(m) & = \sqrt{(1.2-m)^2+(3.8-2m)^2+
		 (4.2-3m)^2+(6.3-4m)^2+
		 (7.2-5m)^2+(7.3-6m)^2} \\
H_2(m) & = \sqrt{\left(1-\frac{1.2}{m}\right)^2+
		 \left(2-\frac{3.8}{m}\right)^2+
		 \left(3-\frac{4.2}{m}\right)^2+
		 \left(4-\frac{6.3}{m}\right)^2+
		 \left(5-\frac{7.2}{m}\right)^2+
		 \left(6-\frac{7.3}{m}\right)^2}\\
P_2(m) & = \sqrt{\left(\frac{1.2-m }{\sqrt{m^2+1}}\right)^2+
		 \left(\frac{3.8-2m}{\sqrt{m^2+1}}\right)^2+
\cdots +
		 \left(\frac{7.2-5m}{\sqrt{m^2+1}}\right)^2+
		 \left(\frac{7.3-6m}{\sqrt{m^2+1}}\right)^2}
\end{align*}
These functions are defined in the Mathematica file named
\texttt{M3\_P1\_OtherOFs.nb}. We find the critical point of these
functions, and use a test to confirm that the point is a local
minimum. Note that each function gives a different optimal value for
the slope of the of the straight line model.

\subsection{Different ways of totaling the distance}
Once the type of residual has been selected we have to have a scheme
for combining the individual residuals into a `total distance'. The
scheme we have been using is to square, add and square root the sum
(we can call this as `pythagorean scheme'). Other schemes
are:\begin{itemize}
\item Just sum the residuals.
\item Cube, sum, and cube root the sum.
\item Take the fourth power, sum, and take the fourth root of the sum.
\item Take the fifth power, sum, and take the fifth root of the sum.
\item etc.
    \end{itemize}
In other words we can define the total distance function, for this set
of data, by any one of the formulae
below, for any whole value of $p$:

\begin{align*}
V_p(m) & = \left[(1.2-m)^p+(3.8-2m)^p+
\cdots+
		 (7.2-5m)^p+(7.3-6m)^p\right]^{\frac{1}{p}} \\
H_p(m) & = \left[\left(1-\frac{1.2}{m}\right)^p+
		 \left(2-\frac{3.8}{m}\right)^p+
\cdots+
		 \left(5-\frac{7.2}{m}\right)^p+
		 \left(6-\frac{7.3}{m}\right)^p\right]^{\frac{1}{p}}\\
P_p(m) & = \left[\left(\frac{1.2-m }{\sqrt{m^2+1}}\right)^p+
		 \left(\frac{3.8-2m}{\sqrt{m^2+1}}\right)^p+
\cdots +
		 \left(\frac{7.2-5m}{\sqrt{m^2+1}}\right)^p+
		 \left(\frac{7.3-6m}{\sqrt{m^2+1}}\right)^p\right]^{\frac{1}{p}}
\end{align*}

Note that for odd values of $p$ the functions $V_p$, $H_p$ and $P_p$
do not have a minimum. They either increase or decrease. They do,
however, pass through the $x$-axis, meaning that there is a value of
$m$ for which the `total distance' between data and model is zero
(this is when the positive residuals cancel out the negative
residuals). So, for odd values of $p$ we find the optimal value of $m$
by finding the zero of the distance function.

