How fitting.

We talked about the fit from data and we even talked about multivariate Gaussians.

The algorithm is amazingly simple.

And on this slide, I'm basically explaining all the problems you have.

I'm going to be fitting very complex high order polynomials that might fit the training set with much more complex functions

Given the following data, my first question is, can this data be fit exactly using a linear function that maps from X to Y?

For this quiz we will assume a degenerative case of 3 data points and just 1 cluster center.

Is just equal to 8.

As for the discussion of most of the learning algorithms, we're going to begin by talking about the cost function for fitting the parameters of the network.

Suppose we're fitting a high order polynomial like that shown here, but to prevent overfitting, we're going to use regularization, like that shown here, so we have this regularization term to try to keep the values of the parameters small.

So this is equal to y plus I'm just adding this whole term to both sides of this equation plus a squared over s squared y is equal to so this term is now gone, so it's equal to this stuff.

In that case, given that we're fitting a high order polynomial, this is a usual overfitting setting.

If x is equal to the square root of y, then x squared is just y.

You learned about what a Gaussian is.

Plus the derivative of this with respect to why is 3 times we're just doing the chain rule the derivative of y with respect to x.

So we're really curious

And then we get psi is equal to x squared plus 3x, plus some function of y.

Right?

Value in Y

And once again here is as if we fit this parameter theta to my cross validation set, which is why I am saving aside a separate test set that I am going to use to get a better estimate of how well my a parameter vector theta will generalize to previously unseen examples.

So, we'll do that for this figure, where maybe d equals 1, were going to be fitting very simple functions where as we are the right of this this may be d equals 4 or relatively may be even larger numbers.

And then y times y, well that's just y squared,

four y x. two times two times y times x.

Suppose we fit a linear regression model with a very high order polynomial, but to prevent overfitting, we are going to use regularization as shown here.

Base times height, x times x.

Well, let's say that I said that y prime plus y is equal to x plus 3.

Let's see.

So this is going to be the integral from t equals a to t equals b, of p of p of x, really, instead of writing x, y, it's x of t, right? x as a function of t, y as a function of t.

So this will be x plus y times x minus y.

If this were true, then we could rewrite this as the partial of psi, with respect to x, plus the partial of psi, with respect to y, times dy, dx, equal to 0.

The only difference between that and that is they put y's here.

So first lets add all the y terms

Plus 20 divided by 5 is 4. x squared divided by x is x. y squared divided by y is y.

But anyway, back to the problem.

So fn prime of x times the second function, g n of y, plus the first function, fn of x, times the derivative of the second function.

Y tile size

Y lens offset

There's the second.

I'll call the plus the constant due to y.

It's the derivative of y with respect to y is 1, times the derivative of y with respect to x, which is just y prime.

And then if we wanted to figure out what h of y is, we get h of y just integrate both sides, with respect to y is equal to y squared plus sorry y squared minus 2y.

And what makes this whole expression equal 0, in this case, it was y is equal to 0.

And a good place to start let's just set y being equal to f of x.

It's going to look something like that right there.

And actually, I'm going to do that right now.