How fitting.

I'm going to be fitting very complex high order polynomials that might fit the training set with much more complex functions

And on this slide, I'm basically explaining all the problems you have.

We talked about the fit from data and we even talked about multivariate Gaussians.

The algorithm is amazingly simple.

Suppose we're fitting a high order polynomial like that shown here, but to prevent overfitting, we're going to use regularization, like that shown here, so we have this regularization term to try to keep the values of the parameters small.

Suppose we fit a linear regression model with a very high order polynomial, but to prevent overfitting, we are going to use regularization as shown here.

As for the discussion of most of the learning algorithms, we're going to begin by talking about the cost function for fitting the parameters of the network.

For this quiz we will assume a degenerative case of 3 data points and just 1 cluster center.

So, that's Jcv of theta because with high bias we won't be fitting.

So, we'll do that for this figure, where maybe d equals 1, were going to be fitting very simple functions where as we are the right of this this may be d equals 4 or relatively may be even larger numbers.

In that case, given that we're fitting a high order polynomial, this is a usual overfitting setting.

And so this degree of polynomial, so the parameter is no longer fit to the test set.

You learned about what a Gaussian is.

And the more general principal is that, once your parameters were fit to some set of data maybe the training set, maybe something else then the error of your hypothesis as measured on that same data set, such as the training error, that's unlikely to be a good estimate of your actual generalization error, that is, of how well the hypothesis will generalize to new examples.

And once again here is as if we fit this parameter theta to my cross validation set, which is why I am saving aside a separate test set that I am going to use to get a better estimate of how well my a parameter vector theta will generalize to previously unseen examples.

So here are examples of two models with, different amounts of Baryonic matter.

left corresponds to a high bias problem, that is, if you are fitting an overly

And the key that distinguishes these two cases is if you have a high bias problem your training set error will also be high as your hypothesis just not fitting the training set well.

So I can look at my first model, my second model, fits with these different values of the regularization parameter and evaluate them on my cross validation set basically measure the average squared error of each of these parameter vectors theta on my cross validation set.

Given the following data, my first question is, can this data be fit exactly using a linear function that maps from X to Y?

And the reason is, what we've done is, we've fit this extra the parameter d, that is this degree of polynomial, and we'll fit that parameter d using the test set.

And by having fit the parameter to the test set, this means that the performance of the hypothesis on that test set may not be a fair estimate of how well the hypothesis is likely to do on examples we haven't seen before.

As we increase the degree of the polynomial, we're going to fit our training set better and better and so, if d equals 1 that ever rose to the high training error.

And in this video, we also stepped through an entire process of giving data set, we have, fitting the parameters, doing parameter estimations.

Often that matter, if we look at the test set error we'll get a pretty similar result as if we were to plot the cross validation error.

In this video, we're going to put together gradient descent with our cost function, and that will give us an algorithm for

We get this function over here that specifies any probability for a value X given a specific mu and sigma squared.

longer fair to evaluate my hypothesis on this test set.

And these are exactly the same calculations as before when we fit a Gaussian but just weighted by the soft correspondence of a data point to each Gaussian.

And that's indeed what's seen through a number of other pieces of evidence.

low order polynomial such as a plus one, when we really needed a higher order polynomial to fit the data.

For example, the size of a house is relative to its price, and initially, we talked about scatter plots, we talked about bar charts but we didn't talk about what's the holy grail of statistics, and that is fitting a line through the data points.

What we've done is we fit that parameter d, and we'll set d equals 4, and we did so using the cross validation set.

And if you will verify them, you will find that the maximum likelihood estimate for sigma squared is the average deviation of data points from the mean mu.

And all the test cases we'll be using will have the property that the lower and upper quartile are well defined elements.

So different cosmological models in different combinations of parameters Make a prediction of a pair that will go through these data points and shown here is a particularly well fitting model that is model that's now pretty much generally accepted.

Now that we can measure kinematics of ellipticals, reflecting their potential wells, we can fit dynamical models and find what their masses are so here is from a large survey of gal , of particular galaxies by a group called Spider unknown and Carvalho unknown collaborators.

And we'll just keep adding them, dot, dot, dot, to a bunch of numbers, to big N minus 1 plus big N, right?

You have to do it. Hurry.

But... but...

Brands like Nike are doing it.

But I've run out of time and I don't want this video to get rejected.

Click on this side to pull it out and click to finish.

But when He broke bread, then we knew Him.