Steepest Descent

Just initialize them to 0.

That's where this occasion comes from in our update step. This would be simultaneous update.

And it turns out there are sometimes other versions of gradient descent that are not batch versions but instead do not look at the entire traning set but look at small subsets of the training sets at the time, and we'll talk about those versions later in this course as well.

Here is a prototypical loss function and the method for interation will be called gradient descent.

What we're going to do is apply gradient descent to minimize our squared error cost function.

But for now, using the algorithm you just learned, now we're using batch gradient descent, you now know how to implement gradient descent, or linear regression.

And, you notice that my line changed a little bit.

How can we optimize these two terms over here?

I have done them all, and the answer is no.

The gradient with respect to W0 is very similar.

Extinguisher activated.

So this update takes place where j 0, and j 1. So you're going to update j, theta0, and update theta1.

You do this until you find yourself with what's called a local minimum, where B resides.

Still dropping.

And I have also moved to a new point on my cost function.

In this video, let's talk about how to fit the parameters of that hypothesis.

So let me say what I mean by that.

Dropping off now.

Final descent.

So, that's the intuition in pictures. Let's

And what you should do is to really implement the simultaneous update of gradient descent.

And with that, with the next video, hopefully, we'll be able to give all the intuitions you need to apply gradient descent.

So, this is what we worked out in the previous videos.

So, let's see how gradient descent works.

Finally, just to give this another name, it turns out that the algorithm that we just went over is sometimes called batch gradient descent.

We'll go down after him.

Rate of decay?

So, this is what gradient descent looks like, and so actually there is no need to decrease alpha overtime.

Recurse into subdirectories

Down on your luck?

And you want to minimize over (theta0 up to theta n) of this J of (theta0 up to theta n).

We want to minimize this, and we're going to use gradient descent.

Later in this course we will talk about that method as well that just solves for the minimum cost function J without needing this multiple steps of gradient descent.

So if alpha is very large, then that corresponds to a very aggressive gradient descent procedure, where we're trying to take huge steps downhill.

Now, how about if the alpha is too large. So here's my function J of theta.

I'm not sure.

The big trials are closer to the true answer.

Money's safe. Let's take her down.

Make for the lower ground!

Finally, unfasten the lifting brackets mounted to each side of the bridge casting

This first time we ran gradient descent, we were starting at this point over here, right?

Hopeful Descent

When he arches his back, he gains altitude. When he pushes his shoulders forward, he goes into a dive.

We're in the pipe. Five by five.