2.1 Linear Regression with Multiple Variables | Multiple features --[Machine Learning | Andrew Ng]
 Welcome back. In this week, we'll learn to make linear regression much faster and much more powerful, and by the end of this week, you'll be two-thirds of the way to finishing this first course. Let's start by looking at a version of linear regression that can look at not just one feature, but a lot of different features. Let's take a look. In the original version of linear regression, you had a single feature x, the size of the house, and you're able to predict y, the price of the house. So the model was FWB of x equals Wx plus b. But now, what if you did not only have the size of the house as a feature with which to try to predict the price, but if you also knew the number of bedrooms, the number of floors, and the age of the home in years, it seems like this would give you a lot more information with which to predict the price. To introduce a little bit of new notation, we're going to use the variables x subscript 1, x subscript 2, x subscript 3, and x subscript 4 to denote the four features. And for simplicity, let's introduce a little bit more notation. We'll write x subscript j, or sometimes I'll just say for short x sub j to represent the list of features. So here, j will go from 1 through 4 because we have four features. I'm going to use lowercase n to denote the total number of features. So in this example, n is equal to 4. As before, we'll use x super stripped i to denote the I've training example. So here, x super strip i is actually going to be a list of four numbers, or sometimes we'll call this a vector that includes all the features of the I've training example. So as a concrete example, x super strip in parentheses 2 will be a vector of the features for the second training example. So it will equal to this 1416, 32, and 40. And technically, I'm writing these numbers in a row, so sometimes this is called a row vector rather than a column vector. But if you don't know what the difference is, don't worry about it. It's not that important for this purpose. And to refer to a specific feature in the I've training example, I will write x super strip i subscript j. So for example, x super strip 2 subscript 3 will be the value of the third feature, that is the number of flaws in the second training example. And so that's going to be equal to two. Sometimes in order to emphasize that this x two is not a number, but it's actually a list of numbers that is a vector, we'll draw an arrow on top of that, just to visually show that is a vector. And over here as well. But you don't have to draw this arrow in your notation. You can think of the arrow as an optional signifier that sometimes use just to emphasize that this is a vector and not a number. Now that we have multiple features, let's take a look at what our model would look like. Previously, this is how we defined the model where x was a single feature. So a single number. But now with multiple features, we're going to define it differently. Instead, the model will be f WB of x equals W1 x1 plus W2 x2 plus W3 x3 plus W4 x4 plus B. Concretely, for housing price prediction, one possible model may be that we estimate the price of the house as 0.1 times x1, the size of the house, plus four times x2, the number of bedrooms, plus 10 times x3, the number of floors, minus two times x4, the age of the house in years plus 80. Let's think a bit about how you might interpret these parameters. If the model is trying to predict the price of the house in thousands of dollars, you can think of this B equals 80 as saying that the base price of a house starts off at maybe $80,000, assuming it has no size, no bedrooms, no floor and no age. And you can think of this 0.1 as saying that maybe for every additional square foot, the price will increase by $0.1,000 or by $100 because we're saying that for each square foot, the price increases by 0.1, you know, times $1,000, which is $100. And maybe for each additional bathroom, the price increases by $4,000. And for each additional floor, the price may increase by $10,000. And for each additional year of the house's age, the price may decrease by $2,000 because the parameter is negative two. And in general, if you have n features, then the model will look like this. Here again is the definition of the model with n features. What we're going to do next is introduce a little bit of notation to rewrite this expression in a simpler but equivalent way. Let's define w as a list of numbers that lists the parameters w1, w2, w3, all the way through wn. In mathematics, this is called a vector. And sometimes to designate that this is a vector, which just means a list of numbers, I'm going to draw a little arrow on top. You don't always have to draw this arrow. And you can do so or not in your own notation. So you can think of this little arrow as just an optional signifier to remind us that this is a vector. If you've taken a linear algebra class before, you might recognize that this is a row vector as opposed to a column vector. But if you don't know what those terms means, you don't need to worry about it. Next, same as before, b is a single number and not a vector. And so this vector w together with this number b are the parameters of the model. Let me also write x as a list or a vector, again, a row vector that lists all of the features x1, x2, x3 up through xn. This is again a vector. So I'm going to add a little arrow up on top to signify. So in the notation up on top, we can also add little arrows here and here to signify that that w and that x are actually these lists of numbers that they're actually these vectors. So with this notation, the model can now be rewritten more succinctly as f of x equals the vector w dot and this dot refers to a dot product from linear algebra of x, the vector plus the number b. So what is this dot product thing? Well, the dot products of two vectors of two lists of numbers w and x is computed by taking the corresponding pairs of numbers w1 and x1, multiplying that w2 x2, multiplying that w3 x3, multiplying that all the way up to wn xn, multiplying that and then summing up all of these products. Writing that out, this means that the dot product is equal to w1 x1 plus w2 x2 plus w3 x3 plus all the way up to wn xn. And then finally we add back in the b on top. And you notice that this gives us exactly the same expression as we had on top. So the dot product notation lets you write the model in a more compact form with fewer characters. The name for this type of linear regression model with multiple input features is multiple linear regression. This is in contrast to univariate regression, which had just one feature. And by the way, you might think this algorithm is called multivariate regression, but that term actually refers to something else that we won't be using here. So I'm going to refer to this model as multiple linear regression. And so that's it for linear regression with multiple features, which is also called multiple linear regression. In order to implement this, there's a really neat trick called vectorization, which will make it much simpler to implement this and many other learning algorithms. Let's go on to the next video to take a look at what is vectorization.
[{"start": 0.0, "end": 7.2, "text": " Welcome back. In this week, we'll learn to make linear regression much faster and much"}, {"start": 7.2, "end": 12.620000000000001, "text": " more powerful, and by the end of this week, you'll be two-thirds of the way to finishing"}, {"start": 12.620000000000001, "end": 17.42, "text": " this first course. Let's start by looking at a version of linear regression that can"}, {"start": 17.42, "end": 23.36, "text": " look at not just one feature, but a lot of different features. Let's take a look."}, {"start": 23.36, "end": 30.64, "text": " In the original version of linear regression, you had a single feature x, the size of the"}, {"start": 30.64, "end": 40.84, "text": " house, and you're able to predict y, the price of the house. So the model was FWB of x equals"}, {"start": 40.84, "end": 47.879999999999995, "text": " Wx plus b. But now, what if you did not only have the size of the house as a feature with"}, {"start": 47.88, "end": 53.6, "text": " which to try to predict the price, but if you also knew the number of bedrooms, the"}, {"start": 53.6, "end": 58.440000000000005, "text": " number of floors, and the age of the home in years, it seems like this would give you"}, {"start": 58.440000000000005, "end": 62.5, "text": " a lot more information with which to predict the price."}, {"start": 62.5, "end": 67.36, "text": " To introduce a little bit of new notation, we're going to use the variables x subscript"}, {"start": 67.36, "end": 76.0, "text": " 1, x subscript 2, x subscript 3, and x subscript 4 to denote the four features. And for simplicity,"}, {"start": 76.0, "end": 81.72, "text": " let's introduce a little bit more notation. We'll write x subscript j, or sometimes I'll"}, {"start": 81.72, "end": 88.72, "text": " just say for short x sub j to represent the list of features. So here, j will go from"}, {"start": 88.72, "end": 95.8, "text": " 1 through 4 because we have four features. I'm going to use lowercase n to denote the"}, {"start": 95.8, "end": 102.92, "text": " total number of features. So in this example, n is equal to 4. As before, we'll use x super"}, {"start": 102.92, "end": 112.12, "text": " stripped i to denote the I've training example. So here, x super strip i is actually going"}, {"start": 112.12, "end": 120.2, "text": " to be a list of four numbers, or sometimes we'll call this a vector that includes all"}, {"start": 120.2, "end": 128.72, "text": " the features of the I've training example. So as a concrete example, x super strip in"}, {"start": 128.72, "end": 135.68, "text": " parentheses 2 will be a vector of the features for the second training example. So it will"}, {"start": 135.68, "end": 144.68, "text": " equal to this 1416, 32, and 40. And technically, I'm writing these numbers in a row, so sometimes"}, {"start": 144.68, "end": 149.32, "text": " this is called a row vector rather than a column vector. But if you don't know what"}, {"start": 149.32, "end": 155.16, "text": " the difference is, don't worry about it. It's not that important for this purpose. And to"}, {"start": 155.16, "end": 164.44, "text": " refer to a specific feature in the I've training example, I will write x super strip i subscript"}, {"start": 164.44, "end": 173.64, "text": " j. So for example, x super strip 2 subscript 3 will be the value of the third feature,"}, {"start": 173.64, "end": 178.6, "text": " that is the number of flaws in the second training example. And so that's going to be"}, {"start": 178.6, "end": 186.56, "text": " equal to two. Sometimes in order to emphasize that this x two is not a number, but it's"}, {"start": 186.56, "end": 192.56, "text": " actually a list of numbers that is a vector, we'll draw an arrow on top of that, just to"}, {"start": 192.56, "end": 200.28, "text": " visually show that is a vector. And over here as well. But you don't have to draw this arrow"}, {"start": 200.28, "end": 207.35999999999999, "text": " in your notation. You can think of the arrow as an optional signifier that sometimes use"}, {"start": 207.36, "end": 213.24, "text": " just to emphasize that this is a vector and not a number. Now that we have multiple features,"}, {"start": 213.24, "end": 218.68, "text": " let's take a look at what our model would look like. Previously, this is how we defined"}, {"start": 218.68, "end": 225.48000000000002, "text": " the model where x was a single feature. So a single number. But now with multiple features,"}, {"start": 225.48000000000002, "end": 236.64000000000001, "text": " we're going to define it differently. Instead, the model will be f WB of x equals W1 x1"}, {"start": 236.64, "end": 247.27999999999997, "text": " plus W2 x2 plus W3 x3 plus W4 x4 plus B. Concretely, for housing price prediction, one possible"}, {"start": 247.27999999999997, "end": 254.32, "text": " model may be that we estimate the price of the house as 0.1 times x1, the size of the"}, {"start": 254.32, "end": 263.15999999999997, "text": " house, plus four times x2, the number of bedrooms, plus 10 times x3, the number of floors, minus"}, {"start": 263.16, "end": 269.68, "text": " two times x4, the age of the house in years plus 80. Let's think a bit about how you might"}, {"start": 269.68, "end": 274.8, "text": " interpret these parameters. If the model is trying to predict the price of the house in"}, {"start": 274.8, "end": 282.18, "text": " thousands of dollars, you can think of this B equals 80 as saying that the base price"}, {"start": 282.18, "end": 288.36, "text": " of a house starts off at maybe $80,000, assuming it has no size, no bedrooms, no floor and"}, {"start": 288.36, "end": 296.0, "text": " no age. And you can think of this 0.1 as saying that maybe for every additional square foot,"}, {"start": 296.0, "end": 304.42, "text": " the price will increase by $0.1,000 or by $100 because we're saying that for each square"}, {"start": 304.42, "end": 313.24, "text": " foot, the price increases by 0.1, you know, times $1,000, which is $100. And maybe for"}, {"start": 313.24, "end": 320.24, "text": " each additional bathroom, the price increases by $4,000. And for each additional floor,"}, {"start": 320.24, "end": 326.16, "text": " the price may increase by $10,000. And for each additional year of the house's age, the"}, {"start": 326.16, "end": 334.52, "text": " price may decrease by $2,000 because the parameter is negative two. And in general, if you have"}, {"start": 334.52, "end": 340.74, "text": " n features, then the model will look like this."}, {"start": 340.74, "end": 346.72, "text": " Here again is the definition of the model with n features. What we're going to do next"}, {"start": 346.72, "end": 351.72, "text": " is introduce a little bit of notation to rewrite this expression in a simpler but equivalent"}, {"start": 351.72, "end": 360.16, "text": " way. Let's define w as a list of numbers that lists the parameters w1, w2, w3, all the way"}, {"start": 360.16, "end": 367.32, "text": " through wn. In mathematics, this is called a vector. And sometimes to designate that"}, {"start": 367.32, "end": 371.88, "text": " this is a vector, which just means a list of numbers, I'm going to draw a little arrow"}, {"start": 371.88, "end": 378.48, "text": " on top. You don't always have to draw this arrow. And you can do so or not in your own"}, {"start": 378.48, "end": 384.28, "text": " notation. So you can think of this little arrow as just an optional signifier to remind"}, {"start": 384.28, "end": 390.4, "text": " us that this is a vector. If you've taken a linear algebra class before, you might recognize"}, {"start": 390.4, "end": 395.52, "text": " that this is a row vector as opposed to a column vector. But if you don't know what"}, {"start": 395.52, "end": 401.2, "text": " those terms means, you don't need to worry about it. Next, same as before, b is a single"}, {"start": 401.2, "end": 409.91999999999996, "text": " number and not a vector. And so this vector w together with this number b are the parameters"}, {"start": 409.91999999999996, "end": 419.91999999999996, "text": " of the model. Let me also write x as a list or a vector, again, a row vector that lists"}, {"start": 419.92, "end": 428.16, "text": " all of the features x1, x2, x3 up through xn. This is again a vector. So I'm going to"}, {"start": 428.16, "end": 437.36, "text": " add a little arrow up on top to signify. So in the notation up on top, we can also add"}, {"start": 437.36, "end": 445.48, "text": " little arrows here and here to signify that that w and that x are actually these lists"}, {"start": 445.48, "end": 453.6, "text": " of numbers that they're actually these vectors. So with this notation, the model can now be"}, {"start": 453.6, "end": 463.92, "text": " rewritten more succinctly as f of x equals the vector w dot and this dot refers to a"}, {"start": 463.92, "end": 472.0, "text": " dot product from linear algebra of x, the vector plus the number b. So what is this"}, {"start": 472.0, "end": 478.6, "text": " dot product thing? Well, the dot products of two vectors of two lists of numbers w and"}, {"start": 478.6, "end": 488.48, "text": " x is computed by taking the corresponding pairs of numbers w1 and x1, multiplying that"}, {"start": 488.48, "end": 499.24, "text": " w2 x2, multiplying that w3 x3, multiplying that all the way up to wn xn, multiplying"}, {"start": 499.24, "end": 507.08, "text": " that and then summing up all of these products. Writing that out, this means that the dot"}, {"start": 507.08, "end": 522.08, "text": " product is equal to w1 x1 plus w2 x2 plus w3 x3 plus all the way up to wn xn. And then"}, {"start": 522.08, "end": 530.36, "text": " finally we add back in the b on top. And you notice that this gives us exactly the same"}, {"start": 530.36, "end": 537.84, "text": " expression as we had on top. So the dot product notation lets you write the model in a more"}, {"start": 537.84, "end": 544.5600000000001, "text": " compact form with fewer characters. The name for this type of linear regression model with"}, {"start": 544.5600000000001, "end": 550.5600000000001, "text": " multiple input features is multiple linear regression. This is in contrast to univariate"}, {"start": 550.56, "end": 556.3599999999999, "text": " regression, which had just one feature. And by the way, you might think this algorithm"}, {"start": 556.3599999999999, "end": 561.68, "text": " is called multivariate regression, but that term actually refers to something else that"}, {"start": 561.68, "end": 568.0799999999999, "text": " we won't be using here. So I'm going to refer to this model as multiple linear regression."}, {"start": 568.0799999999999, "end": 573.4799999999999, "text": " And so that's it for linear regression with multiple features, which is also called multiple"}, {"start": 573.4799999999999, "end": 580.1199999999999, "text": " linear regression. In order to implement this, there's a really neat trick called vectorization,"}, {"start": 580.12, "end": 585.44, "text": " which will make it much simpler to implement this and many other learning algorithms. Let's"}, {"start": 585.44, "end": 612.44, "text": " go on to the next video to take a look at what is vectorization."}]