en_ub3UlaYRt9I.srt

0
00:00:00,659 --> 00:00:02,670
Hello, and welcome!

1
00:00:02,670 --> 00:00:07,140
In this video, we’ll be covering non-linear regression basics.

2
00:00:07,140 --> 00:00:09,849
So let’s get started!

3
00:00:09,849 --> 00:00:18,900
These data points correspond to China's Gross Domestic Product (or GDP) from 1960 to 2014.

4
00:00:18,900 --> 00:00:25,010
The first column, is the years, and the second, is China's corresponding annual gross domestic

5
00:00:25,010 --> 00:00:29,390
income in US dollars for that year.

6
00:00:29,390 --> 00:00:32,050
This is what the data points look like.

7
00:00:32,049 --> 00:00:35,140
Now, we have a couple of interesting questions.

8
00:00:35,140 --> 00:00:40,210
First, “Can GDP be predicted based on time?”

9
00:00:40,210 --> 00:00:45,350
And second, “Can we use a simple linear regression to model it?”

10
00:00:45,350 --> 00:00:52,239
Indeed, if the data shows a curvy trend, then linear regression will not produce very accurate

11
00:00:52,239 --> 00:00:59,330
results when compared to a non-linear regression -- simply because, as the name implies, linear

12
00:00:59,330 --> 00:01:03,290
regression presumes that the data is linear.

13
00:01:03,290 --> 00:01:09,520
The scatterplot shows that there seems to be a strong relationship between GDP and time,

14
00:01:09,520 --> 00:01:13,160
but the relationship is not linear.

15
00:01:13,160 --> 00:01:20,640
As you can see, the growth starts off slowly, then from 2005 onward, the growth is very

16
00:01:20,640 --> 00:01:21,640
significant.

17
00:01:21,640 --> 00:01:27,570
And finally, it decelerates slightly in the 2010s.

18
00:01:27,570 --> 00:01:32,390
It kind of looks like either a logistical or exponential function.

19
00:01:32,390 --> 00:01:40,060
So, it requires a special estimation method of the non-linear regression procedure.

20
00:01:40,060 --> 00:01:45,890
For example, if we assume that the model for these data points are exponential functions,

21
00:01:45,890 --> 00:01:59,030
such as y ̂ = θ_0 + θ_1 〖θ_2〗^x, our job is to estimate the parameters of the model,

22
00:01:59,030 --> 00:02:06,930
i.e. θs, and use the fitted model to predict GDP for unknown or future cases.

23
00:02:06,930 --> 00:02:13,410
In fact, many different regressions exist that can be used to fit whatever the dataset

24
00:02:13,410 --> 00:02:15,030
looks like.

25
00:02:15,030 --> 00:02:22,640
You can see a quadratic and cubic regression lines here, and it can go on and on to infinite

26
00:02:22,640 --> 00:02:24,030
degrees.

27
00:02:24,030 --> 00:02:29,580
In essence, we can call all of these "polynomial regression," where the relationship between

28
00:02:29,580 --> 00:02:37,640
the independent variable x and the dependent variable y is modelled as an nth degree polynomial

29
00:02:37,640 --> 00:02:40,160
in x.

30
00:02:40,160 --> 00:02:44,680
With many types of regression to choose from, there’s a good chance that one will fit

31
00:02:44,680 --> 00:02:46,370
your dataset well.

32
00:02:46,370 --> 00:02:52,740
Remember, it’s important to pick a regression that fits the data the best.

33
00:02:52,740 --> 00:02:57,290
So, what is polynomial Regression?

34
00:02:57,290 --> 00:03:00,710
Polynomial regression fits a curved line to your data.

35
00:03:00,710 --> 00:03:14,560
A simple example of polynomial, with degree 3, is shown as y ̂ = θ_0 + θ_1x + θ_2x^2

36
00:03:14,560 --> 00:03:24,380
+ θ_3x^3 or to the power of 3, where θs are parameters  to be estimated that makes the model fit perfectly

37
00:03:24,380 --> 00:03:27,350
to the underlying data.

38
00:03:27,350 --> 00:03:33,709
Though the relationship between x and y is non-linear here, and polynomial regression

39
00:03:33,709 --> 00:03:41,959
can fit them, a polynomial regression model can still be expressed as linear regression.

40
00:03:41,959 --> 00:03:47,230
I know it's a bit confusing, but let’s look at an example.

41
00:03:47,230 --> 00:03:59,520
Given the 3rd degree polynomial equation, by defining x_1 = x and x_2 = x^2 or x to the power of 2 and so on,

42
00:03:59,520 --> 00:04:08,310
the model is converted to a simple linear regression with new variables, as y ̂ = θ_0+

43
00:04:08,310 --> 00:04:18,180
θ_1x_1 + θ_2x_2 + θ_3x_3. This model is linear in the parameters to

44
00:04:18,180 --> 00:04:20,450
be estimated, right?

45
00:04:20,450 --> 00:04:28,030
Therefore, this polynomial regression is considered to be a special case of traditional multiple

46
00:04:28,030 --> 00:04:29,720
linear regression.

47
00:04:29,720 --> 00:04:36,980
So, you can use the same mechanism as linear regression to solve such a problem.

48
00:04:36,980 --> 00:04:45,320
Therefore, polynomial regression models CAN fit using the model of least squares.

49
00:04:45,320 --> 00:04:51,110
Least squares is a method for estimating the unknown parameters in a linear regression

50
00:04:51,110 --> 00:04:57,200
model, by minimizing the sum of the squares of the differences between the observed dependent

51
00:04:57,200 --> 00:05:03,150
variable in the given dataset and those predicted by the linear function.

52
00:05:03,150 --> 00:05:08,430
So, what is “non-linear regression” exactly?

53
00:05:08,430 --> 00:05:14,520
First, non-linear regression is a method to model a non-linear relationship between the

54
00:05:14,520 --> 00:05:18,830
dependent variable and a set of independent variables.

55
00:05:18,830 --> 00:05:25,940
Second, for a model to be considered non-linear, y ̂ must be a non-linear function of the

56
00:05:25,940 --> 00:05:30,720
parameters θ, not necessarily the features x.

57
00:05:30,720 --> 00:05:36,790
When it comes to non-linear equation, it can be the shape of exponential, logarithmic,

58
00:05:36,790 --> 00:05:41,210
and logistic, or many other types.

59
00:05:41,210 --> 00:05:47,220
As you can see, in all of these equations, the change of y ̂ depends on changes in the

60
00:05:47,220 --> 00:05:51,410
parameters θ, not necessarily on x only.

61
00:05:51,410 --> 00:05:59,070
That is, in non-linear regression, a model is non-linear by parameters.

62
00:05:59,070 --> 00:06:05,560
In contrast to linear regression, we cannot use the ordinary "least squares" method to fit

63
00:06:05,560 --> 00:06:13,840
the data in non-linear regression, and in general, estimation of the parameters is not easy.

64
00:06:13,840 --> 00:06:20,479
Let me answer two important questions here: First, “How can I know if a problem is linear

65
00:06:20,479 --> 00:06:23,410
or non-linear in an easy way?”

66
00:06:23,410 --> 00:06:28,340
To answer this question, we have to do two things:

67
00:06:28,340 --> 00:06:33,710
The first is to visually figure out if the relation is linear or non-linear.

68
00:06:33,710 --> 00:06:40,830
It’s best to plot bivariate plots of output variables with each input variable.

69
00:06:40,830 --> 00:06:48,400
Also, you can calculate the correlation coefficient between independent and dependent variables,

70
00:06:48,400 --> 00:06:56,290
and if for all variables it is 0.7 or higher there is a linear tendency, and, thus, it’s

71
00:06:56,290 --> 00:07:00,520
not appropriate to fit a non-linear regression.

72
00:07:00,520 --> 00:07:06,889
The second thing we have to do is to use non-linear regression instead of linear regression when

73
00:07:06,889 --> 00:07:12,199
we cannot accurately model the relationship with linear parameters.

74
00:07:12,199 --> 00:07:18,800
The second important questions is, “How should I model my data, if it displays non-linear

75
00:07:18,800 --> 00:07:21,229
on a scatter plot?”

76
00:07:21,229 --> 00:07:28,229
Well, to address this, you have to use either a polynomial regression, use a non-linear

77
00:07:28,229 --> 00:07:34,430
regression model, or "transform" your data, which is not in scope for this course.

78
00:07:34,430 --> 00:07:35,070
Thanks for watching.