Case Study (Car Price) :¶

Linear Regression Model¶

Overview¶

- Importing the relevant libraries

- Loading data

- Declaring Dependent and Independent variables

- Scaling the data

- Train & Test Split

- Creating & Fitting Regression Model

- Predictions (x_train)

- Scatter Plot (Comparing Predictions with Targets)

- Residual Plot (Checking for anomalies)

- Finding the R-Squared

- Finding the Intercept

- Creating a Summary Table (Feature & Weights)

- Testing

- Predictions (x_test)

- Scatter Plot (Comparing Predictions with Targets)

- Creating a Table (Comparong Predictions with Targets)

- Solution: Reset the Index

- Creating a Table (Comparing Predictions with Targets) 

- Exploring Descriptive Statistics

- Sorting the table by difference%

Importing the relevant libraries¶

import numpy as np
import pandas as pd
import statsmodels.api as sm
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
import seaborn as sns
sns.set()

Loading data¶

url = "https://datascienceschools.github.io/Machine_Learning/CaseStudy/LinearRegression/carprice_editted3.csv"

df = pd.read_csv(url)

df.head()

Declaring Dependent and Independent variables¶

- The target (dependent variable) is 'log price'
- The inputs are everything except the dependent variable, so we can simply drop it

target = df['log_price']

inputs = df.drop(['log_price'],axis=1)

Scaling the data¶

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

scaler.fit(inputs)

inputs_scaled = scaler.transform(inputs)

Train & Test Split¶

- Split the variables with an 80-20 split -> test_size=0.2

- random_state = 365

from sklearn.model_selection import train_test_split

x_train, x_test, y_train, y_test = train_test_split(inputs_scaled, target, test_size=0.2, random_state=365)

Creating & Fitting Regression Model¶

model = LinearRegression()

model.fit(x_train,y_train)

LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)

Predictions (x_train)¶

- Store in y_hat as this is the 'theoretical' name of the predictions

y_hat = model.predict(x_train)

Scatter Plot (Comparing Predictions with Targets)¶

 - The simplest way to compare the targets (y_train) and the predictions (y_hat) 

    -> is to plot them on a scatter plot

- The closer the points to the 45-degree line, the better the prediction

- Sometimes the plot will have different scales of the x-axis and the y-axis

- In order for x-axis and y-axis to be the same, add

        plt.xlim(6,13)
        plt.ylim(6,13)

plt.scatter(y_train, y_hat)

plt.xlabel('Targets (y_train)',size=18)
plt.ylabel('Predictions (y_hat)',size=18)

plt.xlim(6,13)
plt.ylim(6,13)

plt.show()

Residual Plot (checking for anomalies)¶

- We can plot the PDF of the residuals and check for anomalies

- In the best case scenario this plot should be normally distributed

- In our case we notice that there are many negative residuals (far away from the mean)

- Given the definition of the residuals (y_train - y_hat), negative values imply
  that y_hat (predictions) are much higher than y_train (the targets)

sns.distplot(y_train - y_hat)

plt.title("Residuals PDF", size=18)

Text(0.5, 1.0, 'Residuals PDF')

Finding the R-Squared¶

# Note that this is NOT the adjusted R-squared
# in other words... find the Adjusted R-squared to have the appropriate measure :)

model.score(x_train,y_train)

0.744996578792662

Finding the Intercept¶

model.intercept_

9.415239458021299

Creating a Table (Feature & Weights)¶

summary = pd.DataFrame(inputs.columns.values, columns=['Features'])

summary['Weights'] = model.coef_

summary

Testing¶

- Testing will be done on a dataset that the algorithm has never seen

- Our test inputs are 'x_test', while the outputs: 'y_test' 

- We SHOULD NOT TRAIN THE MODEL ON THEM, we just feed them and find the predictions

- If the predictions are far off, we will know that our model overfitted

Prediction (x_test)¶

- To obtain the actual prices, we take the exponential of the log_price

        -> np.exp(y_hat_test)

y_hat_test = model.predict(x_test)

Scatter Plot ( Comparing Predictions with Targets)¶

- Create a scatter plot with the test targets and the test predictions

- You can include the argument 'alpha' which will introduce opacity to the graph

plt.scatter(y_test, y_hat_test, alpha=0.2)

plt.xlabel('Targets (y_test)',size=18)
plt.ylabel('Predictions (y_hat_test)',size=18)

plt.xlim(6,13)
plt.ylim(6,13)

plt.show()

Creating a Table (Comparing Predictions with Targets)¶

df_performance = pd.DataFrame(np.exp(y_hat_test), columns=['Prediction'])

df_performance['Target'] = np.exp(y_test)

df_performance.head()

Solution: Reset the Index¶

y_test = y_test.reset_index(drop=True)

Creating a Table (Comparing Predictions with Targets)¶

- Residual -> the difference between the targets and the predictions

- OLS is an algorithm minimizing the total sum of squared errors (residuals)

- Difference% -> the absolute difference in percentage

df_performance = pd.DataFrame(np.exp(y_hat_test), columns=['Prediction'])

df_performance['Target'] = np.exp(y_test)

df_performance['Residual'] = df_performance['Target'] - df_performance['Prediction']

df_performance['Difference%'] = np.absolute(df_performance['Residual']/df_performance['Target']*100)

df_performance.head()

Exploring Descriptive Statistics¶

df_performance.describe()

Sorting the table by difference%¶

- To see all rows, use 

    -> pd.options.display.max_rows = 999


- To make the dataset clear, display the result with only 2 digits after the dot 

    -> pd.set_option('display.float_format', lambda x: '%.2f' % x)

pd.options.display.max_rows = 999

pd.set_option('display.float_format', lambda x: '%.2f' % x)

df_performance.sort_values(by=['Difference%'])

	log_price	Mileage	EngineV	Brand_BMW	Brand_Mercedes-Benz	Brand_Toyota	Body_sedan	Body_van	Engine Type_Gas	Engine Type_Petrol	Registration_yes
0	8.342840	277	2.0	1	0	0	1	0	0	1	1
1	8.974618	427	2.9	0	1	0	0	1	0	0	1
2	9.495519	358	5.0	0	1	0	1	0	1	0	1
3	10.043249	240	4.2	0	0	0	0	0	0	1	1
4	9.814656	120	2.0	0	0	1	0	0	0	1	1

	Features	Weights
0	Mileage	-0.448713
1	EngineV	0.209035
2	Brand_BMW	0.014250
3	Brand_Mercedes-Benz	0.012882
4	Brand_Mitsubishi	-0.140552
5	Brand_Renault	-0.179909
6	Brand_Toyota	-0.060550
7	Brand_Volkswagen	-0.089924
8	Body_hatch	-0.145469
9	Body_other	-0.101444
10	Body_sedan	-0.200630
11	Body_vagon	-0.129887
12	Body_van	-0.168597
13	Engine Type_Gas	-0.121490
14	Engine Type_Other	-0.033368
15	Engine Type_Petrol	-0.146909
16	Registration_yes	0.320473

	Prediction	Target
0	10685.501696	NaN
1	3499.255242	7900.0
2	7553.285218	NaN
3	7463.963017	NaN
4	11353.490075	NaN

	Prediction	Target	Residual	Difference%
0	10685.50	2300.00	-8385.50	364.59
1	3499.26	2800.00	-699.26	24.97
2	7553.29	2500.00	-5053.29	202.13
3	7463.96	6400.00	-1063.96	16.62
4	11353.49	9150.00	-2203.49	24.08

	Prediction	Target	Residual	Difference%
count	774.000000	774.000000	774.000000	774.000000
mean	15946.760167	18165.817106	2219.056939	36.256693
std	13133.197604	19967.858908	10871.218143	55.066507
min	1320.562768	1200.000000	-29456.498331	0.062794
25%	7413.644234	6900.000000	-2044.191251	12.108022
50%	11568.168859	11600.000000	142.518577	23.467728
75%	20162.408805	20500.000000	3147.343497	39.563570
max	77403.055224	126000.000000	85106.162329	512.688080

	Prediction	Target	Residual	Difference%
698	30480.85	30500.00	19.15	0.06
742	16960.31	16999.00	38.69	0.23
60	12469.21	12500.00	30.79	0.25
110	25614.14	25500.00	-114.14	0.45
367	42703.68	42500.00	-203.68	0.48
369	3084.69	3100.00	15.31	0.49
769	29651.73	29500.00	-151.73	0.51
272	9749.53	9800.00	50.47	0.52
714	23118.07	22999.00	-119.07	0.52
630	8734.58	8800.00	65.42	0.74
380	3473.79	3500.00	26.21	0.75
648	21174.10	21335.00	160.90	0.75
308	8967.74	8900.00	-67.74	0.76
665	17858.02	18000.00	141.98	0.79
379	17654.84	17800.00	145.16	0.82
719	11391.95	11500.00	108.05	0.94
102	28625.56	28900.00	274.44	0.95
94	7724.17	7800.00	75.83	0.97
561	6429.03	6500.00	70.97	1.09
242	7597.39	7500.00	-97.39	1.30
528	18555.09	18800.00	244.91	1.30
61	7396.87	7300.00	-96.87	1.33
19	16178.14	16400.00	221.86	1.35
280	12327.10	12499.00	171.90	1.38
311	51287.19	52055.25	768.06	1.48
723	6009.63	6100.00	90.37	1.48
49	4973.17	4900.00	-73.17	1.49
114	27716.14	27300.00	-416.14	1.52
636	28498.91	28950.00	451.09	1.56
612	2953.17	3000.00	46.83	1.56
47	26425.14	25999.00	-426.14	1.64
23	13111.91	12900.00	-211.91	1.64
31	12858.08	12650.00	-208.08	1.64
91	13421.16	13200.00	-221.16	1.68
329	7327.18	7200.00	-127.18	1.77
549	3816.33	3750.00	-66.33	1.77
252	9721.50	9900.00	178.50	1.80
387	44173.72	44999.00	825.28	1.83
267	40753.58	40000.00	-753.58	1.88
467	22262.80	22711.65	448.85	1.98