CH7 - Linear Regression

Chapter 7: Linear Regression

Linear regression is a key tool in engineering statistics for modeling and predicting relationships between variables. It allows us to analyze and quantify the relationship between an outcome (dependent variable) and one or more input factors (independent variables).

In this chapter, we will introduce the basics of linear regression, interpret its results, and discuss how to evaluate its effectiveness. This knowledge is crucial for engineers who need to understand data trends and make informed decisions.

---

What is Linear Regression?

Linear regression models the relationship between a dependent variable $Y$ and one or more independent variables $X_1,X_2,\dots ,X_n$ using the equation:

$$Y={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+\cdots +{\beta }_n{X}_n+ϵ$$

Where:

• ${\beta }_0$ is the intercept (value of $Y$ when all $X_i=0$),
• ${\beta }_1,{\beta }_2,\dots ,{\beta }_n$ are the slopes (rate of change of $Y$ with respect to each $X_i$),
• $ϵ$ is the error term accounting for variability not explained by the model.

---

Steps in Linear Regression Analysis

1. Formulate the Model:
   Choose dependent and independent variables based on the problem.

2. Fit the Model:
   Use data to estimate the coefficients ${\beta }_0,{\beta }_1,\dots ,{\beta }_n$ through the least squares method.

3. Evaluate the Model:
   Analyze the model’s accuracy and reliability using statistical metrics.

---

Interpreting the Coefficients

• Slope (${\beta }_i$):
  The expected change in $Y$ for a one-unit increase in $X_i$ while keeping other variables constant.
  Example: In predicting material strength, if ${\beta }_1=2$, every 1°C increase in temperature results in a 2-unit increase in strength.

• Intercept (${\beta }_0$):
  The predicted value of $Y$ when all $X_i=0$.

• Error Term ($ϵ$):
  Represents random noise or factors not included in the model.

---

ANOVA Table: Analysis of Variance

We use an ANOVA table to assess the overall effectiveness of the regression model:

Source	SS (Sum of Squares)	df (Degrees of Freedom)	Mean Square (MS)	F-Ratio
Regression	SSR	$k$ (number of predictors)	$MSR=\frac{SSR}{k}$	$F=\frac{MSR}{MSE}$
Residual (Error)	SSE	$n-(k+1)$	$MSE=\frac{SSE}{n-(k+1)}$	$\times$
Total	SST	$n-1$	$\times$	$\times$

• Null Hypothesis ($H_0$): All slopes ${\beta }_i=0$ (no relationship between $X$ and $Y$).
• Alternative Hypothesis ($H_1$): At least one slope ${\beta }_i\ne 0$ (at least one predictor contributes to the model).

---

Key Metrics for Model Evaluation

1. R-Squared ($R^2$):
   The proportion of variability in $Y$ explained by the model. A value close to 1 indicates a good fit.

2. Adjusted R-Squared:
   Adjusts $R^2$ for the number of predictors, penalizing overfitting.

3. F-Statistic:
   Tests the overall significance of the model. A high $F$ value (and low p-value) suggests the model is useful.

4. P-Value for Coefficients:
   Tests the significance of individual predictors. A p-value < 0.05 indicates the predictor is statistically significant.

---

Assumptions of Linear Regression

1. Linearity:
   The relationship between $X$ and $Y$ is linear.

2. Independence:
   Observations are independent.

---

Engineering Applications

• Quality Control:
  Predicting defects based on process parameters.

• Material Testing:
  Modeling stress-strain relationships.

• Design Optimization:
  Analyzing relationships between design variables and performance metrics.

By understanding and applying linear regression, engineers can make data-driven decisions to improve processes, optimize designs, and predict outcomes with confidence.