Regression analysis is a core tool used to model the linear relationship between a dependent variable and one or more explanatory (independent) variables.

2.1. Simple Linear Regression (SLR)

SLR models the relationship between two variables.

• Model: Return on Asset = α + β (Return on Benchmark) + Error term
• Parameters: α (alpha) is the intercept, and β (beta) is the slope coefficient, which measures the sensitivity of the asset’s return to the benchmark’s return.
• Assumptions: The error term is assumed to be normally distributed with a mean of zero and constant variance.
• Estimation: The Ordinary Least Squares (OLS) method is used to find the line that minimizes the sum of the squared errors, providing the best linear unbiased estimate (BLUE).
• Goodness-of-Fit: The coefficient of determination (R²) measures the percentage of variation in the dependent variable explained by the independent variable. It ranges from 0 (no linear relationship) to 1 (perfect fit). R² is the square of the correlation coefficient.

Applications:

• Estimating the Characteristic Line: A regression of a security’s excess return against the market’s excess return. The resulting alpha (α) is interpreted as a measure of abnormal performance, and the beta (β) measures systematic market risk.
• Controlling Portfolio Risk: Using stock index futures to hedge a portfolio. Regression is used to estimate the beta of the portfolio relative to the futures contract to determine the optimal hedge ratio. This involves substituting price risk with basis risk (the risk that the relationship between the cash and futures price will change).

2.2. Multiple Linear Regression (MLR)

MLR extends SLR by including two or more independent variables to explain the dependent variable.

• Model: y = β₀ + β₁x₁ + β₂x₂ + … + βₖxₖ + ε
• Interpretation: Each coefficient (βₖ) represents the average change in the dependent variable for a one-unit change in the corresponding independent variable, holding all other variables constant.
• Diagnostics:
  • F-test: Tests the overall significance of the model by evaluating the null hypothesis that all regression coefficients are jointly equal to zero.
  • t-test: Tests the statistical significance of each individual independent variable.
  • Adjusted R²: A modified version of R² that accounts for the number of independent variables, providing a more accurate measure of fit for models with multiple variables.

Applications:

• Estimating Empirical Duration: Regression is used to estimate the sensitivity of an asset’s price to changes in interest rates. In a multiple regression, the return on the S&P 500 can be added as a second variable to estimate duration while controlling for market movements.
• Style Analysis (Sharpe Benchmarks): A fund’s returns are regressed on the returns of several style-based indexes (e.g., large-cap value, small-cap growth). The regression coefficients, constrained to sum to one, represent the fund’s average exposure to each investment style. The R² indicates how much of the fund’s performance is explained by its style.

2.3. Specialized Regression Models

Regression with Categorical Variables

• Concept: Used when an explanatory variable represents group membership (e.g., industry sector, credit rating).
• Method: Dummy variables (taking values of 0 or 1) are created to represent categories. For K categories, K-1 dummy variables are needed.
• Testing: The Chow test (an F-test) can determine if the dummy variables are collectively significant, which can identify structural breaks or regime shifts in data.
• Categorical Dependent Variables: When the outcome is a category (e.g., bond default vs. no default), standard regression is inappropriate. The following models are used instead:
  • Linear Probability Model: An OLS regression where the dependent variable is a 0/1 dummy. It is simple but has significant flaws.
  • Probit Regression Model: Predicts the probability of an outcome using the cumulative standard normal distribution function.
  • Logit Regression Model: Predicts probability using the logistic distribution.

Quantile Regression

• Purpose: Used when data contains outliers or is skewed, as classical OLS (which models the mean) may not describe the full relationship.
• Method: Aims to minimize the weighted sum of absolute deviations from a specific quantile (e.g., the median), rather than minimizing squared errors. This makes it robust to outliers.
• Application: Allows for analyzing how independent variables affect the dependent variable across its entire distribution (e.g., at the 10th, 50th, and 90th percentiles), providing a much richer picture than a single OLS estimate.

Robust Regression

• Purpose: Aims to produce parameter estimates that are not sensitive to outliers or small changes in data assumptions.
• Method: Employs techniques that give less weight to extreme observations. Methods include:
  • M-estimators: A generalization of OLS that minimizes a function of the residuals (e.g., Huber or Tukey weighting functions).
  • Least Trimmed of Squares (LTS): Discards a certain percentage of observations with the largest residuals before performing a least-squares estimation on the remaining data.
• Application: Used to generate “resistant” estimates of beta that are less affected by extreme market events.