Let's Talk About Statistics - Introduction to Statistics for Software Quality (No.14 Prediction: Interpreting Quality with Regression Analysis)

In the fourteenth installment of "Let's Talk About Statistics," we challenge ourselves to "predict the future" using a technique called regression analysis.
In this series, we have been using past data to grasp the overall picture and relationships, and this time, we'll leverage that insight to learn how to predict "what will happen in the future" with concrete numbers.

For example—

• Estimate future defect counts based on the relationship between past testing effort and the number of bugs
• Predict the number of review comments based on the number of code lines in a module
• Predict a decline in user satisfaction based on changes in response time

In this way, regression analysis is a statistical tool for stepping from "relationships" into "estimating the future."
In software quality environments, there are increasing situations where, in addition to heuristics, quantitative predictions support decision-making.

This time, we will cover the basics of regression analysis—simple linear regression—and provide a compact explanation from theory to practice to implementation in Python.
I hope you'll feel, "Statistics can also be used for prediction!"

Regression analysis is a statistical method for predicting a certain variable (the dependent variable) using other variables (independent variables).
Simple correlation analysis only captures tendencies such as "there is / there is no relationship," but regression analysis models that relationship as a formula and makes it possible to estimate and predict specific numerical values.

For example—

• If you know the testing effort (independent variable), you can predict the future number of bugs (dependent variable)
• If you know a module's lines of code or complexity, you can estimate the number of review comments or defects
• Model the relationship between skill or years of experience and defect occurrence to analyze team risk

Regression analysis is used for the following purposes:

1. Prediction
   Based on the values of independent variables, predict the values of the dependent variable.
   Examples:

   • Lines of code in a module → Predict the number of latent defects in that module
   • Testing design effort → Predict the number of defects that will be found

2. Factor Analysis
   Quantitatively evaluate which independent variables have how much impact on the dependent variable.
   Examples:

   • Compare the magnitude of the effects of complexity, code volume, and developer skill on defect count

There are various types of regression analysis, but the most basic is simple linear regression.
This is an extremely simple model for "predicting one dependent variable (Y) from one independent variable (X)."

The basic formula for simple linear regression is as follows:

$$Y = \beta_0 + \beta_1 X + \varepsilon$$

The meanings of these symbols are:

• $Y$: the variable to predict (dependent variable)
• $X$: the explanatory variable (variable used for prediction)
• $\beta_0$: intercept (the predicted value of Y when X is 0)
• $\beta_1$: regression coefficient (how much Y changes when X increases by one unit)
• $\varepsilon$: error term (the deviation that the model cannot explain)

This equation assumes a linear relationship between X and Y and aims to draw a single line (the regression line) for prediction.
Intuitively, the goal is to "draw the line that best fits the scatterplot of the data" so that Y can be predicted from X.

How do we find the line that "fits best"?
The answer is the Least Squares Method.

Residuals and Their Minimization

A "residual" is the difference between the actual observed value and the predicted value by the regression line for each data point.

$$\varepsilon_i = y_i - (\beta_1 x_i + \beta_0)$$

We adjust the regression line so that these residuals are as small as possible.
However, simply summing the residuals cancels out positive and negative values, so we minimize the sum of squared residuals:

Objective of the Minimization

$$\text{Objective:} \quad \sum_{i=1}^{n} (y_i - \beta_1 x_i - \beta_0)^2 \quad \text{minimized}$$

In this way, you obtain the optimal line that minimizes the deviation between all data points and the line.
Mathematically, you can derive $\beta_0$ (intercept) and $\beta_1$ (slope) using differentiation, but tools like Python can compute them automatically.

In software development, we can assume there is a certain relationship between testing effort (time or person-days) and the number of bugs found.
Below is an example of creating a model to predict future bug counts from testing effort using past project data.

Past data used

Based on this data, we perform prediction using a linear regression model.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression

# Japanese font setting (for Windows environments)
plt.rcParams['font.family'] = 'Meiryo'

# Past data (example: testing effort [person-days] and number of bugs)
x = np.array([5, 10, 15, 20, 25]).reshape(-1, 1)  # Explanatory variable: testing effort
y = np.array([4, 6, 10, 14, 16])                  # Dependent variable: number of bugs found

# Create and train simple regression model
model = LinearRegression()
model.fit(x, y)

# Retrieve regression coefficient and intercept
a = model.coef_[0]
b = model.intercept_
print(f"回帰式：バグ件数 = {a:.2f} × テスト工数 + {b:.2f}")

# Prediction and visualization
x_pred = np.linspace(0, 30, 100).reshape(-1, 1)
y_pred = model.predict(x_pred)

plt.scatter(x, y, label='実測値（観測データ）', color='blue')
plt.plot(x_pred, y_pred, label='回帰直線', color='red')
plt.xlabel('テスト工数（人日）')
plt.ylabel('バグ件数')
plt.title('テスト工数とバグ件数の回帰分析')
plt.legend()
plt.grid(True)
plt.show()

When you run the above program, the following result is output:

In this example, a positive correlation is assumed: the more testing effort, the more bugs found.
Of course, in reality, other factors such as test quality and specification complexity also play a role, but it is effective as a simple predictive model focusing on a single factor.

The indicator used to evaluate how well the created regression model "fits (i.e., how useful it is)" is the coefficient of determination ($R^2$).

• $R^2$ takes a value between 0 and 1, with values closer to 1 indicating higher explanatory power of the model.

For example, if $R^2 = 0.75$:

• You can read it as "75% of the variance in Y is explained by X."
• The remaining 25% is considered to be due to other variables not used for explanation or random error.

The coefficient of determination serves as a baseline for objectively evaluating the "confidence level of predictions."
However, a high $R^2$ does not always mean a "good model," so it should be judged in conjunction with other indicators.

Example Check in Python (continued)

r2 = model.score(x, y)
print(f"決定係数 R^2 = {r2:.2f}")

Calculating the above yields "Coefficient of Determination $R^2$ = 0.98."
In this way, you can numerically confirm how well the model fits the actual data.

To confirm that the model is not only a "good fit" but also statistically sound, residual analysis is indispensable.

Residuals are the differences between the actual observed values and the predicted values from the regression model:

$$\varepsilon_i = y_i - \hat{y}_i$$

In an ideal regression model, it is important that these residuals are randomly scattered with no patterns.

• No trends or patterns in the residual plot (e.g., curvilinear bias or abnormal distributions)
• Normality of residuals (check with histograms or Q-Q plots for normal distribution)
• Homoscedasticity (uniform scatter of residuals relative to predicted values)
• Absence of outliers (no extremely large residuals)

Plotting a Residual Plot in Python (continued)

import seaborn as sns

# Calculate residuals
y_pred = model.predict(x)
residuals = y - y_pred

# Residual plot
plt.scatter(y_pred, residuals)
plt.axhline(y=0, color='red', linestyle='--')
plt.xlabel('予測値')
plt.ylabel('残差')
plt.title('残差プロット')
plt.grid(True)
plt.show()

From the above graph, evaluate the following points:

From this residual plot, you can infer the following improvement and verification needs:

• No clear patterns are observed in the residuals, but homoscedasticity and normality need verification.
• Conduct a Q-Q plot or Breusch-Pagan test to validate model assumptions.
• Consider polynomial regression or adding features if necessary.

Even with a high coefficient of determination, a model with patterned residuals may not be trustworthy.
Therefore, it is fundamental to always confirm model validity by pairing "coefficient of determination ($R^2$)" with "residual analysis."

In practice, factors influencing quality or defect occurrence are rarely limited to just one.
For example, in addition to "testing effort," factors such as "developer proficiency," "specification complexity," and "whether a review was conducted" often intertwine.
Therefore, multiple regression analysis, which handles multiple explanatory variables simultaneously, is necessary.

The general form of a multiple regression model is:

$$Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \dots + \beta_k X_k + \varepsilon$$

Points to note here:

• Multicollinearity: When explanatory variables are highly correlated with each other, estimates of each coefficient become unstable.
  Example: If review count and testing effort always increase proportionally, it becomes impossible to distinguish their individual effects.
• Adjusted Coefficient of Determination (Adjusted $R^2$): When the number of variables increases, the ordinary $R^2$ tends to be overestimated. Using Adjusted $R^2$ allows more reliable comparison of models' explanatory power.
• Practical Relevance: Even statistically significant explanatory variables may be impractical to control or improve (e.g., weather during development) and may need to be excluded.

• Select explanatory variables that are theoretically causal to the dependent variable.
• Quantitatively assess multicollinearity using VIF (Variance Inflation Factor) and consider variable reduction (e.g., stepwise methods) if necessary.
• Remember to perform residual analysis and cross validation to avoid overfitting and verify generalization performance.

Multiple regression analysis is a fundamental tool for capturing "complex phenomena involving multiple factors" in practice.
In quality management settings, it is also valued for simultaneously evaluating the influence of multiple factors such as "production conditions" and "testing methods."

Regression analysis is a very useful tool, but when using it, you need to understand some important limitations and caveats.

Having a correlation does not necessarily mean there is a causal relationship.

• Example: Ice cream sales and heatstroke incidence may be correlated, but it does not imply causation (both are influenced by a common factor: rising temperatures).

⇒ In practice, it is important to adopt a causal inference perspective (e.g., interventions or controls).

Predictions that go beyond the range of observed data (extrapolation) can suffer significant accuracy degradation.

• Models are based only on regularities within the "observed range."
• Applying them to unknown regions can produce unreasonable predictions.

⇒ In practice, you should be aware of the "valid range of the data."

If the input data quality and reliability are poor, the model results will naturally be poor.

• Neglecting preprocessing steps such as handling missing values, outliers, or input errors leads to unreliable analysis.
• Especially in quality management, pay attention to measurement errors and variability in data collection conditions.

⇒ "Data cleansing" and "EDA (Exploratory Data Analysis)" before analysis are indispensable.

• Insufficient sample size: A small dataset can lead to overfitting and statistical instability.
• Overlooking nonlinear relationships: Complex relationships that cannot be expressed by a linear model may exist.
• Risk of overreliance on the model: Even a highly accurate model may not translate into actionable business insights (interpretability and operational feasibility must be evaluated).

• Correlation indicates "how much two variables change together."
• The correlation coefficient $r$ ranges from −1 to +1, with values closer to the extremes indicating a stronger linear relationship.
• However, a strong correlation does not necessarily mean causation.

• Regression analysis is a method for "building a model to predict one variable (dependent variable) from another (independent variable)."
• Simple regression uses one independent variable; multiple regression uses several to predict the dependent variable.
• In practice, it is widely used for "quality prediction," "effort estimation," "reliability analysis," and more.

Regression analysis is also an effective tool in software quality management and project management.
Below are some representative use cases.

• Overview: Use metrics such as module lines of code (LOC), number of changes, and complexity as explanatory variables to predict future defect counts.
• Purpose: Identify modules at risk of quality degradation in advance and focus reviews and testing on them to improve quality and reduce costs.

• Overview: Predict effort for design, implementation, and testing phases based on development size indicators such as screen count, requirement count, and function points.
• Purpose: Use past performance data to achieve more objective and reproducible estimates, supporting reducing estimation variance and advanced process management.

• Overview: Extract and predict defect trends and delay risks from past project records and defect history.
• Purpose: Anticipate future defect occurrences and delay risks in advance, and plan early countermeasures to improve quality.

Regression analysis is a powerful tool for quantitatively predicting the future using software quality data.
However, careless application can lead to misunderstandings, so verifying statistical assumptions and integrating domain knowledge are indispensable.

• Regression analysis is useful for "predicting numerical values."
• Simple regression is the most basic model and is commonly used in practice.
• Use it cautiously, being aware of its differences from correlation and potential causal pitfalls.

In the next issue, No.15, we will introduce practical applications of statistical quality control as a summary of this series.

I’ve compiled statistical information here.

I hope you find it useful for your data analysis.