Let's Talk About Statistics: Introduction to Statistics for Software Quality (No.8 Intuition and Computation of Probability: Understanding the Nature of 'Chance')

In the eighth installment of "Let's Talk About Statistics," we will discuss an intuitive understanding of "probability," basic methods of calculation, and probability distributions that are useful in quality management settings.

"Why did that bug happen?"
"Wasn't it just that it happened not to be caught in testing?"
— The tool for handling such phenomena of "chance" scientifically is probability.

In this article, we focus on the following three representative distributions:

• Binomial Distribution (Binomial Distribution)
• Poisson Distribution (Poisson Distribution)
• Normal Distribution (Normal Distribution)

To properly understand these "probability distributions," it is also necessary to address the underlying Probability Theory (Probability Theory). A probability distribution is a "model built on Probability Theory," serving as a mathematical tool to represent phenomena such as "how many successes occur out of a number of trials" or "how much variation there is."

Therefore, in the first part of this article, we will first overview the basics of Probability Theory (trials, events, and the definition of probability), and in the latter part, we will introduce representative probability distributions (discrete and continuous).

"Probability Theory" is a theoretical framework for mathematically describing and predicting the occurrence probabilities of random events.
While statistical inference is a practical methodology for drawing conclusions from actual data, Probability Theory provides the theoretical foundation for it.

To understand Probability Theory, it's crucial to grasp the basic framework of "what you consider when thinking about probability." For this, we first confirm two concepts: "trial" and "event."

• Trial (Trial): An experiment or action whose outcome can change by chance
  Examples: drawing one lottery ticket, loading a website once for a test

• Event (Event): A possible occurrence as a result of a trial
  Examples: "drawing a winning lottery ticket," "a load error occurs"

(P(A) = \frac{\text{Number of cases corresponding to event A}}{\text{Total number of trial outcomes}})

For example, if 2 out of 10 tickets are winning, the probability of drawing a winning ticket is

[
P(\text{winning}) = \frac{2}{10} = 0.2
]

• Probability distributions (discrete and continuous)
• Conditional probability
• Independence
• Expectation, variance, standard deviation
• Law of Large Numbers / Central Limit Theorem (※1), etc.

Thus, understanding Probability Theory provides the power to underpin the premises of statistical inference and serves as the theoretical foundation for answering the question, "Why can we understand a population from a sample?"

What comes to mind when you hear the word "probability"? Dice, gacha (capsule toy machines), weather forecasts, lotteries… All these are phenomena influenced by "chance," but we tend to dismiss them as "unpredictable events."

However, probability is a tool for capturing the "regularities within chance." In statistics, probability is used to quantitatively make judgments such as "How likely is this to happen?" or "Can we really say it's impossible?" In fact, every day we unconsciously make probabilistic judgments as we act.

• "It looks like it's going to rain today, so I'll take an umbrella."
• "This feature might have bugs before release."
• "Because there were many pointed out issues in the last review, there should be fewer this time."

It is precisely this ability to estimate 'the possible likelihood of occurrences' and make decisions that is the first step in probabilistic decision-making.

In software development, the following situations are related to probability:

• Frequency of failures (probability of bug occurrence per unit time)
• Test detection capability (probability of discovering defects)
• Risk assessment (probability of failure × impact)

In other words, "probability" is not just a mathematical concept; it's a common language for expressing quality numerically.

So, what is "probability" in the first place? The most basic definition is:

[
P = \frac{\text{Number of favorable patterns}}{\text{Total number of patterns}}
]

Suppose you run 10 test cases and detect a bug in 2 of them. Then, what is the "probability that a bug is found in a single test case"?

• Number of favorable patterns: 2 (the two cases where a bug was found)
• Total number of patterns: 10 (the total number of test cases)

[
P(\text{bug detection}) = \frac{2}{10} = 0.2
]

In this way, probability expresses as a ratio "how likely the desired outcome is relative to the whole."

A probability distribution describes in the form of a "distribution" how a probabilistically varying value appears. Below, we classify and introduce representative distributions.

Among the many probability distributions, in this installment we will explain the "Binomial Distribution," "Poisson Distribution," and "Normal Distribution." These three distributions are the core that connects descriptive statistics and inferential statistics. By understanding where and when to use each, you can broaden the scope of your statistical analysis and apply it more smoothly in practice.

● Binomial Distribution

• A model for cases where there are multiple binary trials, such as "success or failure."
  → Example: If the probability of finding a bug is 30%, how many bugs will be discovered in 10 reviews?

● Poisson Distribution

• A model for the frequency of rare event occurrences per unit time or area.
  → Example: If an average of 2 bugs occur per day, what is the probability of 5 bugs on a given day?

● Normal Distribution

• The distribution that data from nature and practice most commonly follow.
  → Due to the Central Limit Theorem, many quantitative real-world data (effort, time, etc.) tend to approach a normal distribution.

"Success or failure," "bug present or not"—this is a probability model used in scenarios of repeating binary trials. For example, when considering whether bugs are found in 100 tests, you want to find the distribution of the number of successes when you repeat n trials with success probability p—the model you use here is the Binomial Distribution.

The Binomial Distribution is defined as follows:

[
P(X = k) = \binom{n}{k} p^k (1 - p)^{n - k}
]

• (n): Number of trials (e.g., 100 test cases)
• (k): Number of successes (e.g., number of bugs found)
• (p): Probability of success in a single trial (e.g., probability of finding a bug in one test)

The probabilities in a Binomial Distribution are given by the Probability Mass Function (※3) (PMF).

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import binom

plt.rcParams['font.family'] = 'Meiryo'

n = 100  # Number of trials
p = 0.1  # Success probability

x = np.arange(0, 31)
pmf = binom.pmf(x, n, p)

plt.bar(x, pmf, color='skyblue', edgecolor='black')
plt.title("二項分布（n=100, p=0.1）")
plt.xlabel("バグ検出数")
plt.ylabel("確率")
plt.tight_layout()
plt.show()

This graph represents the probability distribution of the number of bug detections (k) when 100 tests are conducted. The most likely number is around 10 bugs, but the possibility of 20 or more is not zero.

Explanation of the Binomial Distribution Plotting Code

• scipy.stats.binom.pmf(x, n, p)
  → Calculates the Probability Mass Function (PMF) of the Binomial Distribution.
  For each (x), it computes the probability that the number of successes equals (x).

• x = np.arange(0, 31)
  → Sets the range of success counts to display (here, number of bug detections) from 0 to 30.

• plt.bar(...)
  → Plots the probabilities as a bar graph.

This graph allows you to visually understand how the number of bugs detected out of 100 tests is distributed probabilistically.

When the probability of detecting a certain bug is 10% and you run 100 tests:

• You can calculate the probability of detecting exactly 10 bugs,
• the probability of detecting 20 or more bugs, etc.

Let's calculate the probability of detecting 20 or more bugs:

from scipy.stats import binom

n = 100
p = 0.1
P_20_or_more = 1 - binom.cdf(19, n, p)  # cdf(19) means P(X<=19), so 1 minus this gives P(X>=20)

print(f"20件以上検出される確率：{P_20_or_more:.4f}")

The calculation result is "Probability of detecting 20 or more bugs: 0.0020".

Thus, the Binomial Distribution is extremely effective for capturing the overall picture of repeated trials of "occurrence or non-occurrence."

"How many bugs occur in one day?" "How many crashes occur in 1000 lines of code?"
— In these cases, when dealing with the count of occurrences within a certain period or region, the Poisson Distribution comes into play.

The Poisson Distribution is defined as follows:

[
P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!}
]

• (\lambda): Average number of occurrences per fixed period (or region) (e.g., average number of bugs)
• (k): Actual number of occurrences (0, 1, 2, …)

This distribution is used when the following conditions are met:

• You want to know the count of rare event occurrences within a fixed period or region.
• You can assume that events occur independently.
• The occurrence rate per unit time (or region) is constant.

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import poisson

plt.rcParams['font.family'] = 'Meiryo'

λ = 3  # Average of 3 per day
x = np.arange(0, 11)
pmf = poisson.pmf(x, mu=λ)

plt.bar(x, pmf, color='lightgreen', edgecolor='black')
plt.title("ポアソン分布（λ=3）")
plt.xlabel("1日に発生するバグ数")
plt.ylabel("確率")
plt.tight_layout()
plt.show()

This graph represents the probabilities of 0–10 bug occurrences under the assumption of an average of 3 bugs per day. You can intuitively see that around 2–4 bugs are most common, while 6 or more bugs are rare.

Explanation of the Poisson Distribution Plotting Code

• λ = 3
  → Average number of bugs per day (3 bugs/day).

• scipy.stats.poisson.pmf(x, mu=λ)
  → Calculates the Probability Mass Function (PMF) of the Poisson Distribution.
  For each x, it computes the probability of x bug occurrences in one day.

• x = np.arange(0, 11)
  → Defines the range of occurrences to display (0–10).

• plt.bar(...)
  → Plots the calculated probabilities as a bar graph.

Consider a system where an average of 3 failures occur per day.
In this case, the following questions can be quantitatively assessed:

• "What is the probability of 5 failures today?"
• "What is the probability of 20 or more failures in a week?"

Let's calculate the probability of 5 failures in one day:

from scipy.stats import poisson
P_5 = poisson.pmf(5, mu=3)
print(f"1日に5件の障害が発生する確率：{P_5:.4f}")

The calculation result is "Probability of 5 failures occurring in one day: 0.1008".

In this way, the Poisson Distribution helps quantitatively evaluate rare but plausible problems (bugs, failures, crashes). In practice, it is extremely useful as foundational information for prioritizing preventive measures and risk management.

Test execution time, review time, rework effort, etc.—many practical data exhibit variability that spreads around the mean. The prototypical form of this variability is the Normal Distribution (Normal Distribution).

The Normal Distribution is defined as follows:

[
f(x) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp!\Bigl(-\frac{(x - \mu)^2}{2\sigma^2}\Bigr)
]

• (\mu): Mean
• (\sigma): Standard deviation

This function draws a bell curve symmetrically around the mean. Here, (f(x)) is the Probability Density Function (※4) (PDF: probability density function), representing "how densely data points are located around that value" (note that it is not the probability itself).

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm

plt.rcParams['font.family'] = 'Meiryo'

mu = 50
sigma = 10
x = np.linspace(mu - 4*sigma, mu + 4*sigma, 100)
y = norm.pdf(x, mu, sigma)

plt.plot(x, y, color='darkorange')
plt.title("正規分布（μ=50, σ=10）")
plt.xlabel("値")
plt.ylabel("確率密度")
plt.grid(True)
plt.tight_layout()
plt.show()

This graph visualizes the Normal Distribution with mean 50 and standard deviation 10. It shows a smooth variability with a peak around the mean and decreasing probability density for extremely small or large values.

Explanation of the Normal Distribution Plotting Code

• scipy.stats.norm.pdf(x, mu, sigma)
  → Calculates the Probability Density Function (PDF) of the Normal Distribution.
  It computes the "height (density)" of the continuous distribution for the specified mean μ and standard deviation σ.

• x = np.linspace(mu - 4*sigma, mu + 4*sigma, 100)
  → Divides the range of values (horizontal axis) into 100 points over the interval mean ± 4σ.

In practice, the following variables are considered to approximate a normal distribution:

• Review time: very fast or very slow reviews are rare; most cluster around the average
• Test execution time: few outliers; distributed around the mean
• Rework effort: aside from extreme rework, values tend to center around the mean

By assuming a normal distribution for such variables, you can perform control charts, process capability indices, and confidence interval estimation.

Below is a simulation of how "review times around an average of 30 minutes" actually vary, compared with the theoretical normal distribution. In practice, such comparisons allow you to visually detect outliers and trends.

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm

plt.rcParams['font.family'] = 'Meiryo'

# Mean and standard deviation
mu = 30
sigma = 5

# Generate 100 review time data points following a normal distribution
review_times = np.random.normal(mu, sigma, 100)

# Visualize with a histogram
plt.hist(review_times, bins=15, density=True, alpha=0.6, color='skyblue', edgecolor='black')

# Overlay the theoretical normal distribution curve
x = np.linspace(mu - 4*sigma, mu + 4*sigma, 100)
y = norm.pdf(x, mu, sigma)
plt.plot(x, y, 'r--', label="理論的な正規分布")

plt.title("レビュー所要時間の分布（平均30分, σ=5）")
plt.xlabel("時間（分）")
plt.ylabel("確率密度")
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()

By looking at this graph, you can intuitively understand how closely review times follow a normal distribution and where exceptionally short or long reviews lie.

• Central Limit Theorem: Regardless of the population distribution, the mean of a sufficiently large sample approaches a normal distribution.
• Assumption in many statistical methods: Techniques such as t-tests, regression analysis, and confidence interval estimation (※5) assume normality.
• Intuitive properties: For example, "68% of values fall within ± one standard deviation of the mean."

• If data are extremely skewed, a normal distribution may not be appropriate.
• It is important to verify normality by using histograms and box plots.
• When necessary, you can make data approximately normal by log transformation or Box-Cox transformation.

The normal distribution is a "lens for interpreting variability." In the field of software quality, skillfully using this lens leads to accurate decision-making and improvement.

• You can model bug occurrence rates and variability of test results.
• You can quantitatively judge "Is this result due to chance or is it an anomaly?"
• It serves as the foundation for inferential statistics such as control charts and confidence intervals.

• Probability is a tool for quantitatively capturing "chance."
• Binomial Distribution: Effective for yes/no trials.
• Poisson Distribution: Effective for the count of rare events.
• Normal Distribution: Applicable to many natural variabilities.

Next time, we will explain "The Normal Distribution and its Surroundings: The Meaning and Limitations of the 3σ Rule." We will also delve into the commonly used "±3σ" and process capability in practice.

Here's a compilation of statistical resources.

I hope you find it useful for data analysis.