Statistics for Digital Signal Processing

Welcome to a foundational exploration of Statistics for Digital Signal Processing (DSP). Understanding how statistics apply to signals is crucial for anyone studying or working with digital data. This guide will break down key concepts like precision, accuracy, various distributions, and noise generation, making complex topics accessible for students.

Understanding Statistics in Digital Signal Processing: An Overview

Digital Signal Processing (DSP) often deals with signals that contain random components or noise. Statistics provide the tools to understand, characterize, and often mitigate these random elements. We'll look at how statistical measures describe signals and the underlying processes that generate them.

Signal and Graph Terminology

Before diving into statistics, let's clarify some basic terms:

• Signal: A description of how one parameter relates to another. Think of a sensor reading over time.
• Continuous Signal: Both parameters (e.g., time and amplitude) can assume a continuous range of values.
• Discrete Signal (or Digitized Signal): Signals formed from quantized parameters, meaning their values are limited to specific steps.
• Domain: Refers to the type of parameter represented on the horizontal axis of a graph (e.g., time, frequency).

Precision and Accuracy in DSP Measurements

When measuring signals in DSP, two critical concepts are precision and accuracy. An example involving an oceanographer measuring water depth with sonar illustrates these points clearly.

• Accuracy: This describes how close the mean of your measurements is to the true value. Poor accuracy results from systematic errors—errors that are repeated in the same manner each time a measurement is conducted. Averaging individual measurements does not improve accuracy; instead, accuracy depends on proper system calibration.
• Precision: This refers to the width of the distribution of your measurements, often expressed by standard deviation, signal-to-noise ratio, or coefficient of variation. Good precision means good repeatability. Poor precision results from random errors that change each time a measurement is repeated. Averaging several measurements always improves precision, as it helps reduce the impact of random noise.

Mean and Standard Deviation: Core Statistical Measures

For a signal with N samples, the fundamental statistical measures are:

• Mean (μ): The average value of a signal. It's calculated as the sum of all samples divided by N. This measures the DC (direct current) portion of the signal.
• Standard Deviation (σ): A measure of how far the signal fluctuates from its mean. It quantifies the AC (alternating current) portion of a signal and is a measure of random noise. It's calculated as the square root of the variance.
• Variance (σ²): The square of the standard deviation, representing the power of the signal's fluctuation.
• RMS (Root-Mean-Square): Measures both the AC and DC components of a signal. If a signal has no DC component, its RMS value is equal to its standard deviation.

Limitations of Standard Calculation:

• If the mean (μ) is much larger than the standard deviation (σ), subtracting very close numbers can lead to excessive round-off errors.
• A running statistics approach, where all samples are involved in each new calculation, can address this. An alternative formula exists: σ² = ( (1/N) * Σ(x_i²) ) - μ².

Signal-to-Noise Ratio (SNR) and Coefficient of Variation (CV)

In many applications, the mean (μ) represents what is being measured, while the standard deviation (σ) represents noise and other interference.

• Signal-to-Noise Ratio (SNR): Calculated as μ / σ. A higher SNR indicates a clearer signal relative to its noise.
• Coefficient of Variation (CV): Calculated as (σ / μ) * 100%. This expresses the standard deviation as a percentage of the mean, useful for comparing variability across different scales.

Statistical Noise vs. Underlying Process

Understanding the difference between the statistics of an acquired signal and those of the underlying process is crucial.

• Statistical Noise: The acquired signal's statistics change each time the experiment is repeated due to random fluctuations.
• Underlying Process: The probabilities of the underlying process that generates the signal are constant.

When calculating the standard deviation (σ) of an underlying process from an acquired signal, a slightly different formula is used (1/(N-1) * Σ(x_i - μ)² instead of 1/N * Σ(x_i - μ)²) to compensate for the fact that calculating the mean from limited samples reduces the estimated σ.

Nonstationary Signals in Digital Signal Processing

Nonstationary signals are not merely a result of statistical noise; they arise from an underlying process that is actively changing. If the mean (μ) of a signal slowly changes, it interferes with accurately calculating the standard deviation (σ).

Solution: Break the signal into short sections, calculate statistics (especially σ) for each section, and then average these σ values to get a more accurate overall measure.

Histograms, PMFs, and PDFs: Visualizing Signal Distributions

Visualizing the distribution of signal values is fundamental. Histograms, Probability Mass Functions (PMFs), and Probability Density Functions (PDFs) are key tools for this.

The Histogram: A First Look at Data Distribution

• Histogram: Displays the number of samples having each possible value. If H_i is the number of samples with value i, and N is the total number of samples, then ΣH_i = N.
• Problem with Histograms for Floating-Point Data: For signals represented in floating-point notation, the number of possible levels vastly exceeds the number of samples, meaning most levels would have zero samples. Counting each possible quantization level becomes impractical.
• Solution: Binning: To overcome this, the range of signal values is divided into a convenient number of