- Round Robin Testing Comparing Laboratories may not be as simple as it may seem.
- “Fake Data” Many decades ago, when I was a young engineer, I was called to explain why my analysis of some metal fatigue data was at odds with an analysis done by another engineer.
- Statistical Relativity and the Coriolis Effect The Coriolis Effect is well known to engineers and provides a parable for the relationship between a sample statistic and the population parameter that it estimates.
- Six-nines (0.999999) Reliability? Risk calculations are alluringly simple in principle, but significantly more complex in practice, making them vulnerable to misuse.
- Probabilistic Life Prediction Isn’t as Easy as It Looks

There’s much more to it than simply replacing constants with probability distributions. - Monty Hall Problem explained – It only
*seems*like it shouldn’t make a difference to switch doors. Here’s why switching doors wins twice as often. No fancy math necessary! - Bayesian thinking considers not only what the data have to say, but what your expertise tells you as well.
- High Flight I have been lucky to have spent my 40 plus years as a practicing engineer in Flight Propulsion.
- Answer-shopping is the dubious practice of comparing different approaches to problem-solving and choosing the answer you like rather than the most plausible.
- Outliers are often infuriating, but they can be very informative too.
- Probability and Likelihood – looking thorough the other end of the telescope.
- \(\chi^2\) (chi-square) and the LogLikelihood Ratio
- The
**Weibull**model is remarkable for being able to provide insight in so many situations. are dangerous, especially large, small, and in-between ones.”*“p-values*

– Frank E. Harrell Jr., Prof. of Biostatistics and Department Chair, Vanderbilt University- Generalized Linear Models

Generalized linear models link the independent variable with the*probability*of observing the dependent variable. - POD “floor” and POD “ceiling”

Some data do*not*support a POD curve that goes to zero on the left, or to one on the right. - 2 + 2 = 5

Just because you can make a statistical statement doesn’t make it true, no matter how much you wish that it were. - Parameter Estimates are NOT parameter values.

There is a profound difference between the mathematical behavior of a function whose parameter values are*defined*(e.g. the FORM/SORM paradigm) and the same function whose parameter values must be*estimated*from data. - How â vs a POD Models Work

POD (Probability of Detection) is the probability that a signal, (â, “ahat”) will be larger than the decision threshold. - How hit/miss POD Models Work

POD (Probability of Detection) as a function of size is less straightforward for binary (yes/no) data when compared with data having a continuous response (â). - How WELL do POD Models Work?

In reality we only get to see ONE collection of data, and from that must estimate the most likely model for the unseen and unknown and unknowable “truth.” - How the LogLikelihood Ratio Criterion Works (an animated example)

Constructing Confidence Bounds on Probability of Detection Curves based on how likely some alternatives to the*maximum*likelihood would be. - MIL-HDBK-1823A “Nondestructive Evaluation System Reliability Assessment”

2009 release of 2007 Update describes procedures for acquiring NDE data and statistical methods for analyzing it to produce*POD(a)*curves, 95% confidence bounds, noise analysis, and noise*vs*detection trade-off curves, and includes worked-out examples using real Hit/Miss and \(\hat a \; vs\; a\) data. - History of MIL-HDBK-1823A Algorithms The Handbook has a 30+ year history. It began to standardize how flight propulsion manufacturers computed reliability since we all agreed that it would be unprincipled to compete in the market based on safety issues.
- POD Short Course/Workshop

This two-day short course is based on the new (2007) MIL-HDBK-1823 and uses the**mh1823 POD software**. The course provides the latest methods for measuring your NDE system’s effectiveness and the workshop will use these state-of-the-art techniques to analyze your enterprise data. - Themes …

… I’m not a philosopher – but I, like you, do occasionally ruminate on the human condition. - Hubris

“I don’t need to understand your problem to solve it.” - The Great Misunderstanding

Both statisticians and engineers recognize the mathematical competence of the other, and this is the cause of The Great Misunderstanding. - Quantitative Nondestructive Evaluation

It isn’t the smallest crack you can find that’s important – it’s the largest one you can miss. - False Positives and the ROC Curve …

The relationship between POD and False Positives depends on more than the inspection itself. It also depends on the frequency of defectives in the population being inspected. - Will …

If you think you are beaten, you are … - Probability and Statistics …

… are**not**one and the same. The differences are not nuanced. They are Apples and Oranges. - Reading List

I am often asked to recommend a “good statistics text.” Here are a few that I refer to often. - Two Secrets of Success
- Monte Carlo Oversights

Most Engineering Monte Carlo simulations ignore the distinction between parameter values, and*estimates*of parameter values, resulting in a gross underestimation of the probability of “low-probability” events. - Repeated Inspections

Repeated inspections do*not*improve Probability of Detection (POD). - Central Limit Theorem Fine-Print

Readers have requested further explanation of when the CLT does*not*apply. - Pseudo-Proof that 2 equals 1

Seemingly logical steps can lead to a silly conclusion. Unfortunately, not all silliness is as self-evident as this example. - The “Most Probable Point” is a fiction

First Order, and Second Order Reliability Methods (FORM/SORM) are based on a demonstrably false premise of a “Most Probable Point.” - Contrasting the Statistical with the Mathematical Properties of NESSUS/FORM

Engineers see reliability as an optimization problem on a known response surface. Statisticians view it differently. - “Choosing” the Right Distribution

There is considerable folklore about choosing statistical distributions, as you might select the appropriate club from your golf bag. - Frequentists and Bayesians

There is a continuing debate among statisticians over the proper definition of probability. - “Probabilistics”

There is more to Monte Carlo simulation than replacing constants with probability densities. - Bivariate Normal

Here is a simple algorithm for sampling from a bivariate normal distribution. - Did you know … ?
- Goodness-of-Fit

Goodness-of-Fit tests, like Anderson-Darling, tell you when you*don’t*have a normal distribution. - R-squared…

… is an often misused goodness-of-fit metric, where bigger isn’t always better. - Other Measures

R-squared isn’t the only way to judge how well the model works. - Chronology of Crack Initiation

Tongue-in-cheek view contains insights. - Curse of Dimensionality

Direct-sampling Monte Carlo requires the number of samples per variable to increase exponentially with the number of variables to maintain a given level of accuracy. - Convergence in Distribution

We engineers are familiar with convergence to a point, but what of convergence to a*distribution?* - Extreme Value Distributions

The largest, or smallest, observation in a sample has one of three possible distributions. This is another example of “convergence in distribution.” - Joint, Marginal, and Conditional Probability

We engineers often play fast and loose with joint, marginal, and conditional probabilities – to our detriment. - Correlation:

It’s a lot more – and less – than you may think. - Outliers …

Often infuriating, these can be very informative too. - Wrong Grid?

Choosing the wrong grid can undermine your analysis, mislead your audience, and make you look foolish. - Bayesian Thinking

… including an example from NDE - Random Fatigue Limit (RFL) on a P/C

Pascual and Meeker’s RFL solves an old problem: how to have a runout model go through (rather than under) all the runout data. - IntraOcular Trauma Test

Sometimes the best Goodness-of-Fit test is the easiest. - Central Limit Theorem

Why is the*Average*of nearly anything always*Normal*? - Hiking the Grand Canyon, rim-to-rim!

Words and pictures are insufficient. - Bayesian Updating

We use Bayesian Statistics every day without knowing it. - Sums of Random Variables

Sometimes you need to know the distribution of some combination of things. Here’s an example. - Distributional Inter-Relationships

There are myriad probability distributions. But did you know that most are related to one another, and ultimately related to the Normal?

Last updated Wednesday, 6 December, 2017

© 2018 Charles Annis, P.E. Statistical Engineering 561-352-9699