Background
Overview:
MILHDBK1823A tells how to plan an NDE experiment, design and
fabricate reliability demonstration specimens, acquire the system
performance data, and provides statistical methods for analyzing the
data to produce POD(a) curves, 95% confidence bounds, noise
analysis, and noise vs. detection tradeoff curves. It presents
workedout examples using real Hit/Miss and â data, and serves as a
user’s manual for the mh1823 POD software.
These methods are statistical bestpractices and have universal
applicability – NDE of engines, airframes, ground vehicles – subject
to the following limitations:

The NDE systems must produce output that can be reduced to either a
quantitative signal, â, or a binary response, hit/miss. (Images
therefore will require some preprocessing to provide either â or
Hit/Miss as input to these analysis methods.)

The specimens must have targets with measurable characteristics,
like size or chemical composition. This precludes amorphous targets
like corrosion unless a specific measure can be associated with it,
such that other corrosion having that same measure will produce the
same output from the NDE equipment.

This mh1823 POD software assumes that the input data are correct.
That is, if the size is X, then that is the true size. If the
response is Y, then that is the true response. Situations where
these conditions cannot be ensured (e.g. where target sizing is only
approximate) will necessarily provide only approximate results. (The
problem of accurate crack sizing is discussed in Handbook Appendix
I.1 Departures from Underlying Assumptions – Crack Sizing and POD
Analysis of Images.)

This mh1823 POD software assumes that a POD curve goes to zero on
the left, and to one on the right. Data for which min(POD) > 0
(perhaps due to signal contamination by excessive background noise),
or max(POD) < 1 (resulting from random misses not related to target
size) cannot be correctly represented by a model for which min(POD)
= 0 and max(POD) =1. (See MILHDBK1823, Appendix I4 "Asymptotic
POD Functions.")
http://www.statisticalengineering.com/mh1823/QNDE/PODfloorceiling.htm
If the input data do not meet MILHDBK1823 requirements, the mh1823
POD software may still produce an answer, but it will be WRONG.
MILHDBK1823 History:

Mid 1980s: A working group from the USAF, UDRI, GEAE, P&W, and Garrett,
later AlliedSignal, now Honeywell, produced MILHDBK1823, "Nondestructive Evaluation System
Reliability Assessment." While it would be some years before an official
publication was available, the draft became the de facto world standard for
establishing quantitatively the effectiveness of inspections by measuring POD
(Probability of Detection). Charles Annis, P.E. was the primary author and
overall editor of the first edition of MILHDBK1823. That first Working Group
included Chuck Annis and Steve Cargill at P&W, Dennis Corbly and Jim Griffiths
at GE, Al Berens and Pete Hovey of UDRI, and Vicki Panhuise at AlliedSignal.
Sharon Vukelich was the primary Air Force contributor.

1993: NATO AGARD (North Atlantic Treaty Organization, Advisory Group
for Aerospace Research and Development) sponsored 2day POD Short
Course based on MILHDBK1823 that Chuck Annis and Sharon Vukelich
presented in Ankara, Turkey, Lisbon, Portugal, Patras, Greece, and
Ottawa, Canada.

Late 1990s: USAF officially publishes MILHDBK1823, 30 April, 1999,
and again in April 2004. By then the Handbook had been the de facto
world standard for nearly two decades.

Early 2000s: ModelAssisted POD  The MAPOD Working Group was formed
in Austin in 2003 with impetus from James Malas, Ph.D., Chief,
Nondestructive Evaluation Branch, Materials and Manufacturing
Directorate, Air Force Research Laboratories. Bruce Thompson
and Bill Meeker were key contributors.

2006: Under AFRL sponsorship, Chuck Annis began work to incorporate
two decades of NDE progress in an update of MILHDBK1823, under
Agreement: 06S50801016C1, Prime contract: F3361503D5204.

The work was largely funded by USAF AFRL/MLLP, Nondestructive Evaluation
Branch, Materials and Manufacturing Directorate, James C. Malas, Ph.D.,
Branch Chief. Special thanks to Charles Buynak, and to Jeremy Knopp who
was the technical monitor. Numerous colleagues provided counsel for the
update including Steve Cargill of Aerospace Structural Integrity, Floyd
Spencer at Sandia National Laboratories, Al Berens at UDRI, and Bill Meeker
and Bruce Thompson of Iowa State.

The work was completed in February, 2007, although it was not officially
released by the Air Force until April, 2009.

The deliverables of the 20062007 contract to update MILHDBK1823
included the draft of the updated Handbook and algorithms necessary
to implement the methods described in the Handbook. Software was not a deliverable item.
However it was soon obvious that delivering descriptions of statistical
algorithms, with no convenient implementation of them, would be of limited
utility, so I wrote the mh1823 POD software suite to make the handbook's
methods more widely usable. The software is © Charles Annis, P.E. and is not
in the Public Domain, however it can be obtained, gratis, by sending a request to:
Charles.Annis@StatisticalEngineering.com, with "mh1823" in the title block.
The software is in use worldwide, including some university programs.

7 April, 2009: The 2007 update was released by the USAF. Download a
copy here.
NOTE: The mh1823 POD algorithms use R, the most powerful statistical
and graphics engine available anywhere, for all data manipulation,
statistical analysis, and graphics. Because R is opensource (and
free), and because all of the algorithms and methods developed here
are based on modern, well documented statistical best practices
described in the open literature, there is nothing proprietary in
this mh1823 POD software. Since there are no restrictions on its
use, the mh1823 POD software can be used as a universal standard for
performing Probability of Detection (POD) analysis.
Click here to see the mh1823 POD software version history.