Overview

MIL-HDBK-1823A tells how to plan an NDE experiment, design and fabricate reliability demonstration specimens, acquire the system performance data, and provides statistical methods for analyzing the data to produce \(POD(a)\) curves, 95% confidence bounds, noise analysis, and noise vs detection trade-off curves. It presents worked-out examples using real Hit/Miss and \(\hat a\) data, and serves as a user’s manual for the mh1823 POD software.

These methods are statistical best-practices and have universal applicability – NDE of engines, airframes, ground vehicles, railroads – subject to the following limitations:

1. The NDE systems must produce output that can be reduced to either a quantitative signal, \(\hat a\), or a binary response, Hit/Miss. (Images therefore will require some pre-processing to provide either \(\hat a\) or Hit/Miss as input to these analysis methods.)
2. The specimens must have targets with measurable characteristics, like size or chemical composition. This precludes amorphous targets like corrosion unless a specific measure can be associated with it, such that other corrosion having that same measure will produce the same output from the NDE equipment.
3. The mh1823 POD software assumes that the input data are correct. That is, if the size is \(X\), then that is the true size. If the response is \(Y\), then that is the true response. Situations where these conditions cannot be ensured (e.g. where target sizing is only approximate) will necessarily provide only approximate results. (The problem of accurate crack sizing is discussed in Handbook Appendix I.1 Departures from Underlying Assumptions – Crack Sizing and POD Analysis of Images.)
4. This mh1823 POD software assumes that a POD curve goes to zero on the left, and to one on the right. Data for which \(min(POD) \gt 0\) (perhaps due to signal contamination by excessive background noise), or \(max(POD) \lt 1\) (resulting from random misses not related to target size) cannot be correctly represented by a model for which \(min(POD) = 0\) and \(max(POD) = 1\). * (See MIL-HDBK-1823, Appendix I-4 “Asymptotic POD Functions.”)

 
If the input data do not meet MIL-HDBK-1823 requirements, the mh1823 POD software may still produce an answer, but it will be WRONG.

* Note: The Workshop mh1823 POD software can compute \(POD\) “floor” or “ceiling” (\(min(POD) \gt 0\) or \(max(POD) \lt 1\)), situations not covered in the Handbook. The advanced software is included in the mh1823 Workshop.

MIL-HDBK-1823 History:

• Mid 1980s: A working group from the USAF, UDRI, GEAE, P&W, and Garrett, later Allied-Signal, now Honeywell, produced MIL-HDBK-1823, “Nondes tructive Evaluation System Reliability Assessment.” While it would be some years before an official publication was available, the draft became the de facto world standard for establishing quantitatively the effectiveness of inspections by measuring POD (Probability of Detection).

 
Charles Annis, P.E. was the primary author and overall editor of the first edition of MIL-HDBK-1823. That first Working Group included Chuck Annis and Steve Cargill at P&W, Dennis Corbly and Jim Griffiths at GE, Al Berens and Pete Hovey of UDRI, and Vicki Panhuise at Allied-Signal. Sharon Vukelich was the primary Air Force contributor.

• 1993: NATO AGARD (North Atlantic Treaty Organization, Advisory Group for Aerospace Research and Development) sponsored 2-day POD Short Course based on MIL-HDBK-1823 that Chuck Annis and Sharon Vukelich presented in Ankara, Turkey, Lisbon, Portugal, Patras, Greece, and Ottawa, Canada.
• Late 1990s: USAF officially publishes MIL-HDBK-1823, 30 April, 1999, and again in April 2004. By then the Handbook had been the de facto world standard for nearly two decades.
• Early 2000s: Model-Assisted POD – The MAPOD Working Group was formed in Austin in 2003 with impetus from James Malas, Ph.D., Chief, Nondestructive Evaluation Branch, Materials and Manufacturing Directorate, Air Force Research Laboratories. Bruce Thompson and Bill Meeker were key contributors.
• 2006: Under AFRL sponsorship, Chuck Annis began work to incorporate two decades of NDE progress in an update of MIL-HDBK-1823, under Agreement: 06-S508-010-16-C1, Prime contract: F33615-03-D-5204.
• The work was largely funded by USAF AFRL/MLLP, Nondestructive Evaluation Branch, Materials and Manufacturing Directorate, James C. Malas, Ph.D., Branch Chief. Special thanks to Charles Buynak, and to Jeremy Knopp who was the technical monitor. Numerous colleagues provided counsel for the update including Steve Cargill of Aerospace Structural Integrity, Floyd Spencer at Sandia National Laboratories, Al Berens at UDRI, and Bill Meeker and Bruce Thompson of Iowa State.
• The work was completed in February, 2007, although it was not officially released by the Air Force until April, 2009.
• The deliverables of the 2006-2007 contract to update MIL-HDBK-1823 included the draft of the updated Handbook and algorithms necessary to implement the methods described in the Handbook.
• Software was not a deliverable item. However it was soon obvious that delivering descriptions of statistical algorithms, with no convenient implementation of them, would be of limited utility, so I wrote the mh1823 POD software suite to make the Handbook’s methods more widely usable. The software is Copyright Charles Annis, P.E. and is not in the Public Domain, however it can be obtained, gratis, by CALL ME – I’m still setting up the site email. The software is in use world-wide, in university programs, private and government laboratories, and by individual practitioners.
• 7 April, 2009: The 2007 update was released by the USAF. Download a copy here.

NOTE: The mh1823 POD algorithms use R, the most powerful statistical and graphics engine available anywhere, for data manipulation, statistical analysis, and graphics. Because R is open-source (and free), and because all of the algorithms and methods developed here are based on modern, well documented statistical best practices described in the open literature, there is nothing proprietary in this mh1823 POD software. Since there are no restrictions on its use, the mh1823 POD software can be used as a universal standard for performing Probability of Detection (POD) analysis.

Click here to see the mh1823 POD software version history.