Every number has a source
Validation
Every entry below is a calculation SPCPlot pins to a fixture, with the source, the data, the method, the numbers and the automated test that asserts them. Almost all of them are published worked examples reproduced; where a calculation is simple enough that no publisher ever needed to work it out, the fixture is a table you can check by hand instead, and the entry says so. The comparisons run in continuous integration on every commit, so a red test is visible on the commit rather than discovered by a reader.
17 of the 18 planned tools appear here. A tool is added to this page when a test pins it to something a reader can check — a publisher's worked example, or arithmetic transparent enough to verify unaided — and not before. An absent tool is not a tool that failed: it is a tool whose calculation has not shipped yet, or one whose published sources print a method without printing an answer.
Reproducing a worked example establishes that the arithmetic agrees with the source's arithmetic on the source's data. It is not a certification. SPCPlot is not approved or endorsed by NIST, AIAG, ISO, ASTM, ANSI or ASQ, and claims conformance to no standard — see the terms.
I-MR chart — NIST/SEMATECH individuals example
Open the tool →- Source
- NIST/SEMATECH e-Handbook of Statistical Methods §6.3.2.2, "Individuals Control Charts"Public domain (US Government work)
- Data
- Ten batches of a chemical process, flow rate: 49.6, 47.6, 49.9, 51.3, 47.8, 51.2, 52.6, 52.4, 53.6, 52.1. This is also the dataset the tool loads on first visit, so the figure on the tool page IS this comparison.
- Method
- MRᵢ = |xᵢ − xᵢ₋₁|; σ̂ = MR̄ / d₂ with d₂ = 1.128 as NIST states it; individuals limits x̄ ± 3σ̂.
- Asserted by
- src/plots/imr/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| x̄ (centre line) | 50.81 | 50.81 | yes |
| MR̄ | 1.8778 | 1.8778 | yes |
| UCL, using NIST’s d₂ = 1.128 | 55.8041 | 55.8041 | yes |
| LCL, using NIST’s d₂ = 1.128 | 45.8159 | 45.8159 | yes |
| UCL, using the exact d₂ = 1.1283792This is what the tool ships by default. It differs from NIST’s printed limit by 0.0017 — the fifth significant figure — purely because NIST computed from a three-decimal d₂. Both are on the figure’s disclosure line, and the "Constant precision" option switches between them. | not published | 55.8024 | yes |
| Points out of control (Nelson test 1) | none — NIST’s figure shows a stable process | 0 | yes |
What this does not establish
- NIST does not print moving-range chart limits for this example, so the MR panel’s D₄·MR̄ upper limit is derived and is not cross-checked against a published answer.
- Reproducing this example establishes that the individuals arithmetic agrees with NIST on this dataset. It says nothing about the other seventeen tools, and nothing about conformance to any standard.
Control chart constants — two independent published tables
Open the tool →- Source
- CRAN SixSigma package vignette "Shewhart Constants for Control Charts" (4 dp), and the AIAG-provenance table hosted by MIT (3 dp)Public (GPL package documentation), and a publicly hosted table
- Data
- Every constant printed for subgroup sizes 2 to 10 in both tables: d₂, d₃, c₄, A₂, A₃, B₃, B₄, D₃, D₄ and E₂.
- Method
- c₄ = √(2/(n−1))·Γ(n/2)/Γ((n−1)/2) in closed form; d₂ and d₃ from the moments of the range distribution by numerical integration; every other factor derived from those three. Nothing is tabled.
- Asserted by
- src/spc/__tests__/constants.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| d₂ at n = 2 | 1.1284 | 1.1284 | yes |
| d₃ at n = 2 | 0.8525 | 0.8525 | yes |
| c₄ at n = 2 | 0.7979 | 0.7979 | yes |
| A₂ at n = 2 | 1.8800 | 1.8800 | yes |
| D₄ at n = 2 | 3.2665 | 3.2665 | yes |
| d₂ at n = 10 | 3.0775 | 3.0775 | yes |
| c₄ at n = 10 | 0.9727 | 0.9727 | yes |
| B₃ at n = 10 | 0.2837 | 0.2837 | yes |
| D₃ at n = 10 | 0.2230 | 0.2230 | yes |
| D₄ at n = 3The two published tables disagree with each other here. The exact value is 2.574591, which rounds to 2.5746 — matching the four-decimal source — and to 2.575, which does NOT match the three-decimal source’s 2.574. That table truncated, or derived D₄ from its own three-decimal inputs. This is the only cell in either table this site does not reproduce, and it is asserted as an exception rather than papered over. | 2.5746 (4 dp source) / 2.574 (3 dp source) | 2.5746 / 2.575 | see note |
| E₂ at n = 2E₂ = 3/d₂. From the printed d₂ = 1.128 that is 2.6596, which rounds to 2.660 as the table prints. From the exact d₂ it is 2.6587, which rounds to 2.659. Both are correct answers to different questions, and this one-unit difference is what moves the NIST I-MR limit above. | 2.660 | 2.659 | see note |
2 rows marked “see note”: these are cases where the published sources disagree with each other or with their own rounding. They are listed rather than resolved silently.
What this does not establish
- The published tables cover subgroup sizes 2 to 25. The tool computes to n = 100 on exactly the same footing, but those rows cannot be cross-checked against any printed source and are marked on the figure.
- d₄ — the median moving-range constant — is computed here but is not printed by either source, so it is checked against its own closed form at n = 2 (√2·Φ⁻¹(0.75)) and nothing else.
- Agreement with two independent tables is evidence the arithmetic is right. It is not a certification, and neither publisher has reviewed this site.
Pareto chart — arithmetic checked by hand, not reproduced from a publisher
Open the tool →- Source
- No published worked example exists, and none is claimed. The nearest thing to a primary source is Juran’s own correction of the name: “The Non-Pareto Principle; Mea Culpa”, Quality ProgressCited for the origin of the idea only — no number on this page comes from it. (The hosted file is dated 1974 and the article is normally cited as 1975; the discrepancy is recorded rather than resolved.)
- Data
- Five hundred customer complaints in seven categories — late delivery 200, wrong quantity 150, damaged packaging 75, cosmetic scratch 40, missing documentation 20, wrong label 10, billing error 5. Every count divides into 500 exactly, so every percentage is a whole number and the entire chart can be checked on the back of an envelope. This is the dataset the tool loads on first visit, and it is entered alphabetically so that the ordering below is evidence the sort ran.
- Method
- Aggregate by category; sort by count descending with ties broken by first appearance; cumulative % = running total ÷ 500 × 100. The vital few is the first bar whose cumulative total reaches or passes the reference percentage.
- Asserted by
- src/plots/pareto/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| Grand total | 500 (200+150+75+40+20+10+5) | 500 | yes |
| Plotted order, from an alphabetical paste | descending by count | descending by count | yes |
| Share of the largest category | 200/500 = 40% | 40.0% | yes |
| Cumulative % at the third bar | (200+150+75)/500 = 85% | 85.0% | yes |
| Cumulative % at the last barSet to exactly 100 rather than to the accumulated division. Every count is non-negative, so the last bar completing the total is a statement of fact — but it is the difference between a figure reading 100.0 and one reading 99.99999999999999. | 100% | 100.0% | yes |
| Vital few at the 80% lineTwo categories give 70%, three give 85%; the boundary is “reaches or passes”, and the figure reports the cumulative total actually reached rather than the 80% that was asked for. | 3 categories, reaching 85% | 3 categories, reaching 85.0% | yes |
| The same 500 complaints as a raw occurrence logOne row per complaint, no Count column, interleaved rather than grouped. The tally path and the frequency-table path must produce the same bars from the same underlying facts. | identical chart | identical chart | yes |
What this does not establish
- This entry reproduces nothing. It asserts that the shipped code agrees with a hand computation on a dataset chosen so the hand computation is exact — which is all that can be established about arithmetic with no convention in it, and is not the same kind of evidence as the NIST reproduction above.
- What the fixture cannot check is the part of a Pareto chart that is a judgement rather than a calculation: which categories your data-collection form allows in the first place, and where you set the “other” threshold. Both change what the reader concludes, and both are printed on the figure for that reason.
- The 80/20 relationship is an observation about some datasets, never an assumption this code makes about yours. The reference line is a setting, and on this very dataset the first two categories come to 70%, not 80%.
p chart — NIST/SEMATECH proportions example (recomputed, not published)
Open the tool →- Source
- NIST/SEMATECH e-Handbook of Statistical Methods §6.3.3.1, "Proportions Control Charts"Public domain (US Government work) — data and method printed, answers not
- Data
- Thirty wafers, fifty chips inspected on each, as the fractions defective NIST prints: 0.24, 0.30, 0.16, 0.20, 0.08, 0.14, 0.32, 0.18, 0.28, 0.20, 0.10, 0.12, 0.34, 0.24, 0.44, 0.16, 0.20, 0.10, 0.26, 0.22, 0.40, 0.36, 0.48, 0.30, 0.18, 0.24, 0.14, 0.26, 0.18, 0.12. Multiplying each by fifty gives a whole number of defective chips every time, which is the transcription check. This is the dataset the np chart page loads on first visit.
- Method
- As NIST states it: p̂ᵢ = Dᵢ/n; p̄ = (ΣDᵢ)/(mn) = (Σp̂ᵢ)/m; then UCL = p̄ + 3√(p̄(1−p̄)/n), CL = p̄, LCL = p̄ − 3√(p̄(1−p̄)/n). Out of control means strictly beyond a limit.
- Asserted by
- src/plots/defectives/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| ΣD and Σn from the printed fractions | 30 samples × 50 chips | 347 defective out of 1500 | yes |
| p̄, by NIST’s two equivalent forms (ΣDᵢ)/(mn) and (Σp̂ᵢ)/mThe two forms coincide only because n is constant here. Once the subgroup sizes differ they are different statistics, and this tool uses the pooled one. | stated equal; no value printed | 0.2313333 and 0.2313333 | yes |
| σ_p = √(p̄(1−p̄)/50) | not printed | 0.0596353 | yes |
| UCL = p̄ + 3σ_p | not printed — NIST shows a chart image only | 0.4102391 | yes |
| LCL = p̄ − 3σ_pPositive here, so nothing is clamped. At this defective rate the limit only reaches zero for subgroups below about thirty-nine chips. | not printed | 0.0524275 | yes |
| Subgroups beyond a limitRecorded because this project’s own validation report lists only sample 23. Recomputing from the data finds two: sample 15 at 0.44 and sample 23 at 0.48, against a limit of 0.4102391. Sample 21, at 0.40, is inside it. The report’s list is incomplete and is not used. | not printed | 2 — samples 15 and 23 | yes |
What this does not establish
- This is a recomputation, not a reproduction. NIST prints the data and the formulas for this example but no numeric answer, so there is nothing to compare the last column against — the evidence here is that the shipped code applies the source’s stated method to the source’s transcribed data, not that the source agrees with the result.
- The dataset has a constant subgroup size, so it exercises none of the stair-step behaviour that the p chart exists for. That path is covered by fixtures in the same test file, checked against the algebra (σᵢ ∝ 1/√nᵢ, and the clamp threshold nᵢ < 9(1−p̄)/p̄) rather than against any publication — no public worked example of a variable-n p chart with printed limits was found.
- Reproducing arithmetic on one dataset says nothing about conformance to any standard, and no such conformance is claimed.
np chart — the same NIST example as counts, and the n·p identity
Open the tool →- Source
- NIST/SEMATECH e-Handbook of Statistical Methods §6.3.3.1, "Proportions Control Charts"Public domain (US Government work) — data and method printed, answers not
- Data
- The same thirty wafers, expressed as the counts they came from: 12, 15, 8, 10, 4, 7, 16, 9, 14, 10, 5, 6, 17, 12, 22, 8, 10, 5, 13, 11, 20, 18, 24, 15, 9, 12, 7, 13, 9, 6 defective chips out of fifty. This is the dataset the tool loads on first visit, so the figure on the page is this comparison.
- Method
- np̄ = n·p̄ with p̄ = ΣDᵢ/Σnᵢ; limits np̄ ± 3√(np̄(1−p̄)), floored at 0 and capped at n. The handbook prints the proportions form; the count form is the same statement multiplied through by n, which is the relation the last row checks.
- Asserted by
- src/plots/defectives/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| np̄ = n·p̄ | not printed | 11.5666667 | yes |
| σ_np = √(np̄(1−p̄)) | not printed | 2.9817631 | yes |
| UCL = np̄ + 3σ_np | not printed | 20.5119559 | yes |
| LCL = np̄ − 3σ_np | not printed | 2.6213774 | yes |
| The identity: 50 × the p-chart UCL√(np̄(1−p̄)) = n·√(p̄(1−p̄)/n), so the two charts are the same chart in different units and must agree exactly. The test suite asserts this on every point, which is what allows one engine to serve both pages without the two answers drifting. | implied by the formulas, not printed | 20.5119559 against 20.5119559 | yes |
| Subgroups beyond a limitThe same two the p chart flags, as the identity above requires. Sample 21, at exactly twenty defectives against a limit of 20.5119559, is not flagged: out of control means beyond the limit, strictly. | not printed | 2 — samples 15 and 23 | yes |
What this does not establish
- A recomputation, not a reproduction: NIST prints no numeric answer for this dataset. What the entry establishes is internal — the stated method applied to the transcribed data, and the exact agreement of the two chart forms.
- The np chart is only defined when the inspected quantity is constant, which it is here. Nothing on this page speaks to what the tool does when it is not; it draws a stepped chart with a warning naming the p chart, and that behaviour is covered by tests rather than by a publication.
- No conformance to any standard is claimed, by this entry or anywhere else on this site.
Run chart — the four runs-test p-values, against a published example
Open the tool →- Source
- Minitab support documentation, "All statistics and graphs for Run Chart" and "Methods and formulas for Run Chart"Public vendor documentation. Numeric results and the stated method are used; no text, table or figure from it is reproduced.
- Data
- RECONSTRUCTED, not transcribed — and this is the entry’s main caveat. The source prints the output of an example but not the eleven measurements behind it. The series 12, 9, 11, 8, 6, 10, 7, 13, 15, 14, 5 was built to satisfy every published intermediate count: eleven points, five above the median of 10 and six on or below it, six runs about the median with the longest spanning points 4–7, and seven runs up or down with the longest spanning points 3–5. Those constraints are tight — E(R) = 6.5 to one decimal together with E(V) = 7.0 forces N = 11 and mn = 30 — but they do not uniquely determine the data, only the statistics.
- Method
- Runs about the centre line: m points above, n on or below, N = m + n, a point exactly on the line counted below; E(R) = 2mn/N + 1; Var(R) = 2mn(2mn − N)/(N²(N − 1)); p(clustering) = Φ(z) and p(mixtures) = 1 − Φ(z). Runs up and down: E(V) = (2N − 1)/3; Var(V) = (16N − 29)/90; p(trends) = Φ(z) and p(oscillation) = 1 − Φ(z). Normal approximation, no continuity correction.
- Asserted by
- src/plots/runchart/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| Number of runs about the median | 6 | 6 | yes |
| Expected number of runs about the medianThe exact value is 2·5·6/11 + 1 = 6.4545…, printed by the source at one decimal. The test asserts it at that precision and asserts the exact value separately, rather than widening what was printed. | 6.5 | 6.5 | yes |
| Longest run about the median | 4 (points 4–7) | 4 (points 4–7) | yes |
| Approximate p-value for clustering | 0.385 | 0.385 | yes |
| Approximate p-value for mixtures | 0.615 | 0.615 | yes |
| Number of runs up or down | 7 | 7 | yes |
| Expected number of runs up or down | 7 | 7.0 | yes |
| Longest run up or down | 3 (points 3–5) | 3 (points 3–5) | yes |
| Approximate p-value for trends | 0.500 | 0.500 | yes |
| Approximate p-value for oscillation | 0.500 | 0.500 | yes |
| Clustering p-value under the alternative published E(R) = 2mn/N + ½Two public sources print two different expectations for the number of runs about the centre: 2mn/N + 1 (the Wald-Wolfowitz statistic, used here) and 2mn/N + ½ (a continuity-corrected form printed in the random-number-testing literature). They are not interchangeable. Under the +½ form z moves from −0.29 to +0.03 and the clustering p-value from 0.385 to about 0.51, which does not reproduce the source. This row is here because the choice was made on evidence rather than on preference. | not published | 0.512 | see note |
1 row marked “see note”: these are cases where the published sources disagree with each other or with their own rounding. They are listed rather than resolved silently.
What this does not establish
- THE DATA IS RECONSTRUCTED. The source publishes the statistics and the p-values but not the eleven measurements, so the series was built to match every published count and the p-values were then checked against what was printed. That is weaker than the I-MR entry, where the source publishes its data. It is not nothing: ten printed values reproduce, two of them p-values to three decimals, and they reproduce under only one of the two published forms of E(R).
- Reproducing this example establishes that the four runs-test p-values agree with the reference implementation on a series with these counts. It says nothing about the pattern-rule overlay, whose behaviour on a chart with no control limits is a decision this site made and documented rather than a published one.
- Both tests are normal approximations to discrete distributions and are labelled approximate by their source and on the figure. Below about twenty points, or fewer than ten on one side of the centre line, they are indicative rather than exact and the tool says so.
- No conformance to any standard is claimed, and neither the publisher of the source nor anybody else has reviewed this site.
Anderson-Darling — the D’Agostino & Stephens worked example
Open the tool →- Source
- "Anderson-Darling Test for Normality", SPC for Excel — reproducing the method and the p-value approximation of D’Agostino, R. B. & Stephens, M. A. (eds.), 1986, Goodness-of-Fit Techniques, Marcel DekkerPublic secondary source. The originating book is copyrighted and paywalled; the formulas and the 44 data values are facts, and no text or layout from either is reproduced here.
- Data
- Forty-four birth weights in grams: 3837, 3480, 3334, 3116, 3554, 3428, 3838, 3783, 3625, 3345, 2208, 3034, 1745, 2184, 2846, 3300, 3166, 2383, 3520, 3428, 3380, 4162, 3294, 3630, 2576, 3406, 3208, 3402, 3521, 3500, 3746, 3736, 3523, 3370, 2902, 2121, 2635, 3150, 3920, 3866, 3690, 3542, 3430, 3278. This is also the dataset the tool loads on first visit, so the figure on the tool page IS this comparison.
- Method
- A² = −N − S with S = Σ((2i−1)/N)[ln F(Yᵢ) + ln(1 − F(Y_{N+1−i}))], F the normal CDF at the sample mean and sample standard deviation; A²* = A²(1 + 0.75/n + 2.25/n²); p from the piecewise exponential approximation, branch A²* ≥ 0.6.
- Asserted by
- src/plots/histogram/__tests__/normality.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| A² (unadjusted) | 1.717 | 1.717 | yes |
| A²* = A²(1 + 0.75/n + 2.25/n²) | 1.748 | 1.748 | yes |
| p-value | 0.000179 | 0.000179 | yes |
| Recomputed past the source’s precisionPinned at this precision as well as at the source’s, so a change in the normal CDF or in the summation order shows up as a failing test rather than as a rounding coincidence. | not published | A² = 1.7168461 · A²* = 1.7481059 | yes |
| Verdict at α = 0.05, critical value 0.752 | not normal | normality rejected | yes |
What this does not establish
- The p-value formulas are an approximation, not the null distribution. They are discontinuous at their branch joins — about 0.0025 at A²* = 0.6, and upward, so p briefly RISES as the statistic worsens — and at A²* = 0.752 they return 0.0503 rather than the 0.05 that critical value is meant to mark. Both facts are asserted in the test file and printed under the figure.
- They apply only where μ and σ were estimated from the same data. Supply a known μ and σ and the tool reports the statistic alone, because that is a different null distribution.
- Anderson-Darling gains power with n. Past roughly 300 observations it rejects on departures too small to affect any decision, and below about 20 it rejects almost nothing. Reproducing this example says nothing about either regime.
- Reproducing this example establishes that the statistic, the modification and the polynomial branch all agree with the source on this dataset. It is not a conformance claim and neither publisher has reviewed this site.
Anderson-Darling — NIST’s heat flow meter, and the other modification
- Source
- NIST/SEMATECH e-Handbook §1.4.2.8.3, "Quantitative Output and Interpretation", heat flow meter calibration case studyPublic domain (US Government work)
- Data
- ZARR13.DAT — 195 heat flow meter calibration factors collected by Bob Zarr at NIST in January 1990, transcribed in full into src/plots/histogram/sample.ts.
- Method
- The same A² definition, with NIST’s own small-sample modification A²(1 + 4/n − 25/n²) and NIST’s own α = 0.05 critical value of 0.787.
- Asserted by
- src/plots/histogram/__tests__/normality.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| Standard deviation | 0.022789 | 0.022789 | yes |
| Sample meanSummed exactly, the mean is 9.2614607590. NIST’s printed 9.261460 is that number truncated at six decimals; rounding gives 9.261461. A discrepancy of 7.6e-7 that changes nothing downstream — recorded rather than "corrected", because a last-digit chase is how a real error gets introduced. | 9.261460 | 9.261461 | see note |
| Adjusted A², under NIST’s A²(1 + 4/n − 25/n²) | 0.129 | 0.129 | yes |
| The same data under A²(1 + 0.75/n + 2.25/n²)This is the finding. NIST prints the α = 0.05 critical value 0.752 on the page that DEFINES the test and 0.787 on the page that APPLIES it, because the two use different published constants — and the defining page says so, warning that different constants and therefore different critical values exist and that a reader must know which produced the value they are comparing against. The statistic differs by 1.6 % at n = 195 and by 6 % at n = 44. This tool ships both, names the one in force on the figure, and will not print an "Anderson-Darling statistic" without its factor. | not published | 0.127 | yes |
| Verdict against NIST’s 0.787 | do not reject normality | not rejected | yes |
1 row marked “see note”: these are cases where the published sources disagree with each other or with their own rounding. They are listed rather than resolved silently.
What this does not establish
- NIST prints no p-value for this dataset, so the p the tool shows here is not cross-checked against a published answer. Only the statistic, the standard deviation and the accept/reject decision are.
- The mean disagrees with NIST’s printed value in the sixth decimal, for the reason given above. It is a truncation rather than an arithmetic difference, and the test file asserts both forms so that it cannot be quietly resolved either way.
- Agreement on one well-behaved dataset says the arithmetic matches NIST’s arithmetic on that dataset. It is not a conformance claim.
X̄-S chart — the qcc piston-ring example
Open the tool →- Source
- "A quick tour of qcc", the vignette shipped with the R package qcc (Luca Scrucca). The piston-ring data originates in Montgomery, Introduction to Statistical Quality Control.Public documentation of a GPL package. The measurements are facts; no text, table layout or figure from either the vignette or the textbook is reproduced.
- Data
- The first 25 of 40 subgroups of five piston-ring diameters in millimetres, the trial period the vignette itself uses. The transcription is checked against eight statistics the vignette prints — the six-number summaries of the subgroup ranges and of the subgroup standard deviations — before anything is concluded from it.
- Method
- Centre line s̄; σ̂ = s̄/c₄ with c₄ = √(2/(n−1))·Γ(n/2)/Γ((n−1)/2); s-chart limits s̄ ∓ 3(s̄/c₄)√(1−c₄²), equivalently B₃s̄ and B₄s̄. The same formulas NIST/SEMATECH §6.3.2.1 prints.
- Asserted by
- src/plots/xbar/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| s̄ (centre line of the s chart) | 0.009240037 | 0.009240037 | yes |
| σ̂ = s̄/c₄Reproduced with the EXACT c₄, no rounding option touched — c₄ is closed form, and the source computes it in closed form too. This is the cleanest reproduction on this page. | 0.009829977 | 0.009829977 | yes |
| s-chart UCL | 0.01930242 | 0.01930242 | yes |
| s-chart LCLB₃ is exactly zero below n = 6, so there is no lower limit rather than one lying on the axis. The source prints 0 and this tool prints "none"; the arithmetic is identical and the second is harder to misread. | 0 | none | yes |
| A₃ at n = 5, from the exact c₄ | 1.427 (CRAN constants table, 4 dp: 1.4273) | 1.4273 | yes |
What this does not establish
- The vignette prints the s-chart summary but no X̄-chart three-sigma limits, so the averages limits on this route are cross-checked only through the σ̂ they are built from. The X̄-R entry pins a pair of printed X̄ warning limits directly.
- Reproducing this example establishes that the s-chart arithmetic agrees with a widely used reference implementation on its own data. It says nothing about the other tools here, and nothing about conformance to any standard.
X̄-R chart — the qcc piston-ring example, and what d₂’s precision does to it
Open the tool →- Source
- "A quick tour of qcc", the vignette shipped with the R package qcc (Luca Scrucca), together with NIST/SEMATECH §6.3.2.1 for the method and the factor tablePublic documentation of a GPL package, plus a public-domain US Government work. The measurements are facts; no text, table layout or figure is reproduced.
- Data
- The same first 25 subgroups of five piston-ring diameters, transcription-checked against the eight summary statistics the vignette prints.
- Method
- Centre lines x̄̄ and R̄; σ̂ = R̄/d₂; averages limits x̄̄ ± kσ̂/√n, equivalently x̄̄ ± A₂R̄; range limits R̄ ∓ 3d₃R̄/d₂, equivalently D₃R̄ and D₄R̄. d₂ and d₃ are obtained here by integrating the range distribution.
- Asserted by
- src/plots/xbar/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| R̄ (centre line of the R chart) | 0.02276 | 0.02276 | yes |
| X̄ warning limits at 2σReproduced with the exact d₂. The difference the row below is about is smaller than the five decimals the source printed here, so both precisions land on the same answer. | 73.99242 and 74.00993 | 73.99242 and 74.00993 | yes |
| σ̂ = R̄/d₂, using the source’s tabled d₂Requires "Constant precision: 3 dp". The source tables d₂ to four significant figures while computing d₃ to seven decimals — verifiable in its own options table — so its σ̂ is R̄ divided by a rounded constant. | 0.009785039 | 0.009785039 | yes |
| σ̂ = R̄/d₂, using the exact d₂ this tool ships withThree parts in a hundred thousand away from the printed value, entirely because of the constant’s precision. This is the same T-01 effect the I-MR entry documents, in a second independent source. | not published | 0.009785338 | yes |
| R-chart UCLThe one number on this page that NO precision setting reproduces, and the reason is worth stating: the source computes R̄ + 3d₃(R̄/d₂) with d₂ rounded and d₃ exact. This tool offers "both exact" (0.04812600) and "both rounded to 3 dp" (0.04812282); the published answer sits between them, seven parts in ten million from the exact one. Mixing the two precisions to force a match would be fitting the code to the answer. | 0.04812533 | 0.04812600 | see note |
| Points out of control in the trial period | none — the published charts show a stable process | 0 | yes |
| D₄ at n = 5 (NIST §6.3.2.1 factor table)Twenty-six of the twenty-seven cells in NIST’s A₂/D₃/D₄ table for n = 2…10 reproduce exactly at the three decimals NIST prints. This one does not: the exact factor is 2.1144991, which the four-decimal CRAN table prints correctly as 2.1145 and which rounds to 2.114 at three decimals. NIST appears to have rounded an already-rounded value. Noted here rather than absorbed, in the same spirit as the D₄(3) disagreement on the constants entry. | 2.115 | 2.114 | see note |
2 rows marked “see note”: these are cases where the published sources disagree with each other or with their own rounding. They are listed rather than resolved silently.
What this does not establish
- The R-chart upper limit is NOT reproduced to the precision the source printed, for the reason given in its row. Everything else on this entry is.
- The vignette prints two-sigma warning limits for the averages chart and no three-sigma limits. The three-sigma limits this tool draws by default (73.98805 and 74.01431) are therefore derived rather than cited, although they are the figures every secondary reproduction of this dataset quotes.
- NIST/SEMATECH §6.3.2.1 works no numeric example for either chart. It is cited here for the limit formulas and the factor table, which is a weaker kind of pin than a reproduced answer, and it is labelled as such rather than counted as one.
- Reproducing this example establishes agreement with one reference implementation on one dataset. It is not a conformance claim, and neither publisher has reviewed this site.
c chart — NIST/SEMATECH counts example
Open the tool →- Source
- NIST/SEMATECH e-Handbook of Statistical Methods §6.3.3.1, "Counts Control Charts"Public domain (US Government work)
- Data
- Defects found on 25 wafers: 16, 14, 28, 16, 12, 20, 10, 12, 10, 17, 19, 17, 14, 16, 15, 13, 14, 16, 11, 20, 11, 19, 16, 31, 13. This is also the dataset the tool loads on first visit, so the figure on the tool page IS this comparison.
- Method
- c̄ = Σc / k, the total number of defects over the number of samples. Under the Poisson assumption the variance equals the mean, so σ = √c̄ and the limits are c̄ ± 3√c̄. NIST adds: "If the LCL comes out negative, then there is no lower control limit."
- Asserted by
- src/plots/attribute/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| Σc over 25 wafers | 400 | 400 | yes |
| c̄ (centre line) | 16 | 16 | yes |
| UCL = c̄ + 3√c̄ | 28 | 28 | yes |
| LCL = c̄ − 3√c̄ | 4 | 4 | yes |
| Sample 3, which has exactly 28 defectsThe boundary convention, pinned rather than assumed. Out of control means strictly BEYOND the limit, so a count of 28 against a limit of 28 is not a signal. An implementation using ≥ would report two signals on this data instead of one, and would then fail to reproduce any published c-chart example that contains a point on the limit. | not flagged in NIST’s figure, despite sitting on the UCL | not flagged | yes |
| Points out of control (Nelson test 1) | one — sample 24, at 31 defects | 1 — sample 24 | yes |
| Overdispersion check on the same countsDerived, not cited. The tool adds this check because a count chart has no sigma estimator — σ is √c̄, so nothing absorbs extra variation. On NIST’s own data it does not fire at the 1% level, which is the reassuring answer: the example is consistent with the Poisson model it assumes. | not published — NIST does not test the Poisson assumption here | χ² = 37.875 on 24 df, p = 0.0357 | yes |
What this does not establish
- NIST prints the limits only to whole numbers here, because on this data they are whole numbers. That makes the comparison exact but tests no rounding behaviour.
- NIST prints the three square-root transformations and states that their limits are ȳ ± 3, but works no numeric example of any of them. The transform option is therefore implemented from the stated formulas and is checked against its own exact inverses, not against a published answer.
- Reproducing this example establishes that the c-chart arithmetic agrees with NIST on this dataset. It says nothing about the other tools, and nothing about conformance to any standard.
u chart — reduction to the NIST c-chart example
Open the tool →- Source
- NIST/SEMATECH e-Handbook of Statistical Methods §6.3.3.1, "Counts Control Charts" — used here as a limiting case, because no public source printing u-chart limits for VARYING inspection sizes could be verifiedPublic domain (US Government work)
- Data
- The same 25 wafer defect counts as the c-chart entry above, each declared as covering exactly one inspection unit.
- Method
- ū = Σc / Σn and σᵢ = √(ū/nᵢ), with limits ū ± 3σᵢ recomputed at every point. With every nᵢ = 1 those expressions collapse to c̄ = Σc/k and σ = √c̄, so the u-chart code path must return NIST’s printed c-chart limits exactly.
- Asserted by
- src/plots/attribute/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| ū at one unit per subgroup | 16 (NIST’s c̄) | 16 | yes |
| UCL through the u-chart code path | 28 (NIST’s c-chart UCL) | 28 | yes |
| LCL through the u-chart code path | 4 (NIST’s c-chart LCL) | 4 | yes |
| Limits vary from point to point? | they must not, since every nᵢ is equal | no | yes |
| Stair-step limits on the shipped weld datasetDERIVED, not a reproduction. This is the behaviour the u chart exists for, and there is no public worked example to check it against, so the test asserts every point against the stated formula ū ± 3√(ū/nᵢ) recomputed from the raw data, and separately asserts that the answer DIFFERS from the flat-limit-from-average-n version that this tool refuses to draw. | not published | UCL 1.7314 at n = 22, rising to 2.1669 at n = 8 | yes |
| Subgroups whose lower limit is clamped awayDerived. The lower limit exists exactly when the expected count ū·nᵢ exceeds 9, and the test asserts that equivalence point by point. Where it is clamped the zone tests below the centre line are suppressed rather than reported as passed. | not published | 1 of 24 | yes |
What this does not establish
- This is a limiting case, not a worked example of a u chart. It establishes that the u-chart arithmetic reduces correctly and reproduces a published answer where the two charts must agree; it does not establish that any published source computed the same stair-step limits, because none that could be verified prints any.
- Two secondary web sources presenting variable-n u-chart tables were fetched while looking for a stronger pin. Neither could be confirmed to print internally consistent limits, so neither is cited and neither was used.
- The shipped weld dataset is invented. It is realistic and it demonstrates the staircase, but it is a demonstration and not a reproduction, and no row above claims otherwise.
EWMA chart — NIST/SEMATECH worked example
Open the tool →- Source
- NIST/SEMATECH e-Handbook of Statistical Methods §6.3.2.4, "EWMA Control Charts"Public domain (US Government work)
- Data
- Twenty observations: 52.0, 47.0, 53.0, 49.3, 50.1, 47.0, 51.0, 50.1, 51.2, 50.5, 49.6, 47.6, 49.9, 51.3, 47.8, 51.2, 52.6, 52.4, 53.6, 52.1 — with λ = 0.3, EWMA₀ = 50 and s = 2.0539, all stated by the source. This is the dataset the tool loads on first visit, but NOT at these settings: the tool defaults to λ = 0.2, exact time-varying limits and σ̂ from MR̄/d₂. Switching λ to 0.3, the limit form to asymptotic and the sigma source to a supplied 2.0539 reproduces the figure below.
- Method
- EWMAₜ = λYₜ + (1−λ)EWMAₜ₋₁ from EWMA₀ = 50; s²_ewma = (λ/(2−λ))s²; limits EWMA₀ ± 3·s_ewma. NIST uses the asymptotic (steady-state) form — it does not apply the (1 − (1−λ)^2ᵗ) factor — so reproducing it requires selecting that form.
- Asserted by
- src/plots/ewma/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| λ/(2−λ) | 0.1765 | 0.1765 | yes |
| √(λ/(2−λ)) | 0.4201 | 0.4201 | yes |
| All 20 plotted EWMA values, at NIST’s two decimalsNIST prints twenty-one values; the first is EWMA₀ = 50, which is the starting value rather than a plotted statistic. Every one of the twenty that follow reproduces exactly at the printed precision. | 50.60, 49.52, 50.56, … 51.94, 51.99 | 20 of 20 match | yes |
| UCL = 50 + 3(0.4201)(2.0539) | 52.5884 | 52.5884 | yes |
| LCL = 50 − 3(0.4201)(2.0539), rounded to 4 dpOne unit in NIST’s last printed digit, and it is a display artefact rather than a difference of method. NIST’s own two printed limits are not symmetric about 50: 52.5884 − 50 is 2.5884 and 50 − 47.4115 is 2.5885, so a single half-width cannot have produced both by the same rounding. The half-width this tool computes is 2.58843174, which rounds to 2.5884 and reproduces the UCL. | 47.4115 | 47.4116 | see note |
| The same LCL, truncated to 4 dp instead of roundedTruncating the exact chain reproduces BOTH of NIST’s printed limits, which is the most likely explanation for the asymmetry above. Recorded rather than assumed: the tool rounds, as everything else on this site does. | 47.4115 | 47.4115 | yes |
| Via NIST’s own printed intermediate 0.4201 rather than the exact rootRecorded so that "use the printed intermediate" is never adopted as a fix. Carrying NIST’s rounded 0.4201 forward gives 52.5885 and fails to reproduce NIST’s own UCL; the exact √(0.3/1.7) = 0.42008403 reproduces it. This is T-01 in a new place — a rounded intermediate is not the same claim as the value it was rounded from. | 52.5884 | 52.5885 | see note |
| Points out of control | none — NIST’s figure shows the process in control | 0 | yes |
2 rows marked “see note”: these are cases where the published sources disagree with each other or with their own rounding. They are listed rather than resolved silently.
What this does not establish
- NIST states s = 2.0539 and does not derive it anywhere on the page. It is not the sample standard deviation of the twenty plotted observations (1.96904), nor MR̄/d₂ over them (2.01034), nor the root-mean-square deviation from the target (2.02602). The reproduction therefore supplies it as a known σ₀ rather than estimating it, and the figure says so. What is pinned is the EWMA arithmetic and the limit formula, not a sigma estimator.
- The reproduction uses the asymptotic limit form. The tool ships the exact time-varying form by default, which is narrower over the first several points and is NOT what NIST draws — no row above claims otherwise, and the option that switches between them is named on every figure.
- The drift dataset shipped as the tool’s second example is invented. It demonstrates the case the chart exists for and is asserted in the test file, but it reproduces nothing published and is not listed here.
- Reproducing this example establishes that the EWMA arithmetic agrees with NIST’s on NIST’s data. It establishes nothing about conformance to any standard.
Tabular CUSUM — NIST/SEMATECH worked example
Open the tool →- Source
- NIST/SEMATECH e-Handbook of Statistical Methods §6.3.2.3, "CUSUM Control Charts"Public domain (US Government work)
- Data
- Twenty averages of samples of size 4 from a process with an estimated mean of 325 and a process standard deviation of 1.27, so σ of the averages is 0.635: 324.925, 324.675, 324.725, 324.350, 325.350, 325.225, 324.125, 324.525, 325.225, 324.600, 324.625, 325.150, 328.325, 327.250, 327.825, 328.500, 326.675, 327.775, 326.875, 328.350. NIST prints S_hi, S_lo and the plain cumulative sum for all twenty rows. This is the dataset the tool loads on first visit.
- Method
- S_hi(i) = max(0, S_hi(i−1) + xᵢ − μ̂₀ − k) and S_lo(i) = max(0, S_lo(i−1) + μ̂₀ − k − xᵢ), both starting at zero, with μ̂₀ = 325, k = 0.3175 and h = 4.1959 in measurement units. A signal is an accumulator that exceeds h.
- Asserted by
- src/plots/cusum/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| σ of the averages, from the stated process σ | 0.635 | 0.635 | yes |
| S_hi, all 20 rows at NIST’s two decimals | 0.00 … 3.01, 4.94, 7.45, 10.63, 11.99, 14.44, 16.00, 19.04 | 20 of 20 match | yes |
| S_lo, all 20 rows at NIST’s two decimals | 0.00, 0.01, 0.00, 0.33, … 0.31, 0.00 … | 20 of 20 match | yes |
| The plain cumulative sum Σ(xᵢ − μ̂₀), all 20 rows | −0.07, −0.40, −0.67 … 15.73, 19.08 | 20 of 20 match | yes |
| Groups NIST marks with an asterisk as beyond h | 14, 15, 16, 17, 18, 19, 20 | 14, 15, 16, 17, 18, 19, 20 | yes |
| Group 13, the first observation after the shiftThe boundary convention again (T-18): out of control means strictly beyond h. The same convention decides the c-chart entry above, and NIST’s own asterisking depends on it here. | not marked, at S_hi = 3.01 against h = 4.1959 | not flagged | yes |
| h from NIST’s own stated relation h = dk, d = (2/δ²)·ln((1−β)/α)NIST states α = 0.0027, β = 0.01, δ = 1 and k = 0.3175 on the same page as h = 4.1959. Substituting them into the relation printed beside them gives d = 11.8089 and h = 3.7493. The page’s own arithmetic does not close, and the chart above therefore takes h from the value NIST prints and uses. | 4.1959 | 3.7493 | see note |
| The h that DOES reproduce 4.1959d = 2·ln(1/0.00135) = 13.2153: the ONE-SIDED tail α/2 rather than α, and the (1−β) numerator dropped. Both are defensible design choices and neither is the formula printed beside the number. Recorded as a finding, not adopted — the tool computes the design block exactly as the relation is written and warns when it disagrees with the h in use. | 4.1959 | 4.1959 | yes |
| h as a multiple of σ, against the page’s own rule of thumbNot a defect in the tool and not really one in the source either — the example designs h from α and β rather than from the rule of thumb, and lands at 6.61σ. It is here because a reader who takes 4.1959 for a sigma multiple, as the rule of thumb invites, will build a chart six times more sensitive than NIST’s. The tool prints k and h in both unit systems for that reason. | "choose … h to be around 4 or 5" | 6.6077σ | see note |
2 rows marked “see note”: these are cases where the published sources disagree with each other or with their own rounding. They are listed rather than resolved silently.
What this does not establish
- Two cells of NIST’s printed intermediate "325 − k − x" column have lost their sign: group 9 prints 0.54 where the value is −0.5425, and group 12 prints 0.47 where it is −0.4675. Neither reaches the accumulators — the S_lo column is right — but transcribing that column and summing it would produce a different chart. The fixture pins to the data and the stated method, not to the intermediates.
- The twentieth S_hi is exactly 19.035 in decimal, which sits on a rounding boundary. Accumulated in the order the published formula is written it prints 19.04, as NIST does; regrouped as S_hi(i−1) + (xᵢ − μ̂₀) − k it prints 19.03. The engine uses the published order and the test asserts all sixty printed cells, which is what keeps it there.
- This tool does not draw a V-mask, so nothing about the V-mask construction is reproduced here beyond the design relation quoted above. The reason is on the tool page; in short, the handbook itself records that most users prefer the tabular chart, and that is what is validated.
- The fast-initial-response option is not exercised by this reproduction and is not pinned to any published example. It is asserted only against its own defining properties in the test file.
- Reproducing this example establishes that the tabular CUSUM arithmetic agrees with NIST’s on NIST’s data. It establishes nothing about conformance to any standard.
Acceptance sampling — MIL-STD-105E’s own tabulated OC curves
Open the tool →- Source
- MIL-STD-105E, "Sampling Procedures and Tables for Inspection by Attributes", 10 May 1989 — Table X-C-1 (code letter C, n = 5) and Table X-L-1 (code letter L, n = 200)Public domain (US Government work; the cover page carries "Approved for public release; distribution is unlimited"). ANSI/ASQ Z1.4, the paywalled civilian successor, is not reproduced and is not claimed.
- Data
- The standard prints, for each plan, the lot quality p at nine fixed probabilities of acceptance: Pa = 99, 95, 90, 75, 50, 25, 10, 5 and 1 percent. Three plans are reproduced here: code letter C at AQL 2.5 (n = 5, Ac = 0), code letter C at AQL 10 (n = 5, Ac = 1), and code letter L at AQL 1.0 (n = 200, Ac = 5) — which is the plan the tool loads on first visit, so the figure a visitor sees IS this comparison.
- Method
- Pa = Σ C(n,d)·p^d·(1−p)^(n−d) for d = 0…Ac. The note under Table X-C-1 states the convention: "Binomial distribution used for percent defective computations; Poisson for defects per hundred units". The note under Table X-L-1 is different — "All values given in above table based on Poisson distribution as an approximation to the Binomial" — because §4.12.1 tabulates every sample size above 80 from the Poisson.
- Asserted by
- src/plots/sampling/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| Code letter C, Ac = 0 — p at Pa = 50% (percent defective) | 12.9 | 12.9 | yes |
| Code letter C, Ac = 0 — p at Pa = 10%Hand-checkable: with Ac = 0 the curve is just (1 − p)⁵, so p = 1 − 0.1^0.2. | 36.9 | 36.9 | yes |
| Code letter C, Ac = 1 — p at Pa = 50% | 31.4 | 31.4 | yes |
| Code letter C, Ac = 1 — p at Pa = 1% | 77.8 | 77.8 | yes |
| Code letter C, Ac = 0 — Pa = 50% in DEFECTS per hundred unitsThe same plan, the same Pa, the other block of the same printed table. 12.9 and 13.9 are both right; they answer different questions, which is why the distribution is a visible option rather than a hidden assumption. | 13.9 | 13.9 | yes |
| Code letter L, Ac = 5 — p at Pa = 50%, Poisson as printed | 2.84 | 2.84 | yes |
| Code letter L, Ac = 5 — p at Pa = 10%, Poisson as printed | 4.64 | 4.64 | yes |
| Code letter L, Ac = 5 — p at Pa = 10%, EXACT binomialThis is what the tool ships by default. It differs from the printed 4.64 in the third significant figure, entirely because the standard tabulated sample sizes above 80 from the Poisson. Both numbers are on the figure’s disclosure line depending on which model is selected, and the "OC curve model" option switches between them. | not published | 4.59 | yes |
| Lot 5,000 · level II · AQL 1.0 · normal → the planTable I gives the code letter from the lot size and the inspection level; Table II-A gives the plan. All 105 cells of Table I are asserted in src/plots/sampling/__tests__/tables.test.ts against the printed page. | code letter L; n = 200, Ac = 5, Re = 6 | code letter L; n = 200, Ac = 5, Re = 6 | yes |
What this does not establish
- This reproduces the standard’s own OC arithmetic for three plans. It does not establish that every one of the 1,248 cells of Tables II-A, II-B and II-C is transcribed correctly — that is covered separately, by a two-source machine diff and by a set of structural invariants, both described in src/plots/sampling/__tests__/tables.test.ts.
- MIL-STD-105E was cancelled in 1995. It is used here because it is public domain and because its tables are reported to be those its civilian successor adopted; it is not the document your contract names, and where the two differ your controlled copy governs.
- Nothing here is a conformance claim to ANSI/ASQ Z1.4, to ISO 2859-1 or to any other standard, and no part of any paywalled standard is reproduced anywhere in this product.
- Acceptance sampling produces a ship-or-scrap decision. This tool is a calculation aid; the disposition of a lot, and whether a plan satisfies a contract or a regulated quality system, remain the user’s responsibility.
Process capability — the piston-ring analysis published with the R package qcc
Open the tool →- Source
- Luca Scrucca, "A quick tour of qcc" — the package vignettePublic (GPL package documentation)
- Data
- The first 25 subgroups of five piston-ring diameters — 125 measurements in millimetres — from the `pistonrings` dataset shipped with qcc, analysed against a specification of 73.95 to 74.05 with a target of 74. The same transcription the X̄-R reproduction uses, verified there against eight summary statistics the vignette prints.
- Method
- σ̂ within from R̄/d₂ on the subgroups, which is what the qcc object carries; Cp = (USL−LSL)/6σ̂, Cpl = (x̄̄−LSL)/3σ̂, Cpu = (USL−x̄̄)/3σ̂, Cpk = min(Cpl, Cpu), Cpm = (USL−LSL)/(6√(σ̂² + (x̄̄−T)²)). Cp interval from the chi-square distribution on N − 1 degrees of freedom; Cpk interval from the normal approximation NIST publishes.
- Asserted by
- src/plots/capability/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| Number of observations | 125 | 125 | yes |
| Center (grand mean) | 74.00118 | 74.00118 | yes |
| StdDev — σ̂ within, using the source’s three-decimal d₂ | 0.009785039 | 0.009785039 | yes |
| Cp | 1.70 | 1.70 | yes |
| Cp_l | 1.74 | 1.74 | yes |
| Cp_u | 1.66 | 1.66 | yes |
| Cp_k | 1.66 | 1.66 | yes |
| CpmThe only public worked Cpm value found for this project. NIST prints the Cpm formula but never a number for it, so without this source the Taguchi index would ship unpinned. | 1.69 | 1.69 | yes |
| Cp, 95% interval | 1.49 to 1.91 | 1.49 to 1.91 | yes |
| Cp_k, 95% interval | 1.45 to 1.88 | 1.45 to 1.88 | yes |
| σ̂ within, using the exact d₂ = 2.3259289This is what the tool ships by default. The source tabled d₂ at three decimals (2.326); the exact value moves σ̂ in the seventh significant figure. All five indices still round to the published two decimals either way, which is why this reproduction is robust rather than a coincidence of rounding — and the "Constant precision" option switches between them. | not published | 0.009785338 | yes |
| Pp and Ppk on the same dataDerived, not cited. Recorded because it is the number a customer measuring the shipped parts would compute, and it is lower than Cpk on the same data. | not published — the source reports no overall-sigma family | Pp 1.66, Ppk 1.62 | yes |
What this does not establish
- The vignette reports no Pp or Ppk, so this reproduction pins only the within-subgroup half of the tool. The overall-sigma half is pinned separately by the NIST entry below, whose plain sample s is exactly this engine’s overall sigma.
- The confidence intervals reproduce, but the degrees of freedom used for the Cp interval are N − 1 = 124 rather than the within-subgroup degrees of freedom of 100. That is the source’s choice and it is reproduced deliberately so the printed interval matches; it is an approximation, and the figure names the df it used.
- Reproducing this example establishes that the capability arithmetic agrees with one widely used implementation on one dataset. It says nothing about conformance to any standard.
Process capability — the NIST/SEMATECH index formulas, and what "Cpk" means
Open the tool →- Source
- NIST/SEMATECH e-Handbook of Statistical Methods §6.1.6, "What is Process Capability?"Public domain (US Government work)
- Data
- THERE IS NONE — and that is the honest label on this entry. NIST states summary statistics only: USL = 20, LSL = 8, x̄ = 16, s = 2. This is a reproduction of a formula, not of a dataset, and it is listed as such rather than dressed up as the stronger thing.
- Method
- Ĉp = (USL−LSL)/6s · k̂ = |m − x̄| / ((USL−LSL)/2) · Ĉpk = Ĉp(1 − k̂) · Ĉpu = (USL−x̄)/3s · Ĉpl = (x̄−LSL)/3s, with σ estimated by the PLAIN SAMPLE STANDARD DEVIATION s.
- Asserted by
- src/plots/capability/__tests__/indices.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| Ĉp | 1.0 | 1.0 | yes |
| m, the specification midpoint | 14 | 14 | yes |
| k̂ | 0.3333 | 0.3333 | yes |
| Ĉpk | 0.6667 | 0.6667 | yes |
| Ĉpu | 0.6667 | 0.6667 | yes |
| Ĉpl | 1.3333 | 1.3333 | yes |
| What this tool CALLS the index NIST labels ĈpkThe single most important row on this page. NIST estimates σ by the plain sample standard deviation and never mentions Pp or Ppk anywhere. That is the OVERALL sigma, so the index NIST prints as "Cpk" is what this tool — and the incumbent software your customer uses — calls Ppk. A tool that pinned "Cpk" to this page and then shipped a within-subgroup Cpk would be asserting two different things with one name. Both families are printed side by side on every figure, with the estimator behind each one named. | NIST calls it Cpk | Ppk | yes |
| Cp → reject rate, 1.00 | 0.27 % | 0.27 % | yes |
| Cp → reject rate, 1.33DOES NOT REPRODUCE, at either reading of the column. The table heads this column both "Cp = 1.33" and "USL − LSL = 8σ", and those are different numbers: 2Φ(−3 × 1.33) is 66.1 ppm and 2Φ(−4) is 63.3 ppm. The printed 64 sits between them and rounds from neither. Three of the four rows reproduce exactly; this one is recorded as an exception rather than papered over, and the tool computes the tail area rather than reading a table. | 64 ppm | 66.1 ppm at Cp = 1.33, or 63.3 ppm at the 8σ reading Cp = 4/3 | see note |
| Cp → reject rate, 1.66 | 0.6 ppm | 0.64 ppm | yes |
| Cp → reject rate, 2.00 | 2 ppb | 1.97 ppb | yes |
1 row marked “see note”: these are cases where the published sources disagree with each other or with their own rounding. They are listed rather than resolved silently.
What this does not establish
- This is a formula reproduction. NIST publishes no data for it, so nothing here exercises subgrouping, the within-subgroup sigma, or any parsing — those are covered by the qcc entry above.
- NIST defines Cp, Cpk, Cpl, Cpu and Cpm and does not define Pp or Ppk at all. The mapping from its vocabulary to this tool’s is stated in the table above and printed on every figure; it is an interpretation, and a reader who disagrees with it can read both columns.
- The 1.33 row of the reject-rate table does not reproduce. It is a rounding artefact of a published table, it does not affect any computation this tool performs, and it is listed because a reader checking the tool against the handbook will otherwise find the difference themselves and mistrust the rest.
Box-Cox transformation — the NIST/SEMATECH lambda profile
Open the tool →- Source
- NIST/SEMATECH e-Handbook of Statistical Methods §6.5.2, "Box-Cox Transformation"Public domain (US Government work)
- Data
- Forty-two microwave radiation leakage measurements, which NIST attributes to Johnson & Wichern (1988), Example 4.14. Strongly right-skewed, and one measurement per subgroup — so this dataset also exercises the subgroup-size-1 path.
- Method
- x(λ) = (x^λ − 1)/λ for λ ≠ 0 and ln(x) for λ = 0, maximising f(x,λ) = −(n/2)·ln[Σ(xᵢ(λ) − x̄(λ))²/n] + (λ−1)·Σ ln(xᵢ). Note the n, not n − 1, in that variance.
- Asserted by
- src/plots/capability/__tests__/boxcox.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| f at λ = −2.0 | 7.1146 | 7.1147 | yes |
| f at λ = −0.5 | 92.7855 | 92.7855 | yes |
| f at λ = 0.0 | 104.8276 | 104.8276 | yes |
| f at λ = 0.3 | 106.5069 | 106.5070 | yes |
| f at λ = 1.9 | 68.8106 | 68.8107 | yes |
| All 39 printed rowsA finding, established across the whole table at once: not one of the thirty-nine printed values equals the recomputed value ROUNDED to four places, and every one equals it truncated. The largest gap is 9.6 × 10⁻⁵. That is why the figures above differ from the handbook in the last digit — the handbook truncated — and why the test asserts the exact relationship rather than a loose tolerance. | thirty-nine values from λ = −2.0 to 1.9 | every one reproduces as the computed value TRUNCATED to four decimals | yes |
| λ = −1.0The table runs −2.0 to 1.9 in steps of 0.1, which is forty values, and prints thirty-nine. The gap is between −1.1 and −0.9. Recomputed here, not cited. | ABSENT — the table skips it | 70.5227 | see note |
| λ̂ by maximum likelihood | 0.3 on the printed grid; "0.28 if a second digit of accuracy is calculated" | 0.27594, which is 0.28 to two digits and 0.3 on NIST’s own 0.1 grid | yes |
| Anderson-Darling before and after transformingDerived. This is the product behaviour the transformation exists for: the untransformed data fail the normality gate, so the tool refuses to let a Ppk be read without a prominent warning, and the transformation removes the objection rather than the warning being switched off. | not published | A²* 2.142 → 0.579; rejected → not rejected | yes |
1 row marked “see note”: these are cases where the published sources disagree with each other or with their own rounding. They are listed rather than resolved silently.
What this does not establish
- A claim recorded in this project’s own research notes — that NIST mislabels the λ = −1.2 row as "1.2" — was re-checked against the page and DOES NOT REPRODUCE. Every λ in that column carries its minus sign, and the λ = 1.2 row elsewhere in the table carries its own distinct value. Only the missing λ = −1.0 is real. The non-reproducing claim is recorded here so nobody re-derives it and "fixes" working code to match a defect that is not there.
- NIST publishes no capability index for this dataset and no specification limit for it. The upper limit of 0.5 used in the tool’s figures is invented for demonstration and is not part of any reproduction.
- The λ confidence interval, the rounded-λ option and the shift constant for non-positive data are implemented from their standard definitions and are checked against their own invariants. None of them is pinned to a published worked answer, because none was found.
- THE JOHNSON TRANSFORMATION IS NOT IMPLEMENTED. It is offered by the incumbent software and it belongs in this tool eventually. It is absent because no public worked example printing the fitted family, its four parameters and a resulting capability index could be found to pin an implementation against — and a number that reaches a customer submission through a code path nothing has ever reproduced is exactly what this page exists to prevent. The deferral is recorded here rather than left as a silent gap.
Gage R&R — the ten-part, three-operator, three-trial study, by both published methods
Open the tool →- Source
- "Three Methods to Analyze Gage R&R Studies", SPC for Excel (data, reduced ANOVA table, variance components, average-and-range results and the K factors), together with Minitab support documentation "Interpret the key results for Crossed Gage R&R Study" (both ANOVA tables, with and without the interaction)Public secondary sources. The ninety measurements and the method originate in the AIAG Measurement Systems Analysis manual, which is copyrighted and paywalled — the values are facts and are reproduced on both public pages, and no text, table layout or figure from the manual is reproduced here. Nothing on this site claims conformance to it.
- Data
- Ten parts, three operators, three trials — ninety readings, transcribed in full into src/plots/gagerr/sample.ts and loaded into the grid on first visit, so the figure on the tool page IS this comparison. The transcription check is the printed total sum of squares, 94.6471 to four decimals: one mistyped digit anywhere in ninety values moves it.
- Method
- A two-way crossed random-effects ANOVA with replicates. The part×operator term is dropped when its p-value reaches the threshold (0.25 as the manual states it; 0.05 is the common software default) and its sum of squares is pooled into repeatability. Variance components come from the mean squares of whichever model was fitted. %Contribution divides variances, %StudyVar divides standard deviations, and ndc = 1.41 × (part σ̂ ÷ gage R&R σ̂), TRUNCATED. The average-and-range method takes R̄̄·K₁, √((X̄_diff·K₂)² − EV²/(parts × trials)) and R_p·K₃, and combines them in quadrature.
- Asserted by
- src/plots/gagerr/__tests__/compute.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| SS(Total) — the transcription check | 94.6471 | 94.6471 | yes |
| Full model: F for part, operator and the interactionThe full model divides part and operator by MS(part × operator), not by MS(repeatability). Using the wrong denominator gives 245.614 here — the reduced model’s answer, from a model that was not fitted — which looks right and is not. | 492.291 · 79.406 · 0.434 | 492.291 · 79.406 · 0.434 | yes |
| p for the part × operator interactionAbove both the 0.25 the manual states and the 0.05 the market-leading software defaults to, so both rules drop the term. The figure prints the p-value and the threshold that fired, because the two rules do not always agree. | 0.974 | 0.974 | yes |
| Reduced model: F for part and operator, and MS(repeatability) | 245.614 · 39.617 · 0.03997 | 245.614 · 39.617 · 0.03997 | yes |
| Total Gage R&R: VarComp and %Contribution | 0.0914 · 7.76 % | 0.0914 · 7.76 % | yes |
| Repeatability and reproducibility, as %Contribution | 3.39 % · 4.37 % | 3.39 % · 4.37 % | yes |
| Part-to-part: VarComp and %Contribution | 1.086 · 92.24 % | 1.086 · 92.24 % | yes |
| %StudyVar for Total Gage R&RDERIVED, not reproduced. %StudyVar is √%Contribution by construction, so the published 7.76 % implies 27.86 % and nothing independent is being checked. It is on this page because it is the number a customer form usually asks for, and because 7.76 and 27.86 describing the same gauge is the single most misquoted fact in measurement-systems analysis. | not printed for this dataset by either source | 27.86 % | yes |
| ndc, and what truncation does to itDERIVED. ndc is truncated, not rounded: 4.86 is four distinct categories. Rounding gives five, which is exactly the threshold the widely quoted guidance asks for — so the convention decides the verdict on this gauge. | not printed by either source | 4.8605 → 4 | yes |
| Average-and-range: EV, AV, R&R, PV, TV | 0.202 · 0.230 · 0.306 · 1.104 · 1.146 | 0.202 · 0.230 · 0.306 · 1.104 · 1.146 | yes |
| Average-and-range: %EV, %AV, %R&R, %PV%PV is the one cell where the precision of the K factors reaches the printed digits. Computing K₃ gives 0.3145598 and 96.3745 %, which is what the source printed; using the source’s own rounded K₃ = 0.3146 gives 96.3757 %, which rounds to 96.38. The source printed a rounded constant and computed with a sharper one — the T-01 effect, in a second family of constants. | 17.61 % · 20.04 % · 26.68 % · 96.37 % | 17.61 % · 20.04 % · 26.68 % · 96.37 % | yes |
| K₁, K₂, K₃ — computed rather than tabledK₁ = 1/d₂(trials); K₂ and K₃ are 1/d₂*(m, 1) with d₂*(m, g) = √(d₂² + d₃²/g), which at g = 1 is √(d₂² + d₃²). At two operators that is exactly √2, so K₂ = 0.7071 where plain d₂ would give 0.8862 — a 25 % difference, and the thing a hand-built spreadsheet gets wrong. | 0.5908 · 0.5231 · 0.3146 | 0.5908 · 0.5231 · 0.3146 | yes |
| The two methods on the same ninety readings: ndcTHE FINDING THIS TOOL EXISTS FOR. Same data, same guidance, two published methods, and the gauge passes the "at least five distinct categories" rule under one and fails it under the other. Both numbers are printed on the figure, with the method that produced each. Marked as a disagreement here because that is what it is — not a defect in either method. | not published — neither source computes ndc | 4 by ANOVA, 5 by average-and-range | see note |
| The variance components printed on the vendor pageNOT A DISAGREEMENT WITH THIS CODE — a defect in the source, and the reason two pages are cited instead of one. That page’s two ANOVA tables ARE this dataset (SS 88.3619 to four decimals), but its components table is not: it prints a repeatability variance of 0.0011541 beside its own MS(repeatability) of 0.03997, and in the reduced model those two quantities are the same number. This project’s research report predicted the mismatch; fetching the page confirmed it. The ANOVA tables are used from that page and the components are pinned to the other one, whose table closes to 100 % against its own mean squares. | Total Gage R&R VarComp 0.0020816, repeatability 0.0011541 | 0.0914285 and 0.0399733 | see note |
2 rows marked “see note”: these are cases where the published sources disagree with each other or with their own rounding. They are listed rather than resolved silently.
What this does not establish
- Reproducing these two examples establishes that the arithmetic agrees with two public sources on one dataset. It is not a conformance claim, neither publisher has reviewed this site, and nothing here is approved by the body that publishes the manual the data and the method originate in.
- Neither source prints %StudyVar, %Tolerance or ndc for this dataset, so those three are recomputations under the sources’ stated definitions rather than reproductions of a printed answer. They are labelled as such in the rows above.
- The dataset was published without a tolerance, so the %Tolerance code path is not exercised by this reproduction at all. It is covered by fixtures in the test file, checked against the algebra (%Tolerance = 100 × k·σ̂ ÷ (USL − LSL)) rather than against any publication.
- The study is balanced — every part × operator cell holds three trials. The tool refuses an unbalanced study rather than guessing, and that refusal is covered by tests and by a QA figure, not by this reproduction.
- Accepting or rejecting a gauge is a decision for the user and their customer. This page shows that the numbers match two published sources; it says nothing about whether any particular gauge is fit for any particular purpose.
Gauge bias and linearity — five masters, ten readings each
Open the tool →- Source
- "The Calculations Behind a Gage Linearity Study", SPC for ExcelPublic secondary source, working through the linearity study defined in the AIAG Measurement Systems Analysis manual. The fifty measurements and the formulas are facts; no text, table layout or figure from either is reproduced, and no conformance is claimed.
- Data
- Fifty readings: ten on each of five masters at reference values 7, 9, 11, 13 and 15. This is the dataset the tool loads on first visit, so the figure on the page IS this comparison. The transcription check is the five printed average biases, each to two decimals.
- Method
- Bias = reading − reference. Ordinary least squares of bias on reference value over all fifty points; residual standard deviation s = √(SSE/(n−2)) on n − 2 degrees of freedom; t for the slope is slope ÷ (s/√Sxx); the interval for the fitted line at x is ŷ ± t·s·√(1/n + (x − x̄)²/Sxx).
- Asserted by
- src/plots/gagerr/__tests__/linearity.test.ts
| Quantity | Published / by hand | SPCPlot computes | Agrees |
|---|---|---|---|
| Average bias at each master — the transcription check | 0.49 · 0.16 · 0.02 · −0.28 · −0.61 | 0.49 · 0.16 · 0.02 · −0.28 · −0.61 | yes |
| Fitted line, bias against referenceBoth are exact terminating decimals on this data — Sxy = −52.8 over Sxx = 400 — so the test pins them to twelve decimals as well as to the three the source prints. | bias = −0.132·x + 1.408 | bias = -0.132·x + 1.408 | yes |
| Residual standard deviation | 0.253048 | 0.253048 | yes |
| t for the slope, on 48 degrees of freedom | 10.43 here, 10.433 in the sibling article | 10.433 on 48 | yes |
| t multiplier at 95 %, 48 degrees of freedomThe t quantile is found by bisecting its own CDF, and the test also pins the two degrees of freedom that have a closed form: 12.7062 at df = 1, which is tan(0.475π), and 4.3027 at df = 2. | 2.0106 | 2.0106 | yes |
| Interval on the FITTED LINE at reference 7The band drawn on the figure, at its widest point. It is the interval for the fitted line, not a prediction interval for a future reading — the two differ by a factor of four here, and the figure says which one it is drawing. | 0.3594 to 0.60863 | 0.35937 to 0.60863 | yes |
| %LinearityDERIVED. Linearity = |slope| × process variation and %Linearity = 100 × Linearity ÷ process variation, so the process variation cancels and %Linearity is exactly 100 × |slope|. Worth stating because a reader asked for "%linearity" and hunting for a denominator is looking for something that is not needed. | not printed | 13.20 % | yes |
| Overall bias, and why one master would not have caught thisDERIVED, and the reason the study exists. This gauge reads 0.49 high at the bottom of its range and 0.61 low at the top — both unmistakably significant — while its average bias over the whole range is −0.044 and cannot be distinguished from zero. A bias study at a single master would have passed it. | not printed | -0.0440, p = 0.4952 | yes |
| Residual s as printed in the sibling articleRECORDED, NOT RESOLVED. A second article by the same publisher, on what is evidently the same study — the five average biases, the fitted line and the t statistic all match — prints s = 0.302. That is not the residual standard deviation (0.253048), not the standard deviation of the fifty biases (0.452726) and not √(SSE/(n−1)) (0.250372); all three are asserted against it in the test file so the discrepancy stays a finding rather than becoming a suspicion somebody re-investigates. Every other figure in both articles reproduces exactly. | 0.302 | 0.253048 | see note |
1 row marked “see note”: these are cases where the published sources disagree with each other or with their own rounding. They are listed rather than resolved silently.
What this does not establish
- The regression is pinned; the per-master bias t tests are not. The source prints the five average biases but no t statistic or interval for any individual master, so those are recomputations under the stated method rather than reproductions.
- One printed value does not reproduce, and it is the last row above. The tool ships the residual standard deviation the primary article prints and that this code independently derives; the sibling article’s 0.302 is recorded as unexplained.
- %Bias is not exercised at all by this reproduction, because it needs a process variation and none was published with the data. Enter one and the tool computes it; the arithmetic is covered by tests, not by this page.
- Whether a gauge’s bias and linearity are acceptable is a decision for the user and their customer. Agreement with a published worked example says the arithmetic is right, and nothing more.
Found a disagreement?
A reproducible disagreement with a published source is the most useful message this site can receive. Send the data, the source and the two numbers to help@spcplot.com. If a source turns out to be wrong, that goes on this page too.