Benchmark
Ray-Optics Designer — benchmark
What the ray-tracing engine has been checked against: the closed form, the published value or the independent program, our number, the difference printed as it is, and presets you can open and break yourself.
How the numbers were checked
Ray-Optics Designer traces rays through lenses and mirrors, reports focal lengths and image quality, optimizes designs, and estimates how many units of a design will pass inspection once it is built.
- 1. Checked against exact math. Where a lens or mirror has an answer that can be worked out on paper, the engine returns that answer to the limit of what a computer can store.
- 2. Checked against an independent program by another author. The open-source ray tracer rayoptics, by M. J. Hayford, is run on the same lenses, and its ray positions, focal lengths and aberration sums match ours as closely as two programs doing the same arithmetic can.
- 3. Checked against what glass makers and patents publish. The catalog reproduces the index each maker prints; what is left over is the maker’s own rounding, not our arithmetic. Two lens designs typed in from patents of 1896 and 1902 come back at the focal length those patents state, within what the patents’ own printed digits can pin down.
- 4. Broken on purpose, to show the checks work. Defects were planted in the engine one at a time, and each one had to be caught by a named check before that check counted as evidence.
The numbers are in the record below, one check at a time, with the difference printed as it is rather than rounded into an adjective.
Where a check is a limit, a sampled quantity, or a behavior rather than a number, the row says so.
Check one of those statements yourself before you read the record: this runs in your browser, on the engine the page is about, and one control breaks it.
One focus
The Cassegrain preset, traced in your browser. Move the entering ray height; the crossings stay on one point. Flip the secondary’s radius and they do not.
Secondary radius R₂
Axis crossing z, secondary’s frame
120.00000000000006 mm
Distance from the paraxial focus
1.421e-13 mm
Focal length, every digit the engine holds
399.9999999999998 mm
The crossing is where the traced ray meets the axis, in the secondary mirror’s own frame, and the distance is measured against the engine’s own paraxial focus for the radius on screen (119.99999999999991 mm as shipped). Aperture clipping is off, which is why h may exceed the preset’s own rays.
The same design, in the tool itself.
Break it on purpose: change the secondary mirror’s radius from −80 to +80. The focal length now reads 57.1429 mm and the crossings spread over 0.35 mm (row A28; how to break it, item 1).
Three things this page does not claim, and five more below:
- A correct trace of the lens you typed says nothing about whether the lens in your hand matches what you typed.
- No comparison against a commercial optical-design program appears here. We have not run one, and nothing on this page should be read as a statement about how any commercial program performs. What is here is agreement between independent implementations of the same published methods; the implementations are named in the record and the methods in the references.
- This is a sequential ray tracer: light visits the surfaces in the order you list them. Stray light, scattering and non-sequential transport are a different job and a different tool.
The verification record
Where a value is not published, the page and the tool leave it out rather than invent it; where we have no independent check, the page says so instead of claiming one.
2.0 How to read a row
Equality classes. Every row that compares two numbers carries one; nothing ships in E4. Three rows are not numeric comparisons and carry a word in place of a class: A11 behavior, G4 statistical, O2 no external reference.
| Class | Meaning | What the printed number is limited by |
|---|---|---|
| E0 (bit-exact) | == on IEEE-754 doubles. | Nothing; any nonzero would be a defect. |
| E1 (round-off) | Agreement at the float64 accumulation floor of the quantity (a few ulp; ~10⁻¹³ mm on mm-scale lengths). | Number of operations; ordering of a sum; conditioning of the quantity. |
| E2 (within the source’s stated precision) | The reference is printed to k digits; we agree inside its rounding. | The source’s print precision, quoted in the row. |
| E3 (limit or model check, cause named) | A finite-difference, sampling or series-limit check; the residual’s size and origin are stated. | The named effect (step size, grid, higher-order terms). |
| E4 (reference not verified at source) | Nothing ships in this class. | none |
Reference classes. A closed form or input-free invariance · R independent open-source implementation, named and versioned · P published data (vendor catalog, patent, journal table) · X cross-engine (our two engines against each other, or the reference implementation against one of them) · G reproducibility (determinism, seeds, replay) · O the optimizer against its own merit (descriptive; no external reference, and the rows say so). Where a row names a closed form, its Compared against line ends with the class letter in brackets.
A row’s id is its class letter and its number: A28 is the twenty-eighth row of class A, R7 the seventh comparison against an outside program, PD-15 the fifteenth planted defect.
Engine under test. Browser = the engine in your browser, the one this page’s links open · Server = the engine on our server, behind optimization and tolerancing · Reference = the implementation both were built from, itself checked against the outside programs of group R. Where a row’s number is the Reference’s, the chain to the shipped engines is row X1 (Browser and Server bit-exact on the trace primitives) and row X2 (Browser against Reference within 7.1 × 10⁻¹⁵ on fresh systems).
Oracles run in pinned virtual environments, always in a separate process from our engines: rayoptics 0.9.9 with opticalglass 2.0.2 (both M. J. Hayford, BSD-3-Clause) in one, optiland 0.6.2 (K. Harrison, MIT) in a second (how the numbers are produced).
Records. Each row names the internal record its number comes from. Those records are not public; if you want to see one, ask.
2.1 Group A: closed forms and input-free invariances
28 rowsSystems whose right answer is known before the engine runs: a formula that can be worked out on paper, or a property that has to hold whatever you feed in, such as a quantity that cannot change or a step that undoes itself.
A paraboloid mirror of focal length 100 mm (curvature 1/200, conic constant −1) reflects 20 rays that arrive parallel to its axis, at heights of 1 to 45 mm and random azimuth: all 20 have to pass through the focus, at z = f.
- Measured
- 7.9 × 10⁻¹⁵ mm (bar 10⁻¹¹)
- Compared against
- conic focal property (A)
An ellipsoid mirror with semi-axes a = 100 and b = 60 reflects a ray that leaves one focus: the ray has to arrive at the other focus, and the distance it travels between the two has to equal the long axis, 2a.
- Measured
- 2.8 × 10⁻¹⁴ mm; miss at F₂ 1.9 × 10⁻¹⁶
- Compared against
- conic focal property (A)
A single spherical surface (radius 50, air into glass of index 1.5168, object at −200) is traced at three apertures two ways, by the engine’s vector formulas and by the classical trigonometric (L, U) trace with its own independent set of formulas: the two have to land in the same place.
- Measured
- 2.3 × 10⁻¹³ mm (bar 10⁻¹⁰)
- Compared against
- the classical trigonometric (L, U) trace, an independent formula set (A)
A ray crosses a flat plate 10 mm thick of index 1.5 at 30°: it has to come out displaced sideways by exactly the closed form t·sin(θᵢ − θₜ)/cos θₜ.
- Measured
- 4.4 × 10⁻¹⁶ mm
- Compared against
- closed form (A)
The Lagrange invariant H, a combination of ray height and angle that paraxial optics does not allow to change, is computed at every surface of the TRIPLET and has to come out the same at each one.
- Measured
- 3.3 × 10⁻¹⁶ relative
- Compared against
- Welford (A)
Ten random skew rays are sent through the TRIPLET, and the skew invariant n(Ly − Mx), which an axially symmetric system cannot change, has to hold its value along each of them.
- Measured
- 3.3 × 10⁻¹⁶
- Compared against
- axisymmetry (A)
Trace a skew ray through the TRIPLET test system, then walk the ray that comes out back through the same surfaces, re-solving each hit and each refraction: Snell’s law is reversible, so the returning ray has to come back to the point it was launched from.
- Measured
- 1.2 × 10⁻¹⁶ mm
- Compared against
- reversibility of Snell (A)
The DOUBLET’s marginal ray is traced at a fraction ε of its full height, with ε running from 10⁻² to 10⁻³: third-order theory says the transverse error has to fall away as the third power of the height, and the power is fitted from the trace to see whether it does.
- Measured
- order 3.000 (Δ 2.2 × 10⁻⁵)
- Compared against
- Seidel theory (A)
A spherical surface has one pair of conjugate points that images perfectly at any aperture: with R = 50 and glass of index 1.5168, rays converging toward the first of them, the virtual object at R(n₁+n₂)/n₁, at heights of 2 to 30, all have to cross the axis at the second, R(n₁+n₂)/n₂ = 82.964135021.
- Measured
- spread 1.4 × 10⁻¹⁴ mm; crossing 0.0
- Compared against
- aplanatic points (A)
On ASPH-SINGLET, whose surface the engine has to find by iteration, two things have to hold: the point it settles on lies on the surface the sag formula describes, and the normal it returns is perpendicular to a tangent worked out separately by finite differences.
- Measured
- 8.9 × 10⁻¹⁶ mm; normal 3.2 × 10⁻¹⁰ (finite-difference step)
- Compared against
- self-consistency (A)
A ray meets the boundary from glass of index 1.5168 into air at 60°, past the angle at which light can still get out: the engine has to raise an error that names total internal reflection rather than return a meaningless number.
- Measured
- raises, typed
- Compared against
- Snell (A)
Snell’s law is recomputed from the geometry the trace actually produced, at every surface and for ten skew rays: the residual |n₁ sin θᵢ − n₂ sin θₜ| has to be zero at each one.
- Measured
- 8.3 × 10⁻¹⁷
- Compared against
- Snell (A)
The DOUBLET’s real-ray transverse aberration is fitted near the axis and its ρ³ coefficient extrapolated to ρ → 0, where it has to equal +S_I/(2n′u′), the same aberration reached through the third-order sums, sign included.
- Measured
- 3.8 × 10⁻⁷ relative (ρ → 0 extrapolation)
- Compared against
- Welford + the pinned sign (A)
Two mirrors, radii −200 and −90, 60 mm apart: the engine’s own first-order focal length has to match the elementary power formula φ₁ + φ₂ − τφ₁φ₂, computed separately, at a separation where the answer on paper is exactly 900 mm.
- Measured
- 899.9999999999998 mm from both routes, agreeing to the last bit (900.000000 at six figures).
- Compared against
- closed form (A)
A paraboloid has no spherical aberration at all, so for the Newtonian preset the engine’s third-order S_I, the spherical part and the aspheric increment together, has to come out zero.
- Measured
- 0.0
- Compared against
- Welford ΔS (A)
The Cassegrain preset is built on the pair of conic foci that image perfectly, so the wavefront error the engine computes across its pupil has to be flat zero.
- Measured
- 9.7 × 10⁻¹¹ waves
- Compared against
- conic conjugates (A)
A system symmetric about its stop and working at magnification m = −1 has no coma and no distortion, by symmetry: the engine’s S_II and S_V sums have to be zero.
- Measured
- ≤ 1.7 × 10⁻¹⁸
- Compared against
- symmetry (A)
For a perfect circular pupil the answer is known in closed form and the engine has to reproduce it: a modulation transfer function (2/π)(arccos ν̄ − ν̄√(1 − ν̄²)), the first dark ring of the Airy pattern at 1.22 λf#, and a Strehl ratio of 1.
- Measured
- MTF rms 4.4 × 10⁻⁴; two-route MTF identity 5.6 × 10⁻¹⁷
- Compared against
- Born & Wolf (A)
A thin prism bends a ray by (n − 1) times its angle, so a powered singlet tilted 2° has to send its ray out at exactly that slope, tan((n − 1)θ).
- Measured
- exact
- Compared against
- thin-prism deviation (A)
A flat mirror at 45° folds the beam without changing the optics, so the folded doublet’s image coordinates and optical path lengths have to match the straight doublet’s.
- Measured
- ≤ 4.3 × 10⁻¹⁴ mm
- Compared against
- unfold identity (A)
Light that bounces twice inside a flat plate makes a ghost image, and elementary optics puts its focus 2t/n upstream of the real one: on a plate 6 mm thick of index 1.5, the engine, an independently built ladder of the same bounces and the closed form all have to agree.
- Measured
- −8.000… mm; engine == independent ladder == closed form
- Compared against
- closed form (A)
On a fresh five-surface system, the flux the engine gives each of the 15 ghost pairs has to equal the product written out by hand for that exact sequence of reflections and transmissions, with every Fresnel reflectance taken from the indices as typed.
- Measured
- worst 3.6 × 10⁻¹⁶ relative
- Compared against
- event enumeration (A)
A pair of surfaces built entirely from numbers a computer stores exactly (R 64/32, t 64, n = 2) is exactly afocal: the quantity the engine tests afocality with has to be zero in floating point, and detuning the pair by 2⁻²⁰ has to give a focal length of exactly ∓2²¹.
- Measured
- C == 0.0; EFL = ∓2 097 152 exactly
- Compared against
- exact arithmetic (A)
The tolerancer tilts a plane plate rigidly by 2°, and the lateral shift it reports has to equal the closed form t·sin θ(1 − cos θ/√(n² − sin²θ)) = 1.189879 × 10⁻¹ mm.
- Measured
- 9.9 × 10⁻¹⁶ mm
- Compared against
- closed form (A)
One element of the Cooke is decentered, and the chief-ray image shift the tolerancer reports has to match the first-order kick sum Δ·Σ B_k c_k(n′−n)/n′, with whatever is left over shrinking as the square of the decenter.
- Measured
- 1.5 × 10⁻⁶ relative; order 2.00; −3.901452 mm per mm
- Compared against
- derived closed form (A)
A tolerance on the glass index alone has to move the focal length by exactly what the thick-lens closed form gives, and a budget with every tolerance set to zero has to return every trial at the nominal design, with no spread at all.
- Measured
- 0.0 (59.285787 mm); spread 0.0 over all trials
- Compared against
- closed form / identity (A)
A merit function of two variables whose minimum can be written down is driven through the whole damped least-squares machinery, and it has to land on that minimum exactly.
- Measured
- 0.0 at (1.25, −0.5)
- Compared against
- closed form (A)
The two mirror presets on the engine in your browser: rays arriving parallel to the axis, at the heights the presets were frozen with, have to cross the axis at one point. Cassegrain (R₁ −200 K −1, gap −70, R₂ −80 K −25/9): heights 4/12/20/25 mm, crossings 120.000 mm behind the secondary; sweep h = 4…25 mm in 1 mm steps. Newtonian: heights 5/20/35/50 mm.
- Measured
- Cassegrain span 5.0 × 10⁻¹³ mm (4 rays), 1.3 × 10⁻¹² mm (22 rays), worst distance from the paraxial focus 7.5 × 10⁻¹³ mm; Newtonian span 3.4 × 10⁻¹³ mm. R₂ = +80: span 0.35 mm, EFL 57.142857142857146 (panel prints 57.1429). The values frozen with the presets when they were built (measured then on the reference implementation) are 5.684 × 10⁻¹³ and 3.411 × 10⁻¹³.
- Compared against
- conic focal property (A)
Test prescriptions SINGLET / DOUBLET / TRIPLET / ASPH-SINGLET are the suite’s own fixed systems (model-glass indices exact at the d line). Rows A1–A13 are re-run against the browser engine on every build, at the tolerances printed in each row; the residuals printed are the reference implementation’s, and the browser engine passes the same checks at the bars shown. Rows A10’s normal check and A18’s MTF are E3 on purpose: a finite-difference tangent and an FFT-sampled MTF cannot reach 10⁻¹⁶, and the printed residuals are the step size and the grid, not the engine. Row A13 is a limit, and its 3.8 × 10⁻⁷ is the extrapolation to ρ → 0. Row A28’s 22-ray span is larger than its 4-ray span because an axis crossing amplifies transverse round-off by 1/tan u; at h = 4 mm the exit slope is 0.010, a factor of 100.
2.2 Group R: independent open-source implementations
9 rowsThe same lenses, and the same glass data, put through open-source software written by other authors, and the sets of numbers compared.
The point where each ray meets each surface is compared, for four test systems at five pupil points each: SINGLET, DOUBLET and TRIPLET on axis and at 5° off axis (chief, zonal and skew rays), and ASPH-SINGLET on the Newton path.
- Measured
- worst 1.8 × 10⁻¹⁵ mm (bar 10⁻⁹)
- Compared against
- rayoptics real-ray trace
The DOUBLET’s focal length and back focal distance are computed from the same prescription on both sides and compared.
- Measured
- 0.0 / 1.4 × 10⁻¹⁴ mm
- Compared against
- rayoptics first-order data
The DOUBLET’s five third-order sums S_I to S_V are compared, both sides started from the same marginal and chief rays.
- Measured
- 6.0 × 10⁻¹⁵, normalization factor exactly 1
- Compared against
- rayoptics third-order sums
The optical path a ray accumulates through the DOUBLET, from the first surface to the last, is compared segment by segment.
- Measured
- 3.6 × 10⁻¹⁵ mm
- Compared against
- rayoptics segment sums
For 20 glasses across SCHOTT, OHARA, CDGM and HOYA, in both the Sellmeier and the power-series dispersion forms, the index our catalog computes at the d, F and C lines is compared with the value the outside library computes from its own copy of the same vendor catalogs.
- Measured
- worst 2.22 × 10⁻¹⁶ (one ulp)
- Compared against
- opticalglass 2.0.2 (its own vendor catalogs)
The engine on our server, with nothing in between: its paraxial matrix elements D, A and C are compared on four afocal systems built for this check alone, a 2.5× Galilean, a Keplerian with an erector at M = +3, a cemented expander and a finite-conjugate collimator.
- Measured
- ≤ 8.9 × 10⁻¹⁶ (39/39)
- Compared against
- rayoptics paraxial trace, separate process
The engine in your browser, with nothing in between: it writes each design as a .zmx file and the outside reader opens that file, so the two are compared surface by surface (curvature, thickness, conic, a model glass’s constant index, the toroid parameter) and then on focal length, over 5 fresh designs and 6 presets.
- Measured
- c/t/K 0.0 everywhere; constant-index n bit-exact; EFL worst 8.5 × 10⁻¹⁴ mm (81/81)
- Compared against
- rayoptics
.zmxreader, separate process
The same trip in the other direction: a .zmx doublet fixture is read by our own parser, surface by surface, and the focal length of the system it produces is compared using the outside reader’s own resolved indices.
- Measured
- 0.0
- Compared against
- rayoptics lens-file reader
Eighteen .seq files, 6 written for this check and 12 carried in from an open-source library under BSD-3, are read by our own parser and by two outside readers working separately, and the systems that come out have to agree field by field: surface count, curvature and radius, every gap including the folded shift, conic and asphere terms, the mirror flag, the medium’s type and name, the stop, the entrance pupil, the field points, the wavelengths, which of them is the reference, and the focal length computed at it.
- Measured
- curvature and radius 0.0 on every fixture; wavelengths exact; EFL ≤ 9.6 × 10⁻¹² mm (the file in inches), ≤ 3 × 10⁻¹⁴ elsewhere
- Compared against
- rayoptics 0.9.9 (377/377) and optiland 0.6.2 (439/439), each in a separate process
We use them as witnesses; we do not build on them, and nothing here implies their authors’ endorsement. Rows R1–R4 are the reference implementation against rayoptics; the browser engine is tied to that reference by X2 (within 7.1 × 10⁻¹⁵ on five fresh systems, 0.0 on all but the asphere-Newton path) and to the server engine by X1 (bit-exact). Rows R6 and R7 are shipped engines against rayoptics with no intermediate: the server engine on fresh afocal systems, and the browser engine on files the reader can produce with the Export button. So the ray coordinates in R1 reach the engine in your browser in two steps rather than one, and both steps are rows of this record (X1 and X2, in §2.4) rather than assumptions.
2.3 Group P: published data
5 rowsNumbers somebody else published and we did not: glass makers’ index data, and lens prescriptions typed in from patent scans.
For every glass in the catalog, the index at the d line is recomputed from the dispersion coefficients we ship and compared with the maker’s own printed n_d in the same record, and the Abbe number V_d the same way.
- Measured
- n_d worst 5.457 × 10⁻⁶ (5.5 × 10⁻⁶ at two figures); V_d worst 8.8 × 10⁻³
- Compared against
- vendor catalog values as mirrored in RefractiveIndex.INFO (CC0), pinned commit; vintages SCHOTT 2017-01-20b, OHARA 2017-11-30, CDGM 2022-06, HOYA 2017-04-01
- Limited by
- Vendors print n_d to 5–6 decimals and V_d to 2; the residual is the vendor’s own fit-vs-print rounding, and the worst n_d case sits just above a five-decimal half-unit.
The three check values SCHOTT prints on its N-BK7 datasheet, n_d = 1.51680, n_e = 1.51872 and ν_d = 64.17, are compared with what the shipped record returns, which is the same record and the same coefficient set the thin-film simulator ships.
- Measured
- 3.5 × 10⁻⁸ / 2.0 × 10⁻⁶ / 2.7 × 10⁻³
- Compared against
- SCHOTT N-BK7 datasheet (glass code 517642.251)
A Tessar typed in from US Patent 721,240 (P. Rudolph, Zeiss, 1902): the f = 1 table read from the scan at native resolution and scaled ×100, each radius taking the direction the patent prints, then traced to see whether the focal length comes back at the patent’s own.
- Measured
- EFL 99.25169308949212 mm at the patent’s D line (nominal 100, miss 0.75 mm); the rejected sign reading gives 18.6
- Compared against
- the patent (Google Patents scan, digit-verified 2026-08-13)
- Limited by
- Radii and thicknesses are printed to three decimals of f = 1; half a unit of each printed digit, propagated through the engine, moves EFL by ±2.0 mm worst-case aligned, ±0.9 mm root-sum-square.
A Cooke triplet typed in from US Patent 568,052 Series III (H. D. Taylor, 1896): the patent states its radii by which way each surface bulges, so each one is converted to a directional radius and scaled ×100, with the stop at the rear vertex of L2 as the parent patent’s text directs, and the focal length compared with the patent’s own.
- Measured
- EFL 98.26805838170866 mm (98.27 at the print’s two decimals; nominal 100, miss 1.73 mm: inside the worst case, 1.2× the root-sum-square); the un-converted reading gives 19.5; diaphragm ratio .131/.108 = 1.213 vs the f/6.5 : f/8 ratio 1.231
- Compared against
- the patent + US 540,122 p. 4 (the convention statement)
- Limited by
- The same propagation: ±3.1 mm worst-case aligned, ±1.5 mm root-sum-square; the two dominant sensitivities are R₁, 1.0 mm per 0.05 mm, and R₄, 0.8 mm.
A .seq file written by someone else, for a catalog achromat 25 mm in diameter and 100 mm in focal length made of N-BK7 and N-SF5, is read by our own parser and analyzed, and the focal length and center thickness that come out are compared with what its supplier publishes for that part.
- Measured
- EFL 100.0308127418172 mm (the two outside readers give 100.03081274181721 and 100.03081274181717); center thickness 8.50 exactly
- Compared against
- the supplier’s published product data, read at source 2026-09-02: EFL 100.00 mm ± 1 %, center thickness 8.50 mm
P3 and P4 are E2 by construction: a patent that prints radii to three figures cannot pin a focal length better than a percent or two, and the honest content of the row is the sign-convention vote, not the last digit. The Cooke patent prints n_D and V(C–F) but not n_F/n_C, so the preset is monochromatic; reconstructing the missing lines would be fabrication and is not done. The Tessar’s chromatic panel evaluates only at its three published lines.
2.4 Group X: our two engines, against each other and against the reference
8 rowsThe engine in your browser, the engine on our server, the implementation both were built from and an independent rewrite of it, checked against each other and against values frozen earlier, so an answer does not change when a calculation moves.
The building blocks of a trace, one at a time. Both engines are driven from the same frozen inputs and their results compared item by item: 19 traced results over 9 systems (a sphere, a Cassegrain form with a negative gap, a paraboloid, an ellipsoid, an even asphere found by iteration, two total-internal-reflection cases and 5 aperture-clip cases, one of them a ray landing exactly on the rim), 10 paraxial results, 3 first-order, 3 pupil, 3 afocal, the 231 points of a six-ring hexapolar grid, 21 fan coordinates and 1 transfer.
- Measured
- 234/234 trace leaves
==; all paraxial/pupil/afocal/transfer leaves==; hexapolar sin/cos ≤ 1 ulp (the browser’s and the server’s math libraries)
The engine in your browser is checked against the implementation it was built from, on five systems that are in no suite: a cemented triplet, a two-mirror Cassegrain form, a steep asphere at the edge of the domain (K = +0.5), a pair bracketing total internal reflection, and a real-catalog doublet at the F line, each engine using its own Sellmeier evaluation.
- Measured
- 0.0 on every row except the asphere-Newton path: hits 1.8 × 10⁻¹⁵, directions 2.2 × 10⁻¹⁵, OPL 7.1 × 10⁻¹⁵
Everything the browser does after the trace (finding the entrance pupil, hexapolar spots, ray fans, Seidel sums) was written a second time from the specification alone, and the two versions are compared on three systems, one of them with an internal stop and one with a mirror.
- Measured
- worst 2.5 × 10⁻¹⁴
The same comparison on kinds of system the suites do not cover: toroids swept both ways, the aspheric Seidel increment of a conic mirror, a user-typed index table evaluated at its last point, a polychromatic point-spread function, and the wavefront of a telecentric system.
- Measured
- ≤ 8.9 × 10⁻¹⁶ everywhere except telecentric OPD 2.4 × 10⁻¹¹ (reference-sphere conditioning, named)
The optimizer on our server is compared with the implementation it was built from on four problems built for this check, matching residuals, derivatives and every step taken along the way: recovering a Petzval form, a doublet whose air-gap bound is active, snapping a model glass onto a vendor other than SCHOTT, and an under-determined problem that asks only for a focal length.
- Measured
- 0.0 exactly
The tolerancer on our server is compared with the implementation it was built from on six budgets built for this check: 19 rows covering all eight kinds of tolerance with a HOYA power-series glass; a uniform random draw, which never touches the error function; a plano-convex lens close enough to total internal reflection that 7 of its 300 wedge trials genuinely fail; budgets a two-mirror system has to refuse; and a run with the compensator switched off.
- Measured
- 0.0 on every row; the two fixture comparisons show no difference anywhere.
- Limited by
- The error function is not one of the operations the arithmetic standard requires to be correct to the last bit, so the rows that go through it are held to a relative 10⁻¹⁰ rather than to exact equality, and the uniform rows are held exact. The 10⁻¹⁰ is the bar, not a measured difference.
The nine preset share links, each written by the tool’s own link writer and read back by its own reader to the same design, are loaded into the shipped engine, where the focal length and back focal distance they produce have to reproduce the values frozen when the presets were built, printed here to nine decimals.
- Measured
- EFL: seven presets within half a unit of the ninth decimal (worst Δ 4.9 × 10⁻¹⁰); Newtonian −500 exactly; Cassegrain 399.9999999999998 (Δ −2.3 × 10⁻¹³ mm; the panel prints 400.000). BFL: the same seven within 4.7 × 10⁻¹⁰, Cassegrain 119.99999999999991 (Δ −8.5 × 10⁻¹⁴), with one bookkeeping difference stated: the folded doublet’s frozen BFL (83.329670352) is the unfolded lens’s, measured from its last glass surface; the shipped engine measures from the fold mirror, 20 mm later, and reports 63.329670352158665 (Δ 1.6 × 10⁻¹⁰ against frozen − 20).
A plain design and the same design holding one configuration have to produce byte-identical requests in the browser and identical responses from the server, with one exception stated in the row: the wall clock.
- Measured
- identical (8/8, 7/7)
2.5 Group G: reproducibility and determinism
6 rowsWhether a number stays put: a file exported twice, a published run replayed on a later build, a seeded simulation re-run, four different seeds landing inside the spread chance allows, and two different routes to the same design ending in the same place.
Exporting one design twice has to give byte-identical .zmx files, with no timestamp anywhere and the header carrying the attribution line; a fresh export has to read back as the same design; and the glass lines of an imported lineage file have to survive a re-export byte for byte.
- Measured
- byte-identical; preserved set exact (274/274)
Every run behind the published lens-design guide, 11 optimizations and 5 tolerance runs, is replayed on a later build of the engine, one that had gained a new kind of operand and multi-configuration support in the meantime.
- Measured
- 16/16 digit-for-digit (e.g. R10 merit 0.030444656741558877; T4 yield 0.970, p90 0.05567831729257731)
The guide’s 2,000-trial yield simulation, started from seed 20260831, is run again on the engine build the guide was published with, and every digit has to come back the same.
- Measured
- yield 0.9795, p90 0.055674375803048404, baseline 0.04897491924768563: digit-identical
The same 2,000-trial budget is run from four different seeds, and the four yields have to sit inside the spread that chance alone allows for a sample this size.
- Measured
- passes 1959 / 1962 / 1962 / 1960 of 2000 = 97.95 / 98.10 / 98.10 / 98.00 %; pooled 7843/8000 = 0.980375; binomial SD at p = 0.98, n = 2000 is 0.0031 (0.31 %); observed spread 0.0015 (0.15 %)
A design optimized with an idealized model glass, then snapped to the nearest real catalog glass and re-optimized, has to end where a design that started from that catalog glass ends.
- Measured
- merit ratio snapped-and-reoptimized / from-scratch 0.9999999999995914 (1 − 4.1 × 10⁻¹³); the two curvature solutions within 3.9 × 10⁻¹¹ mm⁻¹
A frozen Monte Carlo run (a doublet, 13 rows of tolerances, seed 20260815, 300 trials, compensated) has to return the same mean 1.499370104 × 10⁻², median 1.498098526 × 10⁻² and first trial 1.403841665 × 10⁻² on the engine on our server as on the implementation it was built from.
- Measured
- observed 0.0
2.6 Group O: the optimizer
3 rowsOne optimization problem whose answer is known in advance, and two rows that describe what the optimizer does rather than compare it against anything outside.
The same check as A27, listed here because it is the optimizer’s: a two-variable quadratic with a minimum that can be written down, driven through the whole damped least-squares path, has to land on that minimum exactly.
- Measured
- 0.0 at (1.25, −0.5)
Every curvature of the Tessar patent geometry is perturbed by ±1 %, which makes the merit 52.3 times worse, and the optimizer is set to recover it: it gets there in 19 iterations and finishes at 0.038 times the merit of the patent geometry itself. There is nothing outside to compare this against, and the merit function is ours, so the row prints a recovery factor rather than agreement with anything.
- Measured
- ×1386 recovery, 19 iterations
The convergence curves of the guide’s first two runs, 36 points and 28 points, each falling at every step from 0.869815991025691 and from 2.555107549821758, have to replay digit for digit.
- Measured
- digit-identical at replay
O2 is on the page as a description of what the optimizer does, not as a benchmark; it has no independent reference and the row says so. The thin-film page’s optimizer case had an analytically enumerable root set; ray optics has none for a real lens, and we do not invent one.
2.7 Planted defects: checks tested on a broken engine
A check that has never been seen to fail proves nothing. So defects were planted in the engine on purpose, one at a time, and the table below records which named check caught each one. Each row is one planted defect, numbered PD-01 to PD-31 so that rows above can name the one they caught. Where the catching check is a row of this page, its id is given; otherwise the check is described in words. The records are the authority.
| id | Round | Planted defect | Caught by | Recorded in |
|---|---|---|---|---|
| PD-01 | Engine known answers | Small-angle refraction, renormalized to hide the unit-length symptom | A3, A4, A6, A7, A9, A11, A12 | Known-answer suite |
| PD-02 | Engine known answers | Seidel S_I sign flipped | A13 | Known-answer suite |
| PD-03 | Engine known answers | Asphere Newton refinement skipped | A10 | Known-answer suite |
| PD-04 | Analyses | Wavefront (OPD) sign flipped | the two wavefront sign pins: defocus W₂₀ = +(|n′|u′²/2)δz and spherical W₄₀ = +S_I/8 | Analysis suite |
| PD-05 | Analyses | Aspheric Seidel constant 4G instead of 8G | A15 | Analysis suite |
| PD-06 | Analyses | A user-typed index table extrapolated silently past its last point | the tabulated-medium range check (must refuse, not extrapolate) | Analysis suite |
| PD-07 | Tolerancing | One random draw per two-axis row instead of two | G6 (the first frozen trial moves: 1.4195 × 10⁻² ≠ 1.4038 × 10⁻²) | Tolerancing suite |
| PD-08 | Tolerancing | Normal draws not truncated at the tolerance | the truncation check (no draw may exceed the tolerance; 4,000 draws; the broken version reached 1.63×) | Tolerancing suite |
| PD-09 | Tolerancing | Compensator silently skipped | the compensator check (a fully compensated last-gap tolerance must leave a spread ~10⁻¹⁴; the broken version left 2.23) | Tolerancing suite |
| PD-10 | Tolerancing | Decenter sign flipped | the decenter identity (a decentered element equals the nominal trace of the shifted ray plus the shift; the broken version missed by 0.46) | Tolerancing suite |
| PD-11 | Tolerancing | Melt model evaluated on the catalog’s rounded n_d/V_d metadata instead of re-deriving them from the dispersion | the V_d re-derivation check (2.9 × 10⁻⁷ vs a 10⁻⁹ bar) | Tolerancing suite |
| PD-12 | .zmx export | A retired spelling of the constant-index glass token | the outside reader crashes at its glass lookup | .zmx export and import checks |
| PD-13 | .zmx export | A model glass with nonzero V_d read back as a constant index | the constant-index type check | .zmx export and import checks |
| PD-14 | .zmx export | Constant-index glass line written with a short tail | the outside reader fails to parse the line | .zmx export and import checks |
| PD-15 | .zmx export | Writer and reader negating every curvature the same way. Our own round trip still passes to 10⁻¹⁰ on the broken bytes. | R7, the outside reader: EFL −36.69 vs +40.77 | .zmx export and import checks |
| PD-16 | .zmx import | The field note fires on an all-zero x-field list | the two field-note checks | .zmx export and import checks |
| PD-17 | .zmx import | The x-field list wins when both x and y lists are present | the two field-note checks | .zmx export and import checks |
| PD-18 | .zmx import | The high-order asphere refusal keyed on array length instead of a nonzero coefficient | the padding check | .zmx export and import checks |
| PD-19 | .zmx import | The refusal bound off by one | the padding check and the check that the refusal names a₁₈ | .zmx export and import checks |
| PD-20 | .zmx import | The import report’s re-fate of ignored lines skipped | one of the two report checks (the other still passed; recorded as such) | .zmx export and import checks |
| PD-21 | Trace parity | Curvature × (1 + 10⁻¹⁵) | X1 (three of the compared values move). The × (1 + 10⁻¹⁷) twin still passes because c·(1 + 10⁻¹⁷) == c: the floor is the input’s own resolution | Browser-vs-server trace comparison |
| PD-22 | Trace parity | Wrong quadratic root (the far side of the sphere: z 98.99 instead of 1.01) | X1 | Browser-vs-server trace comparison |
| PD-23 | Trace parity | Aperture clip predicate ≥ instead of > | X1’s ρ == semi-diameter boundary row | Browser-vs-server trace comparison |
| PD-24 | Ghosts and fan labels | The struck (R/T)(R/T) ghost-flux form | A22 and two sibling ghost-flux checks | Ghosts and fan labels |
| PD-25 | Ghosts and fan labels | Interior transmission T¹ where T² belongs | A22 and the same two (the per-factor fixture sees the exponent; a single plate never could) | Ghosts and fan labels |
| PD-26 | Ghosts and fan labels | Tangential and sagittal swapped in the fan labels | three fan-label checks | Ghosts and fan labels |
| PD-27 | Ghosts and fan labels | Dominance threshold moved 0.6 → 0.5 | the fan-label constant check and two more | Ghosts and fan labels |
| PD-28 | Multi-configuration | Overlay applied after the variables | the optimizer’s per-configuration check | Multi-configuration |
| PD-29 | Multi-configuration | Empty overlay returns a perturbed copy of the design | X8 and three cross-engine checks | Multi-configuration |
| PD-30 | Multi-configuration | Per-configuration overlay wins over the shared variable | the server’s one-configuration check (a design holding one configuration must get the same response as the plain design, the property X8’s server leg pins) and one more server check | Multi-configuration |
| PD-31 | Multi-configuration | A design holding one configuration treated as a multi-configuration set | X8’s browser leg, exactly | Multi-configuration |
| Total | 8 rounds | 31 |
Two rules the records keep. A planted defect only counts once we have checked that its fixture can show it at all: a plate has no interior surfaces, so breaking interior transmission on a plate proves nothing. And a check has to fail when a value is missing, because a check that crashes has not caught anything.
How to break it
Three deliberately wrong inputs next to the right ones, so the reader can see the checks are live rather than pattern-matched.
- Flip the Cassegrain secondary’s radius sign (−80 → +80) in the preset. The focal length readout goes from 400.000 mm to 57.1429 mm, and the axis crossings of parallel rays go from a 5.0 × 10⁻¹³ mm span to 0.35 mm; the layout shows it. This is the vote that settled R₂’s sign when the preset was built (row A28).Open Classical Cassegrain f/8 (two-mirror preset)
- Type the Cooke patent’s radii as printed (+17.0, +94.5, −56.0, −15.9, +362.3, +77.0) instead of converted (+17.0, −94.5, −56.0, +15.9, +362.3, −77.0). EFL reads 19.5 mm, not 98.3: the 1896 patent lists radii by which way each surface bulges, not by sign, and the tool does not guess which convention you meant (row P4).Open Cooke Triplet f/6.5 (Taylor 1895-96, US 568,052 Series III)
- Export a design with a constant-index medium and read the file’s glass line. In
.zmxit is___BLANK 1 0 <n> 0 0 0 0 0 0 0, the V_d = 0 sentinel. Now edit V_d to 55 and re-import: the medium comes back unbound with a note, because a model glass with a nonzero V_d has a dispersion curve the format’s owner does not publish, and the tool will not invent one (manual §13; the check that caught PD-13).
What this page does not claim
- Sequential only. Light visits the surfaces in the order you list them. The ghost view covers exactly one disobedience (a double bounce); scattered light, multiple ghost orders and true non-sequential transport are a different job, and on this site that job belongs to the Illumination Analyzer. More in the manual
- Numerical precision is not design truth. The trace is exact to round-off for the prescription you typed; nothing checks that the prescription is the lens in your hand. More in the manual
- No comparison against a commercial optical-design program appears here. We have not run one, and nothing on this page should be read as a statement about how any commercial program performs. What is here is agreement between independent implementations of the same published methods; the implementations are named in the record and the methods in the references.
- Scalar diffraction, monochromatic pupils. PSF and MTF come from the scalar pupil function: no polarization, no vector high-NA model, and polychromatic results are incoherent sums. Interference and coating physics live in the Thin-Film Coating Simulator. More in the manual
- Seidel is third order. The sums predict; the real rays decide. The fan labels state their fit shares so a label never claims more than the fit. More in the manual
- Glass data has a vintage. The 2017 SCHOTT, OHARA and HOYA and the 2022 CDGM coefficient sets are reproduced exactly (P1); whether they match a maker’s 2026 datasheet is a different question, and the tool prints the vintage beside every glass so you can ask it. More in the manual
- The tolerance model is a model. Truncated-normal and uniform draws at the tolerances you typed, not your shop’s process distributions. The seed makes every yield number reproducible; it does not make it a guarantee. More in the manual
- The optimizer is local, and its recovery factor (O2) is not a benchmark. Damped least squares descends from your starting point to a nearby minimum; it will not find a different design form. O2 says the optimizer descends; it does not say the result is optimal in anyone else’s merit. More in the manual
The manual’s own statement of these limits, with one more on .zmx and .seq coverage, is §18 of the tool documentation.
How the numbers are produced
- Every number on this page is a string rendered from a fixed list; the numbers measured on the shipped engines are checked against them by an automated test on every build, and the rest are transcribed from the record each row names. The page never recomputes its own claims when it renders: a page that recomputes its claims from the tool it is validating proves nothing.
- Every oracle runs in a separate process from the shipped engine. Our server engine’s package name collides with the reference’s; ignoring that would silently compare the engine to itself.
- Every share link on the page is generated by the shipped serializer and parsed back by the shipped parser before it is printed (X7); a link that opened a different design from its row would be worse than no link.
- Full precision where the claim is full precision. A number whose point is that it agrees to sixteen digits is not shortened to four.
- Fresh-system discipline: the cross-checks against the reference implementation and the outside programs are run on constructions that are in neither engine’s own suite (R6, R7, X2, X4, X5, X6 say so in their rows), and the planted-defects table records which checks were seen to fail on a broken engine before they counted.
- The rows measured on the reference implementation say so, and the two rows that tie it to the shipped engines (X1, X2) are printed beside them rather than assumed.
References. rayoptics 0.9.9, M. J. Hayford, BSD-3-Clause, github.com/mjhoptics/ray-optics · opticalglass 2.0.2, same author, BSD-3-Clause · optiland 0.6.2, K. Harrison, MIT, github.com/optiland/optiland, the second .seq reader (fileio/codev/) · RefractiveIndex.INFO database, M. N. Polyanskiy, Sci. Data 11, 94 (2024), CC0 · SCHOTT N-BK7 datasheet · US Patent 721,240 (Rudolph, 1902); US Patent 568,052 (Taylor, 1896); US Patent 540,122 (Taylor, the convention statement) · W. T. Welford, Aberrations of Optical Systems (1986) · M. Born, E. Wolf, Principles of Optics, 7th ed. (1999) · G. H. Spencer and M. V. R. K. Murty, J. Opt. Soc. Am. 52, 672 (1962).
Open the Ray-Optics Designer at /ray-optics-design. The layout, first-order numbers and every widget on this page run in your browser without an account; spot diagrams, ray fans, aberration sums, wavefront analysis, ghost analysis and .zmx and .seq export need a free account; optimization and tolerancing run on our server and need a Pro plan. The manual’s own summary of these checks is /docs/ray-optics-design.