cert-machine · the registry · every contour re-decided at this build

Their contours, re-decided to the last point.

An environmental contour is the curve an offshore designer reads a 50-year sea state off: the line in the (wave height, wave period) plane that a structure must survive. In 2019–2021 nine groups submitted their contours for six metocean datasets to one benchmarking exercise, and the organizers scored each one by counting the hourly observations that fall outside it with a floating-point point-in-polygon test. This page reads every submitted contour as the exact polygon its vertices denote, classifies every one of the 1,840,824 hourly observations INSIDE, OUTSIDE or ON it in exact arithmetic, and reads the paper's printed counts against the exact ones.

tl;dr
  • The finding. 176 numbers printed in the benchmark's two tables read against the exact count: 173 are the exact count to the integer, one more agrees once two observations lying exactly ON a contour are counted outside, and two — the one-year sea-state row of contribution 3 — are not the count of the file the repository holds. Those three files were rewritten on 2020-09-25 to remove 53, 41 and 55 vertices with a NaN coordinate (the repository's own commit says so); the 20-year row of the same contribution, whose files were never touched, agrees to the integer on all three datasets. 30 of the 150 submitted contours are not simple polygons — they cross themselves, 21 times in the worst case — and 49 are not closed in the file, so the polygon that was scored includes a closing edge nobody drew. No observation changes side between the even-odd rule the benchmark used and the nonzero-winding rule, so the printed counts do not depend on that choice; the shape of what was counted does. The paper's expected numbers are certified: 197 and 11.5 for a one- and twenty-year IFORM contour on twenty years of hours are the roundings of 196.862 and 11.524, and the total-exceedance expectations are exactly 20 and 1 because twenty years of hourly states is exactly 20 × 8766 observations.
  • The mechanism. Every coordinate is read as the decimal literal it is written — an hourly "0.2845; 4.7252", a vertex "4.283446918632201; 7.469377172830912" — never as the nearest double. Each polygon is closed from its last vertex to its first, as the benchmark's script closed it. A point is classified by the even-odd crossing count of a horizontal ray, every predicate a sign of a 2×2 determinant: computed first in float64 and trusted only when it clears a proved forward-error bound (48u·M², 8.7 × 10⁻¹¹ at these magnitudes), otherwise re-decided in BigInt at a fixed decimal scale. A point on an edge or a vertex is ON, a third answer the float test does not have. Self-crossings are decided the same way, pair of edges by pair of edges. The IFORM expectation needs Φ⁻¹, which is enclosed: erfc by Laplace's continued fraction, whose consecutive convergents bracket the value, β by bisection that tightens only on a certain sign.
  • Check it. node instruments/ecbench/battery.js — 218 checks, 14 red controls that must fire (a float determinant with the wrong sign, a bowtie, a square traced twice, a fold, a NaN, a column swap, a point 10⁻²⁰ off an edge). node tools/run-ecbench-ledger.js re-hashes the 167 pinned files and re-decides all 150 contours in about six seconds; node tools/fetch-ec-benchmark.js fetches the twelve datasets from the pinned commit and verifies them by digest.
contours decided
150
11 methods × 6 datasets × the return periods each group supplied; 1,840,824 hourly observations classified against each
printed numbers reproduced
173 of 176
to the integer; one more within the two boundary points; two are not the count of the file on record
contours that cross themselves
30
in 6 of the 11 methods; up to 26 crossings in one contour
observations exactly on a contour
2
the float test put both outside; the exact answer is ON, and the printed mean is consistent with either
expected numbers certified
4 of 4
Table 1's 197 · 11.5 · 492 · 12 are the roundings of certified enclosures; the total-exceedance 20 and 1 are exact
predicates that needed exact arithmetic
237,173
of 637,402,203: the float filter decided the rest under a proved bound, and every one of these was a sign a double could not certify
§1 · the table

Two tables, 176 numbers, 173 of them the exact count

173 printed numbers are the exact count 1 agrees once two boundary points are counted outside 2 are not the count of the file the repository holds
Every number in the paper's Tables 5 and 6, one cell each: the three per-dataset counts outside each long-return-period contour, the same above the severity threshold (Hs > 1 m; for the wind-wave sets also u₁₀ > 1 m/s), and the two one-year means. Hover for the printed and exact values.

Sea-state datasets A, B, C (NDBC buoys; 175,320, 175,320 and 175,320 hours, ten provided and ten retained years). Each cell: the exact count of observations outside the polygon for the 20-year contour (+n observations exactly on it), then the printed count in parentheses; T is a total-exceedance construction, M a marginal-exceedance one. † marks a contour that is not a simple polygon. "Above threshold" and "retained only" give the three datasets in order: the count outside with Hs > 1 m, and the count on the ten retained years alone — the out-of-sample number the paper did not print. The one-year rows, which the paper prints only as three-dataset means, are the cells of the figure above.

contr. · methodA · 20-yrB · 20-yrC · 20-yrabove thresholdretained only
1 · ISORM · T153 (153)127 (127)62 (62)117 · 119 · 60107 · 36 · 33
2 · DSCM · M437 (437) †236 (236) †187 (187)401 · 228 · 185275 · 84 · 86
3 · IDSCM · T68 (68)28 (28)33 (33)14 · 26 · 3057 · 4 · 14
4 · HDCM · T1 (1) †32 (32)8 (8)1 · 30 · 41 · 2 · 2
5 · DIFORM · M8,503 (8,503) †12,573 (12,573)29,216 (29,216)47 · 24 · 34,774 · 6,189 · 15,771
6 · IFORM · M27 (27)14 (14)9 (9)20 · 12 · 827 · 9 · 3
7 · IFORM · M79 (79)4 (4) †23 (23)21 · 0 · 262 · 3 · 11
8 · IFORM · M189 (189)368 (368)409 (409)93 · 339 · 40893 · 101 · 183
9 DS · DSCM · M22,504 (22,504) †22,933 (22,933) †20,462 (20,462) †4 · 43 · 1812,078 · 11,830 · 11,045
9 DS s. · DSCM, smoothed · M5,096 (5,096) †7,716 (7,716) †5,376 (5,376) †4 · 42 · 172,976 · 3,667 · 3,086
9 IFORM · IFORM · M22,514 (22,514)22,990 (22,990)20,480 (20,480)8 · 100 · 3512,085 · 11,844 · 11,049

Wind-wave datasets D, E, F (coastDat hindcast; 438,288 hours each, 25 provided and 25 retained years). The threshold column counts observations outside with both u₁₀ > 1 m/s and Hs > 1 m; the contours are the 50-year ones.

contr. · methodD · 50-yrE · 50-yrF · 50-yrabove thresholdretained only
1 · ISORM · T154 (154)86 (86)71 (71)107 · 63 · 11104 · 48 · 65
2 · DSCM · M514 (514) †720 (720)355 (355) †304 · 388 · 236276 · 346 · 206
3 · IDSCM · T154 (154)82 (82)72 (72)106 · 62 · 12104 · 45 · 66
4 · HDCM · T3 (3)8 (8)0 (0)3 · 8 · 01 · 6 · 0
5 · DIFORM · M174 (174)112 (112)182 (182)7 · 18 · 1978 · 48 · 132
6 · IFORM · M3 (3)17 (17)5 (5)3 · 17 · 50 · 14 · 3
7 · IFORM · M3 (3)16 (16)7 (7)2 · 16 · 20 · 11 · 4
8 · IFORM · M509 (509)110 (110)167 (167)449 · 108 · 159263 · 59 · 91
9 DS · DSCM · M17,460 (17,460)17,226 (17,226) †11,217 (11,217)146 · 122 · 417,586 · 7,502 · 5,310
9 DS s. · DSCM, smoothed · M5,092 (5,092)6,782 (6,782)5,440 (5,440)146 · 122 · 411,954 · 2,846 · 2,577
9 IFORM · IFORM · M17,415 (17,415)17,184 (17,184)11,202 (11,202)123 · 107 · 297,562 · 7,484 · 5,306

The organizers' script (create_points_outside_table_abc.py, _def.py, held in the corpus) concatenated the provided and retained data, read each contour file in a column order hard-coded per participant, and asked matplotlib's Path.contains_points which points fall inside. This page reads the column order from each file's header instead — the two readings agree on all 150 files — and decides the same question exactly. Where they disagree it is not the arithmetic: contribution 3's one-year files for A, B and C give 147, 106, 59 outside (mean 104.0) against a printed mean of 82.3, and 26, 100, 39 above the threshold (mean 55.0) against 48.3. The repository's history explains it: those three files were rewritten on 2020-09-25 (commit 07f0ecc3) to drop 53, 41 and 55 rows in which one coordinate was NaN — a vertex with no place in the plane — and a float path with NaN vertices answers something, version by version, that no polygon answers. The 20-year files of the same contribution were never rewritten and their counts (68, 28, 33) are the printed ones to the integer. Nothing is refuted here; a row's provenance is what the bytes cannot reproduce.

§2 · the polygons

30 contours that cross themselves, and the edge nobody drew

0 2 4 6 8 10 12 4 6 8 10 12 14 zero-up-crossing period Tz (s) significant wave height Hs (m) contribution 9 DS · dataset A · 20-yr contour, 60 vertices in file order 21 self-crossings, decided exactly
Contribution 9's direct-sampling 20-year contour for dataset A, drawn vertex to vertex in the order the file lists them. The exact decider finds 21 proper crossings between non-adjacent edges (open diamonds; their positions are drawn in float, their number is decided). The loop is three-cornered because the direct-sampling construction returns one boundary point per sampling direction and most directions land on the same three extremes; the crossings cluster at those corners, where successive directions return nearly the same point and the listed order zigzags at the scale of centimetres.

A point-in-polygon test presumes a polygon. 30 of the 150 files describe a closed polyline whose edges cross: 10 of contribution 2's 12 direct-sampling contours (24 vertices each, 44 consecutive duplicates among them), 9 of contribution 9's 18 direct-sampling contours and 6 of its 18 smoothed ones, 2 of the 12 declustered direct-IFORM contours, and — one crossing each — 2 IFORM contours of contribution 7 and the highest-density contour of contribution 4 for dataset A at 20 years. That last one is instructive: its 1,160 listed vertices trace a simple curve, and the one crossing is the implicit closing edge — 0.111 m long, the longest in the polygon — cutting across the curve's own tail. In 11 of the 30 the closing edge is one of the crossing edges. Under the even-odd rule a self-crossing polygon still has an inside; it is just not the region a reader sees when the curve is drawn, because the loops between crossings alternate. The nonzero-winding rule would fill them. Both were computed for every observation: no observation is classified differently by the two rules, on any contour, so the printed counts do not depend on the choice. What depends on it is what the number is a count of.

49 files do not repeat their first vertex, so the polygon that was counted has one edge the author never listed: the benchmark's test closes the path implicitly. For most it is short. For contributions 5 and 4 it is, on some datasets, the longest edge of the polygon; the counts above are for the closed polygon, which is what the benchmark counted.

Two hourly observations lie exactly on a submitted contour: in dataset E the 1989-06-17 20:00 state (Hs 0.1240 m, u₁₀ 2.4972 m/s) on contribution 2's one-year contour, and in dataset F the 1986-06-27 20:00 state (Hs 0.2280 m, u₁₀ 1.9572 m/s) on contribution 2's one-year contour. A vertex of that contour is given to three decimals and the observation to four, and they coincide. The float test counted both outside, which is how the printed mean 1186.3 arises from exact counts of 1,252, 1,477, 828 outside plus one boundary point each on E and F; the exact classification is ON, and the page says so rather than pick a side.

§3 · the expected numbers

Table 1, certified: 197 is 196.86, and 20 is exactly 20

The paper puts an expected number beside every count, E = n·α_t, with α = 1/(T × 365.25 × 24) the hourly exceedance probability of a T-year contour. For total-exceedance constructions (ISORM, inverse directional simulation, highest density) α_t = α and the expectation is a rational number: on the sea-state sets it is 20 for a one-year contour and 1 for twenty years, exactly, because the full datasets hold 175,320 = 20 × 8766 hourly states; on the wind-wave sets, with 438,288 hours, it is 73048/1461 = 49.9986 and 36524/36525. For IFORM the contour is the image of the circle of radius β = Φ⁻¹(1 − α), so α_t = exp(−β²/2), and Φ⁻¹ is not a rational number. It is enclosed here, and both ends of every enclosure agree to the digits printed: β = 3.6856114265 for the one-year contour and 4.3886106004 for twenty years, giving E = 196.862 and 11.524 — the paper's 197 and 11.5. On the wind-wave sets, 492.141 and 11.994 — its 492 and 12.

The out-of-sample expectation follows: a 20-year total-exceedance contour fitted on the ten provided years should be exceeded n_retained × α times in the retained years — 0.5276, 0.5213 and 0.5337 on A, B, C (the retained parts hold a little over ten years of hours), and 18262/36525 = 0.4999 on each of D, E, F; an IFORM contour about 6.0 and 5.9. The retained-years column of §1 is that test. The highest-density contours of contribution 4 are exceeded 1, 2, 2, 1, 6, 0 times on A–F; the ISORM contours of contribution 1, built for the same exceedance, 107, 36, 33, 104, 48, 65 times. The paper's own caution applies with more force here than to its full-data counts: hourly sea states are serially correlated, so a storm that crosses a contour is counted at every hour it stays outside, and these are counts, not tests.

§4 · the maxima

How high each contour reaches, against how high the sea got

4 8 12 16 20 A · 20-yr B · 20-yr C · 20-yr D · 50-yr E · 50-yr F · 50-yr dataset and the long return period the exercise asked for significant wave height Hs (m) max Hs along a total-exceedance contour max Hs along a marginal-exceedance contour the highest Hs observed in the full dataset
The highest Hs along each long-return-period contour (a linear function of position on a polygon peaks at a vertex, so the maximum is a vertex's literal, exactly), against the highest Hs observed in the full dataset. Filled dots are total-exceedance contours, lighter dots marginal-exceedance ones; the diamond is the observation.

In dataset A the highest observed sea state, 11.7976 m on 2010-02-26 05:00 — in the retained years, 4.70 m above anything in the provided ten (7.0994 m) — sits above the maximum of every one of the eleven 20-year contours, the highest of which reaches 11.70 m. The paper says as much from a figure; here each maximum is a literal read from the file. On the other five datasets between 1 and 9 of the eleven contours reach above the observed maximum.

what this page does NOT claim

No contour is called right or wrong: which construction an offshore designer should use is a modelling question this page does not touch, and a count of exceedances is the benchmark's own metric, taken here on its own terms. The two numbers that do not agree are not an error in the paper's arithmetic — the file they were computed from is not the file on record. A self-crossing polygon is a fact about a file, not necessarily about the method that produced the boundary it lists. The expected-number enclosures certify the paper's formula, not the independence it assumes. The counts compared are the PREPRINT's (2021-01-19, the version the repository holds); the published article (Ocean Engineering 236, 109504) was not read. The datasets are NDBC's and WDCC's and are not re-served here; the contours are held verbatim as the published record, for verification only.