Skip to main content
AirMilesCalc
Menu
Methodology

Great-circle distance, intuitively

Why the shortest path between two points on Earth bends north on a Mercator map, why JFK → HKG flies over the Arctic, and how to derive great-circle distance from the spherical law of cosines.

Updated 2026-06-016 min read
Primary sources · 3
  1. [1] Spherical trigonometry — Bowditch (American Practical Navigator)The reference text for great-circle navigation, published by the U.S. National Geospatial-Intelligence Agency · NGA Pub. 9, 2019 edition · 2019 https://msi.nga.mil/Publications/APN
  2. [2] NGA.STND.0036_1.0.0_WGS84Defines the spherical reference radius derived from the WGS-84 ellipsoid · NGA Standard · 2014 https://earth-info.nga.mil/index.php?dir=wgs84&action=wgs84
  3. [3] ICAO Circular 328 — Modernizing the Global Air Navigation SystemDescribes Performance-Based Navigation and great-circle routing as the operational standard · ICAO · 2012 https://www.icao.int/publications

A great circle is the shortest path between two points on a sphere — and the closest spherical analogue to the geodesic that aircraft actually fly on the WGS-84 ellipsoid. The difference between the two is small enough that "great-circle distance" is the everyday name for what AirMilesCalc computes, even though the underlying math uses the ellipsoid.

6,371 km
Mean spherical radius used in great-circle calculation
Derived from WGS-84
≤ 0.5 %
Spherical-vs-ellipsoidal distance error in worst cases
Empirical, NGA
12,983 km
JFK → HKG great-circle distance (over the Arctic)
Computed with WGS-84 Vincenty
17 %
Rhumb-line excess on the same JFK → HKG pair
Computed

What makes a circle "great"

A great circle is the intersection of a sphere with a plane that passes through the sphere's centre. The equator is a great circle; every meridian is a half great circle joined to its antimeridian. Any line drawn on a sphere that does not pass through the centre's plane is a small circle — a parallel of latitude (other than the equator) is the canonical example.

Great-circle (curved on Mercator) vs rhumb line (straight on Mercator) — JFK to Hong Kong
JFKHKGGreat-circle (≈ 12,983 km, what aircraft fly)Rhumb line (≈ 15,200 km, constant compass bearing)
Source: Computed from WGS-84 Vincenty inverse; rhumb-line course from constant-bearing integration

Why the great-circle path looks curved on a map

Mercator projections preserve angles — they were invented in 1569 for sailors who needed to read a constant compass course as a straight line. The price for that property is dramatic area distortion near the poles, and great-circle routes, which want to head north then turn back south on trans-hemispheric flights, look like dramatic arcs against the projection. On a globe the great-circle route is the straight line.

Great-circle distance vs rhumb-line course for representative long-haul pairs
RouteGreat-circle (km)Rhumb (km)Detour
JFK → LHR5,5555,597+0.8 %
LAX → SYD12,05112,238+1.6 %
JFK → HKG12,98315,231+17.3 %
LHR → AKL18,33023,000+25.5 %
LHR → NRT (Tokyo)9,56411,180+16.9 %
Source: Both computed against WGS-84; rhumb via Mercator inverse

How great-circle distance is computed

The spherical law of cosines gives a one-line answer: cos σ = sin φ₁ sin φ₂ + cos φ₁ cos φ₂ cos Δλ, after which the surface distance is d = R · σ for some chosen Earth radius R. For small distances the law of cosines suffers floating-point cancellation; the Haversine formula is a numerically stable rearrangement that is preferred in production code. AirMilesCalc uses Haversine only as a fallback — Vincenty's ellipsoidal solution is preferred for everything else.

When great-circle is wrong

The great-circle and Vincenty distances disagree by up to 0.5 % because Earth is not a sphere. On a long polar route — like Helsinki → Tokyo — that error reaches roughly 40 km. For a flight plan, 40 km is enough fuel to matter; for casual reference, it is not. AirMilesCalc always reports the ellipsoidal number, and the "great-circle distance" label is a shorthand for the geodesic on WGS-84.

Why pilots care about the difference: sphere-vs-ellipsoid error on selected long-haul routes
JFK → LHR (5,555 km)9 kmLAX → SYD (12,051 km)23 kmLHR → NRT (9,564 km)35 kmHEL → KIX (9,094 km, polar)41 km
Source: AirMilesCalc, Vincenty (WGS-84) minus Haversine (R = 6,371 km)

Frequently asked

Are great circles always the shortest path?
On a perfect sphere, yes. On the WGS-84 ellipsoid that real Earth flights use, the geodesic deviates from the great circle by a small amount that becomes noticeable only on long high-latitude routes. AirMilesCalc reports the geodesic; the difference matters for fuel planning but not for casual reference.
Why do JFK → Hong Kong flights pass over the Arctic?
Because the great-circle (and geodesic) path from a mid-latitude origin to a mid-latitude destination at almost-antipodal longitude crosses high northern latitudes. The Mercator projection makes the route look like an enormous loop; on a globe it is the obvious straight line.
What's the difference between great-circle and 'as the crow flies'?
On Earth, they are the same — both refer to the shortest surface path between two points. The phrase 'as the crow flies' is a colloquial synonym; great-circle is the technical name from spherical navigation.
Why is the Haversine formula 'numerically stable' but the law of cosines isn't?
For two nearby points, sin φ₁ sin φ₂ + cos φ₁ cos φ₂ cos Δλ is very close to 1, and acos of a near-1 number loses precision rapidly. Haversine rearranges the algebra to operate on sin²(half-distance) terms, which stay well away from problematic regions for all real inputs.

Continue reading