Tessellation
Subdivide geometry on the GPU
Definition
Tessellation is the GPU stage that subdivides incoming triangles into denser geometry, usually driven by a displacement map and a camera-distance falloff. It lets a low-poly silhouette gain real depth at close range without paying for the polys far away. Tessellation has largely been displaced in production by Unreal’s Nanite (virtualized geometry) and by high-poly meshes streamed via LOD, but it still appears in terrain and water shaders.
In production
DirectX 11 introduced hardware tessellation via hull and domain shaders, and Unreal Engine 4 exposed it directly in the material editor with a displacement map input and a "Crack Free Displacement" toggle to prevent seams from opening up between tessellated triangle edges — a common bug when adjacent UV islands don't share tessellation factors. Unreal Engine 5 deprecated tessellation entirely in favor of Nanite, which virtualizes geometry at the source-asset level instead of subdividing at runtime, so a ZBrush sculpt with millions of polygons can be imported near-verbatim rather than baked down to a displacement map. Tessellation still shows up in Unity's HDRP for terrain and in ocean/water shaders across engines, where a flat plane needs dense geometry only near the camera to catch wave displacement. The classic tessellation artifact is "popping" at the LOD transition distance where the tessellation factor steps down, which artists hide by tying the falloff to fog or depth-of-field.
