in Shaders, Tutorial, Unity

Journey Sand Shader: Ripples

This is the sixth part of the online series dedicated to Journey Sand Shader.

In this final post, we will recreate the typical sand ripples that appear due to the dune-wind interaction.

This last article is dedicated to sand ripples, which are commonly seen on dunes which have been exposed to wind.

Conceptually, it would have made sense having this tutorial after Journey Sand Shader: Sand Normal. The reason why it has been kept last, is because it is the most complex effect so far. Part of the complexity comes from the way in which normal maps are stored and processed in a Surface shader, which makes a lot of extra steps necessary.

Normal Mapping

In a previous lecture of this online course, we have explored how the dishomogeneous look of the sand was achieved. In Journey Sand Shader: Sand Normal, a very common technique called normal mapping was used to alter how the light interacts with the geometry surface. This is often used in 3D graphics to give the illusion something has a more complex geometry, typically to make curved surfaces appear smoother (below).

To achieve such an effect, each pixel is associated with a normal direction, that indicates its orientation. That, instead of the actual orientation of the mesh, is used in the light calculation.

By reading the normal directions from seemingly random texture, we were able to simulate a grainy look. While not physically accurate, it is still very believable.

Sand Ripples

Sand dunes, however, exhibit another particular feature that cannot be ignored: ripples. Each dune features smaller dunes, which are caused by the interaction with the wind, and held together by the friction between the individual grains of sand.

These ripples are very obvious, and can be seen in most dunes. In the image below, taken in Oman, you can see that the dune in the foreground has a strong wavey pattern.

These ripples change dramatically based not just on the shape of the dune, but also on its composition and the direction and speed of the wind. Most dunes that have a sharp peak, typically present ripples only on one side (below).

The effect presented in this tutorial is designed for more softer dunes, that will have ripples on both sides. This is not always physically accurate, but is realistic enough to be believable and is a first good step towards more refined implementations.

Implementing Ripples…

There are many ways in which these ripples could be added. The cheapest would be to draw then on a texture, but that is not what we will do in this tutorial. The reason is simple: the ripples are not “flat”, but should correctly interact with the light. Simply drawing them would not produce a realistic effect when the camera (or the sun) is moving.

Another way to add those ripples could be editing the geometry of the dune model. But increasing the model complexity is not recommended, as it can take a heavy toll on the overall performance.

As seen in Journey Sand Shader: Sand Normal, we can get around this problem using normal maps. They are effectively drawn onto the surface as a traditional texture would, but they are used in the lighting calculations to simulate a more complex geometry of the one actually present.

The problem has now shifted to a new one: creating these normal maps. Manually drawing them would be exceptionally time-consuming. On top of that, every time a dune is changed, the ripples would have to be re-drawn. This would significantly slow down the asset creation, and so it something that many technical artists try to avoid.

A much more efficient and effective solution is to add these ripples procedurally. This means that, based on the local geometry, the normal direction of a dune is altered to take into account not only the grains of sand, but also the ripples.

Since the ripples will need to simulate a 3D surface, it makes sense for them to be implemented altering the normal direction of each pixel. For this, the easier approach is to rely on a tileable normal map with a wavey pattern. This map will then be combined with the existing normal map previously used for the sand.

Normal Mapping

Up to this point, we have encounter three different normals:

  • Geometry normal: the orientation of each face of the 3D model, which is stored directly in the vertices;
  • Sand normal: calculated in Journey Sand Shader: Sand Normal using a noisy texture;
  • Ripple normal: the new effect that we are discussing in this article.

The example below, taken from Unity’s Surface Shader examples page, show the typical way in which a 3D model’s normal can be overwritten. This requires to change the value of  o.Normal, which is usually done after sampling a texture (usually called a normal map).

That is exactly the technique that we have used to replace the geometry normal with the sand normal in Journey Sand Shader: Sand Normal.

❓ What is UnpackNormal?
Each pixel on a normal map represents a normal direction. That is a unit vector (a vector of length one) which points in the direction where the surface should be oriented. The X, Y and Z components of the normal direction are stored in the R, G and B channels of the texture map.

The components of a unit vectors range from -1 to +1. However, the values stored in a texture ranges from 0 to 1. This means that the values need to be “squished” into a different range, before being copied into a texture, and “stretched” back before being used. These two steps are called normal packing and normal unpacking, and can be done using these very simple equations:

(1)   \begin{equation*} \begin{align} R &= \frac{X}{2} + \frac{1}{2} \\ G &= \frac{Y}{2} + \frac{1}{2} \\ B &= \frac{Z}{2} + \frac{1}{2} \end{align} \end{equation*}

and their counterparts:

(2)   \begin{equation*} \begin{align} X &= 2R - 1 \\ Y &= 2G - 1 \\ Z &= 2B - 1 \end{align} \end{equation*}

Unity comes with a built-in function for (2), called UnpackNormal which is often used to extract normal vectors from normal maps.

You can read more about this on Normal Map Technical Details on the polycount Wiki.

Dune Steepness

The first challenge, however, comes from the fact that the shape of the ripples changes based on the dune inclination. Shallow, flat dunes feature gentle ripples; steep dunes present more pronounced wavey patterns. This means that the steepness of a dune needs to be taken into consideration.

The easiest way to work around this is, to have two different normal maps, for shallow and steep dunes, respectively. The main idea behind this effect is to blend between these two normal maps, based on the steepness of a dune.

Steep Normal Map

Shallow Normal Map

❓ Normal maps and the Blue channel
Many normal maps treat the blue channel slightly differently than the other two.

This is because the information stored in the blue channel is actually redundant. Unit vectors have to have unit length, meaning that once two dimensions (X and Y) have been decided, the value of the third one (Z) is fully determined.

(3)   \begin{equation*} \begin{align} length\left(N\right) &= 1 \\ \sqrt{X^2+Y^2+Z^2} &= 1 \\ \end{align} \end{equation*}

which means:

(4)   \begin{equation*} \begin{align} \sqrt{X^2+Y^2+Z^2} &= 1 \\ X^2+Y^2+Z^2 &= 1^2 \\ Z^2 &= 1 - X^2-Y^2 \\ Z &= \sqrt{1 - X^2-Y^2} \end{align} \end{equation*}

This is reflected directly in the  UnpackNormal function that Unity provides. Based on the Shader API used, it either extract the normal information form three or two channels.

As a default setting, Unity stores normal maps in a formal called DXT5nm, which uses the Alpha and Green channels ( packednormal.wy), instead of the Red and Green ones ( packednormal.xy).

You can read more about other ways to store normals in Normal Map Compression on the polycount Wiki.

 

❓ Why normal maps appear purple-ish?
If you have used normal maps before, you might have seen that many of them are predominantly light purple.

When normal maps encode normal directions in tangent space, each vector represents the new orientation of the surface based on the actual orientation of the geometry. This means that a normal vector that points up, \left[0, 0, 1\right], applies no changes to the way in which light is calculated.

When encoded as a colour, the vector \left[0, 0, 1\right] is converted to \left[0.5, 0.5, 1\right], which is indeed a light purple colour in the RGB colour space.

Normal maps often alter the surface normal only by a small amount; which means why on average that they are expected to be around \left[0, 0, 1\right].

On top of that, some normal maps do not allow the normal vectors to point “inwards”. This means that the B component often ranges not from -1 to +1, but from 0 to +1 instead. As such, all pixels in the final normal maps have at least some blue tint in them.

 

The steepness can be calculated using the dot product, which is commonly used in shader coding to calculate how “aligned” two directions are. In this case, we can take the normal direction of the geometry (below, blue), and compare it with a vector that points towards the sky (below, yellow). The dot product between these two vectors will return values close to 1 when both vectors are aligned (shallow dunes), and 0 when they are 90 degrees apart (steep dunes).

The first problem that we encounter, however, is about the two vectors involved in the operation. The normal vector accessible from the surf function via o.Normal is expressed in tangent space. This means that the frame of reference used to encode the normal direction is relative to the local surface geometry (below). We have briefly touched this subject in Journey Sand Shader: Sand Normal.

The vector that points towards the sky, instead, is expressed in world space. In order for the dot product to work as expected, both the vectors need to be expressed in the same frame of reference. This means that we need to convert one of them, so that they are both expressed in the same space.

Luckily, Unity comes to the rescue with a function called WorldNormalVector, which allows converting a normal vector from tangent space to world space. In order for this function to work, we need to change the Input structure, so that it includes both float3 worldNormal and INTERNAL_DATA, as seen below:

This is explained in Unity’s Writing Surface Shaders, where it stated that:

float3 worldNormal; INTERNAL_DATA – contains world normal vector if surface shader writes to o.Normal.

To get the normal vector based on per-pixel normal map, use WorldNormalVector (IN, o.Normal).

This is often a major source of confusion when writing surface shaders. Basically, the value of o.Normal available in the surf function changes, based on how it is used. If you are only reading it, o.Normal contains the normal vector of the current pixel, in world space. If you are changing its value, o.Normal is in tangent space instead.

If you are writing to o.Normal but still need to access the normal in world space (like we do now), then you can use  WorldNormalVector (IN, o.Normal). This, however, requires the small change to the Input structure seen above.

❓ What is INTERNAL_DATA?
The meaning of INTERNAL_DATA is not well explained in the Unity documentation.

To understand its contribution, it is necessary to understand what the WorldNormalVector function does. It takes a vector, expressed in tangent space, and converts it into world space. This change of frame of reference is, in Linear Algebra, known as a change of basis (more on Wikipedia).

Without venturing too deep into its mathematics, a change of basis can be done simply by multiplying a 3D vector by a specially crafted 3×3 matrix. That matrix is known, unsurprisingly, as the tangent to world matrix, which Unity shortens to TtoW.

What  INTERNAL_DATA data does, is adding the TtoW matrix to the Input structure. This can be seen easily by opening the compiled shader code using the “Show generated code” button in the Inspector:

You will be able to see that INTERNAL_DATA is a shorthand for the following macro which indeed includes the TtoW matrix:

The matrix is included not as a half3x3, but as a three separate  half3 rows.

In the compiled shader code you can also find the definition for  WorldNormalVector, which is a macro that simply performs a multiplication between the input normal vector (expressed in tangent space) with the TtoW matrix:

This could have been done using the matrix multiplication operator mul, but since the TtoW matrix was split into three independent rows, the dot product has been used instead.

In fact, it holds that:

(5)   \begin{equation*} \begin{bmatrix} ToW_{1,1} & ToW_{1,2} & ToW_{1,3} \\ ToW_{2,1} & ToW_{2,2} & ToW_{2,3} \\ ToW_{3,1} & ToW_{3,2} & ToW_{3,3} \end{bmatrix} \cdot \begin{bmatrix} N1\\ N2\\ N3 \end{bmatrix}= \begin{bmatrix} \begin{bmatrix} ToW_{1,1} \\ ToW_{1,2} \\ ToW_{1,3} \end{bmatrix}\cdot\begin{bmatrix} N1\\ N2\\ N3 \end{bmatrix} \\ \begin{bmatrix} ToW_{2,1} \\ ToW_{2,2} \\ ToW_{2,3} \end{bmatrix}\cdot\begin{bmatrix} N1\\ N2\\ N3 \end{bmatrix} \\ \begin{bmatrix} ToW_{3,1} \\ ToW_{3,2} \\ ToW_{3,3} \end{bmatrix}\cdot\begin{bmatrix} N1\\ N2\\ N3 \end{bmatrix} \end{bmatrix} \end{equation*}

You can learn more about the Mathematics of normal mapping on this article from LearnOpenGL.

Implementation

The piece of code below converts the normal from tangent to world space, and calculates the steepness with respect to the up direction.

Now that the steepness of the dune is calculated, we can use it to blend the two normal maps. Both the shallow and steep normal maps (called _ShallowTex and _SteepTex in the snippet below) are sampled. Then, they are blended based on the value of steepness:

As previously discussed in Journey Sand Shader: Sand Normal, correctly combining normal maps is rather tricky, and cannot be done using lerp. While slerp would be the correct function to use in this case, a cheaper version called  normalerp is used instead.

Ripple Blending

If we use the code above, the results might be quite underwhelming. That is because dunes tend to have a very gentle steep, which causes the two normal textures to overmix. To correct that, we can apply a non-linear transformation to the steepness, which would increase the sharpness of the blending:

When blending two textures, pow is often used to modulate their sharpness and contrast. We have how and why this works in an earlier tutorial dedicated to Physically Based Rendering.

Below, you can see two gradients. The one on the top shows colours from black to white, linearly interpolated on the X axis using c = uv.x. Below, the same gradient is presented but using  c = pow(uv.x*1.5)*3.0:

It is easy to see that pow allows creating a sharper transition between black and white. When textures are involved, this reduces their overlapping, creating cleaner edges and producing.

Dune Direction

Everything that has been done so far works perfectly. But there is one final issue that we need to address. The ripples change depending on the steepness, but not on the direction. As discussed before, ripples tend not to be symmetrical, due to the fact that the wind usually blows predominantly in one direction.

To make the ripples even more realistic, we should add other two normal maps (table below). They can be blended based on the relative alignment of the dune with respect to the X axis, or the Z axis.

Steep Shallow
X steep x shallow x
Z steep z shallow z

What we need to implement this final change is to calculate the alignment of a dune with respect to the Z axis. This can be done similarly to the steepness calculation, but instead of using float3 UP_WORLD = float3(0, 1, 0);, we can use float3 Z_WORLD = float3(0, 0, 1); instead.

I will leave this final step to you. And if there is any issue, there is a link to download the complete Unity package at the end of this tutorial.

Conclusion

This final part of the online series about the sand rendering in Journey, we have covered how ripples can be created on the dunes, based on their inclination.

Below, you can see how far we have gone with this series:

The rest of the series is also available on this blog:

I wanted to thank you all for sticking with this rather long series. I hope you had fun learning and reimplementing this shader. And if you end up using it in one of your games or project, please do not hesitate to get in touch with me.

Credits

The videogame Journey is developed by Thatgamecompany and published by Sony Computer Entertainment. It is available for PC (Epic Store) and PS4 (PS Store).

The 3D models of the dunes, backgrounds and lighting settings were made by Jiadi Deng.

The 3D model of the Journey’s player was found on the (now closed) FacePunch forum.

Download Unity Package

Become a Patron!
If you want to recreate this effect, the full Unity package is available for download on Patreon. It includes everything needed, from the shaders to the 3D models.

💖 Support this blog

This websites exists thanks to the contribution of patrons on Patreon. If you think these posts have either helped or inspired you, please consider supporting this blog.

Twitter_logo

📧 Stay updated

You will be notified when a new tutorial is relesed!

📝 Licensing

You are free to use, adapt and build upon this tutorial for your own projects (even commercially) as long as you credit me.

You are not allowed to redistribute the content of this tutorial on other platforms. Especially the parts that are only available on Patreon.

If the knowledge you have gained had a significant impact on your project, a mention in the credit would be very appreciated. ❤️🧔🏻

Write a Comment

Comment

Webmentions

  • Journey Sand Shader: Sand Normal - Alan Zucconi

    […] Part 6. Journey Sand Shader: Sand Ripples […]