Impossible Geometry: Non-Euclidean Cubes

This tutorial will teach you how to create non euclidean cubes in Unity, giving the illusion that each face is a door onto another dimension. This post is part of a series of tutorials on impossible geometries.

antichamber

This effect can be seen in many game, most notoriously Antichamber which uses it extensively.

You can download the Unity package here.

Step 1. Stencil Theory

For this example, all the geometries will be contained within the cube, at the same time. This particular effect is achieved using stencil buffer. The GPU offers a special buffer which can be used to mask and discard pixels. The idea behind non-euclidean cubes is strikingly simple and it requires two steps:

  1. Stencil mask: each face of the cube writes a specific value to the stencil buffer (yellow and green, in this example).
    s1
  2. Stencil geometry: the geometry that has to be shown in the yellow face only draws pixels where the stencil buffer is yellow.

All the objects exists at the same time, but their pixels are only drawn in the respective faces.

Step 2. The stencil mask

The first step is to create a material that initialises the stencil buffer with a specific value. To do this, we use a simple vertex and fragment shader (learn how to use them: Vertex and Fragment Shader). Even if in front of the geometry, is essential that this mask is drawn before the content of the cube. To do this, we change the Queue value to Geometry-100.

Shader "Stencils/StencilMask" {
	Properties {
		_StencilMask("Stencil Mask", Int) = 0
	}
	SubShader {
		Tags {
			"RenderType" = "Opaque"
			"Queue" = "Geometry-100"
		}
		ColorMask 0
		ZWrite off
		Stencil {
			Ref[_StencilMask]
			Comp always
			Pass replace
		}

		Pass {
			CGPROGRAM
			#pragma vertex vert
			#pragma fragment frag

			struct appdata {
				float4 vertex : POSITION;
			};

			struct v2f {
				float4 pos : SV_POSITION;
			};

			v2f vert(appdata v) {
				v2f o;
				o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
				return o;
			}

			half4 frag(v2f i) : COLOR {
				return half4(1,1,0,1);
			}
			ENDCG
		}
	}
}

Lines 10-16 write into the stencil buffer. The line Ref[_StencilMask] indicates that the value we want to write is provided in the property _StencilMask. Comp always indicates that the value has to be written regardless of what was on the stencil buffer already.

Step 3. The stencil geometry

What we need now is a material which draws only if the stencil buffer has been initialised accordingly to a specific value. This is done using the Comp equal condition.

Shader "Stencils/StencilGeometry" {
	Properties {
		_StencilMask("Stencil Mask", Int) = 0

		_Color ("Color", Color) = (1,1,1,1)
		_MainTex ("Albedo (RGB)", 2D) = "white" {}
		_Glossiness ("Smoothness", Range(0,1)) = 0.5
		_Metallic ("Metallic", Range(0,1)) = 0.0
	}
	SubShader {
		Tags { "RenderType"="Opaque" }
		LOD 200
		Stencil {
			Ref[_StencilMask]
			Comp equal
			Pass keep
			Fail keep
		}

		CGPROGRAM
		// Physically based Standard lighting model, and enable shadows on all light types
		#pragma surface surf Standard fullforwardshadows

		// Use shader model 3.0 target, to get nicer looking lighting
		#pragma target 3.0

		sampler2D _MainTex;

		struct Input {
			float2 uv_MainTex;
		};

		half _Glossiness;
		half _Metallic;
		fixed4 _Color;

		void surf (Input IN, inout SurfaceOutputStandard o) {
			// Albedo comes from a texture tinted by color
			fixed4 c = tex2D (_MainTex, IN.uv_MainTex) * _Color;
			o.Albedo = c.rgb;
			// Metallic and smoothness come from slider variables
			o.Metallic = _Metallic;
			o.Smoothness = _Glossiness;
			o.Alpha = c.a;
		}
		ENDCG
	} 
	FallBack "Diffuse"
}

The Stencil block in lines 13-18 can be easily added to many existing shaders, allowing to use virtually any material you want.

Step 4. Putting all together

To recap:

  • A standard cube structure;
  • Each face of the cube is a quad has a different material, which uses the StencilMask shader with a unique value for _StencilMask;
  • All the geometry pieces are inside the cube;
  • Each geometry piece has it’s own material, derived from StencilGeometry and initialised to match the _StencilMask property of its respective face.

You can download the full Unity package here.

Conclusion

Become a Patron!

Non-euclidean cubes are always intriguing and can add an interesting twist to your game.

Despite the name, these cubes are actually euclidean. This technique works nicely for decorative elements, but it fails to provide an efficient solution to the portal effect seen in (guess!) Portal or Parallax. Future tutorials will cover other impossible geometries.

  • Part 1: Non-Euclidean Cubes
  • Part 2: The Portal Effect
  • Part 3: The Physics of Monument Valley

This tutorial is inspired by Noisecrime’s post called “Stencils for portal rendering“.

Comments

19 responses to “Impossible Geometry: Non-Euclidean Cubes”

  1. Hi! Thank you for great tutorials!
    Is there a way to read/write stencil from texture or any other info from mesh (vertex color, world position, normals, etc)? Basically, I want to work with it inside Pass.
    I don’t like the idea of creating unique material for each face.

  2. […] non-euclidean cube. There no real name to it as far as I know, so I decided to took the one from this article because it sounds complicated. I love the magic-trick-feel it has and decided to investigate to […]

  3. Kawika avatar

    So, for example, how would I use what you have shown me to create something like this https://www.youtube.com/watch?v=U3sTSWJF_qA ?
    As in, going through a “gate” can take you to another area. You can find this on 0:45 and 0:50 on the youtube video.

  4. I’m using this trick to great effect already but I started wondering what I would do if I wanted to also render bits poking out of the square from angles other than directly in front. I thought it would be nice to somehow support this, but the more I look at it, the harder it seems.

    It’s like all I’d want to do is to also paint the stencil over bits of the model geometry which are a positive distance from a plane defined by that side of the cube (vertices behind would have negative distance), and there is a shader specifically solving that problem, but which requires knowing the plane in advance, which in my case I don’t because it’s always moving. So it’s like I’d have to have a place to stash the plane location between the runs of two different shaders. 🙁

    At the moment I’m compromising and have a manual switch where I have an additional cube of stencil mask textures in a cubic region in front, and try to turn it on manually, in a very careful way ensuring that nobody will notice how I’m cheating. (Basically, their eyes have to be within the cuboid formed by extending the normals of that face infinitely outwards. If they are not, they will see stencil over the edges of the frame, and bits of the object they shouldn’t be able to see.)

    1. Hey!
      I don’t know if I’m understanding correctly. But what you can do is probably possible with a very simple NORMAL EXTRUSION shader.

      Basically, you can have a shader with two passes. The first one extrudes the geometry based on its normal, and does the stencil pass. If you want to learn more about normal extrusion, I’ve been talking about it here: https://www.alanzucconi.com/2015/06/17/surface-shaders-in-unity3d/ .

      Alternatively, you can attach some quads 90 degress inside the faces. Perhaps rotating them with a script based on LookAt?

  5. I was trying to run it on mobile(Android) but it doesn’t work. I’ve searched for “Disable the Stencil and Depth” but it was unchecked and used OpenGLES3 as an API and deleted #pragma target 3.0. Do you know were I can search further?
    I’ve tested on Samsung Galsy S6 and LG G6.

    1. Hi!
      I haven’t experienced that problems myself.
      I’ll try to look into it!

  6. […] Unity Impossible Geometry: Non-Euclidean Cubes (from Impossible Geometry) […]

  7. This is great! Thank you, very helpful! My only question is how would you add a stencil to a projector using this setup?

    1. I found my own solution by pasting this in place of the stencil:
      Stencil {
      Ref[_StencilMask]
      Comp Equal
      Pass replace
      }

      1. Glad you have solved it! 😀

  8. Were the follow-up tutorials ever created? I can’t find them.

    1. Hey! Unfortunately no, I haven’t had the time to write them yet! T_T

      1. How about now in 2018 😛

        1. I know right! :p

          1. It’s 2019 now..

          2. Hello. I need help with how to make a portal effect with cameras and stencil without using RenderTexture. It is very costly when you create recursive portal rendering with RenderTexture. Valve is said to do this with stencil. Can you train us on this? Thanks.

  9. […] Impossible Geometry: Non-Euclidean Cubes […]

Leave a Reply

Your email address will not be published. Required fields are marked *