Shader Creation

Shader file structure

Shaders are a programmable code for each of the OpenGL standard rendering pipeline stages. These are used to modify the geometry and per-fragment ("pixel") rendering of a given object.

In Overgrowth, they are programmed in the GLSL shader language.

They are generally stored in the Data/GLSL folder. Shaders that go together have the same base filename, but have a different extension for each shader stage.

.vert shader

.vert corresponds to the per-vertex shader stage.

This stage is always required

This comes first in the shader pipeline.

Vertex shaders are used most often for calculating interpolated data for the fragment shader to use (e.g. vertex coloring/lighting), or to calculate things in-shader, but once per vertex instead of once per pixel (which often makes it cheaper).

.frag shader

.frag corresponds to the per-fragment shader stage.

This stage is always required

This comes last in the shader pipeline.

Fragment shaders are used for calculating the color on a per-fragment ("pixel") basis.

Note: a "pixel" is a simple way to think about it, but if you're doing AA, or full-screen effects that span multiple pixels per single fragment, or rendering to a texture, than there isn't always a 1:1 mapping of "pixels on the screen" to fragments. This is useful to keep in mind in special cases, but generally for regular objects, a fragment is a "pixel".

other shader stages

.geom, .tess_eval, and .tess_ctrl are other shader stages that the engine supports. However, the engine only supports these stages in special cases. For the most part there's no need to worry about or try to work with these shader types.

Object file specification

You set the shader you want to use for a given object by editing that object's XML file, and specifying the shader you want in the ShaderName tag.

<ShaderName>my_example_shader</ShaderName>

This will tell the engine to use the Data/GLSL/my_example_shader.vert and Data/GLSL/my_example_shader.frag programs to render this object.

Reusing shader programs

If you want to reuse common parts of a shader program (or make optional parts you can turn on or turn off, per-object) then you can pass a set of #define flags to the shader.

For example:

<ShaderName>my_example_shader #DO_SOMETHING</ShaderName>

Will automatically add this at the top of your shader files:

#define DO_SOMETHING

You can then use #if defined(DO_SOMETHING) or #ifdef DO_SOMETHING to block off optional code.

Overgrowth does this with most of its object shaders, putting a lot of common code inside the Data/GLSL/envobject.* shader programs.

Special shader file names

Right now the engine automatically substitutes certain shader strings with a different/expanded string with extra options in place.

This means, even if an object seems to be using one of the shaders inside the Data/GLSL folder, the engine is actually replacing it with envobject, with an extra set of flags.

As of this writing, here is an exhaustive list of these substitutions:

• cubemap -> envobject #TANGENT
• cubemapobj or cubemapobjchar or cubemapobjitem -> envobject
• cubemapalpha -> envobject #TANGENT #ALPHA
• plant or plant_less_movement -> envobject #TANGENT #ALPHA #PLANT
• plant_foliage -> envobject #TANGENT #ALPHA #PLANT #NO_DECALS
• detailmap4 -> envobject #DETAILMAP4 #TANGENT
• detailmap4tangent -> envobject #DETAILMAP4 #TANGENT #BASE_TANGENT
• MagmaFloor -> envobject #MAGMA_FLOOR
• MagmaFlow -> envobject #MAGMA_FLOW

This means that, for example, even though there are Data/GLSL/MagmaFlow.vert and Data/GLSL/MagmaFlow.frag files, they aren't actually used by the game. The shader code for those is inside Data/GLSL/envobject.*

Making a custom shader

To create or modify an existing shader:

1. Copy the Data/GLSL/envobject.* files and paste them on Data/GLSL folder inside your mod.
2. Rename the envobject.* files to another name, so they won't conflict with those from the game (for our example, we will use EnvobjectEx.*).
3. Edit the object .XML file, and change the ShaderName element to look like this:
   <ShaderName>EnvobjectEx</ShaderName>
4. Start editing Data/GLSL/EnvobjectEx.vert and Data/GLSL/EnvobjectEx.frag to set your own custom behavior.

The envobject.* shader programs that you copied the code from have a lot of extra code in them. It might be useful to start by very carefully cutting out code out that you know doesn't apply.

This will be code between #ifdef or "#if define(...) sections, and their corresponding #endif. Be careful, these can be nested, and it may be hard to figure out what is what, unless you're carefully cutting or selecting line-by-line.

Global shader definitions

These are added to all shaders, based on the game's configuration settings.

They are effectively added as #define FIELD_NAME lines at the beginning of the shader.

• Every flag in the level's Custom Shader script param gets added to all shaders
• Every flag in the level's GPU Particle Field script param gets added when GPU particle fields are drawn
• #SIMPLE_SHADOW - Simple shadow option is enabled. Turns off sun-based shadow calculations
• #SIMPLE_WATER - Simple water option is enabled. Turns off reflections on water
• #SIMPLE_FOG - Simple fog option is enabled. Turns off more complex noise-based fog calculations and uses a simplified fog model instead
• #NO_REFLECTION_CAPTURE - No reflection capture option is enabled. Turns off local cubmap reflections. Force-enabled on low-end GPUs
• #NO_VELOCITY_BUF - Velocity buffer is set to zero. Turns off velocity buffer calculations
• #ALPHA_TO_COVERAGE - Anti-aliasing is greater than zero. Turns on alpha-to-coverage style alpha blending calculations
• #DISABLE_FOG - Disable fog debug option is enabled. Turns off fog calculations
• #NO_DETAILMAPS - No detail maps debug option is enabled. Turns off detail map texture lookups, and only uses the base non-detailed color map
• #NO_DECALS - No decals debug option is enabled. Turns off decal object and environmental blood rendering
• #ALBEDO_ONLY - Albedo only debug option is enabled. Draws only base colors of objects, and disables lighting
• #SSAO_TEST - Ssao debug option is enabled. This is a prototype of SSAO, and is not recommended for normal use
• #VOLUME_SHADOWS - Volume shadow debug option is enabled. This is a prototype of god-rays, and is not recommended for normal use
• #CAN_USE_LIGHT_PROBES - Deprecated: Enabled if probe lighting is enabled (disabled by default)
• #CAN_USE_3D_TEX - Deprecated: Enabled if probe lighting and ambient light volumes option is enabled (disabled by default)

Per-pass shader definitions

These are added to all shaders, based on which drawing pass the game is doing at the time.

They are effectively added as #define FIELD_NAME lines at the beginning of the shader.

• #SHADOW_CASCADE - Set during the shadow drawing pass (TODO: Is this accurate?)
• #DEPTH_ONLY - Set when drawing the depth pre-pass (which is done to reduce fragment over-draw), or when doing shadow draw passes (TODO: Is this accurate?)
• #NO_INSTANCE_ID - Set when drawing the depth pre-pass (which is done to reduce fragment over-draw), or when doing shadow draw passes (TODO: Is this accurate?)

envobj inputs, outputs, uniforms

TODO: Just dumping here for now. Will document/arrange them more clearly later

TODO: This might just be for envobject.frag - Might need to look at envobject.vert, etc

in vec3 world_vert;  // Always
in float alpha;  // #GPU_PARTICLE_FIELD
in vec2 base_tex_coord;  // #DETAIL_OBJECT
in vec3 concat_bone1;  // #CHARACTER
in vec3 concat_bone2;  // #CHARACTER
in vec3 frag_normal;  // #ITEM #TANGENT
in vec3 frag_tangent;  // #TERRAIN #DETAILMAP4
in vec4 frag_tex_coords;  // #TERRAIN
in vec2 frag_tex_coords;  // #DETAIL_OBJECT, #ITEM
in vec2 frag_tex_coords;  // Static non-detail object && !#USE_GEOM_SHADER
in vec2 frag_tex_coords_fs;  // Static non-detail object && #USE_GEOM_SHADER
in vec2 fur_tex_coord;  // #CHARACTER
flat in int instance_id;  // #PARTICLE, #GPU_PARTICLE_FIELD
flat in int instance_id;  // Static non-detail && !#NO_INSTANCE_ID
in mat4 model_mat_inv;  // Static non-detail && #DETAILMAP4 #AXIS_UV
in mat3 model_rotation_mat_inv;  // Static non-detail && #DETAILMAP4
in vec2 morphed_tex_coord;  // #CHARACTER
in vec3 orig_vert;  // #CHARACTER
in mat3 tan_to_obj;  // Static non-detail && #TANGENT && !#USE_GEOM_SHADER
in mat3 tan_to_obj_fs;  // Static non-detail && #TANGENT && #USE_GEOM_SHADER
in vec3 tangent_to_world1;  // #PARTICLE && (#NORMAL_MAP_TRANSLUCENT || #WATER || #SPLAT)
in vec3 tangent_to_world2;  // #PARTICLE && (#NORMAL_MAP_TRANSLUCENT || #WATER || #SPLAT)
in vec3 tangent_to_world3;  // #PARTICLE && (#NORMAL_MAP_TRANSLUCENT || #WATER || #SPLAT)
in mat3 tangent_to_world;  // #DETAIL_OBJECT
in vec2 tex_coord;  // #PARTICLE, #GPU_PARTICLE_FIELD, #CHARACTER
in vec3 vel;  // (#ITEM || #CHARACTER) && !#NO_VELOCITY_BUF

out vec4 out_color;  // Always
out vec4 out_vel;  // !#NO_VELOCITY_BUF

uniform InstanceInfo {  // #PARTICLE and #INSTANCED
    vec4 instance_color[100];
    mat4 instance_transform[100];
};

uniform InstanceInfo {  // #DETAIL_OBJECT
    mat4 transforms[200];
    vec4 texcoords2[200];
};

struct Instance {  // Static objects - !(defined(GPU_PARTICLE_FIELD) || defined(PARTICLE) || defined(DETAIL_OBJECT) || defined(ITEM) || defined(TERRAIN) || defined(CHARACTER) || defined(SKY))
    mat4 model_mat;
    mat3 model_rotation_mat;
    vec4 color_tint;
    vec4 detail_scale;
};
uniform InstanceInfo {  // Static objects - !(defined(GPU_PARTICLE_FIELD) || defined(PARTICLE) || defined(DETAIL_OBJECT) || defined(ITEM) || defined(TERRAIN) || defined(CHARACTER) || defined(SKY))
    Instance instances[100];
};

layout (std140) uniform ClusterInfo {  // Used for clustered forward rendering, for lights and decals. Always present
    uvec3 grid_size;
    uint num_decals;
    uint num_lights;
    uint light_cluster_data_offset;
    uint light_data_offset;
    uint cluster_width;
    mat4 inv_proj_mat;
    vec4 viewport;
    float z_near;
    float z_mult;
    float pad3;
    float pad4;
};

uniform LightProbeInfo {  // Used in lightprobe.frag. TODO: For deprecated light probe tech?
    vec4 center[128];
    mat4 view_mat;
    mat4 proj_mat;
    vec3 cam_pos;
    vec4 ambient_cube_color[6 * 128];
};

uniform samplerCube tex0;  // #SKY - Not sure what it's used for
uniform sampler2D tex0;  // !#SKY && !GPU_PARTICLE_FIELD - Referred to as color_tex (except in #PARTICLE, just "tex0" there - Still a color map, though)
uniform sampler2D tex0;  // #NO_REFLECTION_CAPTURE && !#SKY && !GPU_PARTICLE_FIELD && !#DETAIL_OBJECT - Referred to as ref_cap_cubemap (tex19 is not used in these cases)

uniform samplerCube tex1;  // #SKY - Not sure what it's used for
uniform sampler2D tex1;  // Everything but #SKY - Normal map
uniform sampler2D tex1;  // !#PARTICLE && !#GPU_PARTICLE_FIELD && !#SKY - Referred to as normal_tex

uniform samplerCube tex2;  // #SKY - Not sure what it's used for
uniform samplerCube tex2;  // #PARTICLE - Diffuse cubemap
uniform samplerCube tex2;  // Everything but #SKY and #PARTICLE - Referred to as spec_cubemap

uniform samplerCube tex3;  // #GPU_PARTICLE_FIELD - Not sure what it's used for
uniform samplerCube tex3;  // #PARTICLE - Diffuse cubemap

uniform sampler2DShadow tex4;  // Everything - Referred to as shadow_sampler

uniform sampler2D tex5;  // #PARTICLE - Screen depth texture
uniform sampler2D tex5;  // #PLANT - Referred to as translucency_tex
uniform sampler2D tex5;  // #DETAILMAP4 - Referred to as weight_tex

uniform sampler2D tex6;  // #DETAIL_OBJECT - Referred to as base_color_tex
uniform sampler2D tex6;  // #CHARACTER - Referred to as blood_tex
uniform sampler2D tex6;  // #ITEM - Referred to as blood_tex
uniform sampler2DArray tex6;  // #DETAILMAP4 - Referred to as detail_color

uniform sampler2D tex7;  // #DETAIL_OBJECT - Referred to as base_normal_tex
uniform sampler2D tex7;  // #CHARACTER - Referred to as fur_tex
uniform sampler2DArray tex7;  // #DETAILMAP4 - Referred to as detail_normal

uniform sampler2D tex8;  // #CHARACTER - Referred to as tint_map

uniform sampler2D tex9;  // !#NO_DECALS && #DECAL_NORMALS - Referred to as decal_normal_tex

uniform sampler2D tex10; // !#NO_DECALS - Referred to as decal_color_tex

uniform usamplerBuffer tex11;  // #CAN_USE_LIGHT_PROBES - Referred to as ambient_grid_data

uniform usamplerBuffer tex12;  // #CAN_USE_LIGHT_PROBES - Referred to as ambient_color_buffer

uniform usamplerBuffer tex13;  // !#DEPTH_ONLY - Referred to as cluster_buffer

uniform sampler2D tex14;  // #TERRAIN - Referred to as warp_tex

uniform samplerBuffer tex15;  // !#DEPTH_ONLY - Referred to as light_decal_data_buffer

// TODO: What are each of these use for, in which object types? tex16, tex17, tex18

uniform sampler3D tex16;  // Everything but #SKY and #GPU_PARTICLE_FIELD - Not sure what it's used for

uniform sampler3D tex17;  // Everything but #SKY, #PARTICLE, #DETAIL_OBJECT, and #GPU_PARTICLE_FIELD - Not sure what it's used for

uniform sampler2D tex18;  // Everything but #SKY, #PARTICLE, and #GPU_PARTICLE_FIELD - Not sure what it's used for

uniform sampler2DArray tex19;  // Everything but #SKY, #DETAIL_OBJECT, and #GPU_PARTICLE_FIELD - Referred to as ref_cap_cubemap

uniform vec3 avg_color;  // #DETAIL_OBJECT
uniform vec4 avg_color0;  // #DETAILMAP4
uniform vec4 avg_color1;  // #DETAILMAP4
uniform vec4 avg_color2;  // #DETAILMAP4
uniform vec4 avg_color3;  // #DETAILMAP4
uniform vec3 blood_tint;  // #CHARACTER, #ITEM
uniform mat4 bone_mats[200];  // #CHARACTER #GPU_SKINNING
uniform vec3 cam_dir;  // #GPU_PARTICLE_FIELD
uniform vec3 cam_pos;  // Always - The position of the camera
uniform vec4 color_tint;  // #PARTICLE #INSTANCED
uniform vec3 color_tint;  // #DETAIL_OBJECT, #ITEM
uniform vec4 detail_color_indices;  // #DETAILMAP4
uniform vec4 detail_normal_indices;  // #DETAILMAP4
uniform float fog_amount;  // #SKY
uniform vec3 grid_bounds_max;  // #CAN_USE_LIGHT_PROBES
uniform vec3 grid_bounds_min;  // #CAN_USE_LIGHT_PROBES
uniform float haze_mult;  // !#GPU_PARTICLE_FIELD
uniform float height;  // #DETAIL_OBJECT
uniform mat4 light_volume_matrix[10];  // !#SKY && !#DETAIL_OBJECT && !#GPU_PARTICLE_FIELD
uniform mat4 light_volume_matrix_inverse[10];  // !#SKY && !#DETAIL_OBJECT && !#GPU_PARTICLE_FIELD
uniform int light_volume_num;  // !#SKY && !#DETAIL_OBJECT && !#GPU_PARTICLE_FIELD
uniform float max_distance;  // #DETAIL_OBJECT
uniform mat3 model_rotation_mat;  // #ITEM
uniform mat4 mvp;  // #PARTICLE, #GPU_PARTICLE_FIELD, #CHARACTER
uniform mat4 new_vel_mat;  // #ITEM
uniform mat3 normal_matrix;  // #DETAIL_OBJECT
uniform int num_light_probes;  // #CAN_USE_LIGHT_PROBES
uniform int num_tetrahedra;  // #CAN_USE_LIGHT_PROBES
uniform mat4 old_vel_mat;  // #ITEM
uniform float overbright;  // #DETAIL_OBJECT
uniform mat4 prev_projection_view_mat;  // !#PARTICLE && !#GPU_PARTICLE_FIELD && !#SKY && !#DETAIL_OBJECT
uniform vec4 primary_light_color;  // Always
uniform mat4 projection_matrix;  // #GPU_PARTICLE_FIELD
uniform mat4 projection_view_mat;  // Always
uniform mat4 reflection_capture_matrix[10];  // !#SKY && !#DETAIL_OBJECT && !#GPU_PARTICLE_FIELD
uniform mat4 reflection_capture_matrix_inverse[10];  // !#SKY && !#DETAIL_OBJECT && !#GPU_PARTICLE_FIELD
uniform int reflection_capture_num;  // !#SKY && !#DETAIL_OBJECT && !#GPU_PARTICLE_FIELD
uniform mat4 shadow_matrix[4];  // Always
uniform float size;  // #PARTICLE && !#INSTANCED
uniform int subdivisions_x;  // #CAN_USE_LIGHT_PROBES
uniform int subdivisions_y;  // #CAN_USE_LIGHT_PROBES
uniform int subdivisions_z;  // #CAN_USE_LIGHT_PROBES
uniform float time;  // Always - The time in seconds since the game started. Game update time, not wall clock time
uniform float time_scale;  // #GPU_PARTICLE_FIELD
uniform vec3 tint;  // #SKY - Used for tinting the sky
uniform vec3 tint_palette[5];  // #CHARACTER
uniform float tint_weight;  // #DETAIL_OBJECT
uniform mat4 view_matrix;  // #GPU_PARTICLE_FIELD
uniform vec2 viewport_dims;  // #PARTICLE
uniform vec3 ws_light;  // #GPU_PARTICLE_FIELD, #SKY