- Moved OpenGL library wrapper into platform
- Finished reorganizing PLANNING.md
This commit is contained in:
211
planning/3D_GRAPHICS.md
Normal file
211
planning/3D_GRAPHICS.md
Normal file
@@ -0,0 +1,211 @@
|
||||
|
||||
# 3D Graphics (`gfx3d`)
|
||||
|
||||
## Table of Contents
|
||||
|
||||
<!-- TOC -->
|
||||
* [Home](./CONTENTS.md#planning-documentation-for-fennec)
|
||||
* [3D Graphics (`gfx3d`)](#3d-graphics-gfx3d)
|
||||
* [Table of Contents](#table-of-contents)
|
||||
* [Introduction](#introduction)
|
||||
* [Structures](#structures)
|
||||
* [Stages](#stages)
|
||||
<!-- TOC -->
|
||||
|
||||
|
||||
## Introduction
|
||||
|
||||
  This system handles the rendering of 3D meshes, materials, lights, and post-processing
|
||||
effects.
|
||||
**DirectX will never have official support.**
|
||||
If you would like to make a fork, have at it, but know that I will hold a deep disdain for you.
|
||||
|
||||
The graphics pipeline will have a buffer with a list of objects and their rendering data.
|
||||
This will be referred to as the Object Buffer. There will be two, for both the Deferred and Forward Passes.
|
||||
|
||||
The buffers will be optimized by scene prediction.
|
||||
This involves tracking the meshes and textures directly and indirectly used by a scene.
|
||||
A callback function in the graphics system for scene loading can do this.
|
||||
|
||||
|
||||
Materials and Lighting models will be run via a shader metaprogram to make the pipeline independent of this aspect.
|
||||
This allows the GPU to draw every single deferred rendered mesh in a single draw call for each stage of the renderer.
|
||||
|
||||
Specifications for debugging views via early breaks are included in the stages.
|
||||
|
||||
|
||||
Links:
|
||||
- https://en.wikipedia.org/wiki/Octree
|
||||
- https://www.adriancourreges.com/blog/2015/11/02/gta-v-graphics-study/
|
||||
- https://learnopengl.com/PBR/Lighting
|
||||
- https://learnopengl.com/PBR/IBL/Diffuse-irradiance
|
||||
- https://en.wikipedia.org/wiki/Schlick%27s_approximation
|
||||
- https://pixelandpoly.com/ior.html
|
||||
- https://developer.download.nvidia.com/SDK/10/opengl/screenshots/samples/dual_depth_peeling.html
|
||||
|
||||
|
||||
|
||||
|
||||
## Structures
|
||||
|
||||
Object Structure. The mesh is implicit data.
|
||||
|
||||
```c++
|
||||
struct Object
|
||||
{
|
||||
vec3 location, scale; // A matrix would be 64 bytes, this is instead 28 bytes
|
||||
quat rotation;
|
||||
uint32 material, lighting;
|
||||
}
|
||||
```
|
||||
|
||||
Textures for 3D rendering are stored in various buffers with sizes of powers of 2.
|
||||
Ratios of `1:1` and `2:1` are allowed. The `2:1` ratio is specifically for spherical and cylindrical projection.
|
||||
UVs may be transformed to use a `2:1` as if it were `1:2`.
|
||||
Cubemaps and 3D textures may only be `1:1`, I would be concerned if you are using any other ratio.
|
||||
|
||||
- 8-Bit R Texture `4096, 2048, 1024, 512` (8)
|
||||
- 8-Bit RG Texture `4096, 2048, 1024, 512` (8)
|
||||
- 8-Bit RGB Texture `4096, 2048, 1024, 512` (8)
|
||||
- 8-Bit RGBA Texture `4096, 2048, 1024, 512` (8)
|
||||
- 8-Bit R Cubemap `1024, 512, 256, 128` (4)
|
||||
- 8-Bit RGB Cubemap `1024, 512, 256, 128` (4)
|
||||
|
||||
* 16-Bit HDR R Texture `4096, 2048, 1024, 512` (8)
|
||||
* 16-Bit HDR RGB Texture `4096, 2048, 1024, 512` (8)
|
||||
* 16-Bit HDR RGBA Texture `4096, 2048, 1024, 512` (8)
|
||||
* 16-Bit HDR R Cubemap `1024, 512, 256, 128` (4)
|
||||
* 16-Bit HDR RGB Cubemap `1024, 512, 256, 128` (4)
|
||||
|
||||
- 16-Bit Shadow Texture `4096, 2048, 1024, 512` (8)
|
||||
- 16-Bit Shadow Cubemap `2048, 1024, 512, 256` (4)
|
||||
|
||||
* 8-Bit 3D R Texture `256, 128, 64, 32` (4)
|
||||
* 8-Bit 3D RGB Texture `256, 128, 64, 32` (4)
|
||||
* 16-Bit 3D HDR R Texture `256, 128, 64, 32` (4)
|
||||
* 16-Bit 3D HDR RGB Texture `256, 128, 64, 32` (4)
|
||||
|
||||
Documentation should provide guidelines on categories of Art Assets and the resolution of textures to use.
|
||||
|
||||
Textures are identified by a 32-bit integer.
|
||||
- `8 bits` → the texture buffer
|
||||
- `24 bits` → the layer in the buffer
|
||||
|
||||
Artists should be informed on the texture structure of the engine and its limitations.
|
||||
However, these principles should be followed in other game engines as these are
|
||||
guided by what is most efficient for typical GPU hardware.
|
||||
|
||||
|
||||
Materials are, for the most part, user-defined. Documentation should make the user aware of this.
|
||||
Material buffers will be a sequence of the Material Struct instances.
|
||||
They will at the very least contain the id of their shader.
|
||||
|
||||
Types of materials:
|
||||
- Surface (Vertex & Fragment)
|
||||
- Opaque / Masked
|
||||
- Translucent (Forward)
|
||||
- Volume (Path Traced Volumetrics / Forward)
|
||||
- Light (Lighting Model)
|
||||
- Post-Process
|
||||
|
||||
|
||||
|
||||
## Stages
|
||||
|
||||
This is the set of stages for the graphics pipeline that runs every frame:
|
||||
Unless otherwise specified, each stage will be run on the GPU.
|
||||
|
||||
- BVH
|
||||
- Octree `(8 Bpn, 64 bpn) [6-Layers ≈ 2.1MB]`
|
||||
- Leaf Size and Tree Depth should be calculated by the scene, constraints are as follows:
|
||||
- Min Object Size
|
||||
- Max Object Size
|
||||
- Scene Center
|
||||
- Scene Edge
|
||||
- Buffer has implicit locations due to the tree having 8 children.
|
||||
- Insertions and Updates are done on the CPU
|
||||
- Nodes
|
||||
- Start Index `int32`
|
||||
- Object Count `int32`
|
||||
- Objects
|
||||
- Buffer of Object IDs grouped by Octree Node
|
||||
- Leaf Culling
|
||||
- Starting at each Octree Leaf, traverse upwards.
|
||||
- Insert Visible Leaf IDs
|
||||
- Track using atomic buffer
|
||||
- Generate the Command Buffer for Culled Mesh LODs from the Visible Leaf Buffer
|
||||
- Track counts using atomic buffers
|
||||
- To avoid double counting due to the construction of the Octree output, we have some options
|
||||
- Ignore Leaf Instances based on occurrences of the mesh in the surrounding 8 Quadtree Leaves. This would require
|
||||
a bias towards a specific corner of the filter.
|
||||
- Perform a preprocessing step on the CPU to erase duplicate elements and fix the buffer continuity.
|
||||
- Let the duplicates be rendered.
|
||||
- Generate the Culled Object Buffer with the respective object IDs
|
||||
- Adjust Buffer Size using the counts
|
||||
- Insert by reusing the count buffer, clipped to only contain used meshes
|
||||
|
||||
Debug View: Object ID, Mesh ID, LOD
|
||||
|
||||
- Visibility
|
||||
- Buffer `(15 Bpp, 120 bpp) [1920x1080] ≈ 39.4MB`
|
||||
- Depth Buffer → `D24`
|
||||
- Visibility Info → `RGB32I`
|
||||
- R = Object ID
|
||||
- G = Mesh ID
|
||||
- B = Material ID
|
||||
- Regenerate the Command Buffer for Visible Mesh LODs
|
||||
- Regenerate the Culled Object Buffer
|
||||
|
||||
Debug View: Visibility Buffer
|
||||
|
||||
* G-Buffer Pass `(17 Bpp, 136 bpp) [1920x1080] ≈ 35.3MB`
|
||||
* Depth - Stencil → `D24_S8`
|
||||
* S → used to represent the lighting model.
|
||||
* Diffuse → `RGBA8`
|
||||
* A → Ambient Occlusion
|
||||
* Emission → `RGB8`
|
||||
* Normal → `RGB8`
|
||||
* Specular → `RGB8`
|
||||
* R → Roughness
|
||||
* G → Specularity (sometimes called the Metallicness)
|
||||
* B → Index of Refraction (IOR)
|
||||
|
||||
Debug View: Depth, Stencil, Diffuse, Emission, Normal, Specularity
|
||||
|
||||
- Generate Dynamic Shadows
|
||||
- Generate Dynamic Reflections (Optional)
|
||||
- SSAO (Optional)
|
||||
- Deferred Lighting Pass `(10 Bpp, 80 bpp) [1920x1080] ≈ 2 x 16.3MB + 8.3MB ≈ 24.6MB`
|
||||
- Depth Buffer → `D24`
|
||||
- Lighting Buffer → `RGB16` w/ Mipmapping
|
||||
- Stencil Buffer $rarr; `S8`
|
||||
- Apply Lighting Model
|
||||
|
||||
Debug View: Shadows, Reflections, SSAO, Deferred Lighting
|
||||
|
||||
We can combine all of these into one framebuffer:
|
||||
|
||||
- Depth - Stencil → `D24_S8`
|
||||
- Visibility Info → `RGB32I`
|
||||
- Diffuse → `RGBA8`
|
||||
- Emission → `RGB8`
|
||||
- Normal → `RGB8`
|
||||
- Specular → `RGB8`
|
||||
- Lighting Buffer → `RGB16` w/ Mipmapping
|
||||
- One more slot left open for another
|
||||
|
||||
* Forward Pass
|
||||
* BVH, Same as Above
|
||||
* LOD Selection, Same as Above
|
||||
* Translucent Materials
|
||||
* Dual Depth Peeling
|
||||
|
||||
Debug View: Forward Mask
|
||||
|
||||
- Post Processing
|
||||
- Depth of Field (Optional)
|
||||
- When enabled, the Visiblity Buffer, G-Buffer, and Deferred Lighting Pass will be double layered.
|
||||
- At this point the Lighting Buffers will be Flattened
|
||||
- Bloom (Optional) → Mipmap Blurring `(6Bpp, 48bpp) [1920x1080] ≈ 16.3MB`
|
||||
- Tonemapping (Optional)
|
||||
- HDR Correction
|
||||
Reference in New Issue
Block a user