Renderer/docs/src/progress/renderertodo.md

@page renderertodo TODO

@subpage bigthings Big Things To Build

DONE

(6/10/2023)

Scene

• Generic entity & btree grouping mechanism

Server Data Cell

• Wrapper around scene used by server to manage server scenes and players associated with them Realm
• Manages server data cells including rudimentary spin up/down Realm Manager
• Manages all realms + provides constructors for types of realms Data Cell Manager
• Interface defining method for laying out cells and querying for cells based on real or world coordinates Gridded Data Cell Manager
• Implementation of Data Cell Manager that manages a grid of data cells if using terrain engine Data Cell Location Resolver
• Takes a vector3d and an entity and figures out what data cell that should correspond to
• Breaks the above resolver relationship out of cells so that can support different "worlds" Entity Data Cell Mapper
• Takes an entity and gives the data cell that entity is inside of
• Breaks this out from data cell manager to support multiple worlds Data Cell Physics Manager
• Breaks the physics generation for chunks out of the data cell manager Data Cell Search Utils
• Combine the above classes to provided utilities for searching cells nearby a location for groups of entities

Client Scene Wrapper

• Wrapper around scene used by client
• Specifically adds translating server/client entity IDs to one another

Loading Threads

• Separate client and server versions of loading threads to different files

Server Bone Loader

• Server bone loading for server only collision simulation

Instancing support

Server Scene Management

• Creating and simulating entities per chunk

Server Terrain Management (specifically for collision)

• Generation collision meshes
• Also generate texture array

Terrain Chunk Shader

• Procedural textures for eg rock and dirt

Client Terrain Entity Management (specifically creation/teardown for client)

• Do the basic query stuff for getting terrain

Diagnose weird walking physics

• Character terrain collision messiness

Upgrade terrain editing user experience

• Sphere pallete
• Gradual updates
• Terrain Deletion

Upgrade Terrain Chunk

• Fully connect world chunks

Terrain Editing

• Server interface for editing terrain
• Ability to request terrain change from client

Voxel Data Storage

• Store metadata about types of voxels (rgb offsets, foliage placements, etc)

Foliage Manager upgrades

• Place foliage based on voxel type
• Use timeout queueing system to evaluate voxel locations to place foliage
• Destroy foliage on voxel edit
• Regrow foliage on empty cells after timeout
• Gradually regrow

Chunk Saving System

• File that maps world position to chunk file
• Save chunks to compressed files
• Server load chunks as needed
• Unload chunks after 100 frames of not being used
• Save chunks on unload

Convert client and server terrain entity composition to use 8 chunks instead of 1

Server Diagnose Physics potentially broken

• Server physics calculation not happening <-- hard to test bc gravity tree might not actually be updating client to turn off or something

Foliage Manager upgrades

• Fix popping

(2/25/2024) Generate Tree Entities

• Generate leaf blobs container entity
• Attach individual leaf blob entities to container entity
• Sways in the wind
• Synchronize creation on server and client
• Synchronize swinging on server and client
• Tree types configurable via files Scene Streaming (streaming state of a scene to client)
• Solidify the work that has already been done in this space and PARTICULARLY add good high level documentation Server Content Engine
• On first loading a new-to-server chunk, generate all content (foliage, terrain objects (boulders), structures, etc) entities for the chunk

(2/28/2024) Improve Fonts

• Support multiple fonts in engine
• Leverage https://github.com/SilverTiger/lwjgl3-tutorial/wiki/Fonts to load fonts with java

(2/29/2024) Properly prioritize close trees over far trees

Implement proper Frustum Culling

• Regular Actors
• Instanced Actors

(03/06/2024) Bake in imgui

(03/07/2024) Server frametime bar graph

(03/09/2024) Ability to attach ambient audio emitters to entities

Timekeeping class that defaults to gltf time and falls back to systemCurrentTimeMillis

Methods for sleeping physics bodies if nothing nearby them is dynamic (ie trees if there are no moving creatures near them)

• SAP2 space from ode4j specifically (ended up using BVH space instead)

Overhaul mesh class

• remove unused stuff
• private constructor (this is going to require changing a lot of dependencies)

(03/10/2024) De-dupe render calls via doing mutations in render pipeline status and dont call setting variables to values they are already set to Render pipeline updates to support multiple pipelines defined in different files

• Grouping meshes together independent of actor so don't have to rebind shader programs or VAOs to redraw

(03/12/2024) Foliage Manager upgrades

• Place foliage intelligently

Fix arena mode (terrain be crazy)

(03/17/2024) Optimize instance logic (currently sorting the list of objects to push to buffer each frame nukes the cpu) sort nukes cpu because capacity logic is after sort, so it tries to sort ~600k items every frame before caping The way to optimize this is to completely gut existing code. One draw call per tile. Draw call calles drawInstanced for all blades within that cell. Provide a texture that contains data per blade of grass to the cell on draw. Store positions in that once at creation. For dynamic wind later will need to use UBOs or something like that. Fix grass flickering (it's frustum culling being inconsistent, try commenting it out in InstancedActor and see what happens :| ) (well we won't have that problem anymore lol)

(03/20/2024) Free camera system that can detatch from player entity

(03/20/2024) Half pass at cellular automata fluid dynamics system

• Diffuse density
• Streaming chunks over network
• Basic model creation

(03/21/2024) Fix character movement (allegedly fixed -- maybe by camera stuff?)

• Walking left or right while turning camera is very jittery
• Can lock on moving

Fix Frustum Culling for skybox

Fix Character creation preview not working

(03/23/2024) Physics-controlled objects system

Clean up main method/class

• Include Remotery library

(03/24/2024) Bring LWJGL version up to latest

Automatic Scene unloading

• Tree structure inside scenes for tearing down groups of entities
• Entity decomposition
  • Server handling
  • Client handling when scene should be unloaded

Level loading/saving + Basic Editor

• Spin up voxel level (think arena mode)
• Save voxel level

(04/10/2024) Level loading/saving + Basic Editor

• Basic editor functionality
  • Menu of types of entities to spawn
  • Button to spawn them at cursor

UI Work

• Make ui feel more responsive, whatever that means (answer is make hover-over styling work lol)
• Frames for windows (WIP)
• Need to figure out style-wise what we want to do here (generally minimalist)
• Fix scrollable handling

Terrain editing UI

• Menu to select palette to generate, populated based on data

(04/13/2024) UI Work

• Level editor ability to destroy an entity on server, have it also destroy on client, AND not persist on save
• Environment controls (not persisting in save yet)

Fix Movement Bug where player keeps running after releasing control

(04/21/2024) UI Fixes

• Refactor ui toolkit elements to separate absolute and relative position
• Fix inventory menus

Touch Up working with items

• Make sure the models are appropriately scaled
• Remove all logic from before client-server separation

More Debug menus

• Screen that shows the overall status of client scene
• Number of entities
• Maybe a listing of each one?

Another pass at grass

• Fix shader being camera position independent (if you move the wind moves with you lol)

Make cursor less jittery (ie always up to date with where the camera is facing)

Physics Tweaks

Fix controls not repeating

(04/22/2024) Data Cleanup

• Clean up audio
• Fix ui audio effects having play times that are wayyyy too long

(04/30/2024) Fix grass not generating for closest tiles

• There is no distance check in the ClientFoliageManager

(05/04/2024) Ground Texture Atlas system

• Basic atlas working with marching cubes
• Make it work with triplanar mapping

TODO

Character movement in particular feels off

• Bring back strafing
• Fix interaction with networking
• Potentially facing vector on server misaligned with client facing vector? - Nope! They're perfectly aligned
• May be in the ground move tree itself the hard setting velocity instead of applying a force is causing weirdness

Fix being able to walk off far side of the world (ie in level editor)

Grass System properly LOD

• Have foliage dynamically time out cells to be reconsidered based on distance from player (if close, short cooldown, if far long cooldown)

Data Cleanup

• Clean up creatures
• Remove unused ones
• Fix values for movement now that physics has changed
• Fix characters bouncing when landing from a fall
• Clean up unused models
• Clean up textures
• Move model textures into models
• Recursive model transform data

Clean up Material class

• fix storing textures in the mat class ( pain :c )

Ground Texture Atlas system

• Refactor to block main thread only when creating the actual texture object (load buffered image separately)

More Debug menus

• Screen that shows the overall status of draw cell manager
• Screen that shows the overall status of fluid cell manager
• Screen that shows the overall status of realm 0
• Screen that shows the overall status of realm manager

Revisit first attempt at instancing (its really laggy lol)

• Maybe have draw call happen on top level entity and immediately queue all children recursively

Shader library system

• Abiltiy to include the shader library in individual files (ie implement #include)

Break control handlers into separate files with new logic to transition between control handler states

Transvoxel Algorithm Client Terrain Entity Management (specifically creation/teardown for client)

• Also queries for far out chunks to load far away terrain Server Terrain Management (specifically for collision)
• Handles communicating far out LOD chunks to client as well Terrain Interface Positional Access Interface
• Ability to get terrain at point for interactions with game world eg placing grass/water collision Actually implement transvoxel algo Marching Cubes Texture Overhaul
• Detect opengl max image size
• Construct texture atlas of max size
  • (target 256x256 resolution initially, should give ~1000 types for 8192x8192)
• Prebake all textures into atlas
• Rewrite marching cubes shader to leverage this atlas

Another pass at grass

• Multiple foliage models in same cell

Build a lod system

• Could potentially be held at actor level
• Link different models based on LOD level
• LOD trees aggressively
• LOD foliage cells aggressively

Refactor attach logic to better encapsulate semantic attachment

Light Manager

• Creates and manages light entities
• Uses priority queue mechanism like foliage manager to only draw the most important lights
• Support "flickering" somehow
• Eventually support spot lights?
• Point shadows ???

gltf Support

• Fix bad data with human mesh textures not mapping
• Texture loading from gltf file

Cellular Automata Fluid Dynamics System

• Advect force
• Advect density
• Diffuse density
• Do not bound to single chunks
• Only add compression when it starts to become an issue

skybox work

• make it prettier
• be able to manage its colors through a clean interface

Ray Traced Audio Engine

Documentation Pass on as many files as you can stomache

Generate Tree Entities

• Generate stem
• Ability to specify central stem
• Cubic function for limb dispersion over length
• Generate branch starters from trunk that are not full length
• Have leaves point out of branches at specific angles

Foliage Manager upgrades

• Wind system (environment ubi that defines wind that is lookup'd by individual blades)

Server Micro/Macro transitioning (turning entity into character in macro simulation and vice-versa) Macro level content tracker

• Keep track of chunks that would have trees and include this in the data passed to client

Season tracking

• Have a concept of seasons as loaded from assets/data
  • Have a season manager that keeps track of seasons at the macro scale
  • Maybe make it per biome or something in data?
• Create interface for querying and updating season in a given chunk
• Update foliage system to transition models and colors based on the current season (update maybe every minute in game or something?)

Weather tracking

• Keeps track at macro level of wind direction and level
• Keeps track at macro level of cloud coverage and structure (ie transmits same cloud to client as server)
• Keeps track at macro level of temperature
• Keeps track at macro level of precipitation

Foliage Manager upgrades

• Add wind simulation if relevant

Server Content Engine

• On reloading a chunk with macro objects like structures, apply appropriate modifiers to align micro object with macro object
  • EG if a building is damaged in macro simulation, regenerate the micro one to be damaged before marking chunk as valid

Upgrade terrain editing user experience further

• Leveling tools
• Lock to axis tools
• Server validation for client request to change terrain

Ongoing

Upgrade terrain generation algorithms

• This one should be an ongoing process in general as it is a matter of taste
• Make a route hard coded that throws you straight into a generated world
• This makes it easier to tweak algo and immediately get results

Documentation

Eventually

Generic collision engine to support different instances of engine (eg hitboxes vs terrain vs liquids, etc)

• Major refactoring to happen here Procedural Cliff Texture
• Uses noise or fractals or something to generate infinite textures in shader Terrain Chunks:
• Scale textures to be 1 texture per unit of terrain
• Texture atlasing (512x512) Loot Generator
• System that can generate items that would be appropriate reward given some variables
• ie you tell it 'this is this character's stats, this is the relative level of loot I want to provide'
• it then searches through the lore to generate appropriate weapons, armor, materials, consumables, etc Computational Fluid Dynamics System
• Basic transparent voxels for fluid
• Fluid simulation algorithm
• Vectorize/JNI fluid simulation
• Separate fluid chunks
• Networked fluid chunk transportation (including force vectors for generating particles on client)
• Save fluid chunks

Gameplay:

• Swordsman with movement abilities
• Archer with movement abilities
• Mage with elemental physics based abilities

Fun pie in the sky stuff

Ability to cast spell to extinguish sun, making world permanently dark/night time dynamic camp/house system - npcs will gradually join your camp the longer you stay there

• They also leave to go on quests and missions
• You see the composition of the camp change over time dynamic warfare system
• Guard towers that need to be captured by factions before enabling assault on real settlements
• Raids against villages

Known bugs

• Draw cell manager iso values dont make sense and should scale empty cells based on neighbor cells
• Draw cell manager logic doesn't fill in border cells properly (the logic to check if a border cell exists always succeeds as long as the potential location is within world bounds, not if it actually exists in cache)
• Control handler re-polls for mouse coordiantes for each control handler group it processes, so only the first group gets the mouse movement event properly