Renderer/docs/src/progress/renderertodo.md

@page renderertodo TODO

@subpage bigthings Big Things To Build



# DONE



(6/10/2023)

Scene
 - Generic entity & btree grouping mechanism

Server Data Cell
 - Wrapper around scene used by server to manage server scenes and players associated with them
Realm
 - Manages server data cells including rudimentary spin up/down
Realm Manager
 - Manages all realms + provides constructors for types of realms
Data Cell Manager
 - Interface defining method for laying out cells and querying for cells based on real or world coordinates
Gridded Data Cell Manager
 - Implementation of Data Cell Manager that manages a grid of data cells if using terrain engine
Data Cell Location Resolver
 - Takes a vector3d and an entity and figures out what data cell that should correspond to
 - Breaks the above resolver relationship out of cells so that can support different "worlds"
Entity Data Cell Mapper
  - Takes an entity and gives the data cell that entity is inside of
  - Breaks this out from data cell manager to support multiple worlds
Data Cell Physics Manager
  - Breaks the physics generation for chunks out of the data cell manager
Data Cell Search Utils
 - Combine the above classes to provided utilities for searching cells nearby a location for groups of entities

Client Scene Wrapper
 - Wrapper around scene used by client
 - Specifically adds translating server/client entity IDs to one another

Loading Threads
 - Separate client and server versions of loading threads to different files

Server Bone Loader
 - Server bone loading for server only collision simulation

Instancing support

Server Scene Management
 - Creating and simulating entities per chunk

 Server Terrain Management (specifically for collision)
 - Generation collision meshes
 - Also generate texture array

Terrain Chunk Shader
 - Procedural textures for eg rock and dirt

Client Terrain Entity Management (specifically creation/teardown for client)
 - Do the basic query stuff for getting terrain

Diagnose weird walking physics
 - Character terrain collision messiness

Upgrade terrain editing user experience
 - Sphere pallete
 - Gradual updates
 - Terrain Deletion

Upgrade Terrain Chunk
 - Fully connect world chunks

Terrain Editing
 - Server interface for editing terrain
 - Ability to request terrain change from client

Voxel Data Storage
 - Store metadata about types of voxels (rgb offsets, foliage placements, etc)

Foliage Manager upgrades
 - Place foliage based on voxel type
 - Use timeout queueing system to evaluate voxel locations to place foliage
 - Destroy foliage on voxel edit
 - Regrow foliage on empty cells after timeout
 - Gradually regrow

Chunk Saving System
 - File that maps world position to chunk file
 - Save chunks to compressed files
 - Server load chunks as needed
 - Unload chunks after 100 frames of not being used
 - Save chunks on unload

Convert client and server terrain entity composition to use 8 chunks instead of 1

Server Diagnose Physics potentially broken
 - Server physics calculation not happening <--  hard to test bc gravity tree might not actually be updating client to turn off or something

Foliage Manager upgrades
 - Fix popping

(2/25/2024)
Generate Tree Entities
 - Generate leaf blobs container entity
  - Attach individual leaf blob entities to container entity
 - Sways in the wind
 - Synchronize creation on server and client
 - Synchronize swinging on server and client
 - Tree types configurable via files
Scene Streaming (streaming state of a scene to client)
 - Solidify the work that has already been done in this space and PARTICULARLY add good high level documentation
Server Content Engine
 - On first loading a new-to-server chunk, generate all content (foliage, terrain objects (boulders), structures, etc) entities for the chunk


(2/28/2024)
Improve Fonts
 - Support multiple fonts in engine
 - Leverage https://github.com/SilverTiger/lwjgl3-tutorial/wiki/Fonts to load fonts with java

(2/29/2024)
Properly prioritize close trees over far trees

Implement proper Frustum Culling
 - Regular Actors
 - Instanced Actors

(03/06/2024)
Bake in imgui

(03/07/2024)
Server frametime bar graph

(03/09/2024)
Ability to attach ambient audio emitters to entities

Timekeeping class that defaults to gltf time and falls back to systemCurrentTimeMillis

Methods for sleeping physics bodies if nothing nearby them is dynamic (ie trees if there are no moving creatures near them)
 - SAP2 space from ode4j specifically (ended up using BVH space instead)

Overhaul mesh class
 - remove unused stuff
 - private constructor
  (this is going to require changing a lot of dependencies)

(03/10/2024)
De-dupe render calls via doing mutations in render pipeline status and dont call setting variables to values they are already set to
Render pipeline updates to support multiple pipelines defined in different files
 - Grouping meshes together independent of actor so don't have to rebind shader programs or VAOs to redraw

(03/12/2024)
Foliage Manager upgrades
 - Place foliage intelligently

 Fix arena mode (terrain be crazy)

(03/17/2024)
Optimize instance logic (currently sorting the list of objects to push to buffer each frame nukes the cpu)
 sort nukes cpu because capacity logic is after sort, so it tries to sort ~600k items every frame before caping
 The way to optimize this is to completely gut existing code. One draw call per tile. Draw call calles drawInstanced for all blades within that cell.
 Provide a texture that contains data per blade of grass to the cell on draw. Store positions in that once at creation.
 For dynamic wind later will need to use UBOs or something like that.
 Fix grass flickering (it's frustum culling being inconsistent, try commenting it out in InstancedActor and see what happens :| ) (well we won't have that problem anymore lol)

(03/20/2024)
Free camera system that can detatch from player entity

(03/20/2024)
Half pass at cellular automata fluid dynamics system
 - Diffuse density
 - Streaming chunks over network
 - Basic model creation

(03/21/2024)
Fix character movement (allegedly fixed -- maybe by camera stuff?)
 - Walking left or right while turning camera is very jittery
 - Can lock on moving

Fix Frustum Culling for skybox

Fix Character creation preview not working

(03/23/2024)
Physics-controlled objects system

Clean up main method/class
 - Include Remotery library

(03/24/2024)
Bring LWJGL version up to latest

Automatic Scene unloading
 - Tree structure inside scenes for tearing down groups of entities
 - Entity decomposition
   - Server handling
   - Client handling when scene should be unloaded

Level loading/saving + Basic Editor
 - Spin up voxel level (think arena mode)
 - Save voxel level

(04/10/2024)
Level loading/saving + Basic Editor
 - Basic editor functionality
   - Menu of types of entities to spawn
   - Button to spawn them at cursor

UI Work
 - Make ui feel more responsive, whatever that means (answer is make hover-over styling work lol)
 - Frames for windows (WIP)
 - Need to figure out style-wise what we want to do here (generally minimalist)
 - Fix scrollable handling

Terrain editing UI
 - Menu to select palette to generate, populated based on data

(04/13/2024)
UI Work
 - Level editor ability to destroy an entity on server, have it also destroy on client, AND not persist on save
 - Environment controls (not persisting in save yet)

Fix Movement Bug where player keeps running after releasing control

(04/21/2024)
UI Fixes
 - Refactor ui toolkit elements to separate absolute and relative position
 - Fix inventory menus

Touch Up working with items
 - Make sure the models are appropriately scaled
 - Remove all logic from before client-server separation

More Debug menus
 - Screen that shows the overall status of client scene
  - Number of entities
  - Maybe a listing of each one?

Another pass at grass
 - Fix shader being camera position independent (if you move the wind moves with you lol)

Make cursor less jittery (ie always up to date with where the camera is facing)

Physics Tweaks

Fix controls not repeating

(04/22/2024)
Data Cleanup
 - Clean up audio
  - Fix ui audio effects having play times that are wayyyy too long

(04/30/2024)
Fix grass not generating for closest tiles
 - There is no distance check in the ClientFoliageManager

(05/04/2024)
Ground Texture Atlas system
 - Basic atlas working with marching cubes
 - Make it work with triplanar mapping

(05/05/2024)
Synchronize attack tree over network

Clean up data
 - Tree Model Paths


(05/07/2024)
Ground Texture Atlas system
 - Refactor to block main thread only when creating the actual texture object (load buffered image separately)

First Person Camera


(05/15/2024)
More consistent terrain editing

(05/16/2024)
Reintroduce strafing

(05/??/2024)
First person render pipeline
 - Properly compositing onto main texture
   - Potentially look at storing the framebuffer for the pipeline in the pipeline class itself

(05/19/2024)
Character movement in particular feels off
 - Bring back strafing
 - Fix interaction with networking
  - Potentially facing vector on server misaligned with client facing vector? - Nope! They're perfectly aligned
  - May be in the ground move tree itself the hard setting velocity instead of applying a force is causing weirdness

(05/23/2024)
Viewmodel
 - Fix hands placement
 - Animations defined in data file

(05/24/2024)
Viewmodel
 - Add animations queues in btrees
  - idle
  - jump
  - ground movement
  - land
  - fall
  - attack

Attaching items to hands in first person

Fix grass placement

(05/26/2024)

VERY rudimentary first person documentation to give basic navigation to relevant files.

Fix attacking looping and freezing the character in place
 - Was using delta real time (0.02ms) instead of delta frames in server and client attack trees (1 frame/simulate() call)

Document hitboxes
 - Documented how it works currently and the architecture we want to move towards

Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
 - Need to have an object attached to creature that stores the rigid body
  - When creating the creature, for each hitbox, create shapes for the rigid body
 - Attach the overall object to the creature entity

(05/27/2024)

Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
 - Synchronize hitbox positions each frame

(05/31/2024)
Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
- Write custom callback for the collision engine for just hitboxes

(06/04/2024)
Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
 - Need to have an object attached to creature that stores the rigid body
  - When creating the creature, for each hitbox, create shapes for the rigid body
 - Attach the overall object to the creature entity

(06/07/2024)
Hitboxes work to properly use capsules (constantly destroy/recreate every frane because od4j doesn't allow rescaling :<)

(06/10/2024)
Add flow for demo menu/level loading

(06/11/2024)
Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
 - Properly calculate the capsule that bridges from previous frame to current frame hitbox location
 - Write custom callback for the collision engine for just hitboxes
Fix player model side-jog animations

(06/13/2024)
Fix newly exported model not rendering correctly
 - All bones are passed into the shader every render call, the bone values must be corrupted
 - NIGHTMARE BUG
Fix equipping an item spawning two items

(06/14/2024)
Fix inventory ui not closing when you hit 'i' key (will need to update utility functions to manage input mode so you're not doing it in callback)
Develop debug ui for equip points

# TODO


Demo requirements:
 = Assets = 
Block animation in first person
Block animation in third person
Fix attack animation bone rotations for hand
Clean up equip state data
Audio FX for everything


 = Coding = 
Fix items falling below the ground
Control rebinding menu from title screen
Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
 - Introduce block hitbox (blockbox) type
Enemy AI
Probably some kind of tutorial text
Network-able ui messages
Ability to display video both on title screen as well as in game windows for tutorials
better scaffolding for scripting engine with hooks for equipping items, spawning entities, pausing/resuming play, etc
Ability for private realms to have time start/stop based on the player's feedback <-- sync this up to tutorial ui via script





BIG BIG BIG BIG IMMEDIATE TO DO:
always enforce opengl interface across all opengl calls jesus christ the bone uniform bug was impossible






Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
 - Introduce block hitbox (blockbox) type

Fix voxel type selection menu not showing textures
 - The quads are off screen because the calculation for ndcX/ndcY are putting it wayyy to the right -- will need to revisit calcs for all that

Fix being able to walk off far side of the world (ie in level editor)

Grass System properly LOD
 - Have foliage dynamically time out cells to be reconsidered based on distance from player (if close, short cooldown, if far long cooldown)

Would be nice to be able to cut clients that stream their logs to my server 

Data Cleanup
 - Clean up creatures
  - Remove unused ones
  - Fix values for movement now that physics has changed
   - Fix characters bouncing when landing from a fall
 - Clean up unused models
 - Clean up textures
  - Move model textures into models
 - Recursive model transform data

Clean up Material class
 - fix storing textures in the mat class ( pain :c )

Clean up framebuffer class
 - Comment everything

Overhaul of 'attach' semantics
 - Having different types of attach tree propagation
 - Ability to turn on/off combinations of models at will (already exists, but needs review)

More Debug menus
 - Screen that shows the overall status of draw cell manager
 - Screen that shows the overall status of fluid cell manager
 - Screen that shows the overall status of realm 0
 - Screen that shows the overall status of realm manager
 - Particularly, want a view of all entities in the scene, and the ability to click on a single entity to get an overview of everything on the entity
  - For each behavior tree, ability to click into the tree and view fine details about its state (both pure state variable as well as other relevant variables)

Revisit first attempt at instancing (its really laggy lol)
 - Maybe have draw call happen on top level entity and immediately queue all children recursively

Shader library system
 - Abiltiy to include the shader library in individual files (ie implement #include)

Break control handlers into separate files with new logic to transition between control handler states
   

Transvoxel Algorithm
  Client Terrain Entity Management (specifically creation/teardown for client)
  - Also queries for far out chunks to load far away terrain
  Server Terrain Management (specifically for collision)
  - Handles communicating far out LOD chunks to client as well
  Terrain Interface Positional Access Interface
  - Ability to get terrain at point for interactions with game world eg placing grass/water collision
  Actually implement transvoxel algo
  Marching Cubes Texture Overhaul
  - Detect opengl max image size
  - Construct texture atlas of max size
      - (target 256x256 resolution initially, should give ~1000 types for 8192x8192)
  - Prebake all textures into atlas
  - Rewrite marching cubes shader to leverage this atlas

Another pass at grass
 - Multiple foliage models in same cell

Build a lod system
 - Could potentially be held at actor level
 - Link different models based on LOD level
 - LOD trees aggressively
 - LOD foliage cells aggressively

Refactor attach logic to better encapsulate semantic attachment

Light Manager
 - Creates and manages light entities
 - Uses priority queue mechanism like foliage manager to only draw the most important lights
 - Support "flickering" somehow
 - Eventually support spot lights?
 - Point shadows ???

gltf Support
  - Fix bad data with human mesh textures not mapping
  - Texture loading from gltf file


Cellular Automata Fluid Dynamics System
 - Advect force
 - Advect density
 - Diffuse density
 - Do not bound to single chunks
 - Only add compression when it starts to become an issue


skybox work
 - make it prettier
 - be able to manage its colors through a clean interface

Ray Traced Audio Engine

Documentation Pass on as many files as you can stomache

Generate Tree Entities
 - Generate stem
 - Ability to specify central stem
 - Cubic function for limb dispersion over length
 - Generate branch starters from trunk that are not full length
 - Have leaves point out of branches at specific angles

Foliage Manager upgrades
 - Wind system (environment ubi that defines wind that is lookup'd by individual blades)


Server Micro/Macro transitioning (turning entity into character in macro simulation and vice-versa)
Macro level content tracker
 - Keep track of chunks that would have trees and include this in the data passed to client


Season tracking
 - Have a concept of seasons as loaded from assets/data
   - Have a season manager that keeps track of seasons at the macro scale
   - Maybe make it per biome or something in data?
 - Create interface for querying and updating season in a given chunk
 - Update foliage system to transition models and colors based on the current season (update maybe every minute in game or something?)


Weather tracking
 - Keeps track at macro level of wind direction and level
 - Keeps track at macro level of cloud coverage and structure (ie transmits same cloud to client as server)
 - Keeps track at macro level of temperature
 - Keeps track at macro level of precipitation


Foliage Manager upgrades
 - Add wind simulation if relevant



Server Content Engine
 - On reloading a chunk with macro objects like structures, apply appropriate modifiers to align micro object with macro object
   - EG if a building is damaged in macro simulation, regenerate the micro one to be damaged before marking chunk as valid


Upgrade terrain editing user experience further
 - Leveling tools
 - Lock to axis tools
 - Server validation for client request to change terrain




# Ongoing

Upgrade terrain generation algorithms
 - This one should be an ongoing process in general as it is a matter of taste
 - Make a route hard coded that throws you straight into a generated world
  - This makes it easier to tweak algo and immediately get results

Documentation





# Eventually

Generic collision engine to support different instances of engine (eg hitboxes vs terrain vs liquids, etc)
 - Major refactoring to happen here
Procedural Cliff Texture
 - Uses noise or fractals or something to generate infinite textures in shader
Terrain Chunks:
 - Scale textures to be 1 texture per unit of terrain
 - Texture atlasing (512x512)
Loot Generator
 - System that can generate items that would be appropriate reward given some variables
 - ie you tell it 'this is this character's stats, this is the relative level of loot I want to provide'
 - it then searches through the lore to generate appropriate weapons, armor, materials, consumables, etc
Computational Fluid Dynamics System
 - Basic transparent voxels for fluid
 - Fluid simulation algorithm
 - Vectorize/JNI fluid simulation
 - Separate fluid chunks
 - Networked fluid chunk transportation (including force vectors for generating particles on client)
 - Save fluid chunks




Gameplay:

 - Swordsman with movement abilities
 - Archer with movement abilities
 - Mage with elemental physics based abilities








# Fun pie in the sky stuff
Ability to cast spell to extinguish sun, making world permanently dark/night time
dynamic camp/house system - npcs will gradually join your camp the longer you stay there
  - They also leave to go on quests and missions
  - You see the composition of the camp change over time
dynamic warfare system
  - Guard towers that need to be captured by factions before enabling assault on real settlements
  - Raids against villages





# Known bugs
 - Draw cell manager iso values dont make sense and should scale empty cells based on neighbor cells
 - Draw cell manager logic doesn't fill in border cells properly (the logic to check if a border cell exists always succeeds as long as the potential location is within world bounds, not if it actually exists in cache)
 - Control handler re-polls for mouse coordiantes for each control handler group it processes, so only the first group gets the mouse movement event properly