Renderer/docs/src/progress/renderertodo.md
austin ee15dd2366
All checks were successful
studiorailgun/Renderer/pipeline/head This commit looks good
client-server physics synchronization
2024-08-11 19:31:43 -04:00

779 lines
25 KiB
Markdown

@page renderertodo TODO
@subpage bigthings Big Things To Build
# DONE
(6/10/2023)
Scene
- Generic entity & btree grouping mechanism
Server Data Cell
- Wrapper around scene used by server to manage server scenes and players associated with them
Realm
- Manages server data cells including rudimentary spin up/down
Realm Manager
- Manages all realms + provides constructors for types of realms
Data Cell Manager
- Interface defining method for laying out cells and querying for cells based on real or world coordinates
Gridded Data Cell Manager
- Implementation of Data Cell Manager that manages a grid of data cells if using terrain engine
Data Cell Location Resolver
- Takes a vector3d and an entity and figures out what data cell that should correspond to
- Breaks the above resolver relationship out of cells so that can support different "worlds"
Entity Data Cell Mapper
- Takes an entity and gives the data cell that entity is inside of
- Breaks this out from data cell manager to support multiple worlds
Data Cell Physics Manager
- Breaks the physics generation for chunks out of the data cell manager
Data Cell Search Utils
- Combine the above classes to provided utilities for searching cells nearby a location for groups of entities
Client Scene Wrapper
- Wrapper around scene used by client
- Specifically adds translating server/client entity IDs to one another
Loading Threads
- Separate client and server versions of loading threads to different files
Server Bone Loader
- Server bone loading for server only collision simulation
Instancing support
Server Scene Management
- Creating and simulating entities per chunk
Server Terrain Management (specifically for collision)
- Generation collision meshes
- Also generate texture array
Terrain Chunk Shader
- Procedural textures for eg rock and dirt
Client Terrain Entity Management (specifically creation/teardown for client)
- Do the basic query stuff for getting terrain
Diagnose weird walking physics
- Character terrain collision messiness
Upgrade terrain editing user experience
- Sphere pallete
- Gradual updates
- Terrain Deletion
Upgrade Terrain Chunk
- Fully connect world chunks
Terrain Editing
- Server interface for editing terrain
- Ability to request terrain change from client
Voxel Data Storage
- Store metadata about types of voxels (rgb offsets, foliage placements, etc)
Foliage Manager upgrades
- Place foliage based on voxel type
- Use timeout queueing system to evaluate voxel locations to place foliage
- Destroy foliage on voxel edit
- Regrow foliage on empty cells after timeout
- Gradually regrow
Chunk Saving System
- File that maps world position to chunk file
- Save chunks to compressed files
- Server load chunks as needed
- Unload chunks after 100 frames of not being used
- Save chunks on unload
Convert client and server terrain entity composition to use 8 chunks instead of 1
Server Diagnose Physics potentially broken
- Server physics calculation not happening <-- hard to test bc gravity tree might not actually be updating client to turn off or something
Foliage Manager upgrades
- Fix popping
(2/25/2024)
Generate Tree Entities
- Generate leaf blobs container entity
- Attach individual leaf blob entities to container entity
- Sways in the wind
- Synchronize creation on server and client
- Synchronize swinging on server and client
- Tree types configurable via files
Scene Streaming (streaming state of a scene to client)
- Solidify the work that has already been done in this space and PARTICULARLY add good high level documentation
Server Content Engine
- On first loading a new-to-server chunk, generate all content (foliage, terrain objects (boulders), structures, etc) entities for the chunk
(2/28/2024)
Improve Fonts
- Support multiple fonts in engine
- Leverage https://github.com/SilverTiger/lwjgl3-tutorial/wiki/Fonts to load fonts with java
(2/29/2024)
Properly prioritize close trees over far trees
Implement proper Frustum Culling
- Regular Actors
- Instanced Actors
(03/06/2024)
Bake in imgui
(03/07/2024)
Server frametime bar graph
(03/09/2024)
Ability to attach ambient audio emitters to entities
Timekeeping class that defaults to gltf time and falls back to systemCurrentTimeMillis
Methods for sleeping physics bodies if nothing nearby them is dynamic (ie trees if there are no moving creatures near them)
- SAP2 space from ode4j specifically (ended up using BVH space instead)
Overhaul mesh class
- remove unused stuff
- private constructor
(this is going to require changing a lot of dependencies)
(03/10/2024)
De-dupe render calls via doing mutations in render pipeline status and dont call setting variables to values they are already set to
Render pipeline updates to support multiple pipelines defined in different files
- Grouping meshes together independent of actor so don't have to rebind shader programs or VAOs to redraw
(03/12/2024)
Foliage Manager upgrades
- Place foliage intelligently
Fix arena mode (terrain be crazy)
(03/17/2024)
Optimize instance logic (currently sorting the list of objects to push to buffer each frame nukes the cpu)
sort nukes cpu because capacity logic is after sort, so it tries to sort ~600k items every frame before caping
The way to optimize this is to completely gut existing code. One draw call per tile. Draw call calles drawInstanced for all blades within that cell.
Provide a texture that contains data per blade of grass to the cell on draw. Store positions in that once at creation.
For dynamic wind later will need to use UBOs or something like that.
Fix grass flickering (it's frustum culling being inconsistent, try commenting it out in InstancedActor and see what happens :| ) (well we won't have that problem anymore lol)
(03/20/2024)
Free camera system that can detatch from player entity
(03/20/2024)
Half pass at cellular automata fluid dynamics system
- Diffuse density
- Streaming chunks over network
- Basic model creation
(03/21/2024)
Fix character movement (allegedly fixed -- maybe by camera stuff?)
- Walking left or right while turning camera is very jittery
- Can lock on moving
Fix Frustum Culling for skybox
Fix Character creation preview not working
(03/23/2024)
Physics-controlled objects system
Clean up main method/class
- Include Remotery library
(03/24/2024)
Bring LWJGL version up to latest
Automatic Scene unloading
- Tree structure inside scenes for tearing down groups of entities
- Entity decomposition
- Server handling
- Client handling when scene should be unloaded
Level loading/saving + Basic Editor
- Spin up voxel level (think arena mode)
- Save voxel level
(04/10/2024)
Level loading/saving + Basic Editor
- Basic editor functionality
- Menu of types of entities to spawn
- Button to spawn them at cursor
UI Work
- Make ui feel more responsive, whatever that means (answer is make hover-over styling work lol)
- Frames for windows (WIP)
- Need to figure out style-wise what we want to do here (generally minimalist)
- Fix scrollable handling
Terrain editing UI
- Menu to select palette to generate, populated based on data
(04/13/2024)
UI Work
- Level editor ability to destroy an entity on server, have it also destroy on client, AND not persist on save
- Environment controls (not persisting in save yet)
Fix Movement Bug where player keeps running after releasing control
(04/21/2024)
UI Fixes
- Refactor ui toolkit elements to separate absolute and relative position
- Fix inventory menus
Touch Up working with items
- Make sure the models are appropriately scaled
- Remove all logic from before client-server separation
More Debug menus
- Screen that shows the overall status of client scene
- Number of entities
- Maybe a listing of each one?
Another pass at grass
- Fix shader being camera position independent (if you move the wind moves with you lol)
Make cursor less jittery (ie always up to date with where the camera is facing)
Physics Tweaks
Fix controls not repeating
(04/22/2024)
Data Cleanup
- Clean up audio
- Fix ui audio effects having play times that are wayyyy too long
(04/30/2024)
Fix grass not generating for closest tiles
- There is no distance check in the ClientFoliageManager
(05/04/2024)
Ground Texture Atlas system
- Basic atlas working with marching cubes
- Make it work with triplanar mapping
(05/05/2024)
Synchronize attack tree over network
Clean up data
- Tree Model Paths
(05/07/2024)
Ground Texture Atlas system
- Refactor to block main thread only when creating the actual texture object (load buffered image separately)
First Person Camera
(05/15/2024)
More consistent terrain editing
(05/16/2024)
Reintroduce strafing
(05/??/2024)
First person render pipeline
- Properly compositing onto main texture
- Potentially look at storing the framebuffer for the pipeline in the pipeline class itself
(05/19/2024)
Character movement in particular feels off
- Bring back strafing
- Fix interaction with networking
- Potentially facing vector on server misaligned with client facing vector? - Nope! They're perfectly aligned
- May be in the ground move tree itself the hard setting velocity instead of applying a force is causing weirdness
(05/23/2024)
Viewmodel
- Fix hands placement
- Animations defined in data file
(05/24/2024)
Viewmodel
- Add animations queues in btrees
- idle
- jump
- ground movement
- land
- fall
- attack
Attaching items to hands in first person
Fix grass placement
(05/26/2024)
VERY rudimentary first person documentation to give basic navigation to relevant files.
Fix attacking looping and freezing the character in place
- Was using delta real time (0.02ms) instead of delta frames in server and client attack trees (1 frame/simulate() call)
Document hitboxes
- Documented how it works currently and the architecture we want to move towards
Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
- Need to have an object attached to creature that stores the rigid body
- When creating the creature, for each hitbox, create shapes for the rigid body
- Attach the overall object to the creature entity
(05/27/2024)
Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
- Synchronize hitbox positions each frame
(05/31/2024)
Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
- Write custom callback for the collision engine for just hitboxes
(06/04/2024)
Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
- Need to have an object attached to creature that stores the rigid body
- When creating the creature, for each hitbox, create shapes for the rigid body
- Attach the overall object to the creature entity
(06/07/2024)
Hitboxes work to properly use capsules (constantly destroy/recreate every frane because od4j doesn't allow rescaling :<)
(06/10/2024)
Add flow for demo menu/level loading
(06/11/2024)
Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
- Properly calculate the capsule that bridges from previous frame to current frame hitbox location
- Write custom callback for the collision engine for just hitboxes
Fix player model side-jog animations
(06/13/2024)
Fix newly exported model not rendering correctly
- All bones are passed into the shader every render call, the bone values must be corrupted
- NIGHTMARE BUG
Fix equipping an item spawning two items
(06/14/2024)
Fix inventory ui not closing when you hit 'i' key (will need to update utility functions to manage input mode so you're not doing it in callback)
Develop debug ui for equip points
(06/18/2024)
Block state synchronization between client and server
Highlevel netcode gen updates
- Fix formatting for imports so it doesn't put an extra newline
- Fix formatting for function generation so it DOES put a new line when the function isn't being replaced
- Convert fields to using id variables so all behavior tree classes don't get git updates every time the ids shuffle
- Furthermore, keep tracking of the existing ids for trees and fields and only generate ids for new trees and fields
Fix client gravity tree name
(06/19/2024)
Transvoxel implementation
- Begin work on transvoxel algo
(06/21/2024)
Transvoxel implementation
- First working implementation of mesh generation for transvoxel chunks (architecture of adding it to drawcellmanager still todo)
(06/22/2024)
Transvoxel implementation
- Scaling LODed chunks by lod level
Fix items falling below the ground
Fix server always rotating entity to face client camera -- should only be changing movement vector
Probably some kind of tutorial text
(06/??/2024)
Start working on script engine documentation/design
(06/27/2024)
Animations
- 2H Sword Hold (3rd person)
- 2H Sword Attack (3rd person)
- 2H Sword Block (3rd person)
- 2H Sword Hold (1st person)
- 2H Sword Attach (1st person)
- 2H Sword Block (1st person)
Audio
- Sword Swing
- Sword Sheath
- Sword Unsheath
- Sword Hit Metal
- Sword Hit Flesh
(07/02/2024)
better scaffolding for scripting engine with hooks for equipping items, spawning entities, pausing/resuming play, etc
Redo hitboxes to have capsules and also chaining between frames (but not between swinging the camera around)
- Introduce block hitbox (blockbox) type
- Sour spot, sweet spot for damage hitboxes and hurtboxes
(07/03/2024)
Clean up framebuffer class
- Comment everything
- Error checking
Overhaul opengl error checking generally (as in, actually use it)
- Add all over the place to help debugging graphics issues
- Handle uniforms being sent to shaders that don't have the uniform defined
Extracting pixels from framebuffers
(07/07/2024)
Work on testing
(07/16/2024)
Scene Loading Refactor
Hooking into engine from script-side
(07/19/2024)
Element Manager debugging and logic simplification
Hooks debugging
On add item to inventory hook
Hook manager debugging
(07/20/2024)
Properly drill static classes to script context
Expose framestep control to script side
Show tutorial hints from script side
TextBox element (not editable yet)
Word element
Text Wrap handling
Overflow handling
(07/21/2024)
AI scaffolding
Attacker ai tree
(07/24/2024)
2 Hand katana
Switching between first and third person
Devtools for updating first person attachment rotations
(07/26/2024)
Viewmodel equipped item rotates inverted to bone rotation
Utility object for reducing boilerplate when writing btree transitions that just play an animation then transition
First animations flickering in first person (enforce animation priority requirement)
Debug third person animations flickering (attachments not reflecting animations that were played that frame)
Small data fix
Refactor spawn point to not be global
Synchronize objects between client and server
Synchronize life state between client and server
(07/27/2024)
Small bugfix with blocking
Refactor math to be client/server agnostic in attach utils
Attach utils fixes for first person handling
(07/28/2024)
Tear out first person rendering pipeline
Partially fix first person attachment to viewmodel
Creature data validation
Unify animation format data on disk
Leverage animation masks to block while moving
Remove Airplane movement system
Fix client-attached models to viewmodel drawing on previous frame
Alignment work for human right hand
(07/29/2024)
Option to load all cells on init scene file
Insidious Entity protocol bugfix on server
Refactor network protocols to enforce async/sync split
(07/30/2024)
Fix depth texture for shadow rendering
Fix eyebrow weights on human model
(07/31/2024)
Massive netcode gen refactor
Server synchronization manager
Jump tree synchronization
Fix jump bugginess
(08/01/2024)
Fix server entity camera rotation
Fix world bounds application
Bone debug rendering
(08/02/2024)
State transition packets and support on both client and server
State transition utils interrupt support
Block animation cancels immediately on first person model
Bone-attachment fix
Fix data for viewmodel hand.r equip point
Support audio on state transition
Audio on block state transitions
(08/04/2024)
Strafe/backpedal movement speed multipliers
Properly equipping/unequipping 2hand items
(08/08/2024)
Client surface-audio system
Sound effects on footstep
New sound effects for movement
Jump sound effects
Don't play walking audio when entity is jumping
Server alert client on collision
Sound effect on sword hit
(08/09/2024)
Math overhaul
- Engine defined origin, up, and left vectors
- Redo math for camera calculations
Rotate player models to face correct direction
Remove BLENDER_TRANSFORM token
Redo of creature-spawning logic to support including attached items
- Items do not necessarily send from server if they are in the scene! They must also not have a parent
Ability to serialize/deserialize a creature with equipped items
- Serialize
- Deserialize
- Send to client
- Receive from server
(08/11/2024)
Sending initial synchronized state on player connect to chunk
Pass at client-server physics synchronization
# TODO
BIG BIG BIG BIG IMMEDIATE TO DO:
always enforce opengl interface across all opengl calls jesus christ the bone uniform bug was impossible
Ability to fully reload game engine state without exiting client
- Back out to main menu and load a new level without any values persisting
- Receive a teleport packet from server and flush all game state before requesting state from server again
Unit Testing
- Capture image from opengl to pixel-check
- Ability to click through the ui via scripts
- Add ui tests
Goof off today requirements:
Transvoxel implementation
- Fix draw cell manager requesting far-out chunks
- Properly update to higher LOD meshes as you get closer
Client Terrain Entity Management (specifically creation/teardown for client)
- Also queries for far out chunks to load far away terrain
Server Terrain Management (specifically for collision)
- Handles communicating far out LOD chunks to client as well
Terrain Interface Positional Access Interface
- Ability to get terrain at point for interactions with game world eg placing grass/water collision
Fix not all grass tiles update when updating a nearby voxel (ie it doesn't go into negative coordinates to scan for foliage updates)
Refactor menu clases under electrosphere.client package
Allow texture map to bind multiple model paths to a single set of mesh->textures
Cache texture atlas for terrain
Rework how chunks are written to disk to make them more cache friendly
- IE, write consecutively higher LOD levels the further into the file, so that you can read just the first few bytes if its a far away chunk
Fix voxel type selection menu not showing textures
- The quads are off screen because the calculation for ndcX/ndcY are putting it wayyy to the right -- will need to revisit calcs for all that
Fix being able to walk off far side of the world (ie in level editor)
Grass System properly LOD
- Have foliage dynamically time out cells to be reconsidered based on distance from player (if close, short cooldown, if far long cooldown)
Debug
- Draw all bones with orientations
Would be nice to be able to cut clients that stream their logs to my server
Refactor render flags
Data Cleanup
- Clean up creatures
- Remove unused ones
- Fix values for movement now that physics has changed
- Fix characters bouncing when landing from a fall
- Clean up unused models
- Clean up textures
- Move model textures into models
- Recursive model transform data
Clean up Material class
- fix storing textures in the mat class ( pain :c )
Overhaul of 'attach' semantics
- Having different types of attach tree propagation
- Ability to turn on/off combinations of models at will (already exists, but needs review)
More Debug menus
- Screen that shows the overall status of draw cell manager
- Screen that shows the overall status of fluid cell manager
- Screen that shows the overall status of realm 0
- Screen that shows the overall status of realm manager
- Particularly, want a view of all entities in the scene, and the ability to click on a single entity to get an overview of everything on the entity
- For each behavior tree, ability to click into the tree and view fine details about its state (both pure state variable as well as other relevant variables)
Revisit first attempt at instancing (its really laggy lol)
- Maybe have draw call happen on top level entity and immediately queue all children recursively
Shader library system
- Abiltiy to include the shader library in individual files (ie implement #include)
Break control handlers into separate files with new logic to transition between control handler states
Another pass at grass
- Multiple foliage models in same cell
Build a lod system
- Could potentially be held at actor level
- Link different models based on LOD level
- LOD trees aggressively
- LOD foliage cells aggressively
Refactor attach logic to better encapsulate semantic attachment
Light Manager
- Creates and manages light entities
- Uses priority queue mechanism like foliage manager to only draw the most important lights
- Support "flickering" somehow
- Eventually support spot lights?
- Point shadows ???
gltf Support
- Fix bad data with human mesh textures not mapping
- Texture loading from gltf file
FileUtils sanitation function check if requested file is in game root dir
Cellular Automata Fluid Dynamics System
- Advect force
- Advect density
- Diffuse density
- Do not bound to single chunks
- Only add compression when it starts to become an issue
skybox work
- make it prettier
- be able to manage its colors through a clean interface
Ray Traced Audio Engine
Documentation Pass on as many files as you can stomache
Generate Tree Entities
- Generate stem
- Ability to specify central stem
- Cubic function for limb dispersion over length
- Generate branch starters from trunk that are not full length
- Have leaves point out of branches at specific angles
Foliage Manager upgrades
- Wind system (environment ubi that defines wind that is lookup'd by individual blades)
Server Micro/Macro transitioning (turning entity into character in macro simulation and vice-versa)
Macro level content tracker
- Keep track of chunks that would have trees and include this in the data passed to client
Season tracking
- Have a concept of seasons as loaded from assets/data
- Have a season manager that keeps track of seasons at the macro scale
- Maybe make it per biome or something in data?
- Create interface for querying and updating season in a given chunk
- Update foliage system to transition models and colors based on the current season (update maybe every minute in game or something?)
Weather tracking
- Keeps track at macro level of wind direction and level
- Keeps track at macro level of cloud coverage and structure (ie transmits same cloud to client as server)
- Keeps track at macro level of temperature
- Keeps track at macro level of precipitation
Foliage Manager upgrades
- Add wind simulation if relevant
Server Content Engine
- On reloading a chunk with macro objects like structures, apply appropriate modifiers to align micro object with macro object
- EG if a building is damaged in macro simulation, regenerate the micro one to be damaged before marking chunk as valid
Upgrade terrain editing user experience further
- Leveling tools
- Lock to axis tools
- Server validation for client request to change terrain
# Ongoing
Upgrade terrain generation algorithms
- This one should be an ongoing process in general as it is a matter of taste
- Make a route hard coded that throws you straight into a generated world
- This makes it easier to tweak algo and immediately get results
Documentation
# Eventually
Procedural Cliff Texture
- Uses noise or fractals or something to generate infinite textures in shader
Loot Generator
- System that can generate items that would be appropriate reward given some variables
- ie you tell it 'this is this character's stats, this is the relative level of loot I want to provide'
- it then searches through the lore to generate appropriate weapons, armor, materials, consumables, etc
Computational Fluid Dynamics System
- Basic transparent voxels for fluid
- Fluid simulation algorithm
- Vectorize/JNI fluid simulation
- Separate fluid chunks
- Networked fluid chunk transportation (including force vectors for generating particles on client)
- Save fluid chunks
Gameplay:
- Swordsman with movement abilities
- Archer with movement abilities
- Mage with elemental physics based abilities
# Fun pie in the sky stuff
Ability to cast spell to extinguish sun, making world permanently dark/night time
dynamic camp/house system - npcs will gradually join your camp the longer you stay there
- They also leave to go on quests and missions
- You see the composition of the camp change over time
dynamic warfare system
- Guard towers that need to be captured by factions before enabling assault on real settlements
- Raids against villages
# Known bugs
- Draw cell manager iso values dont make sense and should scale empty cells based on neighbor cells
- Draw cell manager logic doesn't fill in border cells properly (the logic to check if a border cell exists always succeeds as long as the potential location is within world bounds, not if it actually exists in cache)
- Control handler re-polls for mouse coordiantes for each control handler group it processes, so only the first group gets the mouse movement event properly