Welcome to a new blog series highlighting some terrible ideas I’ve tried out. I call them terrible, but they were created for a purpose and maybe they have a use somewhere. If anything this will be a place for me to look back and wonder “what was I thinking?!”
All terrible ideas have a purpose. They don’t just come out of thin air. There’s some problem to be solved. Time constaints. Skill level constraints. Performance. There’s a good reason for every idea. To understand the terrible idea we need to understand what lead to the idea. Maybe that terrible idea might not be so terrible once you understand it.
Background
To set the stage I was fresh off a class on ECS (would recommend Dave Churchill) and had started experimenting with how far could I take ECS in C++.
First we need to set up some background, definitions, and some basic requirements.
ECS in 15 seconds
You got Entities, Components, Systems.
- Entities are things that exist. It’s a really fancy ID that multiple database rows may reference.
- Components are data that an entity has. A component is a database table, the data for an entity is a row.
- Systems are functions that can query and/or manipulate the data in one or more components.
Congrats! You are now the leading expert in ECS at your company.
Requirements
An important step to understand a terrible idea, why did it come into existence, and does it really solve anything is to start with the requirements.
- Developers can define Systems as functions
- Systems should be able to query and iterate over data
- Systems should be able to change data
- Concurrent systems makes games run on more than 1 thread
- Systems should specify the Components they read
- Systems should specify the Components they write
- Systems cannot use any data that is not in their Data Contract (read/write)
- Errors should at compile time
Systems Are Black Holes For Data
Systems need to pull in data from a lot of places to do the right thing.
Maybe gravity needs to be 9.81f. Maybe 1.0f feels better for gameplay.
The InputSystem wants to call on the GameManager to start the PauseMenuScene.
The AnimationSystem wants the AssetManager to load a shader and add some stutter to the user’s experience.
We could add these to ECS to manage, but how do we find these values?
Are they always on entity 0? Do we need to store what entity they are on?
Many ECS implementations stick resources on a special entity to keep everything in ECS.
I’m not sure if EVERY piece of data really needs to be on ECS, so I want the flexibility.
The 10,000 Foot ECS::View
Concurrency alone adds a whole host of problems, but game devices have so many cores just laying around. We’ll actually ruin our brains on concurrency in another blog, but let’s start with that Data Contract requirement.
We need to track which systems write to which components to make sure they don’t overwrite each other.
We also need to track what a system reads to make sure it has the version of the data it wants.
The GravitySystem is going to change the velocity.y of a bunch of entities.
The CollisionSystem is probably going to need that updated data.
Enter ECS::View: a class that defined what Components the view had access to and what was mutable/constant.
Here’s a simple view that can query entities with Gravity, their Movement, and if they are standing on something solid. We also are going to need to change the Movement to drop the entity if it isn’t standing on something solid.
ECS::View<CGravity, ECS::Mutable<CMovement>, CSensors> view
(I don’t think you understand the gravity of the situation. Oh I think I do)
This view can only be a useful contract if it can make sure I don’t write to somewhere I didn’t promise. We need to make sure the system has no access to the rest of the data managed by ECS. There will be a follow up blog about this ECS::View, but this will have to remain as a terrible teaser.
Iterating on Design
With all the background and problem defined we can iterate on solutions to arrive at something useful. Let’s start with adding ECS::View to provide a solid contract. We need to make sure a System cannot access data managed by ECS that isn’t covered by a View.
Define contracts
We need to make sure that a created view is passed to the system.
Let’s start off with creating a view object when we register the System with the ECS World.
I’m going to wait to implement RegisterSystem until we have figured out how we want our game code to look.
void GameScene::Init() {
RegisterSystem(GravitySystem, this->world->CreateView<CGravity, ECS::Mutable<CMovement>, CSensors>());
}
void GravitySystem(ECS::View<CGravity, ECS::Mutable<CMovement>, CSensors> view) {
const float gravity = 9.81f;
// Iterate over all entities with both a Gravity and Movement
for (auto &it : view.iterate<CGravity, ECS::Mutable<CMovement>>()) {
// Does the entity have shoes and are they on something?
if (view.has<CSensors>(it.entity) && view.value<CSensors>(it.entity)->bottomCollision) {
continue;
}
it.value<ECS::Mutable<CMovement>>()->velocity.y -= gravity;
}
}
This isn’t the worst idea I’ve had. We have our contract defined. We’ve really limited the scope of data that GravitySystem has access to.
But we have to define our Contract twice.
Anyone who edits the world.CreateView<> needs to update the ECS::View<>.
People are in a rush or forget where in a 5000 line file they need to update.
This is going to cause a compiler error for someone.
Have you ever had to debug C++ template compiler errors?
If not, go generate 10 paragraphs of Lorem Ipsum. It’s the same thing.
Maybe Type Alias?
We could create a type alias to define the contract once.
We can totally make world.CreateView accept a View<> type and break out the parts of the view.
using GravityViewContract = ECS::View<CGravity, ECS::Mutable<CMovement>, CSensors>;
void GameScene::Init() {
RegisterSystem(GravitySystem, world.CreateView<GravityViewContract>());
}
void GravitySystem(GravityViewContract view) {
// unchanged
}
It’s 3am and I’m debugging gravity trying to remember what data I can access.
I gotta click on GravityViewContract to find out and then scroll all the way back down.
Ok, Intellisense probably will tell me what it is. I’m an IDE person.
But what about vim and emacs in the corner? Poor nano is sobbing.
Remember: code is read at least 10x more than it is written. We need code to be readable.
Though, maybe not in this case. I probably rewrote it 10x and will never read it again.
Smarter Register System
What if we can make RegisterSystem figure out it’s life based solely on the function’s type.
Bevy figured this out for Rust, so I just had to figure it out for C++.
So what we want is something that looks like
void GameScene::Init() {
RegisterSystem(GravitySystem);
}
void GravitySystem(ECS::View<CGravity, ECS::Mutable<CMovement>, CSensors> view) {
// unchanged
}
We’ve defined our contract once. We’ve limited GravitySystem to what it needs.
This is mostly readable. Ignoring the 50 ECS::’s.
We do have to push more work onto RegisterSystem. How bad could it be?
You Forgot Resources
Oh right. I said we need to be able to access resources outside of ECS. Can we just stuff the resources into the Scene class and have the Systems be methods of the class?
void GameScene::Init() {
RegisterSystem(&GameScene::SpriteRenderSystem);
}
void GameScene::SpriteRenderSystem(ECS::View<CSprite> view) {
for (auto &it : view.iterate<CSprite>()) {
it.value<CSprite>().DrawTo(this->GetWindow());
}
}
Cool. That was easy. Wait, you want to know how we get keyboard presses or mouse clicks? Should they be an instance on the class even though they change each frame? Time for another iteration.
You Shall Not Pass All Events
We could pass these kind of dynamic things to the Systems. It solves the problem, but…
void GameScene::InputSystem(ECS::View<CMovables> view, InputEvents* events) { }
void GameScene::GravitySystem(ECS::View<CGravity, ECS::Mutable<CMovement>, CSensors> view, InputEvents* unused_events) { }
void GameScene::SpriteRenderSystem(ECS::View<CSprite> view, InputEvents* unused_events) { }
Almost none of our systems need the input events. What we need here is something dynamic. Something flexible. Something terrible.
Inject Joke Here
We’ve already accepted we want to make RegisterSystem smart and generate Views for us.
Why can’t it also dispatch the correct parameters to any System?
We basically need a dependency injection system.
void GameScene::Init() {
RegisterSystem(&GameScene::InputSystem);
RegisterSystem(&GameScene::GravitySystem);
RegisterSystem(&GameScene::SpriteRenderSystem);
}
void GameScene::InputSystem(ECS::View<CSprite> view, InputEvents* events) { }
void GameScene::GravitySystem(ECS::View<CSprite> view) { }
void GameScene::SpriteRenderSystem(ECS::View<CSprite> view) { }
I know I promised terrible and this seems fairly reasonable. For C++ template terribleness it’s reasonable. 3/10 would be forced to read again.
Something we need to avoid is the game crashing on level 50, because Matt tried to inject something wrong. I just spent 50 hours playing this game and now it crashes constantly. THANKS A LOT MATT.
If You Build It They Will Cry
Turns out C++ templates and constexpr are extremely powerful. Did you know you can use recursion? How about variable template args we can recurse on? Or this entire library of terribleness built in? Oh we are about to have some fun debugging compiler errors over the next 900 hours!
RegisterSystem Beginnings
RegisterSystem now needs to somehow take type information of our method and magic all the parameters in.
It will need to make sure every parameter is a type that we could inject.
It also needs to support class instance functions and also plain functions, because I want a challenge.
A class instance function is basically a plain function that takes a this pointer.
You might assume these types are equal.
void GameScene::GravitySystem(View v) == void GravitySystem(GameScene* this, View v)
C++ has other ideas though. They are not. But we can provide two implementations of a template! We can also turn a class instance function into a plain function.
template<typename DerivedScene>
class BaseScene {
template <typename... Args>
void RegisterSystem(void(DerivedScene::*system)(Args...)) {
std::function<void(DerivedScene*, Args...)> fn = std::mem_fn(system);
_register_system<DerivedScene*, Args...>(fn);
}
template <typename... Args>
void RegisterSystem(void(*system)(Args...)) {
// A plain function implicitly casts to an std::function.
_register_system<Args...>(system);
}
template <typename... Args>
void _register_system(std::function<void(Args...)> system) {
}
}
Protection From Matts
Before we get too far we need to make sure Matt didn’t try to use a type we don’t inject.
Well let’s have some fun with constexpr and static_asset.
Now we can prevent Matt from crashing Matt’s game at level 50.
void RegisterSystem(void(*system)(Args...)) {
static_assert(
(_is_valid_arg<Args>() && ...),
"System has an invalid argument for Update."
);
_register_system<Args...>(system);
}
template <typename Arg>
static constexpr bool _is_valid_arg() {
if constexpr (ECS::is_view_type<Arg>::value) {
return true;
}
if constexpr (std::is_same<Arg, InputEvents*>::value) {
return true;
}
return false;
}
What in the world is this (_is_valid_arg<Args>() && ...) nonsense?
Welcome to the terrible world of parameter packs.
This tells the C++ compiler to repeat the thing on the left for every Args and AND it all together.
Parameter packs get a bit wonky and honestly end up being a source where I tripped a lot trying to figure out.
I give it a 4/10 terrible. It is approaching terrible. In the vicinity of terriblehood. But not fully terrible.
I really don’t want to maintain this huge list if Arg is SomeSupportedType.
You must embrace recursion.
Recurse Into Recursion
We should capture the supported arguments into an object. That will make it easier to manage.
template <typename... ValidArgs>
struct InjectorArgs {
template <typename Arg>
static constexpr bool is_valid_arg() { }
}
(More. More. MORE.)
Now we can to check if Arg is anywhere inside of ValidArgs.
Recursion is the answer.
You can split off one ValidArg off at a time and stick the others aside.
template <typename... ValidArgs>
struct InjectorArgs {
template <typename Arg>
static constexpr bool is_valid_arg() {
// This is special, for later.
if constexpr (ECS::is_view_type<Arg>::value) {
return true;
}
return _valid_arg<Arg, ValidArgs...>();
}
template <typename Arg, typename NextValidArg, typename... OtherValidArgs>
static constexpr bool _valid_arg() {
if constexpr (std::is_same<Arg, NextValidArg>::value) {
return true;
}
// Without this check, C++ has no idea what _valid_arg<> means.
if constexpr (sizeof...(OtherValidArgs) > 0) {
return _valid_arg<Arg, OtherValidArgs...>();
}
return false;
}
}
Going back to the RegisterSystem let’s make use of our new InjectorArgs.
using SystemInjectorArgs = InjectorArgs<DerivedScene*, ECS::World*, InputEvents*>;
void RegisterSystem(void(*system)(Args...)) {
static_assert(
(SystemInjectorArgs::template is_valid_arg<Args>() && ...),
"System has an invalid argument for Update."
);
_register_system<Args...>(system);
}
So at this point I should probably let you know something. There is a theory which states that if ever anyone discovers exactly how C++ templates work, it will instantly disappear and be replaced by something even more bizarre and inexplicable. There is another theory which states that this has already happened.
I have long forgotten about ::template and search engines no longer seem to care
if I add quotes in my search "C++ ::template". It’s magic ok.
It’s a way to tell C++ which version of is_valid_arg is it supposed to use.
This is sort of like when you do GameScene::GravitySystem, but with terribleness.
We Must Go Deeper
I am so sorry for what is about to follow. I know I said code is read 10x more than it is written. The only reason you will read the upcoming code 10x is trying to understand why it should ever exist.
(Your scientists…)
We’re going to need one variable template list of supported types and another of all the actual types.
It’s not like you can do template<typenames... A, typenames... B>. That would be too easy.
We can’t even iterate two variable template lists at the same time.
I got some bad news. Remember how we said we would move CreateView to RegisterSystem?
That’s about to byte us.
Let’s break this problem down to the smallest form and then expand it.
- I just want to get one value to be injected
- I want to inject all the things
First step: let’s just figure out a way to get one parameter from the list of all parameters. In C++, you can define a struct with a template. You can also provide special versions if a template matches a specific pattern. We can just have a special version that can generate our view and inject it.
// Capture all possible argument types within one struct
template <typename... Injectables>
struct _injector {
template<typename Injectable>
struct fetcher {
static Injectable get(std::tuple<Injectables...>& injectables) {
return std::get<Injectable>(injectables);
}
};
template<typename... Vargs>
struct fetcher<ECS::View<Vargs...>> {
static ECS::View<Vargs...> get(std::tuple<Injectables...>& injectables) {
return ECS::View<Vargs...>(std::get<ECS::World*>(injectables));
}
};
};
Why is there a struct within a struct? Oh wait till we zoom out. We are in the 4th dream right now. There is a struct within a struct within a namespace within a struct. We are about to go Full Terrible. But first, let me explain a bit what just happened.
The outer struct is capturing what Arg types we can even inject.
The inner struct is needed, because we need template specialization.
We can specialize the fetcher for when Matt asks for an ECS::View and just create one!
The tuple of Injectables is just a way to hold all the possible values to inject.
Now to .pop() the stack of dreams all the way to the surface.
We need to create some kind of function that takes every possible value and calls our System with the right values.
We can do this with yet another struct that implements operator().
Remember how we had an std::function<> in _register_system like 5 millenia worth of reading ago?
That’s our System we need to wrap.
namespace internal {
template <typename... Injectables>
struct _injector {
// see above.
};
}
// Capture the Types of that our System function wants to use.
template <typename... Args>
class InjectedCallable {
public:
InjectedCallable(std::function<void(Args...)> system) : _system(system) {}
// And then capture the Types of every type of value we can inject.
template<typename... Injectables>
void operator()(std::tuple<Injectables...> injectables) {
typedef internal::_injector<Injectables...> inject;
_system(
inject::template fetcher<Args>::get(injectables)...
);
}
private:
std::function<void(Args...)> _system;
};
(I know some of these words)
Ok, we are in the home stretch. We are about 9/10 on the terrible scale.
Our friend the Parameter pack is back, but with a function.
We just used our fetcher and spread across every Args in our System function.
This gives us a method for every System that has the exact same signature.
Draw The Owl
namespace internal {
template <typename... Injectables>
struct _injector {
template<typename Injectable>
struct fetcher {
static Injectable get(std::tuple<Injectables...>& injectables) {
return std::get<Injectable>(injectables);
}
};
template<typename... Vargs>
struct fetcher<ECS::View<Vargs...>> {
static ECS::View<Vargs...> get(std::tuple<Injectables...>& injectables) {
return ECS::View<Vargs...>(std::get<ECS::World*>(injectables));
}
};
};
}
// Capture the Types of that our System function wants to use.
template <typename... Args>
class InjectedCallable {
public:
InjectedCallable(std::function<void(Args...)> system) : _system(system) {}
// And then capture the Types of every type of value we can inject.
template<typename... Injectables>
void operator()(std::tuple<Injectables...> injectables) {
typedef internal::_injector<Injectables...> inject;
_system(
inject::template fetcher<Args>::get(injectables)...
);
}
private:
std::function<void(Args...)> _system;
};
template <typename... ValidArgs>
struct InjectorArgs {
template <typename Arg>
static constexpr bool is_valid_arg() {
// This is special, for later.
if constexpr (ECS::is_view_type<Arg>::value) {
return true;
}
return _valid_arg<Arg, ValidArgs...>();
}
template <typename Arg, typename NextValidArg, typename... OtherValidArgs>
static constexpr bool _valid_arg() {
if constexpr (std::is_same<Arg, NextValidArg>::value) {
return true;
}
// Without this check, C++ has no idea what _valid_arg<> means.
if constexpr (sizeof...(OtherValidArgs) > 0) {
return _valid_arg<Arg, OtherValidArgs...>();
}
return false;
}
}
template<typename DerivedScene>
class BaseScene {
template <typename... Args>
void RegisterSystem(void(DerivedScene::*system)(Args...)) {
static_assert(
(SystemInjectorArgs::template is_valid_arg<Args>() && ...),
"System has an invalid argument."
);
std::function<void(DerivedScene*, Args...)> fn = std::mem_fn(system);
_register_system<DerivedScene*, Args...>(fn);
}
void RegisterSystem(void(*system)(Args...)) {
static_assert(
(SystemInjectorArgs::template is_valid_arg<Args>() && ...),
"System has an invalid argument."
);
_register_system<Args...>(system);
}
template <typename Arg>
static constexpr bool _is_valid_arg() {
if constexpr (ECS::is_view_type<Arg>::value) {
return true;
}
if constexpr (std::is_same<Arg, InputEvents*>::value) {
return true;
}
return false;
}
template <typename SystemType, typename... Args>
void _register_system(std::function<void(Args...)> system) {
InjectedCallable<Args...> f(system);
_systems.push_back(f);
}
void RunSystems(InputEvents* events) {
for (auto func : _systems) {
DerivedScene* scene = static_cast<DerivedScene*>(this);
func(std::make_tuple(scene, _ecs_world, events));
}
}
}
We finally made it back to BaseScene. The Callable has been created. The System is down. It’s over.
The Good, The Bad, The Terrible
(I’m not proud, I am a little bit)
Would I recommend doing this? Maybe not, at least not if you have other options. It worked. It solved a problem. Maybe it’s not the worst to read. Ok it is pretty bad. There probably is a problem out there where this is useful. There is probably a lot of problems out there where this is just not the best idea. A terrible idea even.
What does this do well?
- Types are validated at compile time and there’s no runtime errors.
- Systems can make use of any number of different parameters.
- Data available to systems can be controlled.
- Subjectively, the Matt creating a Scene has a fairly easy
RegisterSystemAPI. - We can create multiple ECS::View objects for reasons unknown to me. But we could.
So, why would I hesitate to recommend it?
- The static_assert errors are not very helpful. They need to point someone to which Type is wrong.
- Testing this code is not enjoyable. We need to see if code compilers or doesn’t for pass/fail. Essentially we have a “shell test” that runs a command to see if the command succeeds when desired. Have fun integrating that with different toolchains (gcc, clang, msvc) or build systems (make, bazel).
- Not a lot of people are going to want to understand or maintain this code. Being able to understand it is one thing, but wanting to understand it is a different story. There’s always tradeoffs. Maintainability. Performance. Meets requirements. Pick 1.
- I never measured the performance impact of wrapping systems into
std::function. There is most likely overhead for every frame, for every system. Hopefully it’s small. Once again, tradeoffs. - C++ template errors are a huge time sink. You’ll have pages of errors to sift through trying to spot the problem. This injector guarantees someone on your team will spend at least an hour trying to debug a compiler error.
What’s Less Terrible
Starting with dynamic things like InputEvents.
There’s very few amounts of things that are not going to be some member of BaseScene.
I wanted to abstract out SFML inputs in GameManager and use only those abstract events in a System.
I could have done that abstraction in a system. The rendering systems need to deal with SFML anyway.
I could have just used something like GetGameManager()->GetInputs(). Which leads us to Resources.
Storing Resources in a known EntityID like Bevy is quite common. Honestly methods on BaseScene are probably good enough. If we need some to write to Resources concurrently with other reads/writes, then storing in a concurrent ECS may be better. We could also just use locks on resources. Solving concurrency with ECS is going to be another post for another day.
Now for those ECS::View objects. The dynamically generated thorn in my side.
I would take the type alias solution and rearrange things to get the alias right next to the method.
Anyone working on GravitySystem can quickly see the contract. It’s the next line up.
Odds are low that someone will care about the contract’s type way down in RegisterSystem,
but smart IDEs are going to help.
Vim and Emacs users must be pretty good at memorizing otherwise they wouldn’t know how to exit.
Sorry nano folks, but I can’t really help.
using GravityViewContract = ECS::View<CGravity, ECS::Mutable<CMovement>, CSensors>;
void GravitySystem(GravityViewContract view) { }
void GameScene::Init() {
RegisterSystem(GravitySystem, world.CreateView<GravityViewContract>());
}
template<typename... Args>
void RegisterSystem(void(*system)(Args...), Args... args) {
auto fn = std::bind(system, args...);
_systems.push_back(fn);
}
Look how little code we have. Less code == less maintenance. It’s fairly readable. We still have templates, but a lot less of it. I don’t need an entire custom test suite just to validate the universe won’t collapse. It’s also going to be just as fast. It’s still compile time checked. Still going to be some overhead with std::bind, but hopefully tiny. This checks all the boxes I want except for Cool Factor.