Using entities with multiple components

[Driphter]

What would be a performant way to get entities with multiple components with the current implementation?

[Janiczek]

From looking at the source, it seems the best way is getting entities for all the components you want, and then intersecting the results. (ie. use get-all-entities-with-component and clojure.set/intersection)

[markmandel]

Interesting. Not something I've ever had a need of - out of curiosity, what is the use case?

[Driphter]

I'm early into messing around with my current project so it was just stuff like Renderable/Position, Camera/Position, etc. I suppose I could just do it as you have with your pong demo and assume that all components that are depended upon are always present. That would also be the more performant solution as well.

I do find it odd that I can't think of an example in which it would be required... I can't shake the feeling that a well-designed, flexible, and complex ECS would need something like this eventually.

Regardless, I messed around with it anyways and so I'll share what I've found. Btw, all of these are from my ECS wrapper namespace. All the derefs are because I'm using an atom for game-state and I avoid using brute.entity/get-component-type because I'm only using records as components. Also all tests are using a scene with 6505 Renderable/Position entities, 1 Light/Position entity, and 1 Camera/Position entity.

Baseline:

(brute/get-all-entities-with-component @gs Light)

Evaluation count : 346193400 in 60 samples of 5769890 calls.
             Execution time mean : 174.776005 ns
    Execution time std-deviation : 3.162045 ns
   Execution time lower quantile : 171.533099 ns ( 2.5%)
   Execution time upper quantile : 182.020147 ns (97.5%)

I tried using brute.entity/get-all-entities-with-component and clojure.core/intersection first. This was unfortunately super slow.

(defn get-by-components-1
  [gs type1 type2]
  (let [gs @gs]
    (clojure.set/intersection (set (brute/get-all-entities-with-component gs type1))
                              (set (brute/get-all-entities-with-component gs type2)))))

(entity/get-by-components-1 gs Position Light)

Evaluation count : 61860 in 60 samples of 1031 calls.
             Execution time mean : 965.888634 µs
    Execution time std-deviation : 12.187773 µs
   Execution time lower quantile : 934.779758 µs ( 2.5%)
   Execution time upper quantile : 985.124319 µs (97.5%)

Next, I tried some filter trickery. This actually worked pretty well. Nearly half the speed of the baseline!

(defn get-by-components-2
  [gs type1 type2]
  (let [ecs (:entity-components @gs)]
    (filter (get ecs type1)
            (keys (get ecs type2)))))

(entity/get-by-components-2 gs Position Light)

Evaluation count : 167541840 in 60 samples of 2792364 calls.
             Execution time mean : 359.338230 ns
    Execution time std-deviation : 4.064856 ns
   Execution time lower quantile : 351.859327 ns ( 2.5%)
   Execution time upper quantile : 365.904002 ns (97.5%)

Then I began to wonder how fast clojure.set/intersection would be if we maintained a map where the keys were the components and the values were sets of the enties that used them. Here's the modified brute.entity/add-component. There is of course a bit of overhead in latency and memory to consider.

(defn add-component
  [system entity instance]
  (let [type (brute/get-component-type instance)
        system (transient system)
        ctes (:component-type-entities system)
        ecs (:entity-components system)
        ects (:entity-component-types system)]
    (-> system
        (assoc! :component-type-entities (assoc ctes type (-> ctes (get type) (set) (conj entity))))
        (assoc! :entity-components (assoc-in ecs [type entity] instance))
        (assoc! :entity-component-types (update ects entity conj type))
        persistent!)))

(brute/add-component (clojure.core/deref gs) entity light)

Evaluation count : 85659780 in 60 samples of 1427663 calls.
             Execution time mean : 709.378615 ns
    Execution time std-deviation : 9.806567 ns
   Execution time lower quantile : 687.165829 ns ( 2.5%)
   Execution time upper quantile : 727.112794 ns (97.5%)

(entity/add-component (clojure.core/deref gs) entity light)

Evaluation count : 57816720 in 60 samples of 963612 calls.
             Execution time mean : 1.053072 µs
    Execution time std-deviation : 13.917718 ns
   Execution time lower quantile : 1.024814 µs ( 2.5%)
   Execution time upper quantile : 1.079301 µs (97.5%)

Here's using clojure.set/intersection. Even faster than the above filter trickery!

(defn get-by-components-cte-1
  [gs type1 type2]
  (let [ctes (:component-type-entities @gs)]
    (clojure.set/intersection (get ctes type1)
                              (get ctes type2))))

(entity/get-by-components-cte-1 gs Position Light)

Evaluation count : 218439840 in 60 samples of 3640664 calls.
             Execution time mean : 270.226822 ns
    Execution time std-deviation : 3.212845 ns
   Execution time lower quantile : 264.072620 ns ( 2.5%)
   Execution time upper quantile : 276.070598 ns (97.5%)

And the filter trickery is pretty much the same as before, sadly.

(defn get-by-components-cte-2
  [gs type1 type2]
  (let [ctes (:component-type-entities @gs)]
    (filter (get ctes type1) (get ctes type2))))

(entity/get-by-components-cte-2 gs Position Light)

Evaluation count : 163087260 in 60 samples of 2718121 calls.
             Execution time mean : 367.290088 ns
    Execution time std-deviation : 4.732959 ns
   Execution time lower quantile : 359.049880 ns ( 2.5%)
   Execution time upper quantile : 375.423474 ns (97.5%)

Also, getting entities by one component is now 3 times faster!

(defn get-by-component
  [gs type]
  (get (:component-type-entities @gs) type))

(entity/get-by-component gs Light)

Evaluation count : 1017556440 in 60 samples of 16959274 calls.
             Execution time mean : 56.774644 ns
    Execution time std-deviation : 0.696767 ns
   Execution time lower quantile : 55.733089 ns ( 2.5%)
   Execution time upper quantile : 57.901259 ns (97.5%)

I think for now I'm going to stick with getting entities by one component as you have and assume the required components are there for the sake of performance. I'm also going to use my :component-type-entities implementation for the same reason.

If you're interested in using this implementation, I can submit a PR with my changes (in the style of the project, of course).

[UnwarySage]

If I might add the reason for multicomponent filtering, consider a situation like so.

You have Position and Velocity components.

You are making a platformer, and want a system that inflicts gravity.
You get all the components with a position, and assume they have a velocity, and make them change their position and velocity accordingly. Great, enemies, the player and so on all fall appropriately.

Then you add a coin, a pickup. It has a Position, but it doesn't move, so it doesn't have a Velocity. That's a crash.
So you either make a new StaticPosition component, or put in a check to make sure that every entity with a Position has a Velocity with it before messing with gravity.
(Or I suppose you add a Velocity to the coin, but then you can't have them flying over gaps to show the way forward.)

That second approach backfires again if you add laser beams. They have position and velocity, but they shouldn't fall down.
(Negative filtering could be handy, just add a NoGravity component, and a corresponding negative filter in the system,)

In my opinion, it is kind of key for unlocking the biggest advantage of ECS, being able to create new entities out of arbitrary collections of components, and have the system/simulation do as much as it can with the data in the component, and not fall flat because it doesn't look as expected.

Sorry for the necrobump, but started using this and was surprised this wasn't a feature.