Agent actions as entities

If you follow this blog, you may know that we’re currently working on a new simulation game but we’re totally using pure ECS this time. We have just decided on the game name this week. We’re calling it City Hall Simulator! I hope it’s clear what the game entails from the game name alone.

Prototype of the game

We don’t have a Steam page yet as it needs company details and bank information which we don’t have at the moment. We’ll put it up when the game is more developed. For the mean time, you can join our Discord channel. Youtube and Twitter will be up as soon as we have video content to share. With that out of the way, let’s get to why you’re really reading this post!

One of the things that we needed to figure out during prototyping of the game was the implementation of the AI. We’re still going to use the same GOAP system that we used in Academia: School Simulator but translated to Burst compiled ECS that can run in parallel. This is important for us because we need the AI to be as fast as it can be. From my experience in the previous game, AI is the slowest part. I wasn’t able to use DOTS for it because it’s very hard to translate a very expansive OOP code into pure value type ECS that we might as well rewrite the entire game. That’s not an option as the game was already in Early Access for a year already. And so the GOAP system in Academia remained in single threaded OOP land.

So for the new game, we need to have a working GOAP framework that’s Burst compiled and can run in parallel right of the bat. This code is going to be used from start to finish!

I’m not going to discuss the GOAP code that we came up with, though. That would be too long. Instead, I’m going to share to you an ECS pattern that worked for us. The gist is that we modeled the agent actions as separate entities. Buckle up because this is going to be a long wild ride.

Why separate entities for actions

The main reason is it reduces chunk space usage for entities. Actions may have their own data and parameters. If the components of such actions are added to the agent entity, that will add to the archetype size of such entity. Having too few entities for each chunk is not good when you’re iterating on them. Games like ours can have agents that have hundreds of actions. There may come a point where a chunk only contains a single entity if we add the actions components to the agent entity. Not good.

Had we opted for action components in the agent entity, we can’t have multiple actions with the same action type. Sure this can be solved by using DynamicBuffer but that’s another can of worms. How do we decide on the InternalBufferCapacity of each action type? If we do choose a certain number then that amount of items will take up space in the chunk even when the actual number of prepared actions for the agent is less than the capacity.

Having a separate entity for actions would also be more flexible. We can add any number of other components that they might need for their execution.

A simple case study

Let’s say we’re modelling a simple AI system where agents execute actions one after another. After the last action is executed, it just loops back to the first action. An example would be a worker agent that mines gold. This would be its sequence of actions:

  • Move to mine
  • Do mining in the mine
  • Carry gold
  • Move to castle
  • Drop gold

Let’s call this the Sequence AI.

Action Entities

An action would be a separate entity that has a common component and a filter component. The common component would contain the common data that all Sequence AI actions need. Let’s call this component SequenceAction:

public struct SequenceAction : IComponentData {
    public readonly Entity agentEntity;
    public readonly int actionIndex;

    public ActionResult result;
    
    public bool isExecuting;
    public bool started;

    public SequenceAction(Entity agentEntity, int actionIndex) : this() {
        this.agentEntity = agentEntity;
        this.actionIndex = actionIndex;
    }
}

This component needs the agentEntity so it can get data from the entity that is executing the action. The actionIndex is needed so the action knows if it can execute or not. I’ll show this later.

ActionResult is implemented like this:

public enum ActionResult {
    Success,
    Failed,
    Running
}

This is needed so we know if the action is done or not. Success means to move to the next action. Failed would mean to execute the action again. This varies for different systems. In GOAP, an action returning fail would mean to replan again. Running means that the action is not done yet. This is useful for actions that run in multiple frames like Move, Rotate, PlayAnimation, etc.

The boolean flag isExecuting is self explanatory. The boolean flag started is used to check if the action has started already. This is needed because we differentiate Start() from Update(). This will be discussed more on the action processor later.

Aside from SequenceAction, the action entity would also need a filter component. It’s called a filter component as it will be used as a query in a system to only select such action entities. Framed in another way, these are the components that defines the action. For the purposes of our case study here, we’re going to need the following filter components:

  • MoveTo – moves the agent to a certain place
  • PlayAnimation – plays an animation clip
  • AddItem – adds an item to the agent’s inventory
  • RemoveItem – removes an item from the agent’s inventory

Before we implement these components, it’s useful to create an interface to identify that such components are filter actions to our Sequence AI framework. Every filter component should implement this interface. It just looks like this:

public interface ISequenceActionComponent : IComponentData {
}

In our current game, we already have different frameworks that use this pattern and there was a point where it became confusing to identify which framework a certain component belongs. The fix is to use identifying interfaces like ISequenceActionComponent.

The following are the code of each filter component:

public struct MoveTo : ISequenceActionComponent {
    public float3 start;
    public float3 destination;
    
    public Timer timer;
    
    public readonly Place place;

    public MoveTo(Place place) : this() {
        this.place = place;
    }
}

// Struct enum for different places
public readonly struct Place : IEquatable<Place> {
    public static readonly Place CASTLE = new Place(1);
    public static readonly Place MINES = new Place(2);
    
    public readonly byte id;

    public Place(byte id) {
        this.id = id;
    }

    // Implementation for IEquatable is omitted for brevity
}
public readonly struct PlayAnimation : ISequenceActionComponent {
    public readonly AnimationClipId clipId;

    public PlayAnimation(AnimationClipId clipId) {
        this.clipId = clipId;
    }
}
public readonly struct AddItem : ISequenceActionComponent {
    public readonly ItemId itemId;
    public readonly int count;

    public AddItem(ItemId itemId, int count) {
        this.itemId = itemId;
        this.count = count;
    }
}

public readonly struct RemoveItem : ISequenceActionComponent {
    public readonly ItemId itemId;
    public readonly int count;

    public RemoveItem(ItemId itemId, int count) {
        this.itemId = itemId;
        this.count = count;
    }
}

// Struct enum for different items. Only has GOLD for our case study.
public readonly struct ItemId : IEquatable<ItemId> {
    public static readonly ItemId GOLD = new ItemId(1);
    
    public readonly byte id;

    public ItemId(byte id) {
        this.id = id;
    }

    // Omitted implemenation of IEquatable for brevity
}

The member variables of each filter component are self explanatory. We will see their use when we implement the systems that contains the logic for each action.

To create an action entity, we just add SequenceAction component and at least one filter component. More components could be added to the action entity if you need to. You may have some existing systems that you want to reuse but you need a certain component to be added for it to work. I will show later that this pattern can support these other components. Here’s a sample code on how to make an action entity:

Entity agent = entityManager.CreateEntity(agentArchetype);

Entity moveAction = entityManager.CreateEntity(typeof(SequenceAction), typeof(MoveTo));
entityManager.SetComponentData(moveAction, new SequenceAction(agent, 0)); // Zero here because it's the first action
entityManager.SetComponentData(moveAction, new MoveTo(Place.MINES));

Agent Entity

The agent entity will require a component to identify it as an entity that can execute SequenceActions. We will call this component SequenceActionAgent. This is how it looks like:

public struct SequenceActionAgent : IComponentData {
    public readonly int actionCount;
    public int currentActionIndex;
    public ActionResult lastResult;

    public SequenceActionAgent(int actionCount) {
        this.actionCount = actionCount;
        this.currentActionIndex = 0;
        this.lastResult = ActionResult.Success;
    }
}

We need the actionCount so we know when to point action index back to the first action by using the modulo operator. The variable currentActionIndex explains itself. The variable lastResult is needed so we know if the agent would move to the next action or not.

Our worker agent also requires other components.

public readonly struct Movement : IComponentData {
    public readonly float speed;

    public Movement(float speed) {
        this.speed = speed;
    }
}

The Movement component just holds the move speed of the agent.

public struct Animation : IComponentData {
    public int pendingClipIdToPlay;

    public bool playing;

    public void Play(int clipId) {
        this.pendingClipIdToPlay = clipId;
    }

    public bool IsPlaying {
        get {
            return this.playing;
        }
    }
}

This is the Animation component. Let’s just assume that there’s a different framework that’s handling the animation. All we have to do is call Play().

[InternalBufferCapacity(10)]
public struct InventoryItem : IBufferElementData {
    public int count;
    
    public readonly ItemId itemId;

    public InventoryItem(ItemId itemId, int count = 1) {
        this.itemId = itemId;
        this.count = count;
    }
}

InventoryItem is implemented as a buffer element so each entity can hold multiple items. Let’s just say that each entity can hold up to 10 different items.

Our worker agent would be prepared like this:

EntityArchetype agentArchetype = entityManager.CreateArchetype(
    typeof(Translation), 
    typeof(Animation), 
    typeof(InventoryItem),
    typeof(Movement),
    typeof(SequenceActionAgent));

Entity agent = entityManager.CreateEntity(agentArchetype);

// Random speed
Random random = new Random((uint)DateTime.Now.GetHashCode());
entityManager.SetComponentData(agent, new Movement(random.NextFloat(5, 10)));

The systems

We haven’t really started with the meat of the article. This is the part where we code the logic and use the components that we’ve laid out so far. First is we need a system that will identify if a SequenceAction is executing or not.

public class IdentifyExecutingSequenceActionsSystem : JobSystemBase {
    private EntityQuery actionsQuery;

    protected override void OnCreate() {
        this.actionsQuery = GetEntityQuery(typeof(SequenceAction));
    }

    protected override JobHandle OnUpdate(JobHandle inputDeps) {
        Job job = new Job() {
            sequenceActionType = GetComponentTypeHandle<SequenceAction>(),
            allAgents = GetComponentDataFromEntity<SequenceActionAgent>()
        };
        
        return job.ScheduleParallel(this.actionsQuery, 1, inputDeps);
    }
    
    [BurstCompile]
    private struct Job : IJobEntityBatch {
        public ComponentTypeHandle<SequenceAction> sequenceActionType;

        [ReadOnly]
        public ComponentDataFromEntity<SequenceActionAgent> allAgents;
        
        public void Execute(ArchetypeChunk batchInChunk, int batchIndex) {
            NativeArray<SequenceAction> actions = batchInChunk.GetNativeArray(this.sequenceActionType);

            for (int i = 0; i < actions.Length; ++i) {
                SequenceAction action = actions[i];
                SequenceActionAgent agent = this.allAgents[action.agentEntity];

                action.isExecuting = agent.currentActionIndex == action.actionIndex;
                action.started = false; // We set this to false so that Start() would be called
                actions[i] = action; // Modify
            }
        }
    }
}

This system is very simple. It just traverses all action entities and sets isExecuting to true if its actionIndex is equal to its agent’s currentActionIndex. If an action’s actionIndex is not equal to its agent’s currentActionIndex, that means that the it’s not the time for the action to execute yet.

The Action Base System

After identifying which action entities can execute, the systems that will execute such actions will run next. To make the maintainability of such systems better, we’re going to make an abstract base system class such that all action systems can just derive from and just provide the action’s logic. First, let’s define an interface for a processor for the actions:

public interface ISequenceActionProcessor<T> where T : struct, ISequenceActionComponent {
    // Routines before chunk iteration. Methods like ArchetypeChunk.GetNativeArray()
    // can be called here.
    void BeforeChunkIteration(ArchetypeChunk batchInChunk, int batchIndex);
    
    // Start routines. Called when SequenceAction was not yet started. Always called before Update().
    // ActionResult is used to check whether the action is done or not.
    ActionResult Start(in Entity agentEntity, ref T actionComponent, int indexOfFirstEntityInQuery, int iterIndex);

    // Update routines. Called every frame.
    // ActionResult is used to check whether the action is done or not.
    ActionResult Update(in Entity agentEntity, ref T actionComponent, int indexOfFirstEntityInQuery, int iterIndex);
}

Every sequence action system will need to provide an implementation of this interface. I’ll show an example later. Treat this as required methods that action systems need to implement.

Start() would be invoked when SequenceAction.started is false. Initialization or preparation code of the action can be placed here. Note that it has ActionResult as a return type. Some actions need not execute in multiple frames. You just return ActionResult.Success right away when the action is done. This is used for actions that just calls something or sets something and is done.

When Start() returns ActionResult.Running, this means that the action runs in multiple frames. From hereon, Update() would be invoked in each frame until the result is Success or Failed.

The parameters indexOfFirstEntityInQuery and iterIndex would usually be used to create a sortKey to be passed to EntityCommandBuffer.ParallelWriter methods if the action requires it. You will see in the abstract base class system to be shown later where these values come from. An action that uses EntityCommandBuffer.ParallelWriter might use such parameters like this:

public ActionResult Start(in Entity agentEntity, ref T actionComponent, int indexOfFirstEntityInQuery, int iterIndex) {
    // Say the action would create a new entity
    int sortKey = indexOfFirstEntityInQuery + iterIndex;
    this.commandBuffer.CreateEntity(sortKey, this.bulletArchetype);
}

BeforeChunkIteration() is used to extract NativeArrays of other components that were added to the action component aside from SequenceAction and the filter component. You will need access to such arrays because only the filter component is passed to the methods Start() and Update(). You can get access to the arrays of the other components like this:

// This will be set during preparation of the processor struct
public ComponentTypeHandle<SomeOtherComponent> otherComponentType;

// We use NativeDisableContainerSafetyRestriction here so that Unity will not complain that
// this is not set upon creation. The value of this variable would only be populated in 
// BeforeChunkIteration()
[NativeDisableContainerSafetyRestriction]
private NativeArray<SomeOtherComponent> otherComponents;

public void BeforeChunkIteration(ArchetypeChunk batchInChunk, int batchIndex) {
    this.otherComponents = batchInChunk.GetNativeArray(this.otherComponentType);
}

The parameter iterIndex in Start() or Update() can then be used as index to these arrays of other components that you have.

Let’s implement the actual base class for the action execution of our Sequence Actions framework:

[UpdateBefore(typeof(MoveToNextActionSystem))]
    public abstract class SequenceActionBaseSystem<TActionFilter, TProcessor> : JobSystemBase
    where TActionFilter : struct, ISequenceActionComponent
    where TProcessor : struct, ISequenceActionProcessor<TActionFilter> {

    private EntityQuery query;
    protected bool isActionFilterHasArray;

    protected override void OnCreate() {
        this.query = PrepareQuery();
        
        // Action has array if it's not zero sized
        this.isActionFilterHasArray = !TypeManager.GetTypeInfo(TypeManager.GetTypeIndex<TActionFilter>()).IsZeroSized;
    }

    protected virtual EntityQuery PrepareQuery() {
        return GetEntityQuery(typeof(SequenceAction), typeof(TActionFilter));
    }

    protected override JobHandle OnUpdate(JobHandle inputDeps) {
        ExecuteActionsJob job = new ExecuteActionsJob() {
            sequenceActionType = GetComponentTypeHandle<SequenceAction>(),
            actionFilterType = GetComponentTypeHandle<TActionFilter>(),
            isActionFilterHasArray = this.isActionFilterHasArray,
            processor = PrepareProcessor()
        };

        JobHandle handle = this.ShouldScheduleParallel ? 
            job.ScheduleParallel(this.query, 1, inputDeps) : job.Schedule(this.query, inputDeps);
        
        AfterJobScheduling(handle);

        return handle;
    }

    public struct ExecuteActionsJob : IJobEntityBatchWithIndex {
        public ComponentTypeHandle<SequenceAction> sequenceActionType;
        public ComponentTypeHandle<TActionFilter> actionFilterType;
        public bool isActionFilterHasArray;
        public TProcessor processor;

        [NativeDisableParallelForRestriction]
        public ComponentDataFromEntity<SequenceActionAgent> allAgents;
        
        public void Execute(ArchetypeChunk batchInChunk, int batchIndex, int indexOfFirstEntityInQuery) {
            NativeArray<SequenceAction> sequenceActions = batchInChunk.GetNativeArray(this.sequenceActionType);
            NativeArray<TActionFilter> actionFilters = this.isActionFilterHasArray ? batchInChunk.GetNativeArray(this.actionFilterType) :
                default;
            TActionFilter defaultActionFilter = default; // This will be used if TActionFilter has no chunk (it's a tag component)
            
            this.processor.BeforeChunkIteration(batchInChunk, batchIndex);

            for (int i = 0; i < batchInChunk.Count; ++i) {
                SequenceAction sequenceAction = sequenceActions[i];
                if (!sequenceAction.isExecuting) {
                    // Not currently executing
                    continue;
                }
                
                if (this.isActionFilterHasArray) {
                    TActionFilter actionFilter = actionFilters[i];
                    ExecuteAction(ref sequenceAction, ref actionFilter, indexOfFirstEntityInQuery, i);
                    actionFilters[i] = actionFilter; // Modify
                } else {
                    // There's no array for the TActionFilter. It must be a tag component.
                    // Use a default filter component
                    ExecuteAction(ref sequenceAction, ref defaultActionFilter, indexOfFirstEntityInQuery, i);
                }

                if (sequenceAction.result == ActionResult.Failed) {
                    // We set started to false here so that Start() will be invoked again
                    sequenceAction.started = false;
                }

                sequenceActions[i] = sequenceAction; // Modify
                
                // Modify the lastResult of the agent
                SequenceActionAgent agent = this.allAgents[sequenceAction.agentEntity];
                agent.lastResult = sequenceAction.result;
                this.allAgents[sequenceAction.agentEntity] = agent; // Modify
            }
        }

        private void ExecuteAction(ref SequenceAction sequenceAction, ref TActionFilter actionFilter,
            int indexOfFirstEntityInQuery, int iterIndex) {
            // Call Start() if action haven't started yet
            if (!sequenceAction.started) {
                // Call Start() if not yet started
                sequenceAction.result = this.processor.Start(sequenceAction.agentEntity, ref actionFilter,
                    indexOfFirstEntityInQuery, iterIndex);
                sequenceAction.started = true;

                if (sequenceAction.result == ActionResult.Success || sequenceAction.result == ActionResult.Failed) {
                    // Action is done if the result is success or failed
                    return;
                }
            }
            
            // Call Update()
            sequenceAction.result = this.processor.Update(sequenceAction.agentEntity, ref actionFilter,
                indexOfFirstEntityInQuery, iterIndex);
        }
    }

    protected abstract TProcessor PrepareProcessor();

    protected virtual void AfterJobScheduling(in JobHandle handle) {
        // Routines like calling AddJobHandleForProducer() may be placed here
    }

    /// <summary>
    /// There may be times that the action system might not want to schedule in parallel
    /// Like for cases when they write using ComponentDataFromEntity
    /// </summary>
    protected virtual bool ShouldScheduleParallel {
        get {
            return true;
        }
    }

    protected ref readonly EntityQuery Query {
        get {
            return ref this.query;
        }
    }
}

So this is a bit lot. This base system is a generic class that requires 2 types – the filter action and the processor (ISequenceActionProcessor). We need the type of the processor because it will be supplied in a job which you know can’t have reference types. As specified in the discriminator, TProcessor should be a struct that implements ISequenceActionProcessor<TActionFilter>.

Note also that we use chunk iteration here instead of Entities.ForEach because Entities.ForEach does not work for generic systems. This is one of the reasons why we always use chunk iteration. We only use Entities.ForEach for very simple systems.

In OnCreate(), we prepare the query by calling a virtual method called PrepareQuery() which already has a default implementation of querying all entities that has SequenceAction and the filter action component. A concrete action system subclass can override this to provide a different query. This is rarely used but I know that we needed to provide a different query in some cases. That’s why this virtual method was provided.

We also prepared a boolean called isActionFilterHasArray. This is needed during iteration of the action entities in the job. We need this distinction because if the filter action type is just a tag component (no member variables), the Entities API doesn’t allow to call GetNativeArray() for such components. If you do, it will throw an exception. The action filter has an array if it’s not zero sized.

In OnUpdate(), we schedule ExecuteActionsJob which we will explore next. During scheduling we call a virtual getter called ShouldScheduleParallel. This returns true by default but sometimes, you can’t always execute actions in parallel because the logic broke a fundamental rule on when jobs can run in parallel. I’ve wrote about this sometime ago.

Another virtual method that is invoked is AfterJobScheduling(JobHandle). This is used for cases like when the action system uses an EntityCommandBufferSystem. You need to call AddJobHandleForProducer().

ExecuteActionsJob

Let’s get into the real action. Here are the member variables that the job requires:

  • sequenceActionType – Needed so we can get the NativeArray<SequenceAction> from chunks
  • actionFilterType – Needed so we can get the NativeArray<TActionFilter> from chunks
  • isActionFilterHasArray – As discussed, we need this so we know if we can call GetNativeArray() for the filter action or not
  • processor – This is the struct that implements ISequenceActionProcessor. We call its methods in the job. This is set using the PrepareProcessor() abstract method.
  • allAgents – After an action executes, we write the ActionResult to SequenceActionAgent.lastResult

In Execute(), first we get the native arrays of components from chunks. This is standard for chunk iteration. You’ll see the usage of isActionFilterHasArray here. We call GetNativeArray() from the filter action if this is true. Otherwise, it is assigned a default value. We’re not going to use this array if isActionFilterHasArray is false anyway.

We then call ISequenceActionProcessor.BeforeChunkIteration() so that such processor can retrieve what other components from the chunk it needs.

Then we iterate through the action components. Right of the bat, we check if sequenceAction.isExecuting is false which was assigned in IdentifyExecutingSequenceActionsSystem. This means that the action is not currently executing, so we skip it. You may wonder if this is wasteful. From our experience, it’s generally fine. They still run fast even with multiple action systems. Moreover, this is the advice of Joachim Ante of Unity in the DOTS forums. This is what they do internally. This is also in preparation for the feature where components can be enabled/disabled. When that feature becomes available, isExecuting will be refactored into a component and will be enabled/disabled in IdentifyExecutingSequenceActionsSystem. This IsExecuting component will be added to the EntityQuery of this system.

Next is we do an if/else checking for isActionFilterHasArray. If it’s true, we get the value from the NativeArray of filter actions and pass it to ExecuteAction(). The array entry for the filter action is modified as well. If it’s false, a default value is used instead.

Next is we check if the result of action execution is Failed. SequenceAction.result would be set in ExecuteAction(). If the result was Failed, we set started to false so the action would be given the chance to execute again (let’s just say this is our design for this AI framework). The logic varies here in different frameworks. In our GOAP framework, a failed result would mean a replan for a new set of actions.

When calling ExecuteAction(), notice the values passed for indexOfFirstEntityInQuery and iterIndex. The value of indexOfFirstEntityInQuery is the value passed from IJobEntityBatch.Execute() and iterIndex is the index while iterating the components in the chunk. In Start() or Update(), the processor can just use iterIndex to index the other components that it retrieved in BeforeChunkIteration().

Let’s jump to the ExecuteAction() method for now. This is the method that invokes ISequenceActionProcessor.Start() and ISequenceActionProcessor.Update() properly. If SequenceAction.started is false, then it means it hasn’t started yet. We call Start() and assign the result to SequenceAction.result. We set started to true so Start() will not be invoked again. After invoking Start(), we check right away if the result is Success or Failed. If so, the action is done. There’s no need to call Update(). If it’s not Success nor Failed, it can only be Running. The next line of code invokes Update() and stores the result to SequenceAction.result.

After calling ExecuteAction(), we modify the value of the SequenceAction to its array. Notice how we add a comment “// Modify” for each NativeArray modification. This is because we always forgot that these are value types and their values must be assigned back to the array if we are to retain their state/values. It’s not like for classes where you can just call the method that does the mutation and you’re done.

Lastly, we set the action result to SequenceActionAgent.lastResult. This will be needed on the next system that moves the currentActionIndex to the next one.

The remaining methods

After the code of ExecuteActionsJob, you’ll see an abstract method called PrepareProcessor() that needs to return TProcessor. This means that the action systems that derives from this base class needs to implement this method. The action systems will provide the custom logic for Start(), Update(), and BeforeChunkIteration(). I’ll show an example action system class later.

AfterJobScheduling() and ShouldScheduleParallel were already mentioned.

A utility getter called Query is provided for cases when the action subclass needs it. An action subclass may need it for cases when it will override OnUpdate() itself because it needs to schedule a different custom job prior to scheduling ExecuteActionsJob. In one of our use case, we have an action that needs a NativeHashMap of some entities. It needs to schedule a job that populates this NativeHashMap first which is then set to the processor. That processor is then set ExecuteActionsJob which will be scheduled next.

Why make this base system?

As you can see, there’s already a lot here. You don’t want to be repeating such code whenever you want to make an action system. It’s also an aide for maintenance. Someday, you may want to change the rules of what happens depending on the ActionResult. Instead of applying such change to each action system, you just apply it to the base system.

Sample Action Systems

Back to our worker case study, let’s make the action system for the MoveTo action:

public class MoveToSystem : SequenceActionBaseSystem<MoveTo, MoveToSystem.Processor> {
    protected override Processor PrepareProcessor() {
        return new Processor() {
            allTranslations = GetComponentDataFromEntity<Translation>(),
            positionsMap = ResolvePlacesPositionsMap(),
            allMovements = GetComponentDataFromEntity<Movement>(),
            deltaTime = UnityEngine.Time.deltaTime
        };
    }
    
    public struct Processor : ISequenceActionProcessor<MoveTo> {
        [NativeDisableParallelForRestriction]
        public ComponentDataFromEntity<Translation> allTranslations;

        [ReadOnly]
        public NativeHashMap<Place, float3> positionsMap;

        [ReadOnly]
        public ComponentDataFromEntity<Movement> allMovements;

        public float deltaTime;
        
        public void BeforeChunkIteration(ArchetypeChunk batchInChunk, int batchIndex) {
        }

        public ActionResult Start(in Entity agentEntity, ref MoveTo actionComponent, int indexOfFirstEntityInQuery,
            int iterIndex) {
            // Identify start and destination
            Translation translation = this.allTranslations[agentEntity];
            actionComponent.start = translation.Value; // Current position is the start
            
            // Position of the place is the destination
            actionComponent.destination = this.positionsMap[actionComponent.place];

            if (actionComponent.start.Equals(actionComponent.destination)) {
                // Agent is already at the destination. No need to move.
                return ActionResult.Success;
            }
            
            // Compute the duration based on speed
            Movement movement = this.allMovements[agentEntity];
            float duration = math.distance(actionComponent.destination, actionComponent.start) / movement.speed;
            actionComponent.timer = new Timer(duration);
            
            return ActionResult.Running;
        }

        public ActionResult Update(in Entity agentEntity, ref MoveTo actionComponent, int indexOfFirstEntityInQuery,
            int iterIndex) {
            actionComponent.timer.Update(this.deltaTime);

            if (actionComponent.timer.HasElapsed) {
                // Movement is done. Snap to destination.
                this.allTranslations[agentEntity] = new Translation() {
                    Value = actionComponent.destination
                };
                
                return ActionResult.Success;
            }
            
            // Movement is still ongoing. Lerp position.
            float3 newPosition = math.lerp(actionComponent.start, actionComponent.destination, actionComponent.timer.Ratio);
            this.allTranslations[agentEntity] = new Translation() {
                Value = newPosition
            };
            
            return ActionResult.Running;
        }
    }

    private NativeHashMap<Place, float3> ResolvePlacesPositionsMap() {
        // Let's just assume you have this somewhere
        return ...;
    }
}

In our codebase, we’re using {ActionComponentName}System as the format for the action system classes. The processor type can just be a nested struct of the system. This way, we don’t have to create a separate source file for such struct. The action system would otherwise be empty anyway so might as well use the space for the code of the action processor.

The gist of this move action is takes the distance between start and destination then computes the duration using the agent’s move speed. The action’s timer ratio is then used to lerp the position of the agent along the start and destination positions.

What I like about this pattern is that the processor can contain any number and combination of data that the action needs that can be coming from anywhere. For this action system, we need the following data:

  • allTranslations – Needed because the action will be moving the agent
  • positionsMap – The mapping of Place to its position. Let’s assume we have this somewhere else in the code.
  • allMovements – We’ll get the speed of the agent here
  • deltaTime – Used for moving the translation by speed

BeforeChunkIteration() is left empty because we don’t need any other component other than the filter action component (MoveTo).

At Start(), we initialize the variables of MoveTo. The current position is the starting position. The destination is the position of MoveTo.place.

Then we check for an early out. If the start position and destination are equal, then this means that the agent is already at the target place. We return Success to stop the action and move to the next one. If not we continue with rest of the code.

Next is we prepare the duration of the movement via calculation using the agent’s move speed. The duration is simply derived from common speed formula V = d / t. So t (time) is t = d / V. We’re using the speed value from the Movement component of the agent. We then set this duration time to MoveTo’s timer and return Running.

In Update(), we update the timer first with the passed deltaTime. Then we check if it has elapsed. This means that the movement is done. We snap the agent to the destination position and return Success. Otherwise, we set an interpolated position from start to destination using the ratio of the timer and return Running as the action is not yet done.

The actions of this pattern will always get some data from the owner agent of the action. Most of the time, we can run the base action job in parallel if the data we write to is only from the owning agent. Once there’s another set of entity reference that points to another entity other than the owning agent, then we can’t guarantee safety of writing to that other entity. We must override ShouldScheduleParallel to return false in this case. This is because multiple action entities might write to the data pointed to that other entity. This is not safe when run in parallel and must be avoided.

After implementing the Processor struct, we then implement the abstract method PrepareProcessor() which is required by SequenceActionBaseSystem. What we do in this method is just prepare the Processor struct supplying all its required data. Note that this processor is then used to run SequenceActionBaseSystem.ExecuteActionsJob.

I’ll show the action systems for PlayAnimation and AddItem but will no longer discuss them in depth as they are quite simple.

public class PlayAnimationSystem : SequenceActionBaseSystem<PlayAnimation, PlayAnimationSystem.Processor> {
    protected override Processor PrepareProcessor() {
        return new Processor() {
            allAnimations = GetComponentDataFromEntity<Animation>()
        };
    }

    public struct Processor : ISequenceActionProcessor<PlayAnimation> {
        [NativeDisableParallelForRestriction]
        public ComponentDataFromEntity<Animation> allAnimations;
        
        public void BeforeChunkIteration(ArchetypeChunk batchInChunk, int batchIndex) {
        }

        public ActionResult Start(in Entity agentEntity, ref PlayAnimation actionComponent, int indexOfFirstEntityInQuery,
            int iterIndex) {
            Animation animation = this.allAnimations[agentEntity];
            animation.Play(actionComponent.clipId.intId);
            this.allAnimations[agentEntity] = animation; // Modify
            
            return ActionResult.Running;
        }

        public ActionResult Update(in Entity agentEntity, ref PlayAnimation actionComponent, int indexOfFirstEntityInQuery,
            int iterIndex) {
            Animation animation = this.allAnimations[agentEntity];
            return animation.IsPlaying ? ActionResult.Running : ActionResult.Success;
        }
    }
}

This is very easy. It calls Play() and modifies the Animation component at Start(). It waits until the animation is no longer playing before returning Success in Update() (return when animation is finished).

public class AddItemSystem : SequenceActionBaseSystem<AddItem, AddItemSystem.Processor> {
    protected override Processor PrepareProcessor() {
        return new Processor() {
            allInventoryItemBuffers = GetBufferFromEntity<InventoryItem>()
        };
    }

    public struct Processor : ISequenceActionProcessor<AddItem> {
        [NativeDisableParallelForRestriction]
        public BufferFromEntity<InventoryItem> allInventoryItemBuffers; 
        
        public void BeforeChunkIteration(ArchetypeChunk batchInChunk, int batchIndex) {
        }

        public ActionResult Start(in Entity agentEntity, ref AddItem actionComponent, int indexOfFirstEntityInQuery,
            int iterIndex) {
            DynamicBuffer<InventoryItem> items = this.allInventoryItemBuffers[agentEntity];
            
            // Check for existing
            for (int i = 0; i < items.Length; ++i) {
                InventoryItem item = items[i];
                if (item.itemId != actionComponent.itemId) {
                    continue;
                }

                // Already exists, just add the count
                items[i] = new InventoryItem(item.itemId, item.count + actionComponent.count);
                return ActionResult.Success;
            }
            
            // At this point, there's no existing item. We add a new one.
            items.Add(new InventoryItem(actionComponent.itemId, actionComponent.count));
            
            return ActionResult.Success;
        }

        public ActionResult Update(in Entity agentEntity, ref AddItem actionComponent, int indexOfFirstEntityInQuery,
            int iterIndex) {
            return ActionResult.Success;
        }
    }
}

This time, we’re manipulating a DynamicBuffer. To add an item, we check from the DynamicBuffer of items if it already exists. If it does, just add the count to it. If it doesn’t exist yet, we add a new InventoryItem to the buffer.

The action system for RemoveItem would be left as an exercise. I’m sure you can already figure it out on your own.

MoveToNextActionSystem

The last system in our framework is the system that moves the action execution to the next. We do this by simply incrementing SequenceActionAgent.currentActionIndex. This is a very simple system that I’m inclined to use Entities.ForEach().

[UpdateAfter(typeof(IdentifyExecutingSequenceActionsSystem))]
    public class MoveToNextActionSystem : SystemBase {
        protected override void OnUpdate() {
            this.Entities.ForEach(delegate(ref SequenceActionAgent agent) {
                // Move when the last result is success
                if (agent.lastResult == ActionResult.Success) {
                    agent.currentActionIndex = (agent.currentActionIndex + 1) % agent.actionCount;
                }
            }).ScheduleParallel();
        }
    }

What the system does is it moves the currentActionIndex up by one or goes back to zero if it was already at the last index when the lastResult is Success. Why is this a separate system and not implemented in SequenceActionBaseSystem? It’s more efficient this way. Note that for this system, we’re traversing SequenceActionAgent components, not the actions. Cache misses are reduced here. The job that would be generated by this system would also run in parallel.

Notice also that in SequenceActionBaseSystem, we specified an attribute [UpdateBefore(typeof(MoveToNextActionSystem))]. This ensures that action systems execute before MoveToNextActionSystem.

Systems Ordering

The ordering of systems would look like this:

  • IdentifyExecutingSequenceActionsSystem
  • One or more action systems
    • MoveToSystem
    • PlayAnimationSystem
    • AddItemSystem
    • RemoveItemSystem
    • Other action systems
  • MoveToNextActionSystem

Assuming that an action returned Success, the value of SequenceActionAgent.currentActionIndex will be incremented or looped back to zero in MoveToNextActionSystem. On the next frame, when IdentifyExecutingSequenceActionsSystem runs again, it will now set those action entities that has actionIndex equal to the new currentActionIndex as the ones executing (isExecuting = true). Those action entities will be executed once their action systems executes. Then MoveToNextActionSystem checks again if it’s time to move to the next action.

And just like that, our framework is done. Phew!

Entities Preparation

With our Sequence Action AI sorted out, let’s see how we might prepare the agent entity and its action entities for our worker case study:

public class GameControllerSystem : SystemBase {
    protected override void OnCreate() {
        EntityManager entityManager = this.EntityManager;
        
        EntityArchetype agentArchetype = entityManager.CreateArchetype(
            typeof(Translation), 
            typeof(Animation), 
            typeof(InventoryItem),
            typeof(Movement),
            typeof(SequenceActionAgent));

        Entity agent = entityManager.CreateEntity(agentArchetype);

        // Random speed
        Random random = new Random((uint)DateTime.Now.GetHashCode());
        entityManager.SetComponentData(agent, new Movement(random.NextFloat(5, 10)));
        
        // Prepare actions
        int actionCount = 0;
        
        PrepareSequenceAction(ref entityManager, agent, actionCount, new MoveTo(Place.MINES));
        ++actionCount;
        
        PrepareSequenceAction(ref entityManager, agent, actionCount, new PlayAnimation(AnimationClipId.MINE));
        ++actionCount;
        
        PrepareSequenceAction(ref entityManager, agent, actionCount, new AddItem(ItemId.GOLD, 5));
        ++actionCount;
        
        PrepareSequenceAction(ref entityManager, agent, actionCount, new MoveTo(Place.CASTLE));
        ++actionCount;
        
        PrepareSequenceAction(ref entityManager, agent, actionCount, new RemoveItem(ItemId.GOLD, 5));
        ++actionCount;
        
        entityManager.SetComponentData(agent, new SequenceActionAgent(actionCount));
    }

    private static void PrepareSequenceAction<T>(ref EntityManager entityManager, in Entity agent, int actionIndex,
        T actionFilter) where T : struct, ISequenceActionComponent {
        Entity action = entityManager.CreateEntity(typeof(SequenceAction), typeof(T));
        entityManager.SetComponentData(action, new SequenceAction(agent, actionIndex));
        entityManager.SetComponentData(action, actionFilter);
    }
}

First we prepare an archetype for the agent which has all the components we need. We create the agent entity using that archetype and set a random move speed. The actions are prepared using the utility method PrepareSequenceAction() which creates an Entity and adds a SequenceAction component and the passed filter component. The actions are then created in order while maintaining an actionCount so we know the index of the action. The actions are as follows:

  • Move to mine
  • Play mining animation clip
  • Add 5 gold to inventory
  • Move to castle
  • Remove 5 gold from inventory

Finally, we set the SequenceActionAgent component of the agent which needs the action count.

In an actual production code, you don’t do it like this by hand. You should have some kind of editor that is used to lay out what specific actions are added to a certain agent type. Then a parser will then parse this data and prepares the agent entities and its action entities.

Parting Words

What a ride! This is probably the longest article that I have written here. So there you go, I just shared to you one of my major secrets. Note that this is a pattern. You probably should not use the code that was shown here. It’s just a demonstration. We’ve used this pattern a lot in our AI code. Aside from GOAP, we have used it for FSM and Utility AI. Apply it your problems when applicable. If you’re familiar with design patterns, this is probably like the Strategy Pattern but in ECS.

That’s all for now. See you on the next post!

If you liked what you just read, or this blog in general, or you just like me :), please subscribe to my mailing list. I’m giving free stuff sometimes.

2 thoughts on “Agent actions as entities

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s