Playing audio and particles

Topics covered

Networked audio | Networked particles | Animation Events

Usually, visual feedback can be expressed via syncing variables like Animator parameters, positions, and rotations. But sometimes we have the need to play sounds and particles, which are not types that can be automatically set to sync, or that we can send as arguments of Network Commands. So how to do it?

Our use case

This project has a lot of moments where particles and sounds need to play, and we used different strategies for different cases, depending on how fast, repeated, or slow the action is.

The simple way: controlling AudioSource or ParticleSystem directly

The most straightforward solution to play a sound is to use a Network Command. Using Commands, you can remotely invoke methods on AudioSource or ParticleSystem components.

To do that, you could simply open the coherence Configuration panel (from the CoherenceSync), and check the methods you're interested in.

The Config panel showing an object with an AudioSource

The Config panel showing an object with a Particle System

While this is a perfectly fine way of doing things, it requires you to call multiple Network Commands in case you wanted to play a sound and particles at the same time. This could lead to desynchronisation between sound and visuals.

As such, in this project we preferred compacting these calls into methods on their own that are invoked as one Network Command, often without parameters to minimize the data being sent across.

Triggering a sound associated to an action

Connected to the above, let's see how to create our own Network Commands to play sounds (or particles) as a result of an event that happened remotely.

For instance, the Keeper Robot has a series of voices that play whenever it is performing an action. The robots is always controlled by the Simulator, so we need to play sounds on the Clients' devices.

For these sounds, we isolated the sound-playing behavior into Commands of their own. At the end of the KeeperRobot.cs class, we have:

[Command] public void PlayHumSound() => soundHandler.Play(humLoop);
[Command] public void PlayVoiceSound() => soundHandler.Play(voices);
[Command] public void PlayConjure() => soundHandler.Play(objectConjure);
[Command] public void PlayAppear() => soundHandler.Play(objectAppear);

(soundHandler is a script attached to the same gameObject)

Each of these methods is invoked as a Network Command, like so:

_sync.SendCommand(nameof(PlayVoiceSound), MessageTarget.Other);

You can see how we don't play the sound over the network, that would be bandwidth-consuming for no reason, but we just communicate the intention to play it.

Alternative: building an index

Because we only have 4 sounds, we sort of "brute-forced" this, and created an individual Network Command for each sound. This is not a bad idea from the point of view of network traffic: sending a Network Command with no parameter produces less traffic than sending one with.

But it could be unwieldy if we had - say - 100 different sounds to play.

This solution also requires us to bake and produce a new schema if we add or remove one of these Commands. So for a more flexible solution, it could be nice to index the sounds and maybe create a generic Command like:

[Command] public void PlaySound(int id) => soundHandler.Play(sounds[id]);

In this case though, it was ok to go for individual Commands.

Audio and particles tied to animations

There are actions that are really quick or short, and asking to play a sound via a command might result in a mismatch between the visuals (an animation) and the sound, due to network delay.

For instance, it wouldn't make sense to send a Command to inform other Clients to play the sound of a footstep. Chances are, by the time they receive the Command, another two-three footsteps have happened.

So for footsteps, jump, landing, and more; we used a slightly different strategy. Audio and particles are all played locally as part of the animation, using Unity's own Animation Events.

Three Animation Events play sounds and particles in the Walk Animation Clip

A script called PlayAnimationEvents.cs (remember to add it to the same object as the Animator!) listens to these events. An example from it:

public void PlayJumpEffects()
{
    soundHandler.Play(jumpSFX);
    jumpParticles.Play();
    StopRunParticles();
}

This ensures an immediate playback, in sync with the animation. Plus, it produces zero network traffic.

So yes, fun fact: to "network" sounds and particles often you can do without networking anything at all!

One more trick! If you have a state machine blending several clips, you might hear multiple overlapping sounds when a transition happens. One less known trick is to measure the weight of each clip while executing Animation Events, like we do below:

public void PlayStepSound(AnimationEvent evt)
{
    if (evt.animatorClipInfo.weight > 0.5)
    {
        soundHandler.Play(footstepSFX);
    }
}