Two Faces of a Turn Exchange

Introduction

When has a person finished speaking? A voice agent answers that question many times a minute, and almost always from sound alone. The usual formulation is a classifier: given the last few seconds of the user's audio, or its transcript, decide whether the turn is over.^1,2,31Skantze, Turn-taking in conversational systems and human-robot interaction: a review, 20212Castillo-López et al., A survey of recent advances on turn-taking modeling in spoken dialogue systems, 2025 Some models listen to both parties,⁴ but they too only listen.

A conversation has two people in it, and both of them have faces. A long line of work on human interaction holds that the listener is not a passive receiver. Listeners shape a story while it is being told.⁵5Bavelas, Coates and Johnson, Listeners as co-narrators, 2000 And because people answer each other within about 200 ms, while planning even a single word takes about 600 ms, the next speaker has to be preparing while the current one is still talking.^6,76Levinson and Torreira, Timing in turn-taking and its implications for processing models of language, 2015 If that preparation can be seen, it is a signal that no microphone carries.

This post measures it. We tracked the faces in 2,670 recorded conversations from two public corpora, one filmed with both people in the same room and one recorded over video calls, and asked four questions:

• What does the listener's face do before the listener speaks?
• What does the speaker's face do when a turn ends, and how does that differ from a pause?
• Which of these cues survive the move from a room to a video call?
• Does one person's face change what the other person does next?

The short answers. Before taking a turn, the listener's mouth is half open 0.4 s ahead of the first word, against 0.1 s before a one-word “yeah”, and in 43 % of exchanges in the room and 49 % on the call it starts to open while the other person can still be heard talking. A speaker who is done has the jaw closed and the lips pressed 0.3 s after the last sound, while a speaker who is only pausing keeps the jaw half open, and the speaker's eyes return to the partner as the turn ends. All of this holds in the room and on the call. A few slower cues, a smile into the last word among them, appear in only one of the two settings. And the two faces turn out to be only loosely tied to each other: we find small, consistent traces of each side responding to the other's face, and nothing like one face triggering the other.

Two corpora, one measurement

The same-room recordings come from Meta's Seamless Interaction dataset,⁹9Agrawal et al., Seamless Interaction: dyadic audiovisual motion modeling and large-scale dataset, 2025 in which two people talk in a studio with one camera on each. The video calls come from the CANDOR corpus,⁸8Reece et al., The CANDOR corpus: insights from a large multimodal dataset of naturalistic conversation, 2023 in which strangers were paired for an unscripted half hour over video.

Every moment we study is a silence that follows somebody's speech. The person who was talking is the speaker; the other is the listener. If the listener then starts a real utterance, the silence was a turn end. If the speaker goes on first, it was a pause. A backchannel is a short listener sound such as “yeah” or “mm-hm” that does not take the floor, and the gap is the silence between the speaker's last sound and the listener's first word. The same rules, applied to voice-activity and word timings, define the events on both corpora.

One detail of timing matters for everything that follows: where a silence begins. A voice-activity detector, the usual source of such boundaries, closes a segment only once the quiet tail of an utterance has died away. A face timed on that boundary appears to move earlier than it does. We therefore time every turn end and every pause on the sound itself. Time zero is the end of the last 10 ms stretch of the speaker's own audio channel that is within 25 dB of the loudest stretch in the 1.5 s before it, and “the speaker's last sound” below always means that moment. It falls a median 0.12 s before the end of the voice-activity segment in the same-room recordings and 0.11 s before it on the video calls. The listener's first word needs no such correction: its sound starts within 0.05 s of the marked time.

Each face is tracked at 10 frames per second with MediaPipe's face landmarker:¹⁰ 52 expression scores (jaw opening, lip press, smile, blink and so on), head pose, iris position, and the openness of the eyes and the mouth, 65 numbers per frame once head and gaze speed are added. People differ in how expressive they are, and cameras differ in what they resolve, so every feature is rescaled per person against that person's own resting face:

Here \(x_f(t)\) is feature \(f\) at time \(t\), and \(\mu_f^{\,\mathrm{rest}}\) and \(\sigma_f^{\,\mathrm{rest}}\) are its mean and standard deviation over the frames of the same recording in which that person is not speaking. A value of \(z = 0\) is the person's usual resting level and \(z = +1\) is one standard deviation above it.

The effects and intervals that follow use three conventions. A difference between two kinds of event is Cohen's \(d\) on the \(z\) scores above, with the standard deviation pooled over individual events. How well two kinds of event can be told apart is the area under the receiver operating characteristic (ROC) curve, AUC for short, of a classifier scored only on conversations it was not fit on: out-of-fold scores from cross-validation with folds grouped by conversation or, for the recurrent network near the end, a held-out set of conversations. Every interval is a 95 % percentile interval from a cluster bootstrap that resamples whole conversations rather than events, because events from the same pair of people are correlated and resampling them one by one would understate the uncertainty.

Gaze needs a definition of its own, because a camera does not show what a person is looking at. The tracker gives head pose and iris position relative to the camera, so the partner's direction is estimated per person from the frames in which that person is listening, when people mostly look at the speaker: it is the median of head yaw, head pitch and the two iris offsets over those frames. A frame counts as looking at the partner when its Mahalanobis distance to that point, with the covariance taken from the central 80 % of listening frames, is within the 70th percentile of the listening frames. The 70 % is our choice, so a share of frames at the partner means little on its own: a stricter boundary of 50 % lowers every share and shrinks the turn-end differences reported below by about a third. What the post compares are moments measured the same way.

The listener, before speaking

Start with the person who is about to talk. Line up every turn start on the listener's first sound and look backwards: the mouth begins to open about a second ahead, the head begins to move in the last half second, and gaze direction and the brows do nothing until the voice starts. Now do the same for one-word backchannels. The mouth opens almost as far, but only at the last moment.

To compare when the mouth opens without being fooled by how far, we time each kind of sound by the moment its mean opening reaches half of what it will be at the first sound:

where \(\bar m(t)\) is the mean mouth opening \(t\) seconds from the start of the sound. Before a turn, \(t_{1/2}\) is −0.43 s on both corpora. Before a one-word backchannel it is −0.10 s in the room and −0.14 s on the call. The turn is earlier by 0.33 s (95 % interval 0.28 to 0.36 s) and 0.29 s (0.26 to 0.32 s).

What the early opening announces is the length of what is coming more than turn-taking as such. A speaker who resumes after a pause of 2.5 s or more opens just as early as someone starting a turn (half open 0.42 s ahead in the room, 0.30 s on the call), while a one-word reply needs no such run-up. The same contrast is known from breathing. In question–answer pairs, long answers are more often preceded by an inbreath and short ones are produced on the air that is left.¹²12Torreira, Bögels and Levinson, Breathing for answering: the time course of response planning in conversation, 2015 The breath also tends to be deeper before more speech. Readers inhale more deeply and for longer before a longer sentence,³¹ and in three-party conversations a deeper inhalation makes a longer stretch of speech more likely to follow and a very short utterance, the kind that is mostly backchannels, less likely.¹³ The effect in conversation is small, and in the question–answer pairs the depth of the inbreath was unrelated to the length of the answer.¹² Hand-coded video of four-person conversations shows the next speaker's mouth going from closed to open around the end of the current utterance.¹¹11Ishii et al., Prediction of who will be next speaker and when using mouth-opening pattern in multi-party conversation, 2019 Articulatory recordings of read sentences show the mouth opening during the inbreath that precedes speech.³⁰ Our tracks cannot say whether the opening is a breath or a display. For anyone watching, it is informative either way: an open mouth 0.3 s ahead says that more than a single word is coming.¹⁴14A different measure gives a different ordering: Krause and Kawamoto, Anticipatory speech postures during dyadic conversation, 2021, find that the lips take the shape of the first sound earliest for the shortest utterances.

It often starts while the other person is still talking

The run-up has the same shape whether it falls inside the partner's speech or in silence; it is timed to the listener's own first word, not to the partner's last one. Whether the partner is still talking when it begins therefore depends only on the gap. The mouth is a quarter open 0.72 s ahead of the first word in the room and 0.83 s ahead on the call, and the median gap is 0.84 s in both settings. So the opening begins before the speaker's last sound in 43 % of exchanges in the room and 49 % on the call, and in the typical exchange it begins just as that sound ends (0.12 s after it in the room, 0.01 s after it on the call).

The closest published analogue is again respiratory. In Dutch question-and-answer pairs the inbreath before an answer starts a median 56 ms before the question ends,¹² and the most common gap between speakers in conversational corpora is about 200 ms.¹⁵ One consequence is practical: a system that watches the user's face cannot see a reply coming earlier than about a second before it starts, and with the gaps found here that is, slightly more often than not, after the system has itself stopped talking.

The speaker, at the end

Now the other face. Line up every silence on the speaker's last sound, and compare the silences that turned out to be turn ends with those that turned out to be pauses. Anything that happens after somebody speaks again is left out, so past time zero the two curves compare silences that sound identical.

Three things stand out, and they are the same in the room and on the call.

The jaw and the lips say “done”, as the sound ends and not before. At a turn end the jaw is half closed at the moment of the last sound and fully closed 0.3 s later, and the lips press together over the same 0.3 s. In a pause the jaw dips for a moment and then stays half open, in speaking posture, for the whole following second. The posture is not new to phoneticians: magnetic resonance imaging of the vocal tract shows that in pauses within speech the articulators sit close to a “speech-ready” setting, well away from the closed position of rest.¹⁶16Ramanarayanan et al., An investigation of articulatory setting using real-time magnetic resonance imaging, 2013 The brief dip has a counterpart there as well: in about a third of the pauses at strong phrase boundaries in read speech, the lips move toward closure and then reopen.²⁹29Krivokapić, Styler and Parrell, Pause postures: the relationship between articulation and cognitive processes during pauses, 2020 The contrast between keeping a turn and yielding it has been seen in conversation too: in hand-coded video of four-person conversations, a speaker who keeps the turn tends to hold the mouth narrowly open, and one who yields it more often starts to close it or keeps it wide open.¹¹ What these recordings add is the timing, measured frame by frame against the sound, in two-party talk in two settings, and the checks further down that the display is the speaker's own. In a pause that lasts longer than a second the mouth closes too, and the eyes take over as the cue.

The eyes come back. While talking, people look at their partner, by the measure defined above, on 28 % of frames in the room and 35 % on the call, where the partner is the screen. In the last half second before a turn end that share is 13.7 percentage points higher than before a pause in the room (95 % interval 12.0 to 14.9 points) and 13.0 points higher on the call (11.9 to 14.0 points), and the difference keeps growing through the silence, to 31 and 23 points a second later. It is the eyes that come back, not only the head. Measured from the iris position alone, ignoring the head, the difference in the last half second is the same: 14.0 points in the room (12.5 to 15.3) and 13.0 on the call (12.0 to 13.9). From the head angle alone it is smaller, 8.8 and 9.0 points. This is one of the oldest findings in the field, first reported from filmed conversations in 1967,¹⁷ and it has been both confirmed and qualified since: with eye trackers, speakers were looking at the addressee at 61 % of the points where a turn could end.^18,19,2018Kendrick, Holler and Levinson, Turn-taking in human face-to-face interaction is multimodal, 202319Degutyte and Astell, The role of eye gaze in regulating turn taking in conversations: a systematized review, 2021 Eye-tracker shares like that one measure where the eyes point; ours are counted against the 70 % boundary, so the two levels cannot be set side by side, only the return. We find the same return on video calls, where an earlier model study on CANDOR saw no gaze pattern.²²

Before the stop, the face says little. A logistic regression on one second of the speaker's face features tells a coming turn end from a coming pause only weakly while the speaker can still be heard: an AUC of 0.58 in the room and 0.62 on the call for the window ending 0.3 s before the last sound. The score then climbs through the silence, to 0.67 and 0.68 after 0.3 s, 0.71 in both settings after half a second, and 0.77 and 0.74 after a full second.

Is that late signal really the speaker's own, or is the speaker reacting to a listener who has started to answer? Three checks say it is the speaker's own. It survives when only silences that nobody breaks for a full second are compared (AUC 0.63 in the room and 0.61 on the call at 0.3 s into the silence, rising to 0.73 and 0.67 at 1 s). Aligned on the listener's first word instead, the speaker's smile and settled gaze are already in place half a second earlier. And the curves show no step at the moment the listener starts.

Same room and video call

Most of what is timed to the boundary itself is shared between the two settings. The slower, state-like behaviours are not.

On the video calls, speakers smile into the last word of a turn: the smile separates turns that will end from turns that will pause from about a second before the voice stops (d = +0.21 at 0.3 s before it, where a word ends the turn). In the room there is no such lead (d = −0.05); the smile appears only as the voice stops. That agrees with the one other study of the speaker's face on Seamless Interaction, which finds more smiling at the moment of yielding,²¹21Berner et al., Exploring multimodal turn-taking cues in face-to-face conversation using voice activity projection, 2026 and with an experiment in which strangers getting acquainted smiled more over video than in the same room.²⁵ Listeners on the calls also sit with the mouth slightly open as much as two seconds before they answer (d = +0.35, against +0.09 in the room), a readiness posture that is nearly absent face to face.

The room has its own behaviours. The person about to speak looks down and blinks in the last half second. The speaker's head moves more about a second before yielding. And after a turn end the speaker blinks less while waiting for the reply, where a speaker on a call blinks more: the only cue that points in opposite directions.

One corpus per setting cannot say why. Medium, kind of talk and population all differ at once: the calls are free conversation between strangers, the room recordings are prompted and mostly between people who know each other. The one factor we could vary inside a corpus is acquaintance, and it does not explain the cues that set the calls apart: the smile into the last word, the readiness posture and the blink after the stop are the same in 382 Seamless conversations between strangers and 695 between acquainted pairs (every difference is under 0.05 in d, with a 95 % interval that includes zero). Across all 28 statistics compared, 3 differ at the uncorrected 95 % level, where 1 or 2 would be expected by chance. That video calls change the rhythm of conversation is established on its own: the same friends answered each other after about 135 ms in person and 487 ms over video.²⁶

Does one face move the other?

So far each face has been read by itself. The picture of the listener as an active partner makes a stronger claim: that what one person shows changes what the other does. Recordings of natural conversation cannot prove that, because both people hear the same words and can both tell that a sentence is finishing. They can show whether the pattern that influence would leave is there, and how large it is. We ran two exploratory tests, one in silences and one in the middle of turns. Neither was in the analysis plan we fixed in advance.

In a silence: does the speaker hold back when the listener looks ready?

Take every silence that is still open half a second after the speaker's last sound, with the listener quiet throughout. At that moment, read how far the listener's mouth has opened since before the stop (call it readiness), and then follow the silence in steps of 0.1 s to see whether the speaker resumes. The listener's first sound ends the observation. The model is a discrete-time hazard:²⁷

with one intercept \(\alpha_t\) per step, readiness in standard deviations, and controls \(\mathbf{c}_i\) for the speaker's own display (jaw opening, lip press), the length of the turn so far and whether the last word was a hesitation. The odds ratio \(e^{\beta}\) says how the odds of resuming in any one step change per standard deviation of readiness.

In the room, each standard deviation of readiness goes with 10 % lower odds that the speaker resumes in any given step (odds ratio 0.90, 95 % interval 0.89 to 0.92). On the calls the odds are 15 % lower (0.85, 0.83 to 0.86). The speaker's own display does not explain it: the numbers are the same with the speaker's jaw and lips in the model. A listener who looks ready does go with a speaker who stays silent.

By itself that is also what shared knowledge would produce. A listener who can hear that a sentence is complete gets ready, and faces a speaker who was never going to continue. Influence makes two further predictions. First, a mouth can reach the speaker only through the speaker's eyes, so the link should be stronger when the speaker is looking at the listener. It is, slightly: 0.88 against 0.91 in the room, and 0.83 against 0.87 on the calls. The difference is at the edge of what the data resolve (interaction odds ratio 0.95, 95 % interval 0.91 to 1.00, in the room; 0.96, 0.91 to 1.01, on the calls). Second, a speaker who goes on anyway should at least hesitate. That prediction fails: in pauses where the listener looked ready but made no sound, the speaker resumed as quickly as ever (median pause 0.77 s in both the least and the most ready third in the room; 0.75 s in both on the calls).

The relation in the other direction is far stronger. If the speaker looks at the listener throughout the first half second of a silence, rather than not at all, the odds that the listener takes the turn in any given step are 2.3 times as high in the room (95 % interval 2.07 to 2.48) and 1.6 times as high on the calls (1.50 to 1.70), with the listener's own readiness held fixed.

In mid-turn: the look, the nod, the look away

In 2002, Bavelas, Coates and Johnson video-recorded nine pairs of strangers, one telling the other about a close call, and marked by hand where each person looked and when the listener responded.²⁴24Bavelas, Coates and Johnson, Listener responses as a collaborative process: the role of gaze, 2002 They found the same sequence again and again: the speaker looks at the listener, the listener answers with a nod or an “mhm”, and the speaker quickly looks away and carries on, so that the listener's response closes a “gaze window” the speaker opened. Our second test looks for that sequence automatically. A window opens when a speaker's eyes arrive on the partner after at least half a second away. A silent head response is the start of a burst of listener head movement with no listener sound within half a second either side. The hazard model is the same as above, now for the window closing, with a term that switches on once a response has occurred.

Each step of the sequence is there, and each is small. In the second after a speaker's eyes arrive, the listener's head movement rises by 0.14 z in the room and 0.11 z on the calls, against 0.04 and 0.05 z after the eyes leave: a difference of 0.10 z in the room (95 % interval 0.08 to 0.12) and 0.07 z on the calls (0.06 to 0.07), a tenth of a standard deviation or less. Silent head responses come 14.9 times per minute while the speaker is looking and 13.0 times while the speaker talks with the eyes elsewhere, 1.14 times as often (1.10 to 1.18); on the calls the rates are 14.3 and 13.4, 1.06 times as often (1.05 to 1.08). Once the listener's head has answered, the glance ends a little sooner: the odds of the speaker looking away in any tenth of a second are 14 % higher in the room (odds ratio 1.14, 1.08 to 1.18) and 4 % higher on the calls (1.04, 1.02 to 1.05). In the room a spoken “mm-hm” has no detectable effect (0.96, 0.88 to 1.04). On the calls it has the same small effect as a silent head response (1.03, 1.004 to 1.05), so there the glance ends a little sooner after any response, not after a silent one in particular. The usual caution applies with more force here. The end of a phrase can prompt both the nod and the glance away, and the glance is timed with the head-and-eyes measure, which counts a turn of the speaker's head as a look away even when the eyes stay on the listener. The effects are also smaller on the calls, which may be the medium or may be the coarser head tracks of webcam video.

Taken together, the two tests give the same answer from opposite ends of a turn. Each person leaves small, consistent traces of responding to what the other's face is doing, more clearly in the room than on a call. Neither face drives the other. Across turn ends, how strongly the speaker's face says “done” and how strongly the listener's says “ready” are almost uncorrelated (rank correlation 0.00 in the room and +0.09 on the calls), and each side's display is timed to its own voice. The follower in this dance is already moving before the lead has finished, and is mostly keeping its own time.

What a small model can read

The averages above are small in units of a person's own variation. A natural question is how much of the exchange a model can recover from one face. We trained two kinds of model per statistic, each on one person's face only and with no audio: a logistic regression on the mean and the slope of every feature over the last one to three seconds, and a small gated recurrent unit (GRU) network²⁸ over the same frame sequence. Both were evaluated once on held-out conversations.

From the listener's face alone, 0.3 s before the first word, the recurrent network separates a listener who is about to speak from one who is just listening at an AUC of 0.72 in the room and 0.77 on the call. From the speaker's face, a turn end is separated from a pause at 0.68 and 0.66 for the frames up to 0.3 s before the last sound, and at 0.77 and 0.75 three tenths of a second into the silence. Reading the frames in order is worth 0.02 to 0.08 AUC over their summary, most of all for the listener's run-up, whose shape over time is the cue. A model trained in one setting loses 0.03 to 0.04 when scored in the other and stays far above chance, so most of what it reads is common to both.

These numbers are not directly comparable with earlier ones, which answer different questions: on CANDOR, a model given both faces and no audio reaches 68 % balanced accuracy at deciding, in a silence, whether the same speaker continues;²²22O'Connor Russell and Harte, Visual cues enhance predictive turn-taking for two-party human interaction, 2025 on recorded interviews, the speaker's face alone gives an AUC of 0.83 for the end of an utterance once the pause has begun.²³ What is new here is the separation of the two faces and the timing: the listener's own face, ahead of the listener's own first word.

Limits

• Averages, not rules. The effects that are not negligible here run from |d| = 0.15 to 0.35: well resolved at these sample sizes, but far too weak for any one cue to classify a single moment. The example clips were picked to show each cue clearly; most single moments are less clear.
• Two settings, many differences. Medium, topic, acquaintance and population change together between the corpora. Differences between them are differences between these two datasets.
• Resolution. CANDOR's 320 × 240 pixel frames give weaker motion features, about half the effect size of Seamless Interaction on head and gaze movement. Every shared effect has the same direction.
• Where the clock starts. The speaker's last sound is set by a loudness rule (the last 10 ms within 25 dB of the local peak). Any such rule is a choice, and a stricter or looser threshold would move time zero by a few hundredths of a second, and with it every time counted from the speaker's stop. Times counted from the listener's first word do not depend on it.
• Gaze is estimated. “Looking at the partner” is inferred per person from where that person's head and eyes point while listening, with a boundary that takes in 70 % of the listening frames; on a call it means looking at the screen. The tracker gives head angle and iris position, not the direction of gaze, and the two enter the measure as separate numbers, so a head that tilts while the eyes stay on the partner can count as looking away. The turn-end result is the same when only the iris position is used. Measuring where the eyes point would take a dedicated gaze-estimation model.
• Which results were planned. The cue descriptions were first measured on data we had looked at and then tested under a protocol fixed in advance, on conversations set aside for the purpose. With the timing rule used here, 16 of 16 planned tests hold on the 187 Seamless Interaction conversations set aside last and 18 of 18 on the 839 CANDOR ones; on two other Seamless sets scored earlier (71 and 288 conversations), 14 of 16 hold on each (the exceptions concern how still the head is before the stop and how much the listener's cue grows during the silence). The timing rule was settled after those sets had been scored once, so these counts repeat tests that had already been confirmed and are not a fresh confirmation. The coupling tests in the previous section were added afterwards and are exploratory.
• No cause. Nothing here separates a signal meant for the partner from the visible side of getting ready to speak.

Closing thoughts

Detecting the end of a turn from the speaker's audio treats a conversation as one person talking at a time. The faces say otherwise. For about a second around every exchange both people are visibly busy: one is opening the mouth for what comes next, often before the other has finished, and the other is closing the jaw, pressing the lips and looking up. These movements are small, they are the same in a studio and on a webcam, and a small recurrent network is enough to read them well above chance from a single face.

They also set expectations for anything with a face that takes part in a conversation. A mouth that opens only at the instant of speaking looks, to a human, like a backchannel. A jaw that drops to neutral at every phrase break says “done” many times per turn. Eyes that never return leave the end of a turn unmarked. Whether people rely on these cues is a question for experiments. That people produce them, reliably and on both sides of the exchange, is what these 2,670 conversations show.

References

1. Skantze, G. (2021). Turn-taking in conversational systems and human-robot interaction: a review. Computer Speech & Language 67, 101178. doi:10.1016/j.csl.2020.101178
2. Castillo-López, G., de Chalendar, G., and Semmar, N. (2025). A survey of recent advances on turn-taking modeling in spoken dialogue systems. International Workshop on Spoken Dialogue Systems Technology (IWSDS). aclanthology.org/2025.iwsds-1.27
3. Pipecat. Smart Turn: an open-source, audio-only end-of-turn detection model. github.com/pipecat-ai/smart-turn
4. Ekstedt, E., and Skantze, G. (2022). Voice activity projection: self-supervised learning of turn-taking events. arXiv:2205.09812
5. Bavelas, J. B., Coates, L., and Johnson, T. (2000). Listeners as co-narrators. Journal of Personality and Social Psychology 79(6), 941–952. doi:10.1037/0022-3514.79.6.941
6. Levinson, S. C., and Torreira, F. (2015). Timing in turn-taking and its implications for processing models of language. Frontiers in Psychology 6, 731. doi:10.3389/fpsyg.2015.00731
7. Bögels, S., Magyari, L., and Levinson, S. C. (2015). Neural signatures of response planning occur midway through an incoming question in conversation. Scientific Reports 5, 12881. doi:10.1038/srep12881
8. Reece, A., Cooney, G., Bull, P., Chung, C., Dawson, B., Fitzpatrick, C., Glazer, T., Knox, D., Liebscher, A., and Marin, S. (2023). The CANDOR corpus: insights from a large multimodal dataset of naturalistic conversation. Science Advances 9(13), eadf3197. doi:10.1126/sciadv.adf3197
9. Agrawal, V., et al. (2025). Seamless Interaction: dyadic audiovisual motion modeling and large-scale dataset. arXiv:2506.22554. Data released under CC BY-NC 4.0.
10. Lugaresi, C., et al. (2019). MediaPipe: a framework for building perception pipelines. arXiv:1906.08172
11. Ishii, R., Otsuka, K., Kumano, S., Higashinaka, R., and Tomita, J. (2019). Prediction of who will be next speaker and when using mouth-opening pattern in multi-party conversation. Multimodal Technologies and Interaction 3(4), 70. mdpi.com/2414-4088/3/4/70
12. Torreira, F., Bögels, S., and Levinson, S. C. (2015). Breathing for answering: the time course of response planning in conversation. Frontiers in Psychology 6, 284. doi:10.3389/fpsyg.2015.00284
13. Włodarczak, M., and Heldner, M. (2016). Respiratory turn-taking cues. Interspeech 2016, 1275–1279. doi:10.21437/Interspeech.2016-346
14. Krause, P. A., and Kawamoto, A. H. (2021). Predicting one's turn with both body and mind: anticipatory speech postures during dyadic conversation. Frontiers in Psychology 12, 684248. doi:10.3389/fpsyg.2021.684248
15. Heldner, M., and Edlund, J. (2010). Pauses, gaps and overlaps in conversations. Journal of Phonetics 38(4), 555–568. doi:10.1016/j.wocn.2010.08.002
16. Ramanarayanan, V., Goldstein, L., Byrd, D., and Narayanan, S. S. (2013). An investigation of articulatory setting using real-time magnetic resonance imaging. Journal of the Acoustical Society of America 134(1), 510–519. PMC3724797
17. Kendon, A. (1967). Some functions of gaze-direction in social interaction. Acta Psychologica 26, 22–63.
18. Kendrick, K. H., Holler, J., and Levinson, S. C. (2023). Turn-taking in human face-to-face interaction is multimodal: gaze direction and manual gestures aid the coordination of turn transitions. Philosophical Transactions of the Royal Society B 378(1875), 20210473. doi:10.1098/rstb.2021.0473
19. Degutyte, Z., and Astell, A. (2021). The role of eye gaze in regulating turn taking in conversations: a systematized review of methods and findings. Frontiers in Psychology 12, 616471. doi:10.3389/fpsyg.2021.616471
20. Ho, S., Foulsham, T., and Kingstone, A. (2015). Speaking and listening with the eyes: gaze signaling during dyadic interactions. PLoS ONE 10(8), e0136905. doi:10.1371/journal.pone.0136905
21. Berner, W., Cavalcanti, J. C., Åström, K., and Skantze, G. (2026). Exploring multimodal turn-taking cues in face-to-face conversation using voice activity projection. arXiv:2609.14666
22. O'Connor Russell, S., and Harte, N. (2025). Visual cues enhance predictive turn-taking for two-party human interaction. Findings of the Association for Computational Linguistics: ACL 2025, 209–221. arXiv:2505.21043
23. Kurata, F., Saeki, M., Fujie, S., and Matsuyama, Y. (2023). Multimodal turn-taking model using visual cues for end-of-utterance prediction in spoken dialogue systems. Interspeech 2023, 2658–2662. doi:10.21437/Interspeech.2023-578
24. Bavelas, J. B., Coates, L., and Johnson, T. (2002). Listener responses as a collaborative process: the role of gaze. Journal of Communication 52(3), 566–580. talkbank.org
25. Croes, E. A. J., Antheunis, M. L., Schouten, A. P., and Krahmer, E. J. (2019). Social attraction in video-mediated communication: the role of nonverbal affiliative behavior. Journal of Social and Personal Relationships 36(4), 1210–1232. PMC6394910
26. Boland, J. E., Fonseca, P., Mermelstein, I., and Williamson, M. (2022). Zoom disrupts the rhythm of conversation. Journal of Experimental Psychology: General 151(6), 1272–1282.
27. Singer, J. D., and Willett, J. B. (1993). It's about time: using discrete-time survival analysis to study duration and the timing of events. Journal of Educational Statistics 18(2), 155–195.
28. Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. arXiv:1406.1078
29. Krivokapić, J., Styler, W., and Parrell, B. (2020). Pause postures: the relationship between articulation and cognitive processes during pauses. Journal of Phonetics 79, 100953. doi:10.1016/j.wocn.2019.100953
30. Rasskazova, O., Mooshammer, C., and Fuchs, S. (2019). Temporal coordination of articulatory and respiratory events prior to speech initiation. Interspeech 2019, 884–888. doi:10.21437/Interspeech.2019-2876
31. Fuchs, S., Petrone, C., Krivokapić, J., and Hoole, P. (2013). Acoustic and respiratory evidence for utterance planning in German. Journal of Phonetics 41(1), 29–47. doi:10.1016/j.wocn.2012.08.007

Citation

Please cite this work as:

Makatchev, Maxim, "Two Faces of a Turn Exchange", OpenHRI, Nov 2026.

Or use the BibTeX citation:

@article{makatchev2026twofaces,
  author  = {Maxim Makatchev},
  title   = {Two Faces of a Turn Exchange},
  journal = {OpenHRI},
  year    = {2026},
  note    = {https://openhri.ai/research/turn-taking-faces/},
}

Example clips are from Seamless Interaction (Meta), used under CC BY-NC 4.0. CANDOR recordings may not be redistributed, so no CANDOR video is shown; all CANDOR results are aggregate statistics.