Sleep deprivation: thanks to AI, your voice speaks volumes...

30% of French people lack sleep

10 years ago, 10% of the French population slept less than 6 hours a night. Today, 30% of French people suffer from sleep deprivation. With these figures, we can now speak of an epidemic. The causes are manifold: long commuting times, prolonged time at the workplace, night shifts, exposure to screens which not only encroach on sleep time but also delay the onset of sleep... All these situations lead people to experience chronic fatigue, increasing the risk of accidents and exhaustion. If everyone has experienced a lack of sleep following a too-short night's sleep, then everyone has also experienced changes in their own voice the day after the party: a broken voice, words in place of other words...

Detecting the markers of sleep deprivation

A collaboration between researchers from the Vigilance, Fatigue, Sommeil et santé Publique (VIFASOM) team, headed by Professor Damien Léger (Université Paris Cité), with Daniel Pressnitzer (Ecole normale supérieure - PSL), Etienne Thoret, and Thomas Andrillon (Institut du cerveau de Paris), has led to the development of voice analysis methods to detect markers of sleep deprivation. For their study, they worked with a cohort of 22 women who were allowed to sleep just three hours a night, two nights in a row. This sleep deprivation closely mirrored that induced by certain night shift work situations.

In the first phase of the study, Etienne Thoret and Daniel Pressnitzer sought to characterize the different dimensions of the voice that could be affected by sleep deprivation, and to quantify their alterations objectively. They began their study with a purely acoustic analysis of voice recordings before and after sleep deprivation, and calculated, for each recording, sound representations by frequency, frequency modulation and temporal modulation. These different acoustic criteria are related not only to prosody (sentence melody, voice modification, speech rate variation) but also to voice timbre (clear vs. broken voice). These factors, in turn, can be linked to physiological markers such as variation in speech rate, directly related to motor skills, or timbre, directly related to vocal cord vibrations.

This first part of the study having enabled the researchers to confirm the hypothesis that prosody and timbre are indeed affected by sleep deprivation, the team set out to understand how they were affected.

Using machine learning

As conventional methods failed to reveal any clear differences in these sleep-deprivation-induced alterations, the researchers turned to machine learning techniques. While these techniques can detect relationships between excerpts from recordings made before and after sleep deprivation, and the acoustic properties of the sounds that enable them to be discriminated, they are still comparable to "black boxes" whose inner workings we cannot control. 

The researchers therefore trained artificial intelligences (AIs) to perform these discrimination tasks before and after sleep deprivation, to see if they were able to detect differences in the recordings. Initial results showed that, at the individual level, the AI was able to identify differences between voice recordings, thus detecting sleep deprivation and variability between individuals. At a population level, however, the results are not as reliable.

Evaluating fatigue through voice analysis

In a second stage, the research team investigated further and developed methods to understand how these AIs work and what precisely they look for in recordings to detect sleep deprivation. They have also tried to quantify what falls within the scope of prosody and timbre, and checked that the algorithms do not take into account other sounds or noises unrelated to the voice. This advanced research helped refine the study results and produce a sleep deprivation map of the voice for the 22 study subjects, clearly identifying and quantifying alterations in prosody and timbre. Having simultaneously associated each sleep-deprived person's sense of fatigue with analyses of his or her voice, the researchers can now assess a person's degree of fatigue through analysis of his or her voice.

The results of this study now make it possible to envisage protocols for detecting sleep deprivation through AI-enabled voice analysis. Non-invasive and rapid, this method is of particular interest in various sectors of activity where reduced alertness due to sleep deprivation can have serious consequences, particularly in terms of accidents. Following on from this study, voice markers could be correlated with specific physiological factors, paving the way for accessible, non-invasive "sleep stethoscopes". The research team is currently applying the same type of methodology to the development of other vocal biomarkers to help characterize a person's physiological state and detect attention disorders.