Identify the True and False Statements About Facial Expressions


Are there any observable behaviors or cues that can differentiate lying from truth-telling? Nearly all researchers in the field of deception detection concur that in that location is no “Pinocchio’s nose” that can serve as an piece of cake indicator of deception (DePaulo et al., 2003). Nevertheless, many researchers are nevertheless trying to find cues to charade (Levine, 2018; Denault et al., 2020). The “leakage theory” asserts that high-pale lies (the rewards come with serious consequences or there tin can exist astringent punishments) tin can effect in “leakage” of the deception into physiological changes or behaviors (especially microexpressions that last for ane/25 to 1/five s; Ekman and Friesen, 1969; Ekman, 2003; Porter et al., 2011, 2012; Su and Levine, 2016; Matsumoto and Hwang, 2020). Specifically, from the perspective of leakage theory (ten Brinke and Porter, 2012; Ten Brinke et al., 2012a,b), appreciable emotional facial expressions (microexpressions and macroexpressions) can, to some degree, determine who is lying and who is telling the truth (It is a probability problem (see Levine, 2018, 2019). However, fence exists for this possibility. While some researchers (ten Brinke and Porter, 2012; Ten Brinke et al., 2012b; Matsumoto and Hwang, 2018) argued that emotional facial microexpression could be a cue to lies supported their claims by empirical evidence, Burgoon (2018) argued that detecting microexpressions is not the best way of communicable liars. Furthermore, Vrij et al. (2019) fifty-fifty categorized microexpression into pseudoscience.

Fifty-fifty if it can be difficult, or fifty-fifty impossible for human being beings to detect liars based on microexpressions, there do exist some behavioral cues that can, to some degree, differentiate lying from truth-telling (Vrij et al., 2000, 2006). Specially, pupil dilation and pitch are shown to be closely related to lying (Levine, 2018, 2019). Most of the deception researchers concur that lying involves processes or factors such equally arousal and felt emotion (Zuckerman et al., 1981). Therefore, emotional facial expressions tin be valid behavioral cues to charade. Meanwhile, in that location are involuntary aspects of emotional expression. As noted by Darwin, some actions of facial muscles were the most difficult to exist voluntarily controlled and were the hardest to be inhibited (the and then-chosen Inhibition Hypothesis (see too Ekman, 2003). When a strongly felt genuine emotion is present, the related facial expressions cannot be suppressed (Baker et al., 2016). Hurley and Frank (2011) provided show for Darwin’south hypothesis and establish that deceivers could not control some particular elements of their facial expression, such equally eyebrow movements. The liars would feel fear, duping delight, cloy, or appear tense while lying, and would attempt to suppress these emotions past neutralizing, masking, or simulating (Porter and Ten Brinke, 2008). However, the liars could not inhibit them completely and the felt emotion would be “leaked” out in the form of microexpressions, specially under high-stake situations (Ekman and Friesen, 1969).

The merits of emotional leakage is supported by some recent research (Porter et al., 2011, 2012). When liars faux an unfelt emotional facial expression, or neutralize a felt emotion, at least one inconsistent expression would leak and appear transiently (Porter and Ten Brinke, 2008). 10 Brinke and Porter (2012) showed that liars would present unsuccessful emotional masking and certain leaked facial expressions (east.k., “the presence of a smirk”). In improver, they constitute that imitation remorse was associated with (involuntary and inconsistent) facial expressions of happiness and disgust (Ten Brinke et al., 2012a).

In addition to the support for emotional leakage, inquiry also shows that leaked emotions can differentiate lies and truth-telling. Wright Whelan et al. (2014) considered a few cues that had successfully told liars and truth-tellers, including gaze disfavor and caput shakes. They combined the information from each cue to classify individual cases and achieved an accuracy rate equally high as 78%. Meanwhile, Wright Whelan et al. (2015) found non-police and police observers could reach an accuracy of 68 and 72%, respectively, when required to observe deception in high-stake, real-life situation. Matsumoto and Hwang (2018) establish that facial expressions of negative emotions that occurred for <0.40 and 0.50 s could differentiate truth-tellers and liars. These studies all suggested that leaked facial expressions could help homo beings detect liars successfully.

Besides human being inquiry, attempts have also been fabricated to use motorcar learning to automatically detect deception by utilizing leaked emotions. A meta-analysis by Bond and DePaulo (2006) showed that man observers only accomplished a slightly-better-than-chance accuracy when detecting liars. Compared to humans, some previous works with machine learning used the and so-called reliable facial expressions (or involuntary facial expressions) to automatically detect cant and achieved an accuracy higher up 70% (Slowe and Govindaraju, 2007; Zhang et al., 2007). Given that the subtle differences of emotional facial expressions may not exist detected by naïve human observers, computer vision may capture the different and subtle features between lying and truth-telling situations that cannot be perceived by a human being. Su and Levine (2016) found that emotional facial expressions (including microexpressions) could exist effective cues for machine learning to detect high-stake lies, in which the accuracy was much higher than those reported in previous studies (e.g., Bond and DePaulo, 2006). They institute some Action Units (AU, the contraction or relaxation of 1 or more muscles (see Ekman and Friesen, 1976), such equally AU1, AU2, AU4, AU12, AU15, and AU45 (blink), could be potential indicators for distinguishing liars from truth-tellers in high-stake situations. Bartlett et al. (2014) showed that reckoner vision could differentiate deceptive pain facial signals from 18-carat pain facial signals at 85% accuracy. Barathi (2016) developed a system that detected a liar based on facial microexpressions, body language, and speech analysis. They plant that the efficiency of the facial microexpression detector was 82%. Similarly, the automated deception detection system developed by Wu et al. (2018) showed that predictions of microexpressions could be used as features for charade detection, and the system obtained an surface area under the precision-remember curve (AUC) of 0.877 while using various classifiers.

The leakage theory of deception predicts that when lying, peculiarly in high-stake situations, people would be afraid of their lies being detected and therefore result in fear emotions. These fear emotions could then leak and have the potential to exist detected (Levine, 2019). Meanwhile, it is presumed that if the fear associated with deception is leaked, the duration of the leaked fear would exist shorter due to the nature of leaking and repressing (which would be presented as fleeting fear microexpressions). Some may contend that the fearfulness emotions may also appear in truth-telling. Information technology tin be true. Nonetheless, for a truth-teller, the fear of being wrongly treated as a liar would be less leaking, since a truth-teller does not need to try difficult to repress the fear equally liars do. As a issue, the degree of repressing will be different betwixt liars and truth-tellers. On average, the duration of fright (or AUs of fearfulness) in lying situations would be shorter than that in truth-telling situations due to the harder repressing in the former ones.

Stakes may play a vital part while using an emotional facial expression as a cue to discover charade. Participants experience fewer emotions or less cognitive load in laboratory studies (Buckley, 2012). Well-nigh all laboratory experiments are typical of depression stakes and are not sufficiently motivating to trigger emotions giving rise to leakage (in the form of microexpressions). Consequently, liars in laboratory experiments are non as nervous as in real-life high-stake situations, with no or little emotion leakage. Equally noted by Vrij (2004), some laboratory-based studies in which the stakes were manipulated showed that loftier-stake lies were easier to discover than low-pale ones. Frank and Ekman (1997) stated that “the presence of high stakes is central to liars feeling strong emotion when lying.” Therefore, lying in high-stake situations would be more detectable past using emotional facial expression cues, and leaked emotional facial expressions would mostly occur in a high-stake context.

Hartwig and Bond (2014) had an opposite opinion and argued that fifty-fifty in loftier-stake situations, it could still be difficult to tell liars from truth-tellers. They claimed that the context of the high stake would influence both liars and truth-tellers, as liars and truth-tellers might experience similar psychological processes. In other words, loftier-pale situations would crusade inconsistent emotional expressions, similar fear, non merely in liars, just also in truth-tellers. This claim is true to some degree (ten Brinke and Porter, 2012), but high stakes do not necessarily eliminate all the differences betwixt liars and truth-tellers. Even though high-stake situations increment pressure on both liars and truth-tellers, it can be assumed that the caste of increment would be different, and liars would feel much college pressure level than truth-tellers under high stakes. In addition, fabricating a lie requires liars to call up more than and therefore would cause a higher emotional arousal in them than in truth-tellers. Consequently, for liars, the frequency or probability of leaking an inconsistent emotional expression (say, fear) would be higher and thus easier to detect. In theory, the higher the stakes are, the more probable cues associated with deception (e.g., fear) are leaked, and the easier the liars could be identified using these cues.

As well duration, other dynamic features (Ekman et al., 1981; Frank et al., 1993) could besides vary in genuine and simulated facial expressions, such as symmetry. Ekman et al. (1981) manually analyzed the facial asymmetry by using the Facial Activity Coding Organization (FACS) and showed that 18-carat smiles accept more than symmetry when compared to a deliberate grinning. Similarly, the leaked emotional facial expressions of fear while lying and the less leaked ones when telling a truth may besides show different degrees of symmetry. However, the approach Ekman et al. (1981) used could be time-consuming and subjective. Thus, in the current study, we proposed a method that used coherence (a measure of the correlation between two signals/variables) to measure the asymmetry. The more symmetrical the facial movements of the left and right confront, the higher the coefficient of correlation between them. Consequently, the value of coherence (ranges from 0 to ane) can be a measurement of asymmetry or symmetry.

Based on the leakage theory and previous evidence, we hypothesize that (1) emotional facial expressions of fear (fearfulness of existence caught) can differentiate lying from truth-telling in high-pale situations; (2) the duration of AUs of fear in lying would exist shorter than that in truth-telling; (iii) the symmetry of facial movements will be different, as facial movements in lying situations will be more asymmetrical (due to the nature of repressing and leaking).


The Database

The database we used were 32 video clips of xvi individuals telling lies in half of them and truth in the other one-half. All of the video clips were recorded in a high-stake game show. The reason nosotros used the current design was that cues to detect deception could differ from person to person, and what spotted ane liar was ordinarily different from the signals that revealed the side by side liar (Levine, 2019). Consequently, cues may vary from sender to sender. The aforementioned person, however, would display well-nigh the same facial expression design on different occasions. Therefore, the relatively platonic experimental materials should exist composed past the aforementioned individual who tells both lies and truth to exclude or reduce the variation resulted from individual differences.

The video clips recorded individuals’ facial expressions in the game show “the Moment of Truth.” Prior to the show, the contestants took a polygraph test when they answered 50 questions. During the evidence, 21 of the same questions were asked again and the contestants were required to answer them in front of the studio audience. The questions became progressively more than personal as the contestants moved frontwards (an example of an extremely personal question is: Have y’all ever paid for sexual activity?). If the contestant gave the same answer to a question every bit they did in the polygraph test (which means they were telling the truth), they moved on to the next question; lying (as determined by the polygraph) or refusing to reply a question ends the game (see for details). During the game show, most of the people talked emotionally and showed natural emotional facial expressions considering of the high-stake situations they were in. The ground truth was obtained past a pre-show polygraph test that adamant whether an individual was lying or non in the game bear witness. Meanwhile, the stakes in the game evidence tin exist high (the highest proceeds from the show can attain at 500,000 U.s. dollars, and cues to deception will be more pronounced than when there was no such monetary incentive (come across DePaulo et al., 2003).

Popular:   When Should You Refuse to Accept Food in a Shipment

Participants were eight males and 8 females who ended the game with lying. That way, in that location was at least i lying video clip for each participant. The video clips consist of the moments when the individuals were answering the questions, that is, from the cease of the questioning to the end of the answering. To simplify adding, nosotros merged all the truth-telling video clips for each participant into a single ane, and we ended up having one video for each type, truth-telling and lying, for each person. The duration of the video clips ranges from 3 s to 280 s, with an average duration of 56.half-dozen south. Because of the game testify setting that lying ends the game, the truth-telling video clips were much longer than the lying ones (mean
= 105.5 south for truth-telling videos, and
= seven.8 s for lying videos). In full, there were 50,097 frames for truth-telling video clips and 3,689 frames for lying video clips. The median of frames is 199 for lying video clips and is two,872.5 for truth-telling video clips, with a frame rate of thirty f/south.

Using Reckoner Vision to Compare the Features in Video Clips While People Lying or Telling the Truth

Asking people to find out the cues to deception is difficult. Furthermore, naïve homo observers may not be able to perceive the subtle differences of the emotional facial expressions between telling lies and telling the truth. Alternatively, estimator vision may be more capable of doing so. We proposed a method aimed to employ the AUs of fear to discern deceptive and honest individuals in high-pale situations.

Emotional Facial Expressions of Fear

We, get-go, imported the video clips into OpenFace (Baltrusaitis et al., 2018) to conduct computer video analysis. This software automatically detects the face, localizes the facial landmark, outputs the coordination of the landmarks, and recognizes the facial AUs. OpenFace is able to identify eighteen AUs. According to Frank and Ekman (1997), telling a consequential lie results in emotions such equally fright and guilt. Therefore, we focused on the AUs of fear, i.east., AU1, AU2, AU4, AU5, AU20, AU26. For each frame of videos, we obtained presence (0 or one) and intensity (any number from 0 to 5) for each AU from OpenFace. In one case we obtained the AU data from OpenFace, we and so used MATLAB to calculate AUs of the emotional facial expression of fear. It was done past multiplying the output values of presence (0, one) and the value of the intensity (from 0 to 5) for each frame. We and then analyzed the AUs with statistical analysis, and too fabricated classification predictions with car learning.

For statistical analysis, we took the average of each AU beyond all frames in ane condition per participant. Nosotros ended up with one AU value for each condition for each person. Nosotros then bootstrapped the data for statistical analysis.

For motorcar learning, we resampled the data with SMOTE before edifice the model. SMOTE is an over-sampling technique that solves grade imbalance problem by using interpolation to increase the number of instances in the minority class (Chawla et al., 2002). Resampling was necessary because the data are unbalanced, with the video clips of truth much longer than those of deception, l,097 frames vs. 3,689 frames. It was consistent with the real life that lying was non that frequent compared to truth-telling, only it could still affect the reliability and validity of the model.

We then used WEKA (Hall et al., 2009), a auto learning software, to classify the videos into a truth group and a deception grouping. Three dissimilar classifiers were trained
a x-fold cross-validation procedure. The iii classifiers were Random Woods, 1000-nearest neighbors, and Bagging. Random Forest operates by constructing a multitude of decision trees (which is too a better choice for unbalanced datasets (meet Bruer et al., 2020). G-nearest neighbors (lazy.IBK in WEKA) achieves classification past identifying the nearest neighbors to a query case and using those neighbors to determine the course of the query (Cunningham and Delany, 2004). Bagging is a method for generating multiple versions of a predictor and using these to get an aggregated predictor (Breiman, 1996).

The Duration of Fearfulness

We used MATLAB to count the elapsing of AUs of fright (the number of frames when the respective AU was present). Because the frame rates of all the videos were the aforementioned, the number of frames could stand for the elapsing of AU. And then the precise duration was obtained by dividing the total number of frames by frame charge per unit, i.due east., 30.

The Symmetry of Facial Movements

Beh and Goh (2019) proposed a method to detect the changes in the Euclidean distances of facial landmarks to find out microexpressions. We used the distances of ld1 and rd1, which are distances between facial landmarks at the left/right eyebrow and left/right middle (alphabetize 20/25 and alphabetize 40/43, see Figure 1), to investigate the synchronization and symmetry between left and correct facial movements. The MATLAB function of Wcohenrence (wavelet coherence, the values ranged from 0 to ane) was used for this purpose, as this role returns the magnitude-squared wavelet coherence, which is a measure out of the correlation between two signals (herein ld1 and rd1) in the time-frequency domain. If the left and right facial movements have perfect synchronization and symmetry, the value of wavelet coherence would be 1.

Effigy 1

Effigy 1. The 68 facial landmarks and the Euclidean distances of ld1 and rd1.

Summary of Data Processing

All of the aforementioned steps of classifying the truth or charade in the video clips are demonstrated in Effigy 2. Showtime, OpenFace detected the face up, localized the landmarks, output the presence and intensity of AUs. Following that, AUs of fear, as well equally indicators used to summate symmetry in each frame from both lying and truth video clips, were merged into a facial movement description vector (frame by frame). Finally, in the nomenclature stage, classifiers of Random Forest, K-nearest neighbors, and Bagging were trained to discriminate charade and honesty.

Figure 2

Effigy ii. Overview of the procedure of classifying video clips. The model used here for demonstrating the processing flowchart is the tertiary author.


Action Units of Fear Can Differentiate Liars From Truth-Tellers

Machine Learning Classification Results

The whole dataset was split into ii subsets; we arbitrarily selected 12 out of our xvi participants’ data to build the model, i.eastward., data nerveless from 12 participants (42,954 frames, with two,999 frames of lying and the rest of truth-telling) were used for training the model; and the data collected from the remaining four participants (10,832 frames in total, with 690 frames of lying and the residue of truth-telling) were used to test how accurate were the model to make new predictions. Iii classifiers were trained on a dataset of 12 participants to discriminate liars from truth-tellers using feature vectors of AUs of fear (i.east., AU01, AU02, AU04, AU05, AU07, AU20, and AU26, for details of AUs of fearfulness delight, run across All the three classifiers, Random Forest, K-nearest neighbors (IBK), and Bagging, were trained in WEKA
a 10-fold cross-validation procedure. In building the model, the x-fold cross-validation procedure split up all the data from the 12 participants into ten subsets, and the algorithms were trained on 9 subsets and tested on the remaining 10th each time, repeating 10 times. When a classifier was deployed from ten-fold cross-validation, it was applied to the other four participants’ data to summate the accuracy of prediction. To highlight the relative importance of AUs of fear in classification accuracy, nosotros eliminated all other indicators used by Beh and Goh (2019). Tabular array ane shows the performance of machine learning analysis, which was conducted on dataset of 12 participants and tested with the data of the remaining 4 participants.


Table 1. Machine learning performance of the Random Forest, IBK, and Bagging.

Table i reports the percentage of accurateness obtained on the testing dataset. In improver to accuracies, the tabular array reports the weighted average of true-positive rate (TP rate, instances correctly classified as a given class), false-positive rate (FP charge per unit, instances falsely classified as a given grade), precision value (proportion of instances that are truly of a class divided by the total instances classified as that class), recall value (proportion of instances classified as a given course divided past the bodily full in that class), F-measure (a combined measure for precision and recall), precision-recollect curve (PRC) area value (a model performance metrics based on precision and recall), and kappa (which measures the agreement between predicted and observed categorizations). The details of these statistics can be seen in Witten et al. (2016).

In addition, because the size of the dataset is relatively pocket-sized, we did go out-one-person-out cross-validation (LOOCV). LOOCV utilizes each individual person as a “exam” set up and the remaining dataset equally the training set. It is recommended for smaller datasets. The Random Forest algorithm was applied. The results showed that the boilerplate accuracy is still to a higher place 90% (hateful
= ninety.sixteen%, range from 78.74 to 95.78%).

The Differences of AUs of Fear Between Truth-Telling and Lying Video Clips

This assay was carried out by examining the statistical differences of AUs of fright betwixt truth-telling and lying video clips through paired
t-test. To avoid the multiple-testing problem, nosotros practical Bonferroni correction and set
p-value to 0.007. We too calculated Cohen’s d to measure out result size. The results are presented in Tabular array 2. When bootstrapping was used, the
p-value of comparing AU20 in the 2 groups was 0.006 (for AU05, the respective
p-value is 0.008). This analysis revealed that liars and truth-teller had differences in the facial expressions of fear.

Table 2

Table 2. The results of paired
t-examination for comparison the means of values of AUs of fearfulness betwixt truth-telling and lying video clips.

There Were More than Transient Durations of AU of Fear While Lying

Ekman (2003) reported that many people could not inhibit the activity of the AU20 (stretching the lips horizontally) while examining videotapes of people lying and telling the truth. Our results reported in department The Differences of AUs of Fear Betwixt Truth-Telling and Lying Video Clips also found significant differences betwixt truth-telling and lying video clips in values of AU20. Therefore, differences in the elapsing from onset to peak, from peak to get-go, and total durations of AU20 between truth-telling video clips (in which the number of AU20 is 675) and lying video clips (in which the number of AU20 is 47) were analyzed with independent samples
t-examination, using bootstrapping with one,000 iterations. The results showed that there were pregnant differences in the total duration and elapsing from top to offset betwixt truth-telling video clips and lying video clips (twenty.77 vs. 15.21 frames,
= 0.033, effect size = 0.276; 11.35 vs. half dozen.98 frames,
= 0.04, effect size = 0.347). The durations of AU20 in lying video clips were nearly four frames (133 ms) shorter than those in truth-telling video clips on average because the facial movements (herein the AU20) disappeared more than quickly in the lying condition. Figure 3 shows the distribution of full frames, frames from onset to apex, and frames from apex to outset of AU20. The median is 12 in the truth-telling video clips and viii in the lying video clips. For lying video clips, the 95% confidence interval is 10.32 to 20.11 frames for the mean of full elapsing, and nineteen.03 to 22.52 frames for truth-telling video clips. There were 16 (out of 47) AU20s whose durations were less than or equal to six frames (200 ms, one of the ordinarily recognized thresholds differentiating microexpressions and macroexpressions) in the lying video clips, while there were 145 (out of 675) in the truth-telling video clips. There were 32 AU20s whose durations were ≤ 15 frames (500 ms, another microexpression/macroexpression boundary, more details in word) in the lying video clips, and the corresponding number is 407 in the truth-telling video clips.

Popular:   This Activity Lets You Hear the Information


Figure iii. Violin plot for frames of AU20 in truth-telling and lying video clips. IQR, inter-quartile range. *statistically significant (p
< 0.05) differences betwixt lying and truth-telling.

Asymmetries of the Facial Movements Were More Salient in Lying Than Truth-Telling

Nosotros calculated ld1 and rd1, the distance between facial landmarks predicted at the left eyebrow and left eye and the distance between those predicted at the correct countenance and right middle (Beh and Goh, 2019) in each frame. These ii distances represented movements of the left and correct eyebrows. Side by side, we used the MATLAB function Wcohenrence (wavelet coherence) to measure out the correlation betwixt ld1 and rd1 in each video. If the movements were exactly symmetrical (e.g., they have the exact same onset fourth dimension, reach the apex at the same time, and disappear at the same time), the coherence between ld1 and rd1 would be ane. Any asynchrony would consequence in a coherence value of <one, with a smaller coherence value indicating more asymmetry. Effigy four shows the wavelet coherence in truth-telling and lying video clips.


Figure 4. Squared wavelet coherence between the ld1 and rd1 in lying (left
panel) and truth-telling (right
panel) situations. The relative phase relationship is shown as arrows (a rightward arrow indicates 0 lag; a bottom-right arrow indicates a small lead of ld1; a leftward arrow indicates ld1 and ld2 are anti-correlated).

The coherence outputs for each thespian (i.east., the average of coherence between ld1 and rd1) were then imported into the permutation test (see the following link for details: to compare the asymmetry differences between the lying and truth-telling situation. Permutation tests provide elegant ways to command for the overall Blazon I error and are distribution-costless. The results showed that lying and truth-telling situations acquired unlike coherence in facial expressions (the means of coherence are 0.7083 and 0.8096,
= 0.003, outcome size = ane.3144).


The electric current study supported the prediction of leakage theory that leaked fear could differentiate lying from truth-telling. The results of machine learning indicated that emotional facial expressions of fear could differentiate lying from truth-telling in the high-stake game show; the paired comparisons showed significant differences between lying and truth-telling in values of AU20 of fright (AU5 is marginally meaning). The results also substantiated the other 2 hypotheses. The duration of AUs of fear in lying was shorter than that in truth-telling, with a shorter full elapsing and the duration from peak to offset of AU20 of fear when lying compared to telling truth. The 3rd hypothesis predicted that the symmetry of facial movements would be different, and the findings indicated that the facial movements were more asymmetrical in lying situations than in truth-telling situations.

In the current study, the use of machine learning classified deception and honesty. Information technology made up the shortcomings of human being coding and successfully detected the subtle differences between lying and truth-telling. Meanwhile, an objective measure out of asymmetry was proposed. To our all-time knowledge, this is the beginning objective method to measure out the disproportion of facial movements. By using these methods, nosotros were able to find differences betwixt lying and truth-telling, which is the prerequisite for looking for clues of deception.

The machine learning approach could take some disadvantages. For example, the LOOCV is recommended for small datasets, like what nosotros have in the electric current study. However, it yielded a higher variance than x-fold cross-validation. The reason for this loftier variance might exist that the training datasets in LOOCV accept more overlap (each model was trained on an almost identical dataset), which fabricated the outputs from unlike folds highly positively correlated with each other, and hence increases the overall variance (the mean of many highly correlated quantities has college variance than does the mean of many quantities that are not equally highly correlated (run across James et al., 2013, p185). In our data, the variance was represented equally varying accurateness rates when different participants were left out in the training set up; for example, 78.74% accurateness when participant fourteen was left out compared to 95.78% when participant 11 was left out. Bengio and Grandvalet (2004) argued that when independent estimates in cross-validation were correlated, the correlation that is responsible for the overall increase of variance could increase with Thousand in a Grand-fold cross-validation, with leave-one-out existence an extreme case where One thousand is equal to the number of data points. In our dataset, because the like procedure each private generates the same facial expression, it is highly possible that the grooming sets are highly correlated. Future research with a larger sample size would reduce this variance.

The leaked emotions tin can be cues to charade, just they are not deception
per se. They are, however, closely linked with deception. As shown in the results, truth-tellers also experience fear. However, the dynamics of experienced fear of truth-tellers were very unlike from those of liars. Thus, the fear emotion could exist considered every bit a “hot spot” of deceit. Looking for the non-exact “hot spots” of individuals satisfies the demands of rapid evaluation. Some other approaches of deception detection, for example, brain activities, cannot provide existent-time results (Vrij and Fisher, 2020). The results suggested that the “hot spots”—emotional expressions of fear—could distinguish betwixt true and deceptive messages with a reasonable level of accuracy. Using machine learning, nosotros achieved a higher accuracy (above 80%) than the average accuracy achieved by people (54%, run into Bond and DePaulo, 2006). In add-on, we carried out a human deception detection study (Niu, 2021), in which the video clips of the first and the terminal honest answering (both are from the stop of the questioning to the end of the answering. We changed the durations of truth-telling video clips to keep the durations of lying and truth-telling video clips nearly same), the lying video clips are the same. Thirty college students took part in the report. The accurateness of detecting the lies was 0.34; for depression-stake truth-telling video clips (the first honest answering), the accuracy of truth detection was 0.69, and for high-stake truth-telling video clips (the last honest answering), the accuracy of truth detection was 0.64, and the average of deception detection was 0.50. The results showed again that the accuracy for human deception detection was at chance level. Apart from accurateness, there was a big consequence size for the AU of fright (AU20) while differentiating lies from truth.

High-stake lies were used in some previous research. For example, Vrij and Mann (2001) used the videos from media where missing people’s family members announced the missing of their family and asked for aid. In these videos, some of the announcers were telling the truth, while the others were hiding the truth that the people claimed to be missing were murdered by the announcers themselves. I disadvantage of these type of materials is that researchers would non have access to the truth and therefore would non exist able to tell for sure if i is lying or not. Our dataset consists of high-stakes deception videos from a existent game show, in which the veracity of the statements is supported by a polygraph test. That can help us reach a relatively high ecological validity and internal validity. Considering the debate on the reliability of polygraph tests, future research could use materials where the truth is further affirmed. One example would be the game show Golden Assurance, which utilizes a prisoner’southward dilemma setting and the truth becomes obvious after one makes a decision in the game (meet Van den Assem et al., 2012).

Were the facial expressions in lying video clips all microexpressions that last for <0.2 due south? The current results of total duration showed that AU20 on boilerplate lasts for 20.77 frames, i.e., 692 ms, in truth-telling video clips; and 15.21 frames, i.east., 507 ms, in lying clips. The 95% confidence intervals of total elapsing were from 19.03 to 22.52 frames (634–751 ms) while telling truth and were from ten.32 to 20.11 frames (344 ms ~ 670 ms) while lying. In the current study, the hateful was afflicted by farthermost values or outliers (meet Effigy 3). Thus, we used the median, which could be a more than advisable statistic for the elapsing. The median of duration in the truth-telling video clips was 12 (400 ms) and in the lying video clips was 8 (267 ms). Although the duration of (partial) fright was shorter in lying video clips than in truth-telling video clips, near of the durations in lying did not fit into the limits of traditional durations of microexpressions, i.e., <200 ms (meet Shen et al., 2012). In that location were nearly 1/3 AU20s which durations were less than or equal to vi frames (200 ms) in the lying video clips, and simply one/5 of them in the truth-telling video clips were less than or equal to half dozen frames. By using 500 ms every bit the purlieus between microexpressions and macroexpressions (encounter Matsumoto and Hwang, 2018), there were almost 2/3 of the facial expressions that could be named after microexpressions. The results suggested that the leaked emotional facial expressions in real life were much longer (the duration of the apex of leaked emotional facial expressions would be <200 ms). No matter what the duration is, or whether the facial expression is a microexpression or not, the durations of facial expressions were significantly shorter in the lying video clips than in the truth-telling video clips.

Taken together, our findings suggested that deception is detectable past using emotional facial expressions of fearfulness in high-pale situations. Lying in loftier-stake situations will leak facial expressions of fearfulness. The durations of fright were significantly different between lying and truth-telling conditions. Also, the facial movements are more asymmetrical when ane is lying than they are when one is telling the truth.

Our findings prompted that attending to the dynamic features of fear (such as symmetry and elapsing) can improve the power of the people to differentiate liars from truth-teller. Besides, the car learning approach tin be employed to detect real-world deceptive behaviors, especially those high-stake ones in the situations where strong emotions are generated, associated with attempts to neutralize, mask, and faux such emotions (similar piece of work is done in the project of iBorderCtrl, see Crampton, 2019). Certainly, the number of participants (16) in the electric current dataset was relatively small, which could limit the generalization of the results. We consider the current work every bit a preliminary exploration.

Popular:   Human Capital is the Improvement in _____ Produced by _____

Pupil dilation and pitch of oral communication are found to be significantly related to deception past some studies of meta-assay (Zuckerman et al., 1981; DePaulo et al., 2003; Levine, 2019). These cues are closely related to leakage too. The findings of Bradley et al. (2008) indicated that the educatee’s changes were larger when viewing emotionally arousing pictures which also were associated with increased sympathetic activity. Pitch of speech volition be different betwixt honest and deceptive interaction (Ekman et al., 1976; Zuckerman et al., 1981). Future studies should accost all these leaked clues or the “hot spots” of the deception.

Data Availability Statement

The original contributions presented in the report are included in the commodity, the dataset used in the current study tin be obtained on request from the first author ([email protected]).

Ideals Statement

The studies involving human participants were reviewed and approved past the Institutional Review Board (IRB) of Jiangxi University of Traditional Chinese Medicine. The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author Contributions

XS conceived the study, conducted the experiments, analyzed the data, wrote the newspaper, and acquired the funding. GF analyzed the data and revised the manuscript. ZC analyzed the data. CN contributed the materials used in the current study. All authors contributed to the article and canonical the submitted version.


This study was partially supported by the grants from the National Natural Science Foundation of China (No. 31960180, 32000736, 31460251), the Planed Project of Social Sciences in Jiangxi Province (No. 18JY24), and the project of 1050 Young tiptop-notch talent of Jiangxi University of Traditional Chinese Medicine (No. 5141900110, 1141900610).

Conflict of Interest

The authors declare that the inquiry was conducted in the absence of whatever commercial or financial relationships that could be construed as a potential conflict of interest.


Bakery, A., Black, P. J., and Porter, S. (2016). “x. The truth is written all over your face up! involuntary aspects of emotional facial expressions,” in
The Expression of Emotion: Philosophical, Psychological and Legal Perspectives, eds C. Abell and J. Smith (New York, NY: Cambridge Academy Press), 219–244.

Google Scholar

Baltrusaitis, T., Zadeh, A., Lim, Y. C., and Morency, Fifty.-P. (2018). “Openface 2.0: facial behavior assay toolkit,” in
Newspaper Presented at the 2018 13th IEEE International Briefing on Automatic Face and Gesture Recognition (FG 2018)

Google Scholar

Barathi, C. S. (2016). Lie detection based on facial micro expression torso linguistic communication and speech assay.
Int. J. Eng. Res. Technol. 5, 337–343. doi: 10.17577/IJERTV5IS020336

CrossRef Full Text | Google Scholar

Bartlett, One thousand. S., Littlewort, K. C., Frank, M. G., and Lee, K. (2014). Automated decoding of facial movements reveals deceptive pain expressions.
Curr. Biol. 24, 738–743. doi: 10.1016/j.cub.2014.02.009

PubMed Abstruse | CrossRef Full Text | Google Scholar

Beh, K. 10., and Goh, K. Yard. (2019). “Micro-expression spotting using facial landmarks,” in
Paper Presented at the 2019 IEEE 15th International Colloquium on Signal Processing and Its Applications (CSPA)

Google Scholar

Bengio, Y., and Grandvalet, Y. (2004). No unbiased estimator of the variance of g-fold cross-validation.
J. Mach. Acquire. Res. five, 1089–1105. doi: ten.5555/1005332.1044695

CrossRef Total Text | Google Scholar

Bail, C. F. Jr., and DePaulo, B. M. (2006). Accuracy of deception judgments.
Pers. Soc. Psychol. Rev. 10, 214–234. doi: ten.1207/s15327957pspr1003_2

CrossRef Total Text | Google Scholar

Bradley, Thousand. M., Miccoli, Fifty., Escrig, M. A., and Lang, P. J. (2008). The pupil as a measure of emotional arousal and autonomic activation.
45, 602–607. doi: 10.1111/j.1469-8986.2008.00654.x

PubMed Abstruse | CrossRef Full Text | Google Scholar

Bruer, K. C., Zanette, S., Ding, X. P., Lyon, T. D., and Lee, K. (2020). Identifying liars through automatic decoding of children’s facial expressions.
Kid Dev. 91, e995–e1011. doi: 10.1111/cdev.13336

PubMed Abstract | CrossRef Full Text | Google Scholar

Buckley, J. P. (2012). Detection of deception researchers needs to collaborate with experienced practitioners.
J. Appl. Res. Mem. Cogn. 1, 126–127. doi: 10.1016/j.jarmac.2012.04.002

CrossRef Total Text | Google Scholar

Burgoon, J. K. (2018). Opinion: microexpressions are not the best fashion to catch a liar.
Front. Psychol. 9:1672. doi: 10.3389/fpsyg.2018.01672

CrossRef Full Text | Google Scholar

Chawla, N. V., Bowyer, Thou. W., Hall, 50. O., and Kegelmeyer, W. P. (2002). SMOTE: constructed minority over-sampling technique.
J. Artificial Intellig. Res. sixteen, 321–357. doi: 10.1613/jair.953

PubMed Abstract | CrossRef Total Text | Google Scholar

Cunningham, P., and Delany, S. J. (2004). thou-Nearest neighbour classifiers.

Google Scholar

Denault, Five., Dunbar, North. E., and Plusquellec, P. (2020). The detection of deception during trials: ignoring the nonverbal advice of witnesses is not the solution—a response to Vrij and Turgeon (2018).
Int. J. Evid. Proof
24, 3–xi. doi: 10.1177/1365712719851133

CrossRef Full Text | Google Scholar

DePaulo, B. Yard., Lindsay, J. J., Malone, B. Eastward., Muhlenbruck, Fifty., Charlton, M., and Cooper, H. (2003). Cues to deception.
Psychol. Bull. 129, 74–118. doi: 10.1037/0033-2909.129.1.74

CrossRef Full Text | Google Scholar

Ekman, P. (2003). Darwin, charade, and facial expression.
Ann. North. Y. Acad. Sci. 1000, 205–221. doi: x.1196/annals.1280.010

CrossRef Full Text | Google Scholar

Ekman, P., and Friesen, Due west. Five. (1969). Nonverbal leakage and clues to deception.
Psychiatry. 32, 88–106. doi: ten.1080/00332747.1969.11023575

CrossRef Full Text | Google Scholar

Ekman, P., and Friesen, W. V. (1976). Measuring facial move.
Environ. Psychol. Nonverbal Behav. 1, 56–75. doi: x.1007/BF01115465

CrossRef Full Text | Google Scholar

Ekman, P., Friesen, W. Five., and Scherer, Thousand. R. (1976). Torso movement and vocalism pitch in deceptive interaction.
16, 23–27. doi: 10.1515/semi.1976.16.1.23

CrossRef Full Text | Google Scholar

Frank, K. G., and Ekman, P. (1997). The ability to detect deceit generalizes across different types of high-stake lies.
J. Pers. Soc. Psychol. 72, 1429–1439. doi: 10.1037/0022-3514.72.half dozen.1429

PubMed Abstruse | CrossRef Full Text | Google Scholar

Frank, Thou. G., Ekman, P., and Friesen, Westward. Five. (1993). Behavioral markers and recognizability of the smile of enjoyment.
J. Pers. Soc. Psychol. 64, 83–93. doi: 10.1037/0022-3514.64.1.83

PubMed Abstract | CrossRef Full Text | Google Scholar

Hall, Thousand., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. H. (2009). The WEKA information mining software: an update.
ACM SIGKDD Explor. Newslett. 11, 10–18. doi: 10.1145/1656274.1656278

CrossRef Total Text | Google Scholar

Hartwig, 1000., and Bond, C. F. Jr. (2014). Lie detection from multiple cues: a meta-analysis.
Appl. Cogn. Psychol. 28, 661–676. doi: 10.1002/acp.3052

CrossRef Total Text | Google Scholar

Hurley, C. Yard., and Frank, One thousand. G. (2011). Executing facial command during charade situations.
J. Nonverbal Behav. 35, 119–131. doi: ten.1007/s10919-010-0102-ane

CrossRef Full Text | Google Scholar

James, Yard., Witten, D., Hastie, T., and Tibshirani, R. (2013).
An Introduction to Statistical Learning.
New York, NY: Springer.

Google Scholar

Levine, T. R. (2018). Scientific evidence and cue theories in deception research: reconciling findings from meta-analyses and primary experiments.
Int. J. Commun. 12, 2461–2479.

Google Scholar

Levine, T. R. (2019).
Duped: Truth-Default Theory and the Social Science of Lying and Charade. Tuscaloosa, AL: The University of Alabama Press.

Google Scholar

Matsumoto, D., and Hwang, H. C. (2020). Clusters of nonverbal behavior differentiate truths and lies about hereafter malicious intent in checkpoint screening interviews.
Psychiatry Psychol. Police 1–xvi. doi: ten.1080/13218719.2020.1794999

CrossRef Full Text | Google Scholar

Niu, C. (2021).
Building a deception database with high ecological validity
(Master’s thesis). Jiangxi University of Chinese Medicine, Nanchang, Red china.

Google Scholar

Porter, South., and Ten Brinke, L. (2008). Reading betwixt the lies: identifying curtained and falsified emotions in universal facial expressions.
Psychol. Sci. 19, 508–514. doi: 10.1111/j.1467-9280.2008.02116.x

PubMed Abstruse | CrossRef Full Text | Google Scholar

Porter, S., ten Brinke, L., Bakery, A., and Wallace, B. (2011). Would I lie to yous? “Leakage” in deceptive facial expressions relates to psychopathy and emotional intelligence.
Pers. Individ. Diff. 51, 133–137. doi: 10.1016/j.paid.2011.03.031

CrossRef Total Text | Google Scholar

Porter, Southward., Ten Brinke, L., and Wallace, B. (2012). Secrets and lies: involuntary leakage in deceptive facial expressions as a function of emotional intensity.
J. Nonverbal Behav. 36, 23–37. doi: ten.1007/s10919-011-0120-seven

CrossRef Full Text | Google Scholar

Slowe, T. East., and Govindaraju, 5. (2007). “Automatic deceit indication through reliable facial expressions,” in
Paper Presented at the 2007 IEEE Workshop on Automatic Identification Advanced Technologies

Google Scholar

Su, L., and Levine, K. (2016). Does “lie to me” lie to you lot? An evaluation of facial clues to high-stakes deception.
Comp. Vis. Image Sympathise. 147, 52–68. doi: 10.1016/j.cviu.2016.01.009

CrossRef Full Text | Google Scholar

Ten Brinke, Fifty., MacDonald, S., Porter, Southward., and O’connor, B. (2012a). Crocodile tears: facial, verbal and trunk language behaviours associated with 18-carat and fabricated remorse.
Police force Hum. Behav. 36, 51–59. doi: ten.1037/h0093950

PubMed Abstract | CrossRef Full Text | Google Scholar

ten Brinke, L., and Porter, Due south. (2012). Cry me a river: Identifying the behavioral consequences of extremely loftier-stakes interpersonal charade.
Law Hum. Behav. 36, 469–477. doi: 10.1037/h0093929

PubMed Abstract | CrossRef Full Text | Google Scholar

X Brinke, Fifty., Porter, S., and Baker, A. (2012b). Darwin the detective: observable facial muscle contractions reveal emotional high-stakes lies.
Evol. Hum. Behav. 33, 411–416. doi: 10.1016/j.evolhumbehav.2011.12.003

CrossRef Full Text | Google Scholar

Van den Assem, M. J., Van Dolder, D., and Thaler, R. H. (2012). Split or steal? Cooperative behavior when the stakes are big.
Manag. Sci. 58, 2–20. doi: x.1287/mnsc.1110.1413

CrossRef Full Text | Google Scholar

Vrij, A. (2004). “13 guidelines to grab a liar,” in
The Detection of Charade in Forensic Contexts, eds P. A. Granhag and L. A. Strömwall (Cambridge: Cambridge University Press), 287.

Google Scholar

Vrij, A., Akehurst, L., Soukara, S., and Balderdash, R. (2006). Detecting deceit via analyses of exact and nonverbal behavior in children and adults.
Hum. Commun. Res. 30, 8–41. doi: 10.1111/j.1468-2958.2004.tb00723.x

CrossRef Total Text | Google Scholar

Vrij, A., Edward, M., Roberts, Thousand. P., and Bull, R. (2000). Detecting deceit via analysis of verbal and nonverbal behavior.
J. Nonverbal Behav. 24, 239–263. doi: 10.1023/A:1006610329284

CrossRef Full Text | Google Scholar

Vrij, A., and Isle of man, Southward. (2001). Who killed my relative? Police force officers’ ability to detect real-life high-stake lies.
Psychol. Crime Law
7, 119–132. doi: 10.1080/10683160108401791

CrossRef Full Text | Google Scholar

Witten, I. H., Frank, Due east., Hall, M. A., and Pal, C. J. (2016).
Data Mining: Practical Machine Learning Tools and Techniques, 4 Edn. Cambridge, MA: Morgan Kaufmann.

Google Scholar

Wright Whelan, C., Wagstaff, Chiliad., and Wheatcroft, J. G. (2015). High stakes lies: police and non-law accuracy in detecting charade.
Psychol. Crime Police
21, 127–138. doi: 10.1080/1068316X.2014.935777

CrossRef Full Text | Google Scholar

Wright Whelan, C., Wagstaff, G. F., and Wheatcroft, J. M. (2014). High-stakes lies: verbal and nonverbal cues to deception in public appeals for aid with missing or murdered relatives.
Psychiatry Psychol. Law
21, 523–537. doi: 10.1080/13218719.2013.839931

CrossRef Total Text | Google Scholar

Wu, Z., Singh, B., Davis, L., and Subrahmanian, Five. (2018). “Deception detection in videos,”
Paper Presented at the Proceedings of the AAAI Conference on Artificial Intelligence
(New Orleans, LA).

Google Scholar

Zhang, Z., Singh, V., Slowe, T. Due east., Tulyakov, S., and Govindaraju, V. (2007). “Existent-time automatic deceit detection from involuntary facial expressions,” in
Newspaper Presented at the 2007 IEEE Conference on Figurer Vision and Pattern Recognition
(Minneapolis, MN).

Google Scholar

Zuckerman, Grand., DePaulo, B. Yard., and Rosenthal, R. (1981). “Verbal and nonverbal communication of deception,” in
Advances in Experimental Social Psychology, Vol. xiv, eds S. Grand. Miller and B. Leonard (New York, NY: Academic Press), 1–59.

Google Scholar

Identify the True and False Statements About Facial Expressions