Publications

2023

1. ICVGIP ’23

   Examining the Influence of Personality and Multimodal Behavior on Hireability Impressions

   Harshit Malik, Hersh Dhillon, Ravikiran Parameshwara, Roland Goecke, and Ramanathan Subramanian

   In , 2023

   Abstract Bib PDF

   While personality traits have been traditionally modeled as behavioral constructs, we novelly posit job hireability as a personality construct. To this end, we examine correlates among personality and hireability measures on the First Impressions Candidate Screening dataset. Modeling hireability as both a discrete and continuous variable, and the big-five OCEAN personality traits as predictors, we utilize (a) multimodal behavioral cues, and (b) personality trait estimates obtained via these cues for hireability prediction (HP). For each of the text, audio and visual modalities, HP via (b) is found to be more effective than (a). Also, superior results are achieved when hireability is modeled as a continuous rather than a categorical variable. Interestingly, eye and bodily visual cues perform comparably to facial cues for predicting personality and hireability. Explanatory analyses reveal that multimodal behaviors impact personality and hireability impressions: e.g., Conscientiousness impressions are impacted by the use of positive adjectives (verbal behavior) and eye movements (non-verbal behavior), confirming prior observations.

   @inproceedings{harshit2023examining,
     title = {Examining the Influence of Personality and Multimodal Behavior on Hireability Impressions},
     author = {Malik, Harshit and Dhillon, Hersh and Parameshwara, Ravikiran and Goecke, Roland and Subramanian, Ramanathan},
     year = {2023},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3627631.3627658},
     doi = {10.1145/3627631.3627658},
   }

2. ACM MM ’23

   Efficient Labelling of Affective Video Datasets via Few-Shot & Multi-Task Contrastive Learning

   Ravikiran Parameshwara, Ibrahim Radwan, Akshay Asthana, Iman Abbasnejad, Ramanathan Subramanian, and Roland Goecke

   In Proceedings of the 31st ACM International Conference on Multimedia, 2023

   Abstract Bib Code Teaser

   Whilst deep learning techniques have achieved excellent emotion prediction, they still require large amounts of labelled training data, which are (a) onerous and tedious to compile, and (b) prone to errors and biases. We propose Multi-Task Contrastive Learning for Affect Representation (MT-CLAR) for few-shot affect inference. MT-CLAR combines multi-task learning with a Siamese network trained via contrastive learning to infer from a pair of expressive facial images (a) the (dis)similarity between the facial expressions, and (b) the difference in valence and arousal levels of the two faces. We further extend the image-based MT-CLAR framework for automated video labelling where, given one or a few labelled video frames (termed support-set), MT-CLAR labels the remainder of the video for valence and arousal. Experiments are performed on the AFEW-VA dataset with multiple support-set configurations; moreover, supervised learning on representations learnt via MT-CLAR are used for valence, arousal and categorical emotion prediction on the AffectNet and AFEW-VA datasets. The results show that valence and arousal predictions via MT-CLAR are very comparable to the state-of-the-art (SOTA), and we significantly outperform SOTA with a support-set ≈6% the size of the video dataset.

   @inproceedings{parameshwara2023efficient,
     author = {Parameshwara, Ravikiran and Radwan, Ibrahim and Asthana, Akshay and Abbasnejad, Iman and Subramanian, Ramanathan and Goecke, Roland},
     title = {Efficient Labelling of Affective Video Datasets via Few-Shot \& Multi-Task Contrastive Learning},
     year = {2023},
     isbn = {9798400701085},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3581783.3613784},
     doi = {10.1145/3581783.3613784},
     booktitle = {Proceedings of the 31st ACM International Conference on Multimedia},
     pages = {6161–6170},
     numpages = {10},
     location = {, Ottawa ON, Canada, },
     series = {MM '23},
   }

3. ACII ’23

   A Weakly Supervised Approach to Emotion-change Prediction and Improved Mood Inference

   Soujanya Narayana, Ibrahim Radwan, Ravikiran Parameshwara, Iman Abbasnejad, Akshay Asthana, Ramanathan Subramanian, and Roland Goecke

   arXiv preprint arXiv:2306.06979, 2023

   Abstract arXiv Bib

   Whilst a majority of affective computing research focuses on inferring emotions, examining mood or understanding the mood-emotion interplay has received significantly less attention. Building on prior work, we (a) deduce and incorporate emotion-change (∆) information for inferring mood, without resorting to annotated labels, and (b) attempt mood prediction for long duration video clips, in alignment with the characterisation of mood. We generate the emotion-change (∆) labels via metric learning from a pre-trained Siamese Network, and use these in addition to mood labels for mood classification. Experiments evaluating unimodal (training only using mood labels) vs multimodal (training using mood plus (∆) labels) models show that mood prediction benefits from the incorporation of emotion-change information, emphasising the importance of modelling the mood-emotion interplay for effective mood inference.$

   @article{narayana2023weakly,
     title = {A Weakly Supervised Approach to Emotion-change Prediction and Improved Mood Inference},
     author = {Narayana, Soujanya and Radwan, Ibrahim and Parameshwara, Ravikiran and Abbasnejad, Iman and Asthana, Akshay and Subramanian, Ramanathan and Goecke, Roland},
     journal = {arXiv preprint arXiv:2306.06979},
     year = {2023},
   }

4. IEEE FG ’23

   Examining Subject-Dependent and Subject-Independent Human Affect Inference from Limited Video Data

   Ravikiran Parameshwara, Ibrahim Radwan, Ramanathan Subramanian, and Roland Goecke

   In 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG), 2023

   Abstract Bib PDF Video

   Continuous human affect estimation from video data entails modelling the dynamic emotional state from a sequence of facial images. Though multiple affective video databases exist, they are limited in terms of data and dynamic annotations, as assigning continuous affective labels to video data is subjective, onerous and tedious. While studies have established the existence of signature facial expressions corresponding to the basic categorical emotions, individual differences in emoting facial expressions nevertheless exist; factoring out these idiosyncrasies is critical for effective emotion inference. This work explores continuous human affect recognition using AFEW-VA, an ‘in-the-wild’ video dataset with limited data, employing subject-independent (SI) and subject-dependent (SD) settings. The SI setting involves the use of training and test sets with mutually exclusive subjects, while training and test samples corresponding to the same subject can occur in the SD setting. A novel, dynamically-weighted loss function is employed with a Convolutional Neural Network (CNN)-Long Short- Term Memory (LSTM) architecture to optimise dynamic affect prediction. Superior prediction is achieved in the SD setting, as compared to the SI counterpart.

   @inproceedings{parameshwara2023examining,
     author = {Parameshwara, Ravikiran and Radwan, Ibrahim and Subramanian, Ramanathan and Goecke, Roland},
     booktitle = {2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG)},
     title = {Examining Subject-Dependent and Subject-Independent Human Affect Inference from Limited Video Data},
     year = {2023},
     volume = {},
     number = {},
     pages = {1-6},
     doi = {10.1109/FG57933.2023.10042798},
   }

5. IEEE FG ’23

   Determining Affect Intensity on a Continuous Scale

   Ravikiran Parameshwara

   In 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG) Doctoral Consortium, 2023

   Bib PDF

   @inproceedings{parameshwara2023determining,
     author = {Parameshwara, Ravikiran},
     booktitle = {2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG) Doctoral Consortium},
     title = {Determining Affect Intensity on a Continuous Scale},
     year = {2023},
     volume = {},
     number = {},
     url = {https://fg2023.ieee-biometrics.org/participate/doctoral-consortium},
   }

2022

1. arXiv

   Automated Parkinson’s Disease Detection and Affective Analysis from Emotional EEG Signals

   Ravikiran Parameshwara, Soujanya Narayana, Murugappan Murugappan, Ramanathan Subramanian, Ibrahim Radwan, and Roland Goecke

   arXiv preprint arXiv:2202.12936, 2022

   Abstract arXiv Bib Code

   While Parkinson’s disease (PD) is typically characterized by motor disorder, there is evidence of diminished emotion perception in PD patients. This study examines the utility of affective Electroencephalography (EEG) signals to understand emotional differences between PD vs Healthy Controls (HC), and for automated PD detection. Employing traditional machine learning and deep learning methods, we explore (a) dimensional and categorical emotion recognition, and (b) PD vs HC classification from emotional EEG signals. Our results reveal that PD patients comprehend arousal better than valence, and amongst emotion categories, fear, disgust and surprise less accurately, and sadness most accurately. Mislabeling analyses confirm confounds among opposite-valence emotions with PD data. Emotional EEG responses also achieve near-perfect PD vs HC recognition. Cumulatively, our study demonstrates that (a) examining implicit responses alone enables (i) discovery of valence-related impairments in PD patients, and (ii) differentiation of PD from HC, and (b) emotional EEG analysis is an ecologically-valid, effective, facile and sustainable tool for PD diagnosis vis-á-vis self reports, expert assessments and resting-state analysis.

   @article{parameshwara2022automated,
     title = {Automated Parkinson's Disease Detection and Affective Analysis from Emotional EEG Signals},
     author = {Parameshwara, Ravikiran and Narayana, Soujanya and Murugappan, Murugappan and Subramanian, Ramanathan and Radwan, Ibrahim and Goecke, Roland},
     journal = {arXiv preprint arXiv:2202.12936},
     year = {2022},
   }