Your laptop knows you’re sad

This picture has been used for representational purpose

It's the joke that many secretly wish would come true: Your phone automatically shutting down when it realises you're drunk, not allowing you to text an ex. But, in an AI world, that day might not be far away. A Pune-based team of five post-graduate students studying artificial intelligence and machine learning have spent the last year creating software that will collect data from your facial expressions over a period of time and assess whether you are happy or sad, to put it rather simplistically.

Rahul Choubey

The idea for XpressEmoNet, came when Rahul Choubey, Sajal Suryavanshi, Ankur Pathak, Pratik Salvi and Anvesh Reddy, students of Artifical Intelligence and Machine Learning at the Great Learning Institute, had to decide on a PG project last year. With each of them having worked in the IT industry, they understood the stress levels that most individuals face and how this could hamper both mental health and office productivity. And so, the programme was built to help an employer assess an IT professional's mental health through the webcam on their laptop.

Ankur Pathak

The idea is that XpressEmoNet, loaded on the computer, detects an individual's emotions based on facial expressions. It uses photos to detect these emotions and keep a track of an individual's state of mind on a regular basis. Explaining this, Choubey says, "At this stage the software can detect six emotions - sadness, contentment, happiness, fear, surprise and disgust. We have reached this level of accuracy after feeding in a lot data and testing expressions and their accurate description using facial expressions of group members as well as other photos."

Anvesh Reddy

The system, says Suryavanshi, learns and analyses pixels, the building structure of images. "With each change in facial expression, certain edges are created on the face. These are captured as pixels in a photograph. Understanding pixel values can then help the software detect the emotion on face. It also has a face recognition system through which it can detect a certain face in a group in order to further learn about the pixels on that specific face," adds Suryavanshi, who too, has been working in the IT industry for over 11 years now.

Dr Harish Shetty

While brainstorming on an idea for the project, use of artificial intelligence in the mental health space received the maximum consensus as everyone could relate to the problems pertaining to it, the group felt. "After all, using this model, a company can find out an employees' emotions and attend to them as soon as possible," he adds.

Pratik Salvi

The team feels that the workplace enforcement of the AI model, once enough data is collected, will be a good tool for human resource departments and employers, in keeping an enriching atmosphere at work, but also ensuring high productivity. More than that, when the AI detects emotions such as sadness, for eg, it could also trigger an instruction for corrective measures such as a mood-lifting activity or even advising if human or an expert intervention is required.

"We feel keeping a track of these emotions can help any employer design employee centric policies to create a nurturing environment at work. But at the same time, a continuous track-record of an individual's emotions can also give a glimpse of his/her mental health and required steps can be taken at the right time," says Choubey, who having completed his mechanical engineering degree, has been working in the IT industry for over six years now. Considering their backgrounds, the model is currently developed to suit an IT sector atmosphere, however, according to the team it can be altered to suit different work-environments.

Referring to existing systems, Choubey shares that many IT firms already have software installed on employee computers which remind users to blink their eyes every once in a while or to stand up or perform neck exercises. "These ideas too may have sounded futuristic when spoken about initially. But now it has become a regular part of the IT industry where employees spend long hours in front of the screen and suffer related health hazards. The world is quickly driving towards more data and data-based problem solving techniques. Analyzing large data of particular interest and then implementing it using AIML models etc has already made human lives a little easier," he adds. Still, the model isn't ready for the market yet. "Mental health is not something where a lot of concrete work is available by AIML," says Reddy, adding that their project still needs more research, including collaboration with mental health experts.

Prof. Ganesh Ramakrishnan, from the Department of Computer Science Engineering at the Indian Institute of Technology (IIT), Bombay, says, "AI has been used in facial feature recognition extensively. We have also developed products in this space. Facial features can give you a peep into an individual's mind, but it has to be used very carefully, as AI can go wrong. It can provide certain indications, but accuracy may not be 100 per cent. It can be used for assistance but not as an alternative to human inspection. I will oppose using anything fully automated to assess the human mind, firstly because such assessment can lead to stereotyping and secondly because AI can go wrong." He adds that using AI for data collection would also be an invasion of privacy. "Even those working in this much sensitive field, must understand that such a system may be used to assist in ease of operations, but very cautiously. AI tools can potentially bring in more consistency in assessment, however they must be used with extreme care."

Mumbai-based psychiatrist Dr Harish Shetty says the problems with trying to understand mental health on the basis of facial expressions are many. "It's a good idea but can never be foolproof. Expressions and mannerisms may differ in different cultures and also in different sub cultures. To manage so many variables in a program or in AI may not be easy. Algorithms for qualitative variables that are the hallmarks of mental illness are difficult to build. A pilot with a large sample may help reach a reasonable program to suspect depression. Such programs can help in screening that can help detect the illness early," he adds.

But, Dr Shetty says issues of consent and privacy can arise, "Monitoring individuals in this manner first needs consent as it violates human rights, especially in a professional setup. Use of such data should be done responsibly with a positive approach, for example more for inclusion efforts rather than focusing on exclusion of anybody."