Happiness tracking software could gauge mood in photos
1
24 | NewScientist | 27 April 2013 TECHNOLOGY MIND reading can be as simple as slapping a sticker on your forehead. An “electronic tattoo” containing flexible electronic circuits can now record some complex brain activity as accurately as an EEG. The tattoo could also provide a cheap way to monitor a developing fetus. The first electronic tattoo appeared in 2011, when Todd Coleman at the University of California, San Diego, and colleagues designed a transparent patch containing electronic circuits as thin as a human hair. Applied to skin like a temporary tattoo, these could be used to monitor electrophysiological signals associated with the heart and muscles, as well as rudimentary brain activity. To improve its usefulness, Coleman’s group has now optimised the placement of depression could wear the tattoo for an extended period, allowing it to help gauge whether medication is working. “The number one advantage is the medical ease of application,” says Michael Pitts of Reed College in Portland, Oregon. Because its electronic components are already mass-produced, the tattoo can also be made very cheaply. That means it might also lend itself to pregnancy monitoring in developing countries. With help from the Bill & Melinda Gates Foundation, Coleman’s group is working on an unobtrusive version of the tattoo that monitors signals such as maternal contractions and fetal heart rate. n the electrodes to pick up more complex brainwaves. They have demonstrated this by monitoring so-called P300 signals in the forebrain. These appear when you pay attention to a stimulus. The team showed volunteers a series of images and asked them to keep track of how many times a certain object appeared. Whenever volunteers noticed the object, the tattoo registered a blip in the P300 signal. The tattoo was as good as conventional EEG at telling whether a person was looking at the target image or another stimulus, the team told a recent Cognitive Neuroscience Society meeting in San Francisco. The team is now modifying the tattoo to transmit data wirelessly to a smartphone, Coleman says. Eventually, he hopes the device could identify other complex patterns of brain activity, such as those that might be used to control a prosthetic limb. For now, the group is focusing on optimising the tattoo for use in conditions such as depression and Alzheimer’s disease, each of which have characteristic patterns of neural activity. People with Sara Reardon Get your grin on to light up the happiness tracker SMILE – you’re on camera! If you want a quick way to pick out the happiest snaps from a wedding or judge the changing mood of a crowd, now there’s software that can do it for you. Developed by Abhinav Dhall at the Australian National University in Canberra and colleagues, the software analyses all the faces in a photo to give the shot an overall “mood score”. The team used face tracking software to analyse the smiles of the faces in a group by noting the positions of nine spots on the face such as the corners of the mouth and eyes. A machine learning algorithm, trained on photos that had been pre-labelled by humans, then used this data to give each face a smile intensity score. The team also programmed the system to incorporate information from volunteers, who assessed how important the intensity of any individual’s smile was to the overall mood score of a photo. Those who were standing near the centre of a picture were given a stronger weighting, for example, while ADRIAN NEAL/GETTY ‘Tattoo’ reads your brain and bump An electronic patch monitors complex brainwaves, and can even listen in on the heartbeat of a fetus partially obscured faces were less influential. When asked to gauge the happiness level of a photo, the system only deviated from the opinion of a human by around 7 per cent. The software was presented at the Conference on Multimedia Retrieval in Dallas, Texas, last week. Dhall says the aim is to be able to assess the overall mood of a group from a single shot. By looking at a sequence of frames in a video, it could even gauge the mood of a crowd in real time. “If the mood score goes down over the time, we can assume that the group are getting angry,” says Dhall. It could also be used to view albums on Facebook by arranging photos so the happiest ones are shown first, for example. Ranael Kaliouby says the work fits well with what she is doing with her emotion analysis company, Affectiva. “I love the application – namely, understanding and modelling the overall mood of a crowd,” she says. “It gets us one step further to applying emotion measurement technologies in the real world.” Niall Firth n –What are you thinking?– –Listen to this beat– “If the mood score of the crowd in the shot goes down, we can assume people are getting angry” USCD JACOBS
Transcript of Happiness tracking software could gauge mood in photos
24 | NewScientist | 27 April 2013
TECHNOLOGY
MIND reading can be as simple as slapping a sticker on your forehead. An “electronic tattoo” containing flexible electronic circuits can now record some complex brain activity as accurately as an EEG. The tattoo could also provide a cheap way to monitor a developing fetus.
The first electronic tattoo appeared in 2011, when Todd Coleman at the University of California, San Diego, and colleagues designed a transparent patch containing electronic circuits as thin as a human hair. Applied to skin like a temporary tattoo, these could be used to monitor electrophysiological signals associated with the heart and muscles, as well as rudimentary brain activity.
To improve its usefulness, Coleman’s group has now optimised the placement of
depression could wear the tattoo for an extended period, allowing it to help gauge whether medication is working. “The number one advantage is the medical ease of application,” says Michael Pitts of Reed College in Portland, Oregon.
Because its electronic components are already mass-produced, the tattoo can also be made very cheaply.
That means it might also lend itself to pregnancy monitoring in developing countries. With help from the Bill & Melinda Gates Foundation, Coleman’s group is working on an unobtrusive version of the tattoo that monitors signals such as maternal contractions and fetal heart rate. n
the electrodes to pick up more complex brainwaves. They have demonstrated this by monitoring so-called P300 signals in the forebrain. These appear when you pay attention to a stimulus. The team showed volunteers a series of images and asked them to keep track of how many times a certain object appeared. Whenever volunteers noticed the object, the tattoo registered a blip in the P300 signal.
The tattoo was as good as conventional EEG at telling whether a person was looking at the target image or another stimulus, the team told a recent Cognitive Neuroscience Society meeting in San Francisco.
The team is now modifying the tattoo to transmit data wirelessly to a smartphone, Coleman says. Eventually, he hopes the device could identify other complex patterns of brain activity, such as those that might be used to control a prosthetic limb.
For now, the group is focusing on optimising the tattoo for use in conditions such as depression and Alzheimer’s disease, each of which have characteristic patterns of neural activity. People with
Sara Reardon
Get your grin on to light up the happiness trackerSMILE – you’re on camera! If you want a quick way to pick out the happiest snaps from a wedding or judge the changing mood of a crowd, now there’s software that can do it for you.
Developed by Abhinav Dhall at the Australian National University in Canberra and colleagues, the software analyses all the faces in a photo to give the shot an overall “mood score”.
The team used face tracking
software to analyse the smiles of the faces in a group by noting the positions of nine spots on the face such as the corners of the mouth and eyes. A machine learning algorithm, trained on photos that had been pre-labelled by humans, then used this data to give each face a smile intensity score.
The team also programmed the system to incorporate information from volunteers, who assessed how important the intensity of any individual’s smile was to the overall mood score of a photo. Those who were standing near the centre of a picture were given a stronger weighting, for example, while
Ad
riA
N N
EAL/
GET
TY
‘Tattoo’ reads your brain and bumpAn electronic patch monitors complex brainwaves, and can even listen in on the heartbeat of a fetus
partially obscured faces were less influential. When asked to gauge the happiness level of a photo, the system only deviated from the opinion of a human by around 7 per cent. The software was presented at the Conference on Multimedia Retrieval in Dallas, Texas, last week.
Dhall says the aim is to be able to assess the overall mood of a group from a single shot. By looking at a sequence of frames in a video, it
could even gauge the mood of a crowd in real time. “If the mood score goes down over the time, we can assume that the group are getting angry,” says Dhall. It could also be used to view albums on Facebook by arranging photos so the happiest ones are shown first, for example.
Ranael Kaliouby says the work fits well with what she is doing with her emotion analysis company, Affectiva. “I love the application – namely, understanding and modelling the overall mood of a crowd,” she says. “It gets us one step further to applying emotion measurement technologies in the real world.” Niall Firth n
–What are you thinking?–
–Listen to this beat–
“If the mood score of the crowd in the shot goes down, we can assume people are getting angry”
USC
d JA
COBS
130427_N_TechSpread.indd 24 23/4/13 09:58:42
