Parsing Our Silent Language
Parsing the Silent Language is software that analyzes nonverbal language using a Kinect and the C++ framework, Cinder. Coming from a movement background, I was curious why sentiment analysis placed all emphasis on our words when our body language can contradict or add depth to our words. I started this process by first manually, and then algorithmically, categorizing my own body language. However, I was inadvertently programming my own movement tendencies and as a result, the algorithms became biased to my own body. To address the issue, I enlisted the help of several graduate acting students and recorded their skeletal data while they acted out various emotions. I then used this data to train a classifier using machine learning, which now analyzes body language in real time.
Met you at Maker Faire! I was the traceur data scientist 🙂
You might want to also look a pitch, candence etc. in addition to speech and body language.
There are some interesting research papers on detecting sarcasm for example through sound.
Also you may want to try K-fold cross-validation instead of a holdout method.
Hi Alan!
That’s a very good point, adding pitch and cadence would give a much better picture of what I’m analyzing. I’ll also look into into the K-fold cross-validation technique! Thanks for checking this out.