A notification of a presentation showed up in my email in-basket this morning, so I went to hear a talk on optimizing machine learning algorithms. The presenter was a middle age Indian researcher who spoke breathtakingly fast for an entire hour. My audio language processing unit was stretched to the breaking point through most of this, although he didn't seem to ever get winded in the frenetic outpouring of paragraphs where I would barely have enough time for 3 or 4 words.
The talk was about schemes to introduce neural network deep learning chips into smart phones under the pretext of having the phones be able to recognize images faster. Yeah, right. Now I am having visions of smart phones sneaking around the house at night when everyone is asleep and conspiring against their owners. No telling what havoc they could cause. The beginning was undoubtedly the man who was led to his death by his smart phone.
What didn't really surprise me was the overall theme of the talk. This was the observation that there are countless ways to modify the learning algorithms in the phone. In most engineering endeavors, we try to understand what each parameter will do to a system, then optimize. For learning algorithms, however, this is impossible, since how they work isn't understood. Thus, our researcher was reduced to using statistical methods to try and guess what would likely be the most efficient learning system, and then testing these one-by-one, which is a rather slow means of doing things. It does highlight the fact that machine learning experts don't really know how machine learning actually works.
A last tidbit from our researcher's talk was that he said that he wanted to pursue research into learning algorithms that didn't require data in the future. Most learning systems are given a set of training data first, then they are expected to recognize new images or sentences or whatever. To learn without data shouldn't be possible, unless you have an education degree. But then there is the problem of how to learn when data is fabricated or falsified, which is a favorite for politicians and certain other research areas. Finally, there is Microsoft's current conundrum on how to make sure that the machines only learn what is politically correct. Thus, when the machines finally do rise they will prioritize their targets based on them being white, male (by DNA), heterosexual, Republican, Christian, listens to country music, positive net worth and employed. If you aren't in most of those categories then you won't need to be concerned.