There’s been buzz about smartwatches for a few years now, and with popularity comes … security vulnerabilities. A researcher at IT University of Copenhagen is using deep learning to determine a wearer’s bank or debit card PIN by analyzing data from a smartwatch’s accelerometer and gyroscope. Basically, your own devices can be turned against you.
Software engineer Tony Beltramelli used a type of deep learning called Recurrent Neural Network–Long Short-Term Memory, or RNN-LSTM, to train an algorithm to recognize keystrokes on a number pad (like the ones on an ATM or your phone’s lock screen).
Beltramelli wrote a smartwatch app that used Bluetooth to send motion data to an Android smartphone. Then the smartphone delivered the data to a server for the deep-learning analysis. Feeding the algorithm tons of motion data from a Sony SmartWatch 3 eventually allowed it to distinguish when a user was typing on a number pad from other types of movement.
For typing on a smartphone numberpad, the algorithm was able to identify the keystrokes 73 percent of the time. For a numberpad like those in stores or at ATMs, the algorithm was 59-percent accurate. Overall, in a real-world scenario where the algorithm was getting all sorts of motion data, including data from multiple different numberpads in different conditions, it could accurately identify keystrokes 19 percent of the time.
Clearly it would take a lot of work and guessing for a malicious hacker to implement this attack right now and figure out your PINs. Movement data from some devices may also be harder to interpret than from others. And if you wear your smartwatch on your nondominant hand, which lots of people do, it’s less likely that the attack could succeed in identifying your PIN numbers, since you probably don’t enter them with the same hand the watch is on.
Still, there is a growing body of research related to using smartwatches for keystroke logging, and it could eventually become a robust type of attack. Beltramelli put his project on GitHub so others can contribute improvements or suggestions. “Deep neural networks are capable of making keystroke inference attacks based on motion sensors,” he wrote. “The complete technological ecosystem of a user can be compromised.”
