This question originally appeared on Quora.
Answer by Tikhon Jelvis, functional programmer and student:
A big one is correctness. Software tends to be slapdash at best: We like to ship anything that looks like it might not be completely broken. And, for most things, this is fine: If your favorite cat-photo-sharing app doesn’t have five nines of reliability, you’ll be fine.
However, as software becomes increasingly integral to everything, we really need to worry about quality. When failure can lead to hundreds of millions of dollars in damage (the explosion of the Ariane 5), actual deaths (Therac-25), or revealing the identity of dissidents to aggressive governments, software quality suddenly becomes a moral imperative.
Other forms of engineering have very specific ethical guidelines and professional organization. But since most software is decidedly not engineering in this sense, this model doesn’t make sense for all programming. So how do we manage risk and software quality?
Another possibility is enabling too much government power. In the past, a lack of capability has naturally held the government in check. They couldn’t shadow everyone always because they simply didn’t have the manpower. But now? The government can combine pervasive surveillance with data analysis to follow everyone everywhere and listen in on everything. A bit chilling, isn’t it?
As a particular example, we have cameras which excel at face recognition and can read license plates. It’s already practical to, for example, keep track of every single person in a city. In fact, to some extent, this already happens in places like London. Do you feel safer when you know they’re watching you?
Just imagine the Stasi or the KGB with modern technology. Our technology is an incredible force multiplier. Considering how well these governments functioned without it, this is a truly terrifying idea.
More questions on Quora:
