“…anyone who has used a voice-enabled navigation system will have encountered the ‘recalculating’ phenomenon when you miss a turn. I admit that after several of these incidents, I’m convinced the robot speaker sounds increasingly impatient and cynical, leaving off the implied ‘you idiot’ at the end of its directions just to preserve my feelings. I’ve found myself talking back to it, even though I’m fully aware that the device has no microphone and is completely deaf to my remarks.
While I rather doubt we’re in any immediate danger, Turkle’s stories and reports raise in my mind a risk factor that the more natural we try to make these interactions, the more intelligence we might assume these robotic devices have. We might rely on them to behave in intelligent ways far beyond their actual capacity…. The net result of these observations is a caution for designers of ‘intelligent’ tools and applications. The more natural we try to make the systems, the more their users might assume intelligence beyond the system’s capacity.”
From Vint Cerf’s review of Sherry Turkle’s Alone Together: Why We Expect More from Technology and Less from Each Other.