By Roland Piquepaille
This article starts with a quote from Douglas Adams.The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong, it usually turns out to be impossible to get at or repair.
It is true that machines are becoming more complex and "intelligent" everyday. Does this mean that they can exhibit unpredictable behavior like HAL, the supercomputer in 2001: A Space Odyssey? Do we have to fear our PCs?
The last book by Thomas M. Georges, Digital Soul: Intelligent Machines and Human Values, explains how our machines can develop neurosis and what kind of therapy exist.
Autonomous, goal-seeking machines that can reprogram their own goals and subgoals could, in effect, develop "minds of their own" and set off in unpredictable directions. If they create goals that make no sense whatsoever to us, then we may see those choices as "crazy." If you think that nutty people can wreak havoc, just imagine the potential for chaos when a supercomputer in charge of some critical aspect of our lives gets confused about its goals and purpose in life.
A neurotic behavior is usually associated with conflicting instructions sent to our brains.
So what would be the machine equivalent of a neurosis? Imagine that you are driving down a highway in your car and, slowly at first, you begin to apply more and more pressure to both the gas and brake pedals simultaneously. You notice the car's reaction, as it "tries to cope with" the conflicting forces that are simultaneously trying to speed it up and slow it down.
Easy to understand, but what will be the equivalent for our PCs?.
Your PC may experience conflicting demands when you try to run multiple programs, if its operating system is not well designed to handle such conflicts. The result may be that one or more of your programs will behave erratically, or the whole PC may shut down. Because its goal-seeking capabilities are very primitive, however, this is as close as your PC can get to becoming neurotic.
Is there a treatment for a PC's neurosis?
A cognitive approach to machine neuroses would create self-monitoring systems that scan for inconsistent or dangerous orders and would set corrective actions in motion.
If a machine can receive conflicting instructions that will cause confusion about its goals, the machine must have monitoring programs to, first of all, detect conflicts and inconsistencies and then to take corrective action. It could be trained, for example, to deal with certain classes of conflicts by questioning the apparently contradictory instructions or by requesting further information. For conflicts not specifically provided for, the machine could respond unpredictably. Since the causes and remedies of "crazy" machine behavior will eventually lie beyond the understanding of humans, the solution to Douglas Adams's dilemma posed at the beginning of this chapter may well require built-in mechanical psychologists and psychiatrists.
In this article, Georges also looks at the benefits and dangers of nanotechnologies. All in all, a highly recommended article.
Source: Thomas M. Georges, in Darwin Magazine, April 2003