In the late 1980s, somebody in IBM sent out a clever email Christmas card. When the recipient opened the email, a little script inside the card would be activated. The script would read the recipient's list of contacts and forward the Christmas card, script and all, to all those contacts. It was only a matter of hours before the whole corporate network was flooded beyond capacity. Unintended consequences can be expensive!
Around that same time, I helped a young programmer get a phone call directly from the Rochester, Minnesota IBM site manager. The Rochester site had a large mainframe cluster that was used both for design tasks and to run a large factory. Many of the folks who pioneered the VHDL circuit design language were also from Rochester. I was part of the team in Poughkeepsie, NY, that was helping to develop a simulator for VHDL. It was a prototype simulator. where practically every line of code in the VHDL circuit design would get translated into its own little program to be compiled. We had the clever idea of farming the resulting thousands of compile jobs out to the Rochester cluster - after all, it was their language! We got a little help from a guy in San Jose who'd written a very powerful language that gave programmers control over all sorts of esoteric disk drive functions. We wanted all the thousand little separate simulation programs to get linked together when the last compile job finished. So we put a counter in a file on the cluster scratch disk. We needed to serialize access to the counter, so that each decrement would be a integral transaction. The San Jose guy showed us how the lock the whole drive - fast and simple! Yup, that's right, we were locking the scratch disk on one of the biggest mainframe clusters in IBM, thousands of times, like an attack of a swarm of hornets. The site manager was not happy about that at all! We probably shut down the site for no more than half an hour or so. That was probably the most expensive bug in my career! Unintended consequences - at least they didn't fire me!
When I was in high school, around 1970, you could still see the idea around that somehow computers could be so smart that we could delegate crucial decisions to them, on questions like whether to launch a nuclear attack NOW! So many little bits of evidence need to be weighed to determine whether potential enemy nation X has already launched an attack on us - but we need to decide whether to launch a counterattack very promptly, before their missiles land and destroy our counterattack capability. It's hard to see how humans can be relied upon to make such decisions, but why can't the perfect reasoning power of a well oiled machine be employed?
This dream, of using reliable mechanical reasoning to circumvent the strife and corruption of human decision-making processes, goes back at least to Leibniz. Isn't it our deviation from pure logic that brings us all this misery?
The power and reliability of today's logical inference machines is stunning. Yet what we have created with that power is every bit as chaotic and corrupt as any other human institution. By now, computers have become so integrated into every facet of modern industrial living that human and machine chaos and corruption are seamlessly integrated. Somehow mechanical reasoning hasn't quite lived up to the dream of Leibniz.