Question

Interactive Session: People Are We Relying Too Much on Computers to Think for Us?

Does our ever burgeoning dependence on computers foster complacency, suppressing our ability to marshal our mental faculties when required? Although computerization has undoubtedly mitigated malfunctions, work stoppages, and breakdowns, are we concurrently losing our ability to assess alternatives independently and make optimal choices?

At least one technology writer is sure this is exactly what is happening. Nicholas Carr’s book, The Glass Cage: Automation and Us, lays out the case that our overreliance on computers has dulled our reflexes and eroded expertise. Two cognitive failures undermine performance. Complacency—overconfidence in the computer’s ability—causes our attention to wander. Bias—overconfidence in the accuracy of the data we are receiving from the computer—causes us to disregard outside data sources, including conflicting sensory stimuli.

When pilots, soldiers, doctors, or even factory managers lose focus and lack situational awareness, they ignore both suspect data coming from the computer and the external cues that would refute it. The results can be catastrophic. In two instances in 2009, commercial airplane pilots misinterpreted the signals when their autopilot controls disconnected after receiving warnings that the aircraft would stall. Rather than pushing the yoke forward to gain velocity, both pilots heeded faulty control panel data while ignoring environmental cues and pulled back on the yoke, lifting the plane’s nose and decreasing airspeed—the exact opposite of what was required. Loss of automation triggered confusion and panic. Sharply curtailed hands-on flight experience (on a typical passenger flight today, a human pilot mans the controls for just three minutes) resulted in stalled aircraft plunging to earth. Fifty died in Buffalo, New York; 228 perished in the Atlantic Ocean en route to Paris from Rio de Janeiro. The Federal Aviation Administration (FAA) is now pressing airlines to adopt stricter requirements for manual flying hours to offset the risks posed by complacency and bias.

Carr’s critics point out that air travel is now safer than ever, with accidents and deaths steadily declining over decades and fatal airline crashes exceedingly rare. Carr concedes this point but still worries that pilots have come to rely so much on computers that they are forgetting how to fly. Andrew McAfee, a researcher at the MIT Sloan School of Management, points out that people have lamented the loss of skills due to technology for many centuries, but on balance, automation has made the world better off. There may be a high-profile crash, but he believes greater automation, not less, is the solution.

Although humans have historically believed that allocating tasks to machines liberates us from the mundane and enables us to pursue the extraordinary, computers have ushered in an altogether different era. Massive data compilation and complex analytical capabilities now mean that decision making, heretofore the sole province of the human brain, is increasingly being accomplished by computers. Offloading tasks to computers liberates us from complex thinking while requiring us to pursue mundane tasks such as inputting data, observing output, and absentmindedly awaiting equipment failure.

Complacency and bias-induced errors are piling up. For example, computer programs now highlight suspect spots on mammograms. With the compulsion to examine images scrupulously relieved, radiologists are now missing some early-stage tumors not flagged by the program. Australian researchers found that accountants at two international firms using advanced auditing software had a significantly weaker understanding of the different types of risk than did those at a firm using simpler software that required them to make risk assessment decisions themselves. Even the most rudimentary tasks, such as editing and spell checking, are now performed differently. Rather than actively participating, we are observers, waiting to be told to correct an error. Are such short-term efficiencies worth the long-term loss of knowledge and expertise?

What’s more, software programs are shouldering ever more capabilities heretofore thought to be the exclusive domain of the human brain. Sensory assessment, environmental awareness, coordinated movement, and conceptual knowledge are included in programming that has enabled Google to begin testing its driverless cars on public roads. Some argue that this is precisely the direction in which we should be going: autonomous computers with no human oversight or intervention at all. The solution to pilot error during automation failures? A wholly autonomous autopilot. The solution to doctors’ declining diagnostic skills due to complacency and bias? Cut doctors out of the equation altogether.

Carr sees two problems with this thinking. First, complex computer systems require complex interdependencies among databases, algorithms, sensors, software, and hardware. The more mutually dependent elements there are in a system, the greater the potential points of failure and the more difficult they are to find. Second, we have known for more than three decades that humans are spectacularly bad at precisely the job that increased computerization has relegated to them: passive observation. When not actively engaged, our minds tend to drift off to any topic other than the one we are supposed to be monitoring. What’s more, because we now know that “use it or lose it” applies to flying airplanes, diagnosing illnesses, spell-checking, and everything in between, restricting humans to observation reduces experts to rookies, escalating the risk of improper responses to malfunctions.

One solution is to design programs that promote engagement and learning, for example, by returning control to the operator at frequent, but irregular, intervals or by ensuring that challenging tasks are included. If operators must perform and repeat complex manual and mental tasks, the generation effect will be reinforced. Unfortunately, introducing these changes necessarily includes software slowdown and productivity decline. Businesses are unlikely to value long-term expertise preservation and development over short-term profits. Who does this technology benefit in the long run?

Case Study Questions

Identify the problem described in this case study. In what sense is it an ethical dilemma?

Should more tasks be automated? Why or why not? Explain your answer.

Can the problem of automation reducing cognitive skills be solved? Explain your answer.

Answer 1

ANSWER 1:

The problem described in this case is related to the increasing dependence on automation. The world is rapidly adopting automation in every industry and this is leading to less human brain involvement which is making people inefficient at work. Since the human involvement is reducing, people are losing track of their cognitive skills.

It is an ethical dilemma because it is the time where we have to choose whether we need to have more automation or not. More automation may be advantageous and it would make all the operations easier and more efficient. But, it would reduce the need for labour. This would lead to fading of the cognitive skills of the people and those who are earning their living through labour would lose their jobs.

Yes, I think more tasks should be automated.

But, along with the increase in automation, the human skills should also be developed. There should be an integration of automation and manual labour where the tasks are done by machines but it should be mainly operated by humans. It means the machines should work only in the presence and involvement of humans. Sole dependence on the machines should not be promoted. This would ensure that the world gets benefit from the automation and the human skills are also used properly.

I support this idea because the world is facing newer challenges day by day due to the depletion of the nature. So, we need to be more modern and use automation for efficiency in work and minimum exploitation of natural resources. It is the need of the hour. But, we must integrate human skills and involvement with the automation to achieve better results.

ANSWER 2:

Yes, the problem of automation reducing cognitive skills can be solved. We can come up with alternatives where we use automation but it works with the supervision of humans. We should not allow the machines to work amd make all the decisions on their own. An integration of human skills and automation can help in solving this issue. For example, in the aircrafts, the pilots must use the autopilot feature but it should be modernized in such a way that the command still remains in the pilot's hands and he doesn't depend entirely on the machines. Also, some special trainings would help to overcome this issue.