"your instrumentation and outside visual cues"

The thread title is from the video “Children of the Magenta Line”. The lowest level of automation is instruments and visual.

Full Transcript

Okay, automation dependency—as I mentioned earlier today—the reason this segment is in this course is because, as we look at accident history, we find that in 68% of these accidents, automation dependency plays a significant part in leading these crews to either a critical flight attitude or the requirement to extract maximum performance from their planes.

Automation-dependent pilots allowed their airplanes to get much closer to the edge of the envelope than they should have. As we started to study this issue, we decided to take a new tack. So, you’re going to hear a lot more at American Airlines about what we’re going to call levels of automation and technology judgment.

What are levels of automation? Well, the academic types will tell you there are seven levels of automation or something like that. It gives me a headache. I’m a simple guy. I’m going to suggest to you that there are basically three levels of automation in our planes.

The lowest level, you might call manual, is when you’re hand-flying the airplane—the throttles and the controls—and by the inputs and cues you’re getting from your instrumentation and outside visual cues, you’re determining the vertical and lateral path of the plane by flying it.

The next level is when you have the autopilot engaged, and you’re using the flight guidance system to tell the autopilot what to do for short periods of time—flight level change, vertical speed, altitude hold, heading select.

The highest level is when the autopilot is engaged and you’re using the flight management computer (FMC) to tell the autopilot what to do for hours and hours.

So: low, medium, high.

Now the question becomes: which is the appropriate level for the task at hand?

You could say all this automation was put in two-crew airplanes to reduce workload, and that going up a level reduces workload. That would be true in many scenarios—like crossing the North Atlantic.

But you could also say that going down a level can reduce workload in certain scenarios.

What we find in the accident history is that in the vast majority of cases, the crew has lost situational awareness. How did that happen? They became task-saturated. And in most of those accidents, they became task-saturated because they were trying to operate at too high a level of automation in a rapidly changing flight path requirement.

What they needed to do was drop down a level in order to maintain situational awareness and reduce workload.

As an example, suppose you’re coming in on a STAR in an FMC airplane. You’ve probably had that STAR loaded in the computer for an hour, everything going as planned, you’re in LNAV and VNAV and stroking down the arrival.

But then, around 20,000 feet, ATC calls and says, “American so-and-so, change your arrival, turn left heading X, descend and maintain Y, intercept radial Z for the new arrival.”

At this point, is it time to go up a level or drop down a level?

It’s time to drop down a level.

Why? Because he said: turn, descend, intercept. That means: heading select, flight level change. Get the airplane going where he asked, and your head’s up. Then he said intercept a radial: tune it, identify it, present it. You must present raw data per American Airlines procedures. That’s required work.

Later, if you have time, you can type it all into the FMC and hook it up—but that should not be your initial move. Why? Because:

  1. The airplane isn’t going where you want yet.
  2. One person typing means the other must monitor. And both must verify the new flight path is correct. FMCs are not error-resistant.

You can input something wrong, and it will do it. You can input something right, and it might do something wrong due to a database anomaly. It takes two people to verify it—and that takes time. That path-saturates the crew. Drop down a level. Reduce workload. Stay safe.

Another example: Suppose you’re flying into Dallas on the Bowie Arrival on a perfect, clear day. You’ve had it programmed since Colorado Springs. Everything’s synced up and going great.

Then ATC says, “Change your runway to 13R.”

You’re 26 miles out. You see the airport. You can see the pavement. What do you do?

From highest level—LNAV/VNAV—you go straight to the lowest: hand-flying. You line yourself up with the pavement.

Meanwhile, your copilot is trying to help. You’re dealing with traffic, spacing, maintaining visual awareness. Is it a good time to be typing?

No.

Let me ask: what could the computer possibly bring you that you don’t already have? We’ve become, as I call it, children of the magenta. We think we need the magenta line and magenta V-bar, or the plane won’t fly.

Another case: A Fokker aircraft was cleared for 17L, then told to sidestep to 17R. Visual conditions, 10–12 miles out. They saw the airport, but instead of flying it visually, they tried to reprogram—and ended up landing without a clearance.

You’ve got to pick the appropriate level of automation for the task at hand.

To that end, we need to change the culture that tells us to operate at the highest level at all times. We created that culture. The industry did. But it needs to change.

That’s why we have a new course: Human Factors and Safety Training. It will focus on levels of automation and technology judgment.

The top bullet on the slide says: “Be the ball.” That means: stay connected. Don’t disengage just because the autopilot is on. Especially in patterns, configuration changes, and maneuvering—remain mentally and tactilely connected to the aircraft.

Quick example: Bucharest. An A310 with an autothrottle malfunction, nose-high attitude, no visible horizon. One throttle jammed, the other went to idle. The airplane banked over and rolled nearly inverted. If the copilot had his hand on the throttles, 128 people would still be alive.

Autopilots have limitations. Worse, automation-dependent crews often lack confidence in their ability to fly. So they turn to the autopilot in deteriorating situations—and instead of saving them, it kills them.

We’ve seen it in mode confusion, terrain warnings, upsets, and even mid-air conflicts.

Let me tell a story about a 757/767 sim ride. I had a captain and FO, very sharp, nailed their check rides. On the last day, I gave the captain a scenario from Orange County to San Jose, with an MD-80 sim target on a collision course at 5,000 feet.

As the captain coasted toward the beach, he hooked up his autopilot. Then, seeing the MD-80: “Give me vertical speed. Give me flight level change.”

He was trying to avoid a mid-air with the autopilot. That’s not what we trained him for. But it was understandable—he’d been typing since day one.

What should he have done?

Click-click, hand-fly. The pilot with the controls and throttles will do what it takes not to hit a plane he can see.

The autopilot cannot leave altitude in less than 0.23 NM. It’s in the algorithms. It doesn’t understand urgency.

Disconnect. Hold the controls firmly. Test pilot Dale Rantz says: the autopilot may have started correcting—don’t disconnect with a weak grip and have the yoke jump out of your hands. Start there and take control.

The red box on the slide—this one isn’t in your manual. This is from us to you. We must return to the principle of fly the plane first.

We used to say it. We used to do it. But automation crept in. We were told to become “automation managers.” But the accident history says automation managers are plugging themselves into the ground all over this planet.

We are not automation managers.

We are captains and pilots. We will use automation as a tool, but when it doesn’t fly the path we need, we will turn it off and fly.

Example: If you’re on a non-precision approach, in vertical speed, and the plane will descend through MDA—do you push “altitude hold”?

No. You disconnect. You can stop it. The autopilot can’t.

What’s the most asked question in the 757/767 cockpit?

“What’s it doing now?”

If that’s asked in a low-altitude environment, the pilot flying should disconnect. Maintain the intended flight path. Stabilize. Then ask why.

Maintain your flying skills.

If you turn off the autopilot but leave the autothrottles on, are you really flying?

No—you’re just guiding the airplane to the V-bars.

To maintain piloting skills, we must both manipulate the controls and manage the energy. That requires a full cross-check and engagement.

Cecil won’t order you to do this. He’s a leader. But he is saying: as often as you can, when appropriate, practice. Turn off the autopilot and autothrottles. Fly your airplane. Maintain your skills.

So that when something happens, you’ll have the proficiency and confidence to take over and fly any situation successfully.

We haven’t changed procedures. What we are saying is:

When you intend to hand fly to maintain skills, turn off both the autopilot and the autothrottles.

We are pilots. Captains. But also cockpit managers.

We must manage automation—but not be ruled by it.

Increasing automation reduces workload most of the time.

But in some cases, it increases task saturation, reduces situational awareness, and leads to accidents.

Automation cannot generate flexible responses to unanticipated changes.

In those moments—drop down a level.

And fly the plane.

One scenario in the video is given a late change by ATC (Air Traffic Control) of the assigned runway. The question is should the crew reprogram the FMC (Flight Management Computer) or disengage the auto pilot “And fly the plane.” (using “instruments and visual”)

Not perfectly analogous to marine navigation but a similar situation might be approaching the pilot station when the pilot cancels and the ship told to anchor.

Should the bridge team reprogram the ECDIS or just do a round turn and navigate to the anchorage using the instruments and visual?

It doesn’t make sense to ignore the ECDIS, radar and other instruments altogether. It’s about workload, not being some kind of purist.

1 Like

Not even close enough to be considered an analog. ATC in that case is approach control meaning the aircraft is only minutes or moments from landing and has configured for the runway expected or “cleared for” such and such approach. ATC would give a heading and altitude (and possibly a speed restriction) that provides enough time to permit reprogramming and re-briefing the approach and landing.

yea - time is a big factor here. switching to the ship scenario - in general things happen pretty slowly at sea. Don’t think this is a choice between one or the other - more like prioritize the order in which you do or delegate everything.

1 Like

I agree and I think this is the key point. It’s about workload.

In most situations, visual is “good enough” to act as a fast, intuitive filter. That allows the watch officer to prioritize the information which requires the accuracy the instruments provide.

1 Like

In the Dreyfus Model of Skill Acquisition the difference between being competent (level 3) and expert (level 5) is the development of intuition.

In Dreyfus’s paper System 0: the overlooked explanation of expert intuition (pdf) he writes that . “Expertise involves pattern recognition”

That’s a very large part of what the brain does, it subconsciously finds patterns in visual information.

The ability to confidently use visual information is why, in similar situations, the workload for an experienced deck officer is so much lower than for the inexperienced officer.

The value of instruments is obvious but in the training of deck officers the power and usefulness of the human visual system is underrated and overlooked.

Absolutely - pattern recognition is a very powerful tool. But we must also remain cognizant that is events conspire to fool that mechanism, the potential for catastrophic error is present.
For example, when entering a port in reduced visibility we spot what we are convinced is Island A - but in fact it is Island B - the resulting delusion of not knowing our actual position can result in serious consequences. We must be willing to continuously examine our assumptions, no matter how “expert” we are.

2 Likes

In that scenario wouldn’t there be some kind of mismatch between instruments and visual? Are navigation decision going to be made based on this confusion with no cross-check with the instruments?

Failure to cross-check is not an error a competent (stage 3) mariner would make but even a novice (stage 1) could avoid that error by simply following the rules.

Stage What Happens
1. Novice Follows context-free rules; no experience of the whole situation
2. Advanced Beginner Begins recognizing situational patterns; still depends on rules
3. Competent Can plan and make decisions; begins managing complexity
4. Proficient Perceives situations holistically; intuition begins to guide action
5. Expert Performs fluidly and intuitively; no need for explicit rules
2 Likes

From Bowditch (2024 ed) Chapter 11 piloting wrt navigation in restricted waters.

“ECDIS is a good instrument to monitor the vessels track, however, the prudent navigator should continue to actively plot positions.”

The navigator should always remember that reliance on any
single navigation system courts disaster.

That would be rule-based. Doesn’t seem plausible that the OOW would get confused in the low-visibility scenario.

Well, yes - but perhaps I wasn’t clear: The “pull” of “knowing where you are” (or something similar) can be very powerful, leading even pros to ignore other cues.
The most famous example I can think of offhand is the airliner that emerged from the clouds on an instrument approach, saw an illuminated runway dead ahead, and proceeded to land on that runway - 10 miles short of the airport they were actually intending to land on. [This has happened multiple times, btw] They were so used to making the transition from instrument to visual that they ignored all the instrument cues and made the visual approach.
Could this happen to a mariner? I bet it could.

If the navigator is doing it right, of course, then everything works fine - he identifies his visual error and corrects his internal navigation - but not everyone does it right all the time :slight_smile:
My point was that visual errors can be very powerful and cause one to ignore other cues. Edit to add: this problem is MOST likely to affect someone in the higher categories you mention :slight_smile:

1 Like

Of course even experts can make errors but this seem like a bit of a straw man argument, not making the claim that mariners should ignore instruments.

As far as landing on the wrong runway, on the maritime side that’s why deep-sea ships take local pilots.

“straw man argument…” No, that incident really happened (several times) though its applicability to mariners is certainly debatable.

The point I was trying to make was that visual errors CAN be very powerful, and one should remain vigilant and check with every source (even when the visual cue seems good) - which I think was the same thing you were saying all “expert” mariners would do anyway.
I don’t know if that Captain qualifies as an “expert”, but I seem to recall a ferry returning from Nantucket to Hyannis who mis-identified a buoy one night and ran up on the breakwater.

My perspective comes from direct observations of how watch officers perform their duties and my own experience (including federal pilotage in SE Alaska).

Many junior deck officers rely so heavily on instruments that they could stand watch with the wheelhouse windows painted over, an exaggeration but not by much, they’d still need one window for the AB / lookout.

Workload is significantly reduced when officers use visual observations with instrument rather than depending solely on instruments.

That is why most airlines prohibit circle to land approaches as described in the OP. On the other hand, every landing, except by autoland and in very specific criteria with crew, airport, and aircraft certification, involves a visual approach when hand flying once the runway is in sight at or above minimum ceiling and visibility limits. Even at that point, when precision guidance is available it must be used and cross referenced with visual cues. The legitimate opportunity for “seat of the pants” approaches is very limited these days.

1 Like

This is from Stuart E Dreyfus’ paper System 0: the overlooked explanation of expert intuition

Our current educational system almost exclusively values System 1’s declarative knowledge and System 2’s rational decision-making.

That describes the typical maritime academy education.

System 1’s, declarative knowledge is essential for to use tools such as ARPA and ECDIS. System 2 is used in voyage planning. Both necessary! System 0, intuitive, experience based behavior however is often overlooked.

From the same paper:

We probably all have known drivers who,while learning,have found System 1 and 2 thinking so stressful and exhausting that they give up before letting System 0 render driving as natural as walking.

In my experience many junior deck officers find watchstanding “stressful and exhausting” compared to their more experienced peers but they view non-instrument skills as outdated or irrelevant and no longer necessary. Or believe such skills are too difficult to master.

Good judgement is required for situational awareness, but not all the information an experienced mariner relies on can be digitized or displayed on a screen. The “seaman’s eye” remains a critical tool.

1 Like

The phrase from COLREGS works here “all available means appropriate to the prevailing circumstances and conditions” Night and day are different conditions.

We call it a “cross-check” but it’s really a loop in that each observation (visual or instrument) is part of a back-and-forth where the visual observation affects how we use the instruments and the instruments affect how and where we look.

1 Like

I’ve posted this before:

The vector is labeled “18 kts” Channel is 6 miles long and 4 miles wide and marked by 4 buoys. - Daylight, clear weather.

Radar, ECDIS - all nav equipment on and fully operational. Captain, third mate, AB helmsman lookout on the bridge.

Should a junior deck officer be able to make use of visual information in this situation? Or should they be limited to only being capable of keeping the ship icon close to the electronic track-line on ECDIS?

If height of eye permits, one should be able to determine own rough position by dint of sighting the two nearest buoys, and thus the time to alter course. I’m not sure what else you’re asking.

Not only would they not believe it prudent to turn on the buoy they also would depend 100% on the ECDIS for the 20 minutes it takes to transit the buoyed channel. They’d only take their eyes off the ECDIS for a occasional, quick, nervous glance out the window, or a quick look at the ARPA.

Most green deck officers are going to assume you’re some kind of Luddite if you try to explain that visual observation can be useful in cases like this.

all but a brand new 3M should be able to do this from the bridge wing -

1 Like