Design of computerized system contributed to the death of ten USN sailors

On 21 August 2017 the US Navy destroyer John S McCain collided with a civilian tanker near Singapore, resulting in the deaths of ten US sailors, and $100M in damage. The US National Transportation Safety Board report found the probable cause of the collision was "... a lack of effective operational oversight of the destroyer by the US Navy ...". However, also contributing to the accident was the computerized steering system: "Also contributing to the accident was the operation of the steering system in backup manual mode, which allowed for an unintentional, unilateral transfer of steering control.". This would be a useful report for students of safety critical systems to study.

John S McCain Bridge Control Station.
Drawing from IBNS technical manual;
color added by NTSB. Figure 4 of
NTSB/MAR-19/01 PB2019-100970.

The destroyer was equipped with bridge control stations with flat-panel touch screens, and a graphical user interface (GUI), in additional to a conventional steering wheel. The stations had an "emergency override to manual" function activated by what the crew referred to as the "big red button". This was intended to provide manual control in the event of a computer malfunction, and the designers no doubt thought it was foolproof: press the red button and steer the ship with the wheel.

However, as the NTSB detailed, the crew unintentionally
transferred control of steering from one station to another, but interpreted this as a failure of steering. This confusion may have been because the crew were uncomfortable with the automated mode of the system, and preferred to use the backup manual mode. However the backup mode was not intended for normal use, and allowed the control to be transferred without the operator noticing.

The  NTSB recommended crew being instructed to only operate the system in manual mode during an emergency. However, the underlying problem appears to be that the crew did not trust the automated system. This would require training the crew so they felt they could rely on the system, or to redesign the system to provide more intuitive feedback. Part of the intuitive feedback, I suggest, could be via manual controls.

The Bridge Control Stations have a ship's wheel, but this is an input only device, and there are no physical engine throttles, just a GUI display. One way aircraft cockpit interfaces display the operation of the autopilot is by physically moving the throttle levers. Similarly, in aircraft with a control yoke  the control moves in response to auto-pilot commands, and also provides tactile and visual feedback of the control responses put in by the co-pilot. The autopilot can be overridden simply by moving the yoke. If implemented on the ships bridge control, this would provide intuitive feedback as to who is in control, and an intuitive way to take control. The operator would be able to see, and feel, inputs through the wheel and throttles. If they wanted to override the automated system, or another operator, they just need to move the controls.

Some modern aircraft lack the visual and tactile feedback in controls, particularly those using side-stick controllers, rather than a yoke. However, in aircraft pilots receive intensive training in the use of these systems, and are sitting close to each other in the cockpit, so are usually able to see and hear what each other are doing. Even so, conflicting inputs have lead to aircraft accidents.  On a ship the operators are much further apart, which makes coordination much more difficult.

Reference

Collision between US Navy Destroyer John S McCain and Tanker Alnic MC Singapore Strait, 5 Miles Northeast of Horsburgh Lighthouse
August 21, 2017, Marine Accident Report, National Transportation
Safety Board, NTSB/MAR-19/01 PB2019-100970, Notation 58325
Adopted June 19, 2019 URL https://assets.documentcloud.org/documents/6243999/MAR1901.pdf