Lieutenant Mark Jbeily, U.S. Navy, and Christian Heller
In June 2021, the Navy publicly announced a successful aerial refueling test between an autonomous MQ-25A Stingray and a manned F/A-18F Super Hornet. The test marked another milestone in the Stingray program and a significant step toward the broader integration of autonomous and unmanned platforms into carrier air wings. In an October 2020 forcewide memorandum, Chief of Naval Operations Admiral Michael Gilday directed the accelerated development of unmanned platforms to exploit and integrate an “any-sensor/any-shooter” kill-chain concept of operations.
The effective proliferation of unmanned systems supporting the kill chain will require a high degree of autonomy in the execution of combat and other aerial missions. While technical feasibility can be measured and reported in instances such as the Stingray refueling, a significant challenge remains: How to develop trust between the manned aircrew and the autonomous systems they will employ.
A pilot performs a preflight check on an MH-60R Seahawk. Naval aviators learn human-machine trust through these extensive preflight checks and ongoing diagnostics. Unmanned systems must conduct preflight checks for the software and hardware of each functional component to help human aviators build trust. U.S. Navy (Winter Griffith)
Warm and Fuzzy
Two primary trust relationships exist today that enable operational effectiveness: human-human trust among aircrew, and human-machine trust between aircrew and aircraft. Both relationships rely on continuously assessed diagnostics to give aircrew the “warm and fuzzy” feeling that their fellow aviators and their platforms will perform in a predictable, consistent, and effective manner.
For pilots and naval flight officers (NFOs), both trust relationships begin in flight school, where aspiring aircrew begin an extended, regimented, thoroughly documented, and proven training pipeline to earn their coveted Wings of Gold. Student pilots and NFOs go through hundreds of hours of academics, simulators, and flights to train and certify that they have achieved a basic level of knowledge and competence required for higher-level training in fleet replacement squadrons (FRSs) and the fleet. Wings symbolize an institutional trust in an individual’s ability as an aviator and certify that they have met the standards of an established and well-understood process.
Human-machine trust between aviators and their aircraft relies on transparency into system health and ongoing diagnostics. Before any flight, aircrew conduct various preflight checks and built-in tests to develop confidence that this particular aircraft on this particular flight will perform as expected. These extensive and well-documented checks build human-machine trust by relying on human-human trust. Thousands of flight hours in an aircraft develop familiarity between a pilot and his or her aircraft, and a plane captain’s sign-off of preflight checks reinforces that human-machine trust.
Understanding Trust in AI
The foundation for trusting an autonomous wingman exists in the proven processes of human-human and human-machine trust-building already present in naval aviation. Earlier this year, Deputy Secretary of Defense Kathleen Hicks published the DoD “Memorandum on Implementing Responsible AI” (RAI).1 As laid out by the memorandum, RAI includes six tenets, one of which specifically focuses on the trust between human and machine.
Warfighter trust is particularly important to naval aviation’s impetus for manned-unmanned teaming. Building warfighter trust in an unmanned wingman relies primarily on building trust in the AI controlling the autonomous system. While AI models and decisions can be measured with a variety of technical metrics, trust in an AI system is not an internal measurement but rather the relationship formed between the human and the machine. Just as with human wingmen and aircraft, trust does not come prepackaged in the hardware. It is built over time through repeated interactions, transparency, and continuously assessed diagnostics.
Trust with an AI can be subdivided into three dimensions: performance, operations, and ethics.
Performance. Trust in the AI performance derives from the system acting as intended. A high-performing AI must make consistent decisions and be adaptable to changing data when it experiences a situation it has not seen before, like guardrails with human pilots to ensure safe performance of an aircraft. It also must perform at the speed of the warfighter—an unmanned system that makes delayed communications, moves slowly, or acts “right-of-bang” is nearly useless on the battlefield.
Operations. Naval aviators must also trust the AI operations. Factors to consider include understanding and approving of the compliance of the system (e.g., does the unmanned system meet the same certifications and requirements for naval aviation with which operators are familiar), as well as network and data security. Two other components are important for trust in AI operations: first, AI “humility,” which allows aviators to understand what conditions may stress an AI’s decision-making ability, and second, a dynamic understanding of how the AI will respond to that uncertainty
Ethics. Naval aviators operate and make decisions with the highest standards of military ethics in mind, and they must know that their autonomous wingman executes missions with the same standards. Another underlying facet of ethics is privacy, which the Navy must consider. For instance, will verbal communications and interactions between an aviator and an autonomous system be stored for training the next version of an AI? Will that interaction be accessed only by government officials, or by contractors or foreign partners? Transparency in how and why the AI makes decisions also is vital for establishing trust through ethics.
Student naval aviators go through hundreds of hours of education, simulators, and flights to build human-human trust with their manned wingmen. Similar training and certification should be required for autonomous systems. U.S. Navy (Michelle Tucker)
Extending the Foundation for Trust
Naval aviation can draw on its deep well of experience to build trust between aviators and autonomous systems across the dimensions of performance, operations, and ethics. The same requirements for human-human trust-building and human-machine trust-building can be extrapolated to fit the trust-building measures necessary for the future of naval aviation.
Autonomous systems should go through (and are currently going through) multiyear training and certification processes. For instance, the Navy began imagining carrier-based unmanned aerial vehicles (UAVs) in 2006, and the MQ-25, which performs a narrow range of functions, will not reach operating capability until 2024.2 Autonomous systems development and deployment must meet established standards of competency for individual components such as hardware and software packages, as well as holistic mission capabilities, such as the development of a human pilot or an individual aircraft. Autonomous systems can and will go through increasingly difficult levels of assessment to further their capabilities over time, though initial iterations such as the MQ-25 will be greatly constrained in functional mission areas such as refueling, long-range reconnaissance, or intelligence collection. The first time an MQ-25 refueled a Super Hornet it made headlines for novelty; the 100th time it refuels a Super Hornet it will be routine.
As the Navy increasingly adopts autonomous platforms, it will need to adapt training timelines to accommodate human-machine certifications. These likely would be best placed in FRSs or fleet environments as more mature autonomous platforms reach the fleet. There may, however, be potential to realize training efficiencies or cost savings using autonomous aircraft. In primary flight training, student naval aviators first fly as wingmen in the T-6 Texan II. In the earliest stages of formation training, the emphasis is on safe operation near other aircraft and the development of basic “sight picture” fundamentals. Flight leads act as stable platforms for student practice, a role that could be augmented by an autonomous aircraft flying a predetermined circuit or commanded by an instructor pilot in a student’s backseat. In addition to potential man-hour and flight-hour savings, early exposure to autonomous aircraft could build experience and trust that students will carry forward in their careers.
Autonomous wingman systems must have the same ability to communicate system health and platform issues to human pilots as occurs today between humans. Autonomous systems are complex, composed of software, hardware, and integration components for sensors and effectors, engineering operations, mission management controls, communications, maneuvering equipment, and more. Communicating deficiencies to human pilots is a prerequisite for building and maintaining trust with the unmanned system. Emergency procedures must be clearly understood.
Just as a human pilot conducts preflight checks of themselves and the aircraft, an unmanned system must conduct preflight checks for the software and hardware of each functional component. Just as experienced plane captains sign off on preflight checks for aircraft today, human robotics and autonomous systems experts are the ideal individuals to conduct these checks on autonomous systems and build trust. A Navy robotics expert (UAV engineering officer or similar role) could conduct preflight checks of the unmanned system and serve as a human interface for the manned wingman.
Practical Recommendations for the Navy
Man-machine teaming should be built into flight school and other fleet syllabi and training regimens, so student aviators learn to fly with autonomous wingmen at the beginning of their careers. The next generation of aviators must have opportunities and instruction to build the human-machine trust required for future battles. U.S. Navy (Drew Verbis)
To implement manned-unmanned teaming in naval aviation while meeting the guidance set forth by the Department of Defense and Department of the Navy related to autonomy, the Navy should:
Put in place robust and understood certification processes for autonomous systems for both the robotics hardware and the AI software. Much like a winged F/A-18 pilot is not qualified to fly a P-8, AIs should have tightly coupled software, platforms, and missions that are contextually specific. The Navy can accomplish this through the same level of rigid training that human pilots and aircraft go through for initial aviation training, follow-on advanced schooling, and immediate preflight checks. Earning Wings of Gold is a foundational cultural cornerstone for naval aviation, and all pilots—manned or unmanned—should be held to this standard.
Build man-machine team training into flight school, fleet replacement squadrons, and fleet syllabi. While manned aircrews are highly adaptable and can integrate new weapon systems, naval aviation can accelerate trust-building through early and repeated interactions with autonomous systems. For naval aviation to realize the full potential of autonomous combat systems, aviators must already have developed trust and experience with the AI prior to employment in a hostile environment. Just as naval aviators learn to work with wingmen and squadrons in structured training environments, so too must they learn to fly with autonomous wingmen. Such introduction and exposure could be included in Project Avenger, an initiative to employ emerging technologies such as virtual and mixed-reality trainers and AI to revamp aviator training. This program of instruction should already be in development for testing and refinement before unmanned aircraft arrive at the fleet en masse in the coming years.
Identify additional specialties for the squadron and wing levels that are necessary to “represent” the unmanned systems and build trust. The National Security Commission on AI, the Department of Defense AI Strategy, and the Navy’s Intelligent Autonomous Systems Strategy all highlight the need for an AI-ready workforce. These specialty sailors, warrant officers, and officers will serve as robotics and AI experts specifically for unmanned aviation and be human interpreters between man and machine. Naval aviation is taking steps in this direction, recently announcing the creation of the aerial vehicle operator (AVO) warrant officer billet for MQ-25 squadrons. AVOs will go through an abbreviated flight school syllabus as unrestricted line warrant officers who eventually earn Wings of Gold, but unlike naval aviators, they do not need to competitively promote and can therefore become subject matter experts in autonomous system operation.3 The decision to make AVO billets unrestricted line should be especially lauded because it is an organizational signal that autonomous systems have a role to play in combat missions.
Building off the framework laid out by AI experience in private industry, naval aviation should ensure the AVO community is structured around three general responsibilities. AI trainers will fill the role of forward-deployed data scientists and AI engineers, managing, cleaning, and altering data, monitoring changes in data and algorithmic performance, and correcting errors in the software as they arise. AI explainers will work with the AI to understand why it makes the decisions it does, interpret them for human understanding, and monitor consistency and compliance. AI sustainers will ensure the autonomous systems continue to perform with all ethical, legal, and policy requirements. These individuals would likely reside at a higher echelon such as the wing, identify and override decisions when they occur outside preestablished guardrails of AI decision-making, review and approve data and output quality, review and approve updated AI versions before deployment, flag errors in machine judgment, and ultimately ensure the AI is working properly within the commander’s and naval aviator’s requirements.4
A Stingray on board the USS George H. W. Bush (CVN-77) during deck-handling tests in late 2021. Just as with human wingmen, trust with autonomous wingmen must be built over time, through continuous diagnostic assessments and repeated interactions. Boeing
Lethality First, Lethality Always
From automated gunsights to digital flight controls, and now precision-landing mode, increasing levels of automation have continually changed the relationship between aircrew and their platforms. Technology’s relentless advance has meant aviators must constantly learn new ways to squeeze every ounce of lethality from their platforms. What has remained constant is the imperative to fly, fight, and win in aerial combat by any means necessary.
Medal of Honor recipient and World War I ace Captain Eddie Rickenbacker put it bluntly: “Fighting in the air is not a sport. It is scientific murder.”5 Autonomous wingmen present enormous potential to increase naval aviation’s lethality, to conduct aviation missions more effectively and efficiently, and to dominate the air in support of U.S. and allied interests. To realize this potential, naval aviation must take a people-centric approach to integrating autonomous systems. Naval aviation must make concerted efforts to build trust between aircrew and autonomous systems so that as 21st-century combat matures, U.S. naval aviators will win the day.
No comments:
Post a Comment