,,ii #aaith ,,congress #b;d ,session ,s4 #dfhb ,,in ,,the ,,senate ,,of ,,the ,,united ,,states ,june #d1 #bjbf,mr4 ,kelly introduced the following bill2 which was read twice and referred to the ,committee on ,armed ,services ,a ,,bill ,to establish requirements for human judgment in the use of force by autonomous weapon systems used by the ,department of ,defense1 and for other purposes4 #a4 ,short title ,this ,act may be cited as the ,ultimate ,human ,responsibility in ,defense ,systems ,act of #bjbf4 #b4 ,definitions ,in this ,act3 7#a7 ,autonomous weapon system ,the term autonomous weapon system means a weapon system that1 once activated1 can select and engage targets without further intervention by a human operator4 7#b7 ,congressional defense committees ,the term congressional defense committees has the meaning given that term in section #aja7;a7 of title #aj1 ,united ,states ,code4 7#c7 ,ultimate human responsibility ,the term ultimate human responsibility1 with respect to a weapon system1 means the ability of a human commander or operator to exercise informed human agency-- 7,a7 to understand the operational context of the weapon system through training1 capacity building1 and integration of design and testing features that strengthen human understanding and effective oversight2 7,b7 to supervise1 intervene in1 or terminate the use of force by the system1 thereby guaranteeing human responsibility when appropriate given system design2 and 7,c7 to ensure compliance with all applicable ,united ,states law and international law1 including applicable treaties1 weapon system safety rules1 the law of armed conflict1 and rules of engagement4 7#d7 ,use of force ,the term use of force means the application of kinetic lethal or non-lethal military force against a target4 #c4 ,requirement for human judgment in autonomous weapon systems 7a7 ,policy requirement ,the ,secretary of ,defense shall-- 7#a7 ensure that any autonomous weapon system or artificial intelligence capability developed1 acquired1 or operated by the ,department of ,defense-- 7,a7 is designed and employed in a manner that enables commanders and operators to exercise ultimate human responsibility over the use of force2 and 7,b7 is developed1 fielded and employed using ethical and legal principles that would delineate unlawful uses of artificial intelligence2 and 7#b7 develop technical and operational and organizational guidelines around the use of artificial intelligence4 7b7 ,design requirements ,to meet the requirements under subsection 7a71 the ,secretary of ,defense shall ensure that any autonomous weapon system or artificial intelligence capability described in such subsection1 to the maximum extent practicable-- 7#a7 allows human operators to supervise the operation of the weapon system during mission execution when appropriate given system design2 7#b7 includes mechanisms that allow commanders or operators to intervene in or terminate the use of force by the weapon system during mission execution when appropriate given system design2 7#c7 includes fail-safe mechanisms to enable manual control when autonomous systems are degraded1 jammed1 spoofed1 or under adversarial attack when appropriate given system design2 7#d7 provides sufficient information regarding weapon system status and decision logic to enable operators to understand1 oversee1 and exercise functional control over system behavior2 7#e7 maintains records of relevant target selection data1 decision logic1 and human operator actions sufficient to enable post-engagement review of compliance with the requirements under subsection 7a72 and 7#f7 operates consistent with applicable ,united ,states law and international law1 rules of engagement1 the law of armed conflict1 applicable treaties1 and ,department of ,defense policy4 7c7 ,operational oversight 7#a7 ,in general ,the ,secretary of ,defense shall establish procedures to ensure that commanders and operators are rigorously trained to appropriately supervise autonomous weapon systems or artificial intelligence capabilities in operational environments1 including regular proficiency assessments in manual target identification1 threat assessment1 and engagement procedures1 to ensure operators can effectively execute missions using other capabilities when autonomous systems are unavailable or unreliable4 7#b7 ,reporting mechanism ,the procedures established under paragraph 7#a7 shall include a mechanism for operators of autonomous weapon systems or artificial intelligence capability to report1 without fear of reprisal1 concerns regarding weapon system reliability or the adequacy of human-machine interfaces4 #d4 ,review and certification 7a7 ,senior-,level review 7#a7 ,in general ,before fielding an autonomous weapon system capable of independently selecting and engaging targets without human supervision1 the ,director of ,operational ,test and ,evaluation shall conduct a review to certify that the weapon system complies with the requirements of this ,act4 7#b7 ,independent assessment ,each review required under paragraph 7#a7 with respect to an autonomous weapon system shall include an independent adversarial testing 7red-team7 assessment that evaluates the vulnerability of the system to adversarial manipulation under operationally realistic conditions1 including degraded and electronic warfare contested environments 7such as jamming and spoofing71 communication disruption1 and adversarial manipulation4 7b7 ,certification requirement ,no autonomous weapon system described in subsection 7a7 may be fielded by the ,department of ,defense unless the ,director of ,operational ,test and ,evaluation certifies that the weapon system-- 7#a7 allows for ultimate human responsibility with respect to the use of force2 and 7#b7 has been tested to verify secure1 safe1 and reliable operations that meet the reliability standards of the ,department for weapon systems1 including retesting following any significant software update or change in operational environment4 7c7 ,testing standards 7#a7 ,in general ,not later than #ah months after the date of the enactment of this ,act1 the ,director of ,operational ,test and ,evaluation shall publish on a publicly available website standards for the testing and evaluation of artificial intelligence components in autonomous weapon systems1 including benchmarks for reliability1 robustness1 security1 and human-machine team performance4 7#b7 ,form ,the standards required to be published under paragraph 7#a7 shall be published in unclassified form to the maximum extent practicable4 #e4 ,annual reporting to ,congress ,not later than one year after the date of the enactment of this ,act1 and semiannually thereafter until the date that is six years after the date of the enactment of this ,act1 the ,secretary of ,defense shall submit to the congressional defense committees a report that includes-- 7#a7 a summary of autonomous weapon systems in development or fielded by the ,department of ,defense2 7#b7 steps taken by the ,secretary to ensure compliance with the requirements of this ,act2 and 7#c7 such recommendations as the ,secretary may have for legislative or administrative action relating to human judgment in autonomous weapon systems4 #f4 ,autonomous weapon system incident repository 7a7 ,establishment ,the ,secretary of ,defense shall establish and maintain a centralized repository for reporting1 collecting1 and analyzing incidents involving autonomous weapon systems and artificial intelligence capabilities used by the ,department of ,defense4 7b7 ,purpose ,the repository established under subsection 7a7 shall be used-- 7#a7 to document operational incidents1 weapon system failures1 unintended weapon system behaviors1 or near-miss events involving autonomous weapon systems and artificial incident capabilities2 7#b7 to enable the ,department to identify systemic risks1 software errors1 or operational vulnerabilities associated with such weapon systems and capabilities2 and 7#c7 to promote institutional learning and improvement in the design1 testing1 and operational employment of autonomous weapon systems and artificial intelligence capabilities4 7c7 ,reporting requirements ,the ,secretary shall establish procedures requiring the reporting of incidents described in subsection 7b77#a71 including-- 7#a7 operational malfunctions affecting the use of force2 7#b7 unintended or anomalous weapon system behavior2 7#c7 instances in which autonomous weapon systems or artificial intelligence capabilities selected or engaged targets inconsistent with commander intent or rules of engagement1 regardless of whether engagement was ultimately executed2 7#d7 human-machine interaction failures affecting command or operator control2 and 7#e7 other safety-related events determined appropriate by the ,secretary4 7d7 ,safety reporting model ,in establishing the repository under subsection 7a71 the ,secretary shall1 to the extent practicable1 model reporting practices on aviation safety reporting systems used to identify and mitigate systemic risks in complex operational environments4 7e7 ,analysis and dissemination ,the ,secretary shall ensure that information collected through the repository established under subsection 7a7 is analyzed to identify trends and lessons learned1 and that appropriate findings are disseminated across the ,department of ,defense to improve system design1 training1 and operational procedures4 7f7 ,protection of sensitive information ,information contained in the repository established under subsection 7a7 shall be handled in accordance with applicable classification1 operational security1 and national security requirements4 7g7 ,congressional briefing ,not less frequently than semiannually1 the ,secretary shall provide to the congressional defense committees a classified briefing on trends1 systemic risks1 and corrective actions identified through the repository established under subsection 7a74 #g4 ,training pipelines 7a7 ,in general ,the ,secretary of ,defense shall develop formal training pipelines for autonomous systems of the ,department of ,defense4 7b7 ,elements ,the formal training pipelines developed under subsection 7a7 shall include the following3 7#a7 ,specialized training tracks for operators1 commanders1 and system supervisors4 7#b7 ,certification in human-machine teaming1 not just system operation4 7#c7 ,mandatory training in denied and contested environments1 such as loss of global positioning system1 electronic warfare1 and degraded communications4 7#d7 ,integration of autonomous systems into ,combat ,training ,center rotations4 7#e7 ,continuous retraining tied to software updates and system changes4 #h4 ,civilian harm mitigation and response office 7a7 ,establishment ,the ,secretary of ,defense shall establish within the ,office of the ,secretary of ,defense an office to be known as the ,civilian ,harm ,mitigation and ,response ,office 7in this section referred to as the ,office74 7b7 ,purpose ,the purpose of the ,office is to develop1 oversee1 and coordinate ,department-wide policies1 practices1 and capabilities to prevent1 mitigate1 and respond to civilian harm resulting from military operations4 7c7 ,responsibilities ,the ,office shall-- 7#a7 develop and implement policies and guidance relating to civilian harm mitigation and response2 7#b7 oversee the integration of civilian harm mitigation practices into planning1 targeting1 and operational processes2 7#c7 coordinate assessments1 investigations1 and responses to incidents involving civilian harm2 7#d7 support training and doctrine development across the ,department2 and 7#e7 provide regular reports to ,congress on civilian harm mitigation efforts and outcomes4 7d7 ,leadership ,the ,office shall be headed by a senior official designated by the ,secretary of ,defense4 #i4 ,rule of construction ,nothing in this ,act shall be construed to prohibit the development or use of autonomous weapon systems designed to intercept incoming threats where human supervision remains part of the operational design4