How DoD Plans to Leverage Artificial Intelligence and Open Source to Improve Emergency Response and Disaster Relief

John KoenigAll types of natural disasters, from wildfires, to hurricanes, to earthquakes, to tsunamis, to extreme flooding, have caused massive damage to communities, lives, and critical infrastructure in recent years. The global media have brought these tragic situations into our everyday consciousness from all across our country and around the world. The increasing intensity and frequency of disaster events have made disaster response a high priority for the US government.

Some might not know it, but the US military plays a key role in US disaster response strategy and, accordingly, the Department of Defense (DoD) has prioritized its disaster response mission and is investing heavily towards increasing its capabilities and effectiveness. Technology is a big part of all modern DoD missions, and disaster response is no different. The most promising and transformative technology on the horizon for our future, and for the future of the DoD, is Artificial Intelligence (AI).

So how exactly will the DoD leverage Artificial Intelligence technologies in order to meet the demands of the disaster response mission? Is a new Center of Excellence created by DoD, called the Joint Artificial Intelligence Center (JAIC).

Initially launched in January 2018, the JAIC has become a key player in the effort to fulfill the strategy set out in the White House's Executive Order on Maintaining American Leadership in Artificial Intelligence, which was released on February 11, 2019. DoD's outlines the high level steps they will follow in the DoD Artificial Intelligence Strategy document. The executive summary of the document lays out a set of approaches, including decentralized development and experimentation as well as creating "a common foundation of shared data, reusable tools, frameworks and standards, and cloud and edge services." DoD's Artificial Intelligence Strategy lays out a set of Strategic Focus Areas, as well as partnerships "with leading private sector technology companies, academia, and global allies and partners."

Partnership objectives include:

  • Forming open mission initiatives focused on global challenges.
  • Strengthening academic partnerships and seeding new AI innovation districts.
  • Enhancing partnership with U.S. industry.
  • Evolving international alliances and partnerships.
  • Engaging with the open source community.

A key element of the open mission initiatives the DoD AI Strategy calls for forming "open AI missions with academia and industry that will contribute to addressing global challenges of significant societal importance, such as operationalizing AI for humanitarian assistance and disaster relief for wildfires, hurricanes, and earthquakes."

The section on engagement with the open source community states that "we will contribute our data, challenges, research, and technologies to this community and engage with the open-source ecosystem as a vehicle for attracting talent, identifying and advancing new AI technologies that can transform defense, and broadening our accessible technology base."

The section on engaging in partnerships with U.S. industry, it states that "engaging with and strengthening the AI technology ecosystem requires us to experiment with a range of partnership models. These include bold new AI initiatives with large industrial partners, small start-ups, and venture capital firms."

Defense Department Chief Information Officer Dana Deasy and Air Force Lt. Gen. John N.T. Shanahan, the director of the Joint Artificial Intelligence Center, hold a roundtable meeting on DOD’s artificial intelligence strategy at the Pentagon, Feb. 12, 2019. DOD photo by Army Sgt. Amber I. Smith National Mission Initiatives

The first Director of JAIC, Air Force Lieutenant General John N.T. "Jack" Shanahan, has laid out a strategy organized around National Mission Initiatives (NMIs). The first two National Mission Initiatives are Predictive Maintenance (PMx) and Humanitarian Assistance and Disaster Relief (HA/DR). The fact that the humanitarian efforts and disaster response are a top priority for DoD AI development means that major investments will be made in the coming years, as DoD leadership spends the $1 billion allocated for AI in the 2020 budget. Likewise, the 2021 budget for the JAIC itself will double to $208 million.

What are the JAIC and DIU Planning?

Another component of the AI push at DoD is the Defense Innovation Unit (DIU). DIU is a military/civilian cooperative whose mission is to engage with leading-edge private sector companies in order to solve defense problems. From its headquarters in Mountain View, CA (Silicon Valley), DIU is working with most of the top American consumer technology companies that are leading the world in innovation and designing our future. Since DIU was launched by the Obama Administration in 2015, it has been led by a series of high-powered Silicon Valley power players who joined up because they believe in the mission of driving innovation at DoD. The mix of civilian professionals and uniform military personnel at DIU has developed numerous partnerships with leading edge technology companies in order to adopt consumer technology advances for the DoD.

In order to meet General Shanahan's Humanitarian Assistance and Disaster Relief National Mission Initiative, JAIC and DIU decided to announce a series of innovation competitions under the name xView Challenge. The xView Challenge innovation competition focuses on developing a solution to a computer vision problem that could eventually extract useful information from overhead images from satellites or drones - a type of data which DoD produces in great abundance. "DIU's goal in hosting this challenge is to enlist the global community of machine learning experts to tackle a critically hard problem: detecting key objects in overhead imagery in context and assessing damage in a disaster situation," said Mike Haul, DIU AI portfolio director.

The competition aims to identify software that can effectively identify seven types of disasters:

  • Wildfires
  • Landslides
  • Dam collapses
  • Volcanic eruptions
  • Earthquakes/tsunamis
  • Wind and flooding damage.

The xView1 Challenge started in March 2018 and the competition closed in August 2018. This first iteration focused on improving the implementation of computer vision for this task. Building on those improvements, xView2 (September - December 2019) challenges competition participants to geo-locate damaged buildings and provide accurate labels which indicate the level of damage to a building. xView2 is offering $150,000 in total prizes with options available to enter the government acquisitions pipeline after the competition.

The xView2 Challenge focuses on automating the process of assessing building damage after a natural disaster. Credit DIUEach team will have access to the annotated xBD dataset in order to train their algorithms. This dataset is the largest and most diverse disaster imagery dataset in existence and contains 700,000 building annotations of 50,000 building from 15 different countries. Teams will submit their work to DIU for evaluation by US Government employees and the winners presented their work at the NeurIPS Workshop on AI for humanitarian assistance.

The xView2 Challenge is coming on the heels of a similar commercial solicitation which closed in July 2019. Despite a maximum contract award of $10 million for any company who won a contract, it is likely that this new innovation challenge was initiated because the previous solicitation fell short. This competition is backed by an unprecedented partnership of state and federal agencies tasked with the disaster response mission and, hopefully, it will bring more effective solutions than the traditional federal procurement system.

Other AI Solutions for Disaster Response

While DoD has its sights set combining computer vision with its powerful overhead imaging capabilities in order to provide better situational awareness and improve decision-making, there are other AI applications that they will be exploring in the future.

Social media is another ubiquitous data source that has just as much potential, and just as many challenges, as overhead imagery. Social media data is text-based by nature and could be analyzed in real-time in order to learn useful information about conditions on the ground during a disaster.

Social media posts about families stranded on their roofs during a flood, or neighborhoods under threat from a fast moving wildfire, or collapsed buildings after an earthquake could alert disaster response centers about the facts on the ground. AI developers would use a type of algorithms known collectively as Natural Language Processing (NLP) to process and identify critical information from text-based social media posts. However, NLP is a notoriously difficult application for AI to get right. Imagine how the range of different languages would need to be understood in order for this application to be effective. Similarly, local dialects, street slang, and spelling mistakes compound the challenge further.

Another way DoD could harness the power of AI to complete their mission is through the use of sensor networks. Sensor technology is a powerful new paradigm and data source that has been used to solve many problems across the world. A network of low-cost, wireless sensors is used to monitor the physical environment in the area and provide a constant stream of observation data that is often used as input for an AI system.

Sensors have already been in use for years in order to monitor for earthquakes, tsunamis, and volcanic eruptions. In order to aid a disaster response effort, a network of mobile sensors would have to be deployed before, during, or after a disaster event in order to provide useful information for the disaster response center. Furthermore, AI algorithms are capable of analyzing the data from these sensors in real-time and provide alerts about diminishing air quality, earthquake aftershocks, or environmental conditions during a wildfire. However, the challenges involved for this AI application lay in the design, deployment, and implementation of the sensor networks, rather than in the creation of novel AI systems.

Chances for Success

While the drive for AI and the increased priority on humanitarian assistance and disaster response missions are both relatively new occurrences, the outlook for success is relatively good. The federal funding is in place and it appears to be increasing at a rapid rate because support at all levels of the government are fully behind these initiatives. However, AI technology is still nascent and has only recently crept out of the theoretical and into the real world. The challenges technologists face as they sit down at their computers and actually attempt to solve these problems are substantial. Hopefully, new procurement strategies, like the xView2 Competition, will enable the right teams to make the right discoveries at the right times, and result in great advances and DoD success when responding to future disasters.