The goal of my research is to enable the safe operation of autonomous systems, such as unmanned aircraft, autonomous cars, and smart infrastructure. My research focuses on modeling, building, and validating intelligent safety-critical systems. My research combines ideas from decision theory, reinforcement learning, machine learning, game theory, and control theory.
Adaptive Stress Testing
Adaptive Stress Testing (AST) is a novel framework that frames finding the most likely failure scenario as a sequential decision-making problem, allowing existing algorithms such as Monte Carlo tree search (MCTS) and deep reinforcement learning (DRL) to solve them. AST has been used as part of a Federal Aviation Administration (FAA) program to discover scenarios of near mid-air collisions (NMACs) in prototypes of the next-generation Airborne Collision Avoidance System (ACAS X). AST has also been applied to analyze trajectory planning systems for unmanned aircraft and autonomous cars at pedestrian crossings.
Related Publications:
- Lee, R., Mengshoel, O.J., Saksena, A., Gardner, R., Genin, D., Silbermann, J., Owen, M., Kochenderfer, M.J., “Adaptive Stress Testing: Finding Likely Failure Events with Reinforcement Learning”, Journal of Artificial Intelligence Research, 69, pp.1165-1201, 2020
- Koren, M., Alsaif, S., Lee, R., Kochenderfer, M.J., “Adaptive Stress Testing for Autonomous Vehicles”, IEEE Intelligent Vehicles Symposium, 2018
- Lipkis, R., Lee, R., Silbermann, J., Young, T., “Adaptive Stress Testing of Collision Avoidance Systems for Small UASs with Deep Reinforcement Learning”, AIAA Scitech, 2022
- Lee, R., Mengshoel, O.J., Agogino, A.K., Giannakopoulou, D., Kochenderfer, M.J., “Adaptive Stress Testing of Trajectory Planning Systems”, AIAA SciTech Intelligent Systems Conference, 2019
- Julian, K.D., Lee, R., Kochenderfer, M.J., “Validation of Image-Based Neural Network Controllers Through Adaptive Stress Testing”, IEEE International Conference on Intelligent Transportation Systems, 2020
- Moss, R.J., Lee, R., Kochenderfer, M.J., “Adaptive Stress Testing of Trajectory Predictions in Flight Management Systems”, IEEE/AIAA Digital Avionics Systems Conference, 2020
- Durling, M., Herencia-Zapana, H., Meng, B., Meiners, M., Hochwarth, J., Visser, N., Lee, R., Moss, R., Valapil, V., “Certification Considerations for Adaptive Stress Testing of Airborne Software”, IEEE/AIAA Digital Avionics Systems Conference, 2021
Differential Adaptive Stress Testing (DAST) extends the AST approach to analyze differences in failure behavior between two systems. DAST finds failure scenarios where the test system fails but the baseline system does not, making it very useful for comparing two candidate systems or regression testing of successive prototypes. DAST has been used to compare scenarios of near mid-air collisions between ACAS X and the existing Traffic Alert and Collision Avoidance System (TCAS).
Related Publications:
- Lee, R., Mengshoel, O.J., Saksena, A., Gardner, R., Genin, D., Brush, J., Kochenderfer, M.J., “Differential Adaptive Stress Testing of Airborne Collision Avoidance Systems”, AIAA SciTech Modeling and Simulation Technologies Conference, 2018 [Best Paper Award]
Automatic Categorization
Grammar-Based Decision Tree (GBDT) is a machine learning model for automatically categorizing and explaining failure scenarios to help domain experts diagnose the underlying issues of failures. GBDT combines a context-free grammar, temporal logic, expression optimization, and decision tree into a single framework that provides human-interpretability and support for high-dimensional heterogeneous time series data. Decision rules in the tree are Boolean expressions derived from a grammar, allowing much flexibility and user customization. We take each leaf node to be a separate category and the conjunction of branch expressions to be its explanation. GBDT learning relies on existing expression optimization algorithms, such as genetic programming and grammatical evolution. GBDT has been applied to categorize scenarios of near mid-air collision in ACAS X.
Related Publications:
- Lee, R., Kochenderfer, M.J., Mengshoel, O.J., Silbermann, J., “Interpretable Categorization of Heterogeneous Time Series Data”, SIAM International Conference on Data Mining, 2018
Autonomous Vehicle Technologies
We have multiple autonomous vehicle platforms with various computing and sensing capabilities, including the MAX unmanned ground vehicle, X-SCAV half-scale unmanned aircraft, and SWIFT full-scale unmanned electric glider. We also have vehicle and mission simulation capabilities using the NASA Reflection software framework. The vehicles have been deployed on a variety of research and scientific missions. Payload-directed flight (PDF) aims to close the control loop around unconventional sensors such as scientific payloads in order to maximize the return of scientific missions. PDF integrates sensor measurements, performs estimation over models of physical processes, and performs intelligent real-time trajectory planning to maximize information gathering and mission value. In a collaboration with United States Geological Survey (USGS), we have created both hardware and software for conducting autonomous geomagnetic surveys using an unmanned ground vehicle and an unmanned aircraft.
Related Publications:
- Phelps, G., Ippolito, C., Lee, R., Spritzer, J., Yeh, Y.H., “Investigations into Near-Real-Time Surveying for Geophysical Data Collection Using an Autonomous Ground Vehicle” U.S. Geological Survey, 2014
- Lee, R., Ippolito, C., Yeh, Y.H., Spritzer, J., Phelps, G., “Payload-Directed Control of Geophysical Magnetic Surveys”, AIAA Infotech@Aerospace, 2010
- Lee, R., Ippolito, C., “A Perception and Mapping Approach for Plume Detection in Payload Directed Flight”, AIAA Infotech@Aerospace, 2009
Modeling Multiple Human Decision-Makers
Network-Form Game (NFG) is a novel framework for modeling stochastic systems with multiple human actors. The approach combines ideas from game theory, probabilistic models, and reinforcement learning to capture stochastic, bounded rational, and strategic decision-making behavior. NFG models scenarios over a Bayesian network but adds special decision nodes for the actors. The decision nodes are aware of the structure of the scenario but receive limited information from their observation. They behave strategically using level-K thinking. The NFG approach has been applied to model aircraft collision avoidance, human-in-the-loop experiments, and cyber-security of computer networks and smart power grids.
Related Publications:
- Lee, R., Wolpert, D.H., Bono, J., Backhaus, S., Bent, R., Tracey, B., “Counter-Factual Reinforcement Learning: How to Model Decision-Makers That Anticipate the Future”, Decision Making and Imperfection, 474(4), pp.101-128, Springer, 2013
- Lee, R., Wolpert, D.H., “Game Theoretic Modeling of Pilot Behavior During Mid-Air Encounters”, Intelligent Systems Reference Library, 28(4), pp.75-111, Springer, 2012
- Backhaus, S., Bent, R., Bono, J., Lee, R., Tracey, B., Wolpert, D.H., Xie, D., Yildiz, Y., “Cyber-Physical Security: A Game Theory Model of Humans Interacting Over Control Systems”, IEEE Transactions on Smart Grid, 4(4), pp.2320-2327, 2013
- Yan, G., Lee., R., Kent, A., Wolpert, D.H., “Towards a Bayesian Network Game Framework for Evaluating DDoS Attacks and Defense”, ACM Conference on Computer and Communications Security, 2012
- Schlicht, E., Lee., Wolpert, D.H., Kochenderfer, M.J., Tracey, B., “Predicting the Behavior of Interacting Humans by Fusing Data from Multiple Sources”, Conference on Uncertainty in Artificial Intelligence, 2012
- Yildiz, Y., Lee., R., Brat, G., “Using Game Theoretic Models to Predict Pilot Behavior in NextGen Merging and Landing Scenario”, AIAA SciTech Modeling and Simulation Technologies Conference, 2012
Multi-Fidelity Air Traffic Simulator
Multi-fidelity simulator (MFSim) is a fast-time air traffic simulator of the national airspace that simulates aircraft at different levels of abstraction in order to gain computational savings. The simulator is used as a research platform for multi-agent reinforcement learning of air traffic management policies.
Related Publications:
- Agogino, A., Iscen, A., Lee, R., Bowers, D., Tumer, K., Brat, G.P., “Scalable Hierarchical Multifidelity Simulation and Multiagent Optimization of Air Traffic”, AAMAS Workshop on Massive Multiagent Systems, 2015