AdaptiveStressTesting.jl

AdaptiveStressTesting.jl is a software package written in Julia for stress testing safety-critical systems.  The package uses reinforcement learning to search for the highest-probability path from an initially healthy system state to a fault condition.  The package is part of the RLES.jl (Reinforcement Learning Encounter Simulator) framework used to stress test the Next-Generation Airborne Collision Avoidance System (ACAS X).

Related Publications:

  • [DOI] R. Lee, M. J. Kochenderfer, O. J. Mengshoel, G. P. Brat, and M. P. Owen, “Adaptive Stress Testing of Airborne Collision Avoidance Systems,” in Digital Avionics Systems Conference (DASC), Prague, Czech Republic, 2015.
    [Bibtex]
    @InProceedings{Lee2015Adaptive,
    Title = {Adaptive Stress Testing of Airborne Collision Avoidance Systems},
    Author = {Ritchie Lee and Mykel J. Kochenderfer and Ole J. Mengshoel and Guillaume P. Brat and Michael P. Owen},
    Booktitle = dasc,
    Year = {2015},
    Address = {Prague, Czech Republic},
    Doi = {10.1109/DASC.2015.7311450},
    Owner = {rcnlee},
    Timestamp = {2015.05.20}
    }

Network-Form Games Library (libnfg)

Libnfg is a software library written in C++ for implementing Network-Form Games (NFGs). NFGs model systems with multiple human actors whose strategic decisions interact in the context of the system.  The algorithms use game theory, probabilistic inference, and reinforcement learning to predict the dynamics of the overall system.

Related Publications:

  • [DOI] R. Lee and D. Wolpert, “Game Theoretic Modeling of Pilot Behavior During Mid-Air Encounters,” in Decision Making with Imperfect Decision Makers, T. Guy, M. Kárný, and D. Wolpert, Eds., Springer Berlin Heidelberg, 2012, vol. 28, pp. 75-111.
    [Bibtex]
    @InCollection{Lee2012Game,
    Title = {Game Theoretic Modeling of Pilot Behavior During Mid-Air Encounters},
    Author = {Lee, Ritchie and Wolpert, David},
    Booktitle = {Decision Making with Imperfect Decision Makers},
    Publisher = {Springer Berlin Heidelberg},
    Year = {2012},
    Chapter = {4},
    Editor = {Guy, TatianaValentine and Kárný, Miroslav and Wolpert, DavidH.},
    Pages = {75-111},
    Series = {Intelligent Systems Reference Library},
    Volume = {28},
    Doi = {10.1007/978-3-642-24647-0_4},
    ISBN = {978-3-642-24646-3},
    Language = {English},
    Owner = {rcnlee},
    Url = {http://dx.doi.org/10.1007/978-3-642-24647-0_4}
    }
  • [DOI] R. Lee, D. H. Wolpert, J. Bono, S. Backhaus, R. Bent, and B. Tracey, “Counter-Factual Reinforcement Learning: How to Model Decision-Makers That Anticipate the Future,” in Decision Making and Imperfection, T. V. Guy, M. Karny, and D. H. Wolpert, Eds., Springer Berlin Heidelberg, 2013, vol. 474, pp. 101-128.
    [Bibtex]
    @InCollection{Lee2013Counter,
    Title = {Counter-Factual Reinforcement Learning: How to Model Decision-Makers That Anticipate the Future},
    Author = {Lee, Ritchie and Wolpert, David H. and Bono, James and Backhaus, Scott and Bent, Russell and Tracey, Brendan},
    Booktitle = {Decision Making and Imperfection},
    Publisher = {Springer Berlin Heidelberg},
    Year = {2013},
    Chapter = {4},
    Editor = {Guy, Tatiana V. and Karny, Miroslav and Wolpert, David H.},
    Pages = {101-128},
    Series = {Studies in Computational Intelligence},
    Volume = {474},
    Doi = {10.1007/978-3-642-36406-8_4},
    ISBN = {978-3-642-36405-1},
    Language = {English},
    Owner = {rcnlee},
    Url = {http://dx.doi.org/10.1007/978-3-642-36406-8_4}
    }
  • E. J. Schlicht, R. Lee, D. H. Wolpert, M. J. Kochenderfer, and B. Tracey, “Predicting the Behavior of Interacting Humans by Fusing Data from Multiple Sources,” in Conference on Uncertainty in Artificial Intelligence (UAI), Catalina Island, California, 2012.
    [Bibtex]
    @InProceedings{Schlicht2012,
    Title = {Predicting the Behavior of Interacting Humans by Fusing Data from Multiple Sources},
    Author = {Erik J. Schlicht and Ritchie Lee and David H. Wolpert and Mykel J. Kochenderfer and Brendan Tracey},
    Booktitle = uai,
    Year = {2012},
    Address = {Catalina Island, California},
    Owner = {rcnlee},
    Url = {http://www.auai.org/uai2012/papers/74.pdf}
    }
  • G. Yan, R. Lee, A. Kent, and D. Wolpert, “Towards a Bayesian Network Game Framework for Evaluating DDoS Attacks and Defense,” in ACM Conference on Computer and Communications Security (CCS), 2012, pp. 553-566.
    [Bibtex]
    @InProceedings{Yan2012,
    Title = {Towards a {B}ayesian Network Game Framework for Evaluating {DDoS} Attacks and Defense},
    Author = {Yan, Guanhua and Lee, Ritchie and Kent, Alex and Wolpert, David},
    Booktitle = acmccs,
    Year = {2012},
    Organization = {ACM},
    Pages = {553--566},
    Owner = {rcnlee}
    }
  • Y. Yildiz, R. Lee, and G. Brat, “Using Game Theoretic Models to Predict Pilot Behavior in NextGen Merging and Landing Scenario,” in AIAA Modeling and Simulation Technologies Conference (MST), 2012.
    [Bibtex]
    @InProceedings{Yildiz2012,
    Title = {Using Game Theoretic Models to Predict Pilot Behavior in {N}ext{G}en Merging and Landing Scenario},
    Author = {Yildiz, Yildiray and Lee, Ritchie and Brat, Guillaume},
    Booktitle = aiaa_mst,
    Year = {2012},
    Owner = {rcnlee}
    }