ExprOptimization.jl

ExprOptimization.jl is a software package written in Julia for the optimization of expression trees.  The package can be used to learn executable source code from input/output examples, also known as “program induction”.  The package implements a number of tree optimization algorithms including Genetic Programming, Monte Carlo, and Cross-Entropy Method.  The software has been used to learn interpretable rules in multivariate heterogeneous time series datasets.

 

AdaptiveStressTesting.jl

AdaptiveStressTesting.jl is a software package written in Julia for stress testing safety-critical systems.  The package uses reinforcement learning to search for the highest-probability path from an initially healthy system state to a fault condition.  The package is part of the RLES.jl (Reinforcement Learning Encounter Simulator) framework used to stress test the Next-Generation Airborne Collision Avoidance System (ACAS X).

Related Publications:

  • [DOI] R. Lee, M. J. Kochenderfer, O. J. Mengshoel, G. P. Brat, and M. P. Owen, “Adaptive Stress Testing of Airborne Collision Avoidance Systems,” in Digital Avionics Systems Conference (DASC), Prague, Czech Republic, 2015.
    [Bibtex]
    @InProceedings{Lee2015Adaptive,
    Title = {Adaptive Stress Testing of Airborne Collision Avoidance Systems},
    Author = {Ritchie Lee and Mykel J. Kochenderfer and Ole J. Mengshoel and Guillaume P. Brat and Michael P. Owen},
    Booktitle = dasc,
    Year = {2015},
    Address = {Prague, Czech Republic},
    Doi = {10.1109/DASC.2015.7311450},
    Owner = {rcnlee},
    Timestamp = {2015.05.20}
    }

 

Multi-Fidelity Simulator (MFSim)

MFSim is a fast-time national airspace simulator written in Java.  The simulator uses a hierarchical backend that allows aircraft in different regions of the airspace to be simulated at different levels of fidelity.  The simulator has been used as a platform for research in multi-agent reinforcement learning of air traffic automation.

Related Publications:

  • A. Agogino, A. Iscen, R. Lee, D. Bowers, K. Tumer, and G. Brat, “Scalable Hierarchical Multifidelity Simulation and Multiagent Optimization of Air Traffic,” in AAMAS Workshop on Massive Multiagent Systems, Istanbul, Turkey, 2015.
    [Bibtex]
    @InProceedings{Agogino2015,
    Title = {Scalable Hierarchical Multifidelity Simulation and Multiagent Optimization of Air Traffic},
    Author = {Adrian Agogino and Atil Iscen and Ritchie Lee and Devin Bowers and Kagan Tumer and Guillaume Brat},
    Booktitle = {{AAMAS} Workshop on Massive Multiagent Systems},
    Year = {2015},
    Address = {Istanbul, Turkey},
    Owner = {rcnlee}
    }

 

Network-Form Games Library (libnfg)

Libnfg is a software library written in C++ for implementing Network-Form Games (NFGs). NFGs model systems with multiple human actors whose strategic decisions interact in the context of the system.  The algorithms use game theory, probabilistic inference, and reinforcement learning to predict the dynamics of the overall system.

Related Publications:

  • [DOI] R. Lee and D. Wolpert, “Game Theoretic Modeling of Pilot Behavior During Mid-Air Encounters,” in Decision Making with Imperfect Decision Makers, T. Guy, M. Kárný, and D. Wolpert, Eds., Springer Berlin Heidelberg, 2012, vol. 28, pp. 75-111.
    [Bibtex]
    @InCollection{Lee2012Game,
    Title = {Game Theoretic Modeling of Pilot Behavior During Mid-Air Encounters},
    Author = {Lee, Ritchie and Wolpert, David},
    Booktitle = {Decision Making with Imperfect Decision Makers},
    Publisher = {Springer Berlin Heidelberg},
    Year = {2012},
    Chapter = {4},
    Editor = {Guy, TatianaValentine and Kárný, Miroslav and Wolpert, DavidH.},
    Pages = {75-111},
    Series = {Intelligent Systems Reference Library},
    Volume = {28},
    Doi = {10.1007/978-3-642-24647-0_4},
    ISBN = {978-3-642-24646-3},
    Language = {English},
    Owner = {rcnlee},
    Url = {http://dx.doi.org/10.1007/978-3-642-24647-0_4}
    }
  • [DOI] R. Lee, D. H. Wolpert, J. Bono, S. Backhaus, R. Bent, and B. Tracey, “Counter-Factual Reinforcement Learning: How to Model Decision-Makers That Anticipate the Future,” in Decision Making and Imperfection, T. V. Guy, M. Karny, and D. H. Wolpert, Eds., Springer Berlin Heidelberg, 2013, vol. 474, pp. 101-128.
    [Bibtex]
    @InCollection{Lee2013Counter,
    Title = {Counter-Factual Reinforcement Learning: How to Model Decision-Makers That Anticipate the Future},
    Author = {Lee, Ritchie and Wolpert, David H. and Bono, James and Backhaus, Scott and Bent, Russell and Tracey, Brendan},
    Booktitle = {Decision Making and Imperfection},
    Publisher = {Springer Berlin Heidelberg},
    Year = {2013},
    Chapter = {4},
    Editor = {Guy, Tatiana V. and Karny, Miroslav and Wolpert, David H.},
    Pages = {101-128},
    Series = {Studies in Computational Intelligence},
    Volume = {474},
    Doi = {10.1007/978-3-642-36406-8_4},
    ISBN = {978-3-642-36405-1},
    Language = {English},
    Owner = {rcnlee},
    Url = {http://dx.doi.org/10.1007/978-3-642-36406-8_4}
    }
  • E. J. Schlicht, R. Lee, D. H. Wolpert, M. J. Kochenderfer, and B. Tracey, “Predicting the Behavior of Interacting Humans by Fusing Data from Multiple Sources,” in Conference on Uncertainty in Artificial Intelligence (UAI), Catalina Island, California, 2012.
    [Bibtex]
    @InProceedings{Schlicht2012,
    Title = {Predicting the Behavior of Interacting Humans by Fusing Data from Multiple Sources},
    Author = {Erik J. Schlicht and Ritchie Lee and David H. Wolpert and Mykel J. Kochenderfer and Brendan Tracey},
    Booktitle = uai,
    Year = {2012},
    Address = {Catalina Island, California},
    Owner = {rcnlee},
    Url = {http://www.auai.org/uai2012/papers/74.pdf}
    }
  • G. Yan, R. Lee, A. Kent, and D. Wolpert, “Towards a Bayesian Network Game Framework for Evaluating DDoS Attacks and Defense,” in ACM Conference on Computer and Communications Security (CCS), 2012, pp. 553-566.
    [Bibtex]
    @InProceedings{Yan2012,
    Title = {Towards a {B}ayesian Network Game Framework for Evaluating {DDoS} Attacks and Defense},
    Author = {Yan, Guanhua and Lee, Ritchie and Kent, Alex and Wolpert, David},
    Booktitle = acmccs,
    Year = {2012},
    Organization = {ACM},
    Pages = {553--566},
    Owner = {rcnlee}
    }
  • Y. Yildiz, R. Lee, and G. Brat, “Using Game Theoretic Models to Predict Pilot Behavior in NextGen Merging and Landing Scenario,” in AIAA Modeling and Simulation Technologies Conference (MST), 2012.
    [Bibtex]
    @InProceedings{Yildiz2012,
    Title = {Using Game Theoretic Models to Predict Pilot Behavior in {N}ext{G}en Merging and Landing Scenario},
    Author = {Yildiz, Yildiray and Lee, Ritchie and Brat, Guillaume},
    Booktitle = aiaa_mst,
    Year = {2012},
    Owner = {rcnlee}
    }