AI/ML for Network Security: The Emperor has no Clothes https://trusteeml.github.io/ Arthur S. Jacobs UFRGS, Brazil asjacobs@inf.ufrgs.br Roman Beltiukov UCSB, USA rbeltiukov@ucsb.edu Walter Willinger NIKSUN Inc., USA wwillinger@niksun.com Ronaldo A. Ferreira UFMS, Brazil raf@facom.ufms.br Arpit Gupta UCSB, USA arpitgupta@ucsb.edu Lisandro Z. Granville UFRGS, Brazil granville@inf.ufrgs.br ABSTRACT Several recent research efforts have proposed Machine Learning (ML)-based solutions that can detect complex patterns in network traffic for a wide range of network security problems. However, without understanding how these black-box models are making their decisions, network operators are reluctant to trust and deploy them in their production settings. One key reason for this reluctance is that these models are prone to the problem of underspecification, defined here as the failure to specify a model in adequate detail. Not unique to the network security domain, this problem manifests itself in ML models that exhibit unexpectedly poor behavior when deployed in real-world settings and has prompted growing interest in developing interpretable ML solutions (e.g., decision trees) for łexplainingž to humans how a given black-box model makes its deci- sions. However, synthesizing such explainable models that capture a given black-box model’s decisions with high fidelity while also be- ing practical (i.e., small enough in size for humans to comprehend) is challenging. In this paper, we focus on synthesizing high-fidelity and low- complexity decision trees to help network operators determine if their ML models suffer from the problem of underspecification. To this end, we present Trustee, a framework that takes an existing ML model and training dataset as input and generates a high-fidelity, easy-to-interpret decision tree and associated trust report as out- put. Using published ML models that are fully reproducible, we show how practitioners can use Trustee to identify three common instances of model underspecification; i.e., evidence of shortcut learning, presence of spurious correlations, and vulnerability to out-of-distribution samples. CCS CONCEPTS • Networks →Network security; • Computing methodologies →Machine learning; • Security and privacy; Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. CCS ’22, November 7ś11, 2022, Los Angeles, CA, USA. © 2022 Association for Computing Machinery. ACM ISBN 978-1-4503-9450-5/22/11...$15.00 https://doi.org/10.1145/3548606.3560609 ACM Reference Format: Arthur S. Jacobs, Roman Beltiukov, Walter Willinger, Ronaldo A. Ferreira, Arpit Gupta, and Lisandro Z. Granville. 2022. AI/ML for Network Secu- rity: The Emperor has no Clothes. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (CCS ’22), Novem- ber 7ś11, 2022, Los Angeles, CA, USA. ACM, New York, NY, USA, 15 pages. https://doi.org/10.1145/3548606.3560609 KEYWORDS Network Security; Artificial Intelligence; Machine Learning; Ex- plainability; Interpretability; Trust; 1 INTRODUCTION In the last few years, we have witnessed a growing tension in the network-security community. Recent research has demonstrated the benefits of Artificial Intelligence (AI) and Machine Learning (ML) models over simpler rule-based heuristics in identifying com- plex network traffic patterns for a wide range of network security problems (see recent survey articles such as [9, 46, 55, 62]). At the same time, we have seen reluctance among network security researchers and practitioners when it comes to adopting these ML- based research artifacts in production settings (e.g., see [2, 4, 58]). The black-box nature of most of these proposed solutions is the primary reason for this cautionary attitude and overall hesitance. More concretely, the inability to explain how and why these models make their decisions renders them a hard sell compared to existing simpler but typically less effective rule-based approaches. This tension is not unique to network security problems but ap- plies more generally to any learning models, especially when their decision-making can have serious societal implications (e.g., health- care, credit rating, job applications, and criminal justice system). At the same time, this basic tension has also driven recent efforts to łcrack openž the black-box learning models, explaining why and how they make their decisions (e.g., łinterpretable MLž [51], łexplainable AI (XAI)ž [59], and łtrustworthy AIž [12]). However, to ensure that these efforts are of practical use in particular applica- tion domains of AI/ML such as network security is challenging and requires further qualifying notions such as (model) interpretability or trust (in a model) [40] and also demands solving a number of fundamental research problems in these new areas of AI/ML. In this paper, we first provide such a qualification that is moti- vated by the needs of the field of network security as application domain of AI/ML and equates łan end user having trust in an AI/ML modelž with łan end user being comfortable with relinquishing con- trol to the modelž [40]. Given this specific definition of what it 1537

CCS ’22, November 7–11, 2022, Los Angeles, CA, USA. Arthur S. Jacobs et al. Select Model Collect Data x y Train x y Test Model Evaluation Explain Analyze High-fidelity & Low-complexity DT Extraction Trust Report Generation Model design and training Evaluate model with test data TRUSTEE Figure 1: Trustee overview. means for an AI/ML model to engender trust, we next address a number of fundamental research challenges related to the problem of quantitatively deciding when an end user is comfortable with relinquishing control to a given AI/ML model. To this end, a partic- ular focus of this paper is on determining whether or not a given AI/ML model suffers from the problem of underspecification [17]. Here, the problem of underspecification in modern AI/ML refers to determining whether the success of a trained model (e.g., high accuracy) is indeed due to its innate ability to encode some essential structure of the underlying system or data or is simply the result of some inductive biases that the trained model happens to en- code. In practice, inductive biases typically manifest themselves in instances of shortcut learning strategies [28], signs of spurious cor- relations [3], or an inherent inability for out-of-distribution (o.o.d.) generalizations (i.e., test data distribution is different from training data distribution). The implication of such inductive biases is that their presence in trained AI/ML models prevents these models from being credible or trustworthy; that is, generalize as expected in deployment scenarios. Thus, for establishing the specific type of trust in an ML model considered in this paper, it is critical to be able to identify these inductive biases, and this paper takes a first step towards achieving this ambitious goal. To detect underspecification issues in learning models for net- work security problems, we develop Trustee (TRUSt-oriented decision TreE Extraction). This framework provides a means for carefully inspecting black-box learning models for the presence of inductive biases. Figure 1 shows how Trustee augments the traditional ML pipeline to examine the trustworthiness of a given ML model. Specifically developed with the application domain of network security in mind, Trustee takes a given black-box model and the dataset that has been used to train that model as input and outputs a łwhite-boxž model in the form of a high-quality decision tree (DT) explanation. Importantly, in synthesizing this DT, Trustee’s focus is first and foremost on ensuring its practical use which, in turn, requires leveraging domain-specific observations to strike a balance between model fidelity (i.e., accuracy of the DT with respect to the black- box model), model complexity, and model stability. Here, complexity refers to both the size of the DT and to aspects of the tree’s branches. In particular, when viewing the tree’s branches as decision rules, we are concerned with their explicitness and intelligibility; that is, we require these rules to be readily recognizable by domain experts and be largely in agreement with the experts’ domain knowledge. Model stability, on the other hand, is concerned with the correctness, coverage and stability of the decision rules; that is, we require them to correctly describe how the given black-box model makes a significant number of its decisions and also want them to be largely insensitive to the particular data samples that Trustee used in the process of selecting its final DT explanation. We achieve this insensitivity or stability by implementing a heuristic method that selects from among a number of different candidate DTs the one that has the highest mean agreement. Here, the agreement between two different DTs is a measure of how often the two DTs will make the same decision for the same input data [30, 60]. In practical terms, implementing this heuristic reduces the likelihood that Trustee outputs a misleading DT explanation. Trustee also outputs a trust report associated with the DT ex- planation, which operators can consult to determine whether there is evidence that the given black-box model suffers from the problem of underspecification. If such evidence is found, the information provided in the trust report can be used to identify components of the traditional ML pipeline (e.g., training data and model selection) that need to be modified in an effort to improve upon an ML model that Trustee has found to be untrustworthy. While our work contributes to the rapidly growing ML literature on model explainability/interpretability and is inspired by ongoing developments in this area, our efforts and objectives differ from existing approaches in a number of significant ways. For one, given the inherent complexity of learning problems for networking, existing approaches for replacing black-box models with łwhite-boxž models that are inherently explainable in the first place (e.g., decisions trees) are in general impractical. Moreover, local interpretability methods [31, 48, 53] are not suitable for examining the various instances of the underspecification problem. At the same time, although our effort is motivated by prior studies that focus on global interpretability [6, 7, 37], these works are either only applicable to a specific class of learning models (e.g., reinforcement learning) or suffer from poor fidelity. Through various case studies, we illustrate in Section 7 how op- erators can use Trustee’s DTs and associated trust reports to detect the presence of inductive biases. More specifically, we use published ML models that are reproducible (i.e., code base and datasets are publicly available) to show how network operators can use the information provided by Trustee to detect instances of shortcut learning strategies, obtain evidence of overfitting and/or whether the model relies on spurious correlations to make its decisions, or determine the model’s inability to generalize to out-of-distribution data. 2 BACKGROUND AND RELATED WORK The application domain of AI/ML considered in this paper is the area of network security. In this section, we first discuss the unique challenges that this area poses for utilizing the latest advances in AI/ML. In particular, we focus on important recent AI/ML concepts such as łinterpretable MLž and łexplainable AIž and discuss their relevance for our work. 2.1 Challenges in ML for Network Security Beyond the already-mentioned trust issue, there are a number of other reasons why the area of network security is a particularly challenging application domain for AI/ML. Networking-related datasets in general and cybersecurity-specific datasets in particular typically contain information about what is being communicated 1538

AI/ML for Network Security: The Emperor has no Clothes CCS ’22, November 7–11, 2022, Los Angeles, CA, USA. over a network (e.g., packet-level traffic traces) or provide insight into how networks enable such information exchanges. As such, the datasets often raise serious end user-specific privacy concerns or reveal provider-specific details that many companies consider to