Browsing by Author "Mohler, George"

Now showing 1 - 10 of 50
  • Thumbnail Image
    Item
    A note on the multiplicative fairness score in the NIJ recidivism forecasting challenge
    (Springer Nature, 2021) Mohler, George; Porter, Michael D.; Computer Science, Luddy School of Informatics, Computing, and Engineering
    Background: The 2021 NIJ recidivism forecasting challenge asks participants to construct predictive models of recidivism while balancing false positive rates across groups of Black and white individuals through a multiplicative fairness score. We investigate the performance of several models for forecasting 1-year recidivism and optimizing the NIJ multiplicative fairness metric. Methods: We consider standard linear and logistic regression, a penalized regression that optimizes a convex surrogate loss (that we show has an analytical solution), two post-processing techniques, linear regression with re-balanced data, a black-box general purpose optimizer applied directly to the NIJ metric and a gradient boosting machine learning approach. Results: For the set of models investigated, we find that a simple heuristic of truncating scores at the decision threshold (thus predicting no recidivism across the data) yields as good or better NIJ fairness scores on held out data compared to other, more sophisticated approaches. We also find that when the cutoff is further away from the base rate of recidivism, as is the case in the competition where the base rate is 0.29 and the cutoff is 0.5, then simply optimizing the mean square error gives nearly optimal NIJ fairness metric solutions. Conclusions: The multiplicative metric in the 2021 NIJ recidivism forecasting competition encourages solutions that simply optimize MSE and/or use truncation, therefore yielding trivial solutions that forecast no one will recidivate.
  • Thumbnail Image
    Item
    Adversarial Attacks and Defense Mechanisms to Improve Robustness of Deep Temporal Point Processes
    (2022-08) Khorshidi, Samira; Mohler, George; Al Hasan, Mohammad; Raje, Rajeev; Durresi, Arjan
    Temporal point processes (TPP) are mathematical approaches for modeling asynchronous event sequences by considering the temporal dependency of each event on past events and its instantaneous rate. Temporal point processes can model various problems, from earthquake aftershocks, trade orders, gang violence, and reported crime patterns, to network analysis, infectious disease transmissions, and virus spread forecasting. In each of these cases, the entity’s behavior with the corresponding information is noted over time as an asynchronous event sequence, and the analysis is done using temporal point processes, which provides a means to define the generative mechanism of the sequence of events and ultimately predict events and investigate causality. Among point processes, Hawkes process as a stochastic point process is able to model a wide range of contagious and self-exciting patterns. One of Hawkes process’s well-known applications is predicting the evolution of viral processes on networks, which is an important problem in biology, the social sciences, and the study of the Internet. In existing works, mean-field analysis based upon degree distribution is used to predict viral spreading across networks of different types. However, it has been shown that degree distribution alone fails to predict the behavior of viruses on some real-world networks. Recent attempts have been made to use assortativity to address this shortcoming. This thesis illustrates how the evolution of such a viral process is sensitive to the underlying network’s structure. In Chapter 3 , we show that adding assortativity does not fully explain the variance in the spread of viruses for a number of real-world networks. We propose using the graphlet frequency distribution combined with assortativity to explain variations in the evolution of viral processes across networks with identical degree distribution. Using a data-driven approach, by coupling predictive modeling with viral process simulation on real-world networks, we show that simple regression models based on graphlet frequency distribution can explain over 95% of the variance in virality on networks with the same degree distribution but different network topologies. Our results highlight the importance of graphlets and identify a small collection of graphlets that may have the most significant influence over the viral processes on a network. Due to the flexibility and expressiveness of deep learning techniques, several neural network-based approaches have recently shown promise for modeling point process intensities. However, there is a lack of research on the possible adversarial attacks and the robustness of such models regarding adversarial attacks and natural shocks to systems. Furthermore, while neural point processes may outperform simpler parametric models on in-sample tests, how these models perform when encountering adversarial examples or sharp non-stationary trends remains unknown. In Chapter 4 , we propose several white-box and black-box adversarial attacks against deep temporal point processes. Additionally, we investigate the transferability of whitebox adversarial attacks against point processes modeled by deep neural networks, which are considered a more elevated risk. Extensive experiments confirm that neural point processes are vulnerable to adversarial attacks. Such a vulnerability is illustrated both in terms of predictive metrics and the effect of attacks on the underlying point process’s parameters. Expressly, adversarial attacks successfully transform the temporal Hawkes process regime from sub-critical to into a super-critical and manipulate the modeled parameters that is considered a risk against parametric modeling approaches. Additionally, we evaluate the vulnerability and performance of these models in the presence of non-stationary abrupt changes, using the crimes and Covid-19 pandemic dataset as an example. Considering the security vulnerability of deep-learning models, including deep temporal point processes, to adversarial attacks, it is essential to ensure the robustness of the deployed algorithms that is despite the success of deep learning techniques in modeling temporal point processes. In Chapter 5 , we study the robustness of deep temporal point processes against several proposed adversarial attacks from the adversarial defense viewpoint. Specifically, we investigate the effectiveness of adversarial training using universal adversarial samples in improving the robustness of the deep point processes. Additionally, we propose a general point process domain-adopted (GPDA) regularization, which is strictly applicable to temporal point processes, to reduce the effect of adversarial attacks and acquire an empirically robust model. In this approach, unlike other computationally expensive approaches, there is no need for additional back-propagation in the training step, and no further network isrequired. Ultimately, we propose an adversarial detection framework that has been trained in the Generative Adversarial Network (GAN) manner and solely on clean training data. Finally, in Chapter 6 , we discuss implications of the research and future research directions.
  • Thumbnail Image
    Item
    Adversarial Attacks on Deep Temporal Point Process
    (IEEE, 2022) Khorshidi, Samira; Wang, Bao; Mohler, George; Computer Science, Luddy School of Informatics, Computing, and Engineering
    Temporal point processes have many applications, from crime forecasting to modeling earthquake aftershocks sequences. Due to the flexibility and expressiveness of deep learning, neural network-based approaches have recently shown promise for modeling point process intensities. However, there is a lack of research on the robustness of such models in regards to adversarial attacks and natural shocks to systems. Precisely, while neural point processes may outperform simpler parametric models on in-sample tests, how these models perform when encountering adversarial examples or sharp non-stationary trends remains unknown. Current work proposes several white-box and blackbox adversarial attacks against temporal point processes modeled by deep neural networks. Extensive experiments confirm that predictive performance and parametric modeling of neural point processes are vulnerable to adversarial attacks. Additionally, we evaluate the vulnerability and performance of these models in the presence of non-stationary abrupt changes, using the crimes dataset, during the Covid-19 pandemic, as an example.
  • Thumbnail Image
    Item
    Advocacy in Mental Health Social Interactions on Public Social Media
    (2022-02) Cornet, Victor P.; Holden, Richard J.; Bolchini, Davide; Brady, Erin; Mohler, George; Hong, Michin; Lee, Sangwon
    Health advocacy is a social phenomenon in which individuals and collectives attempt to raise awareness and change opinions and policies about health-related causes. Mental health advocacy is health advocacy to advance treatment, rights, and recognition of people living with a mental health condition. The Internet is reshaping how mental health advocacy is performed on a global scale, by facilitating and broadening the reach of advocacy activities, but also giving more room for opposing mental health advocacy. Another factor contributing to mental health advocacy lies in the cultural underpinnings of mental health in different societies; East Asian countries like South Korea have higher stigma attached to mental health compared to Western countries like the US. This study examines interactions about schizophrenia, a specific mental health diagnosis, on public social media (Facebook, Instagram, and Twitter) in two different languages, English and Korean, to determine how mental health advocacy and its opposition are expressed on social media. After delineation of a set of keywords for retrieval of content about schizophrenia, three months’ worth of social media posts were collected; a subset of these posts was then analyzed qualitatively using constant comparing with a proposed model describing online mental heath advocacy based on existing literature. Various expressions of light mental health advocacy, such as sharing facts about schizophrenia, and strong advocacy, showcasing offline engagement, were found in English posts; many of these expressions were however absent from the analyzed Korean posts that heavily featured jokes, insults, and criticisms. These findings were used to train machine learning classifiers to detect advocacy and counter-advocacy. The classifiers confirmed the predominance of counter-advocacy in Korean posts compared to important advocacy prevalence in English posts. These findings informed culturally sensitive recommendations for social media uses by mental health advocates and implications for international social media studies in human-computer interaction.
  • Thumbnail Image
    Item
    Analyzing the Impacts of Public Policy on COVID-19 Transmission: A Case Study of the Role of Model and Dataset Selection Using Data from Indiana
    (Taylor & Francis, 2021) Mohler, George; Short, Martin B.; Schoenberg, Frederic; Sledge, Daniel; Computer and Information Science, School of Science
    Dynamic estimation of the reproduction number of COVID-19 is important for assessing the impact of public health measures on virus transmission. State and local decisions about whether to relax or strengthen mitigation measures are being made in part based on whether the reproduction number, Rt , falls below the self-sustaining value of 1. Employing branching point process models and COVID-19 data from Indiana as a case study, we show that estimates of the current value of Rt , and whether it is above or below 1, depend critically on choices about data selection and model specification and estimation. In particular, we find a range of Rt values from 0.47 to 1.20 as we vary the type of estimator and input dataset. We present methods for model comparison and evaluation and then discuss the policy implications of our findings.
  • Thumbnail Image
    Item
    Applications of Data Mining in Healthcare
    (2019-05) Peng, Bo; Mohler, George; Dundar, Murat; Zheng, Jiang Yu
    With increases in the quantity and quality of healthcare related data, data mining tools have the potential to improve people’s standard of living through personalized and predictive medicine. In this thesis we improve the state-of-the-art in data mining for several problems in the healthcare domain. In problems such as drug-drug interaction prediction and Alzheimer’s Disease (AD) biomarkers discovery and prioritization, current methods either require tedious feature engineering or have unsatisfactory performance. New effective computational tools are needed that can tackle these complex problems. In this dissertation, we develop new algorithms for two healthcare problems: high-order drug-drug interaction prediction and amyloid imaging biomarker prioritization in Alzheimer’s Disease. Drug-drug interactions (DDIs) and their associated adverse drug reactions (ADRs) represent a significant detriment to the public h ealth. Existing research on DDIs primarily focuses on pairwise DDI detection and prediction. Effective computational methods for high-order DDI prediction are desired. In this dissertation, I present a deep learning based model D 3 I for cardinality-invariant and order-invariant high-order DDI pre- diction. The proposed models achieve 0.740 F1 value and 0.847 AUC value on high-order DDI prediction, and outperform classical methods on order-2 DDI prediction. These results demonstrate the strong potential of D 3 I and deep learning based models in tackling the prediction problems of high-order DDIs and their induced ADRs. The second problem I consider in this thesis is amyloid imaging biomarkers discovery, for which I propose an innovative machine learning paradigm enabling precision medicine in this domain. The paradigm tailors the imaging biomarker discovery process to individual characteristics of a given patient. I implement this paradigm using a newly developed learning-to-rank method PLTR. The PLTR model seamlessly integrates two objectives for joint optimization: pushing up relevant biomarkers and ranking among relevant biomarkers. The empirical study of PLTR conducted on the ADNI data yields promising results to identify and prioritize individual-specific amyloid imaging biomarkers based on the individual’s structural MRI data. The resulting top ranked imaging biomarkers have the potential to aid personalized diagnosis and disease subtyping.
  • Thumbnail Image
    Item
    Assessing GAN-based approaches for generative modeling of crime text reports
    (IEEE, 2020-11) Khorshidi, Samira; Mohler, George; Carter, Jeremy G.; School of Public and Environmental Affairs
    Analysis and modeling of crime text report data has important applications, including refinement of crime classifications, clustering of documents, and feature extraction for spatio-temporal forecasts. Having better neural network representations of crime text data may facilitate all of these tasks. This paper evaluates the ability of generative adversarial network models to represent crime text data and generate realistic crime reports. We compare four state of the art GAN algorithms in terms of quantitative metrics such as coherence, embedding similarity, negative log-likelihood, and qualitatively based on inspection of generated text. We discuss current challenges with crime text representation and directions for future research.
  • Thumbnail Image
    Item
    CDASH: Community Data Analytics for Social Harm Prevention
    (IEEE, 2018-09) Pandey, Saurabh; Chowdhury, Nahida; Patil, Milan; Raje, Rajeev R.; Shreyas, C. S.; Mohler, George; Carter, Jeremy; Computer and Information Science, School of Science
    Communities are adversely affected by heterogeneous social harm events (e.g., crime, traffic crashes, medical emergencies, drug use) and police, fire, health and social service departments are tasked with mitigating social harm through various types of interventions. Smart cities of the future will need to leverage IoT, data analytics, and government and community human resources to most effectively reduce social harm. Currently, methods for collection, analysis, and modeling of heterogeneous social harm data to identify government actions to improve quality of life are needed. In this paper we propose a system, CDASH, for synthesizing heterogeneous social harm data from multiples sources, identifying social harm risks in space and time, and communicating the risk to the relevant community resources best equipped to intervene. We discuss the design, architecture, and performance of CDASH. CDASH allows users to report live social harm events using mobile hand-held devices and web browsers and flags high risk areas for law enforcement and first responders. To validate the methodology, we run simulations on historical social harm event data in Indianapolis illustrating the advantages of CDASH over recently introduced social harm indices and existing point process methods used for predictive policing.
  • Thumbnail Image
    Item
    The challenges of modeling and forecasting the spread of COVID-19
    (National Academy of Sciences, 2020-07-02) Bertozzi, Andrea L.; Franco, Elisa; Mohler, George; Short, Martin B.; Sledge, Daniel; Computer and Information Science, School of Science
    The coronavirus disease 2019 (COVID-19) pandemic has placed epidemic modeling at the forefront of worldwide public policy making. Nonetheless, modeling and forecasting the spread of COVID-19 remains a challenge. Here, we detail three regional-scale models for forecasting and assessing the course of the pandemic. This work demonstrates the utility of parsimonious models for early-time data and provides an accessible framework for generating policy-relevant insights into its course. We show how these models can be connected to each other and to time series data for a particular region. Capable of measuring and forecasting the impacts of social distancing, these models highlight the dangers of relaxing nonpharmaceutical public health interventions in the absence of a vaccine or antiviral therapies.
  • Thumbnail Image
    Item
    Classifying the Unknown: Identification of Insects by Deep Open-set Bayesian Learning
    (bioRxiv, 2021-09-17) Badirli, Sarkhan; Picard, Christine J.; Mohler, George; Akata, Zeynep; Dundar, Murat
    Insects represent a large majority of biodiversity on Earth, yet only 20% of the estimated 5.5 million insect species are currently described (1). While describing new species typically requires specific taxonomic expertise to identify morphological characters that distinguish it from other potential species, DNA-based methods have aided in providing additional evidence of separate species (2). Machine learning (ML) is emerging as a potential new approach in identifying new species, given that this analysis may be more sensitive to subtle differences humans may not process. Existing ML algorithms are limited by image repositories that do not include undescribed species. We developed a Bayesian deep learning method for the open-set classification of species. The proposed approach forms a Bayesian hierarchy of species around corresponding genera and uses deep embeddings of images and barcodes together to identify insects at the lowest level of abstraction possible. To demonstrate proof of concept, we used a database of 32,848 insect instances from 1,040 described species split into training and test data. The test data included 243 species not present in the training data. Our results demonstrate that using DNA sequences and images together, insect instances of described species can be classified with 96.66% accuracy while achieving accuracy of 81.39% in identifying genera of insect instances of undescribed species. The proposed deep open-set Bayesian model demonstrates a powerful new approach that can be used for the gargantuan task of identifying new insect species.