Browsing by Author "Wang, Dali"

Now showing 1 - 7 of 7
  • Thumbnail Image
    Item
    ElasticBroker: Combining HPC with Cloud to Provide Realtime Insights into Simulations
    (arXiv, 2020) Li, Feng; Wang, Dali; Yan, Feng; Song, Fengguang; Computer and Information Science, School of Science
    For large-scale scientific simulations, it is expensive to store raw simulation results to perform post-analysis. To minimize expensive I/O, "in-situ" analysis is often used, where analysis applications are tightly coupled with scientific simulations and can access and process the simulation results in memory. Increasingly, scientific domains employ Big Data approaches to analyze simulations for scientific discoveries. However, it remains a challenge to organize, transform, and transport data at scale between the two semantically different ecosystems (HPC and Cloud systems). In an effort to address these challenges, we design and implement the ElasticBroker software framework, which bridges HPC and Cloud applications to form an "in-situ" scientific workflow. Instead of writing simulation results to parallel file systems, ElasticBroker performs data filtering, aggregation, and format conversions to close the gap between an HPC ecosystem and a distinct Cloud ecosystem. To achieve this goal, ElasticBroker reorganizes simulation snapshots into continuous data streams and send them to the Cloud. In the Cloud, we deploy a distributed stream processing service to perform online data analysis. In our experiments, we use ElasticBroker to setup and execute a cross-ecosystem scientific workflow, which consists of a parallel computational fluid dynamics (CFD) simulation running on a supercomputer, and a parallel dynamic mode decomposition (DMD) analysis application running in a Cloud computing platform. Our results show that running scientific workflows consisting of decoupled HPC and Big Data jobs in their native environments with ElasticBroker, can achieve high quality of service, good scalability, and provide high-quality analytics for ongoing simulations.
  • Thumbnail Image
    Item
    FQL: An Extensible Feature Query Language and Toolkit on Searching Software Characteristics for HPC Applications
    (Springer, 2020) Zheng, Weijian; Wang, Dali; Song, Fengguang; Computer and Information Science, School of Science
    The amount of large-scale scientific computing software is dramatically increasing. In this work, we designed a new query language, named Feature Query Language (FQL), to collect and extract HPC-related software features or metadata from a quick static code analysis. We also designed and implemented an FQL-based toolkit to automatically detect and present software features using an extensible query repository. A number of large-scale, high performance computing (HPC) scientific applications have been studied in the paper with the FQL toolkit to demonstrate the HPC-related feature extraction and information/metadata collection. Different from the existing static software analysis and refactoring tools which focus on software debug, development and code transformation, the FQL toolkit is simpler, significantly lightweight and strives to collect various and diverse software metadata with ease and rapidly.
  • Thumbnail Image
    Item
    Geometrically Matched Multi-source Microscopic Image Synthesis Using Bidirectional Adversarial Networks
    (Springer, 2022) Zhuang, Jun; Wang, Dali; Computer Science, Luddy School of Informatics, Computing, and Engineering
    Microscopic images from multiple modalities can produce plentiful experimental information. In practice, biological or physical constraints under a given observation period may prevent researchers from acquiring enough microscopic scanning. Recent studies demonstrate that image synthesis is one of the popular approaches to release such constraints. Nonetheless, most existing synthesis approaches only translate images from the source domain to the target domain without solid geometric associations. To embrace this challenge, we propose an innovative model architecture, BANIS, to synthesize diversified microscopic images from multi-source domains with distinct geometric features. The experimental outcomes indicate that BANIS successfully synthesizes favorable image pairs on C. elegans microscopy embryonic images. To the best of our knowledge, BANIS is the first application to synthesize microscopic images that associate distinct spatial geometric features from multi-source domains.
  • Thumbnail Image
    Item
    OpenGraphGym: A Parallel Reinforcement Learning Framework for Graph Optimization Problems
    (Springer, 2020-06-15) Zheng, Weijian; Wang, Dali; Song, Fengguang; Krzhizhanovskaya, Valeria V.; Závodszky, Gábor; Lees, Michael H.; Dongarra, Jack J.; Sloot, Peter M. A.; Brissos, Sérgio; Teixeira, João; Computer and Information Science, School of Science
    This paper presents an open-source, parallel AI environment (named OpenGraphGym) to facilitate the application of reinforcement learning (RL) algorithms to address combinatorial graph optimization problems. This environment incorporates a basic deep reinforcement learning method, and several graph embeddings to capture graph features, it also allows users to rapidly plug in and test new RL algorithms and graph embeddings for graph optimization problems. This new open-source RL framework is targeted at achieving both high performance and high quality of the computed graph solutions. This RL framework forms the foundation of several ongoing research directions, including 1) benchmark works on different RL algorithms and embedding methods for classic graph problems; 2) advanced parallel strategies for extreme-scale graph computations, as well as 3) performance evaluation on real-world graph solutions.
  • Thumbnail Image
    Item
    Use of impact fees to incentivize low-impact development and promote compact growth
    (2013-07-15) Lu, Zhongming; Noonan, Douglas S.; Crittenden, John C.; Jeong, Hyunju; Wang, Dali
    Low-impact development (LID) is an innovative stormwater management strategy that restores the predevelopment hydrology to prevent increased stormwater runoff from land development. Integrating LID into residential subdivisions and increasing population density by building more compact living spaces (e.g., apartment homes) can result in a more sustainable city by reducing stormwater runoff, saving infrastructural cost, increasing the number of affordable homes, and supporting public transportation. We develop an agent-based model (ABM) that describes the interactions between several decision-makers (i.e., local government, a developer, and homebuyers) and fiscal drivers (e.g., property taxes, impact fees). The model simulates the development of nine square miles of greenfield land. A more sustainable development (MSD) scenario introduces an impact fee that developers must pay if they choose not to use LID to build houses or apartment homes. Model simulations show homeowners selecting apartment homes 60% or 35% of the time after 30 years of development in MSD or business as usual (BAU) scenarios, respectively. The increased adoption of apartment homes results from the lower cost of using LID and improved quality of life for apartment homes relative to single-family homes. The MSD scenario generates more tax revenue and water savings than does BAU. A time-dependent global sensitivity analysis quantifies the importance of socioeconomic variables on the adoption rate of apartment homes. The top influential factors are the annual pay rates (or capital recovery factor) for single-family houses and apartment homes. The ABM can be used by city managers and policymakers for scenario exploration in accordance with local conditions to evaluate the effectiveness of impact fees and other policies in promoting LID and compact growth.
  • Thumbnail Image
    Item
    X-Composer: Enabling Cross-Environments In-SituWorkflows between HPC and Cloud
    (ACM, 2021-07) Li, Feng; Wang, Dali; Yan, Feng; Song, Fengguang; Computer Information and Graphics Technology, School of Engineering and Technology
    As large-scale scientific simulations and big data analyses become more popular, it is increasingly more expensive to store huge amounts of raw simulation results to perform post-analysis. To minimize the expensive data I/O, "in-situ" analysis is a promising approach, where data analysis applications analyze the simulation generated data on the fly without storing it first. However, it is challenging to organize, transform, and transport data at scales between two semantically different ecosystems due to the distinct software and hardware difference. To tackle these challenges, we design and implement the X-Composer framework. X-Composer connects cross-ecosystem applications to form an "in-situ" scientific workflow, and provides a unified approach and recipe for supporting such hybrid in-situ workflows on distributed heterogeneous resources. X-Composer reorganizes simulation data as continuous data streams and feeds them seamlessly into the Cloud-based stream processing services to minimize I/O overheads. For evaluation, we use X-Composer to set up and execute a cross-ecosystem workflow, which consists of a parallel Computational Fluid Dynamics simulation running on HPC, and a distributed Dynamic Mode Decomposition analysis application running on Cloud. Our experimental results show that X-Composer can seamlessly couple HPC and Big Data jobs in their own native environments, achieve good scalability, and provide high-fidelity analytics for ongoing simulations in real-time.
  • Thumbnail Image
    Item
    XScan: An Integrated Tool for Understanding Open Source Community-Based Scientific Code
    (Springer, 2019) Zheng, Weijian; Wang, Dali; Song, Fengguang; Computer and Information Science, School of Science
    Many scientific communities have adopted community-based models that integrate multiple components to simulate whole system dynamics. The community software projects’ complexity, stems from the integration of multiple individual software components that were developed under different application requirements and various machine architectures, has become a challenge for effective software system understanding and continuous software development. The paper presents an integrated software toolkit called X-ray Software Scanner (in abbreviation, XScan) for a better understanding of large-scale community-based scientific codes. Our software tool provides support to quickly summarize the overall information of scientific codes, including the number of lines of code, programming languages, external library dependencies, as well as architecture-dependent parallel software features. The XScan toolkit also realizes a static software analysis component to collect detailed structural information and provides an interactive visualization and analysis of the functions. We use a large-scale community-based Earth System Model to demonstrate the workflow, functions and visualization of the toolkit. We also discuss the application of advanced graph analytics techniques to assist software modular design and component refactoring.