Browsing by Subject "cloud computing"

Now showing 1 - 3 of 3
  • Thumbnail Image
    Item
    ElasticBroker: Combining HPC with Cloud to Provide Realtime Insights into Simulations
    (arXiv, 2020) Li, Feng; Wang, Dali; Yan, Feng; Song, Fengguang; Computer and Information Science, School of Science
    For large-scale scientific simulations, it is expensive to store raw simulation results to perform post-analysis. To minimize expensive I/O, "in-situ" analysis is often used, where analysis applications are tightly coupled with scientific simulations and can access and process the simulation results in memory. Increasingly, scientific domains employ Big Data approaches to analyze simulations for scientific discoveries. However, it remains a challenge to organize, transform, and transport data at scale between the two semantically different ecosystems (HPC and Cloud systems). In an effort to address these challenges, we design and implement the ElasticBroker software framework, which bridges HPC and Cloud applications to form an "in-situ" scientific workflow. Instead of writing simulation results to parallel file systems, ElasticBroker performs data filtering, aggregation, and format conversions to close the gap between an HPC ecosystem and a distinct Cloud ecosystem. To achieve this goal, ElasticBroker reorganizes simulation snapshots into continuous data streams and send them to the Cloud. In the Cloud, we deploy a distributed stream processing service to perform online data analysis. In our experiments, we use ElasticBroker to setup and execute a cross-ecosystem scientific workflow, which consists of a parallel computational fluid dynamics (CFD) simulation running on a supercomputer, and a parallel dynamic mode decomposition (DMD) analysis application running in a Cloud computing platform. Our results show that running scientific workflows consisting of decoupled HPC and Big Data jobs in their native environments with ElasticBroker, can achieve high quality of service, good scalability, and provide high-quality analytics for ongoing simulations.
  • Thumbnail Image
    Item
    Privacy and Security Risks and Requirements with Healthcare IT: Hitting a Home Run Instead of a Foul Ball
    (2012-11-13) Hook, Sara Anne; Jones, Josette
    Using baseball as a metaphor, this practical, engaging session will explore the security and privacy risks with a number of technologies for storing, handling and communicating health information and highlight the legal obligations and technological requirements for collecting, preserving and producing health information as part of an electronic discovery process.
  • Thumbnail Image
    Item
    X-Composer: Enabling Cross-Environments In-SituWorkflows between HPC and Cloud
    (ACM, 2021-07) Li, Feng; Wang, Dali; Yan, Feng; Song, Fengguang; Computer Information and Graphics Technology, School of Engineering and Technology
    As large-scale scientific simulations and big data analyses become more popular, it is increasingly more expensive to store huge amounts of raw simulation results to perform post-analysis. To minimize the expensive data I/O, "in-situ" analysis is a promising approach, where data analysis applications analyze the simulation generated data on the fly without storing it first. However, it is challenging to organize, transform, and transport data at scales between two semantically different ecosystems due to the distinct software and hardware difference. To tackle these challenges, we design and implement the X-Composer framework. X-Composer connects cross-ecosystem applications to form an "in-situ" scientific workflow, and provides a unified approach and recipe for supporting such hybrid in-situ workflows on distributed heterogeneous resources. X-Composer reorganizes simulation data as continuous data streams and feeds them seamlessly into the Cloud-based stream processing services to minimize I/O overheads. For evaluation, we use X-Composer to set up and execute a cross-ecosystem workflow, which consists of a parallel Computational Fluid Dynamics simulation running on HPC, and a distributed Dynamic Mode Decomposition analysis application running on Cloud. Our experimental results show that X-Composer can seamlessly couple HPC and Big Data jobs in their own native environments, achieve good scalability, and provide high-fidelity analytics for ongoing simulations in real-time.