scinet logo
The World's Fastest Network

SC 2004 Bandwidth Challenge Entries

SCinet Premier Sponsors:

A Framework for Cyberinfrastructure

Contact person:

Kovatch, Patricia, SDSC

Author(s):

Phil Andrews, SDSC
Bryan Banister, SDSC
Puneet Chaudhary, IBM
Steve Cutchin, SDSC
Robert Harkness, SDSC
Roger Haskin, IBM
Christopher Jordan, SDSC
Patricia Kovatch, SDSC
Brad Kroeger, SDSC
Martin Margo, SDSC
Nathaniel Mendoza, SDSC
Richard Moore, SDSC
Mike Packard, SDSC
Jim Wyllie, IBM
Kenneth Yoshimoto, SDSC

Keywords:

Visualization • Computation • Novel approach • New technology • Real world application • Data intensive

Abstract:

To achieve the promise of grid computing, HPC applications need coordinated access to the set of resources that comprise cyberinfrastructure: superior compute platforms, concerted job reservations, high performance on-demand remote data access, visualization tools and access to archival storage. The TeraGrid cyberinfrastructure offers these distinctive resources to HPC applications. For example, a scientist can schedule a computation and visualization in automatic succession at different sites using the Grid Universal Remote metascheduler, without moving any files from one site to the other. A parallel file system that spans sites allows data to be shared without duplicating the the hardware and data at each site. New features to the General Parallel File System (GPFS) allow files to be shared without unifying each site's username space. The resulting data can then be stored in archival storage. Using tens of Terabytes of StorCloud SAN-attached storage on the showroom floor along with computation and visualization resources at various TeraGrid sites, a new computation and visualization will be displayed to attendees at the conference.

High Speed TeraByte Transfers for Physics

Contact person:

Bunn, Julian, Caltech

Author(s):

Julian James Bunn, Caltech
R. Les Cottrell, SLAC
Harvey Newman, Caltech
Phil Demar, FNAL
Sylvain Ravot, Caltech
Yang Xia, Caltech
Gary Buhrmaster, SLAC

Keywords:

New technology • Computation • Novel approach • High latency • Large number of sites • Real world application • Data intensive

Abstract:

The Caltech-FNAL-SLAC entry will demonstrate high speed transfers of physics data between host labs and collaborating institutes. Caltech and FNAL are major participants in the CMS collaboration at CERN's Large Hadron Collider (LHC). SLAC is the host accelerator site for the BaBar collaboration. We are using state of the art WAN infrastructure and Grid-based Web Services based on the LHC Tiered Architecture. Our demonstration will show a typical real-time event analysis application that requires the transfer of large physics datasets. For this we will use NLR 10GE waves, monitoring the WAN performance using the MonALISA agent-based system. The analysis software will use a suite of Grid-enabled Analysis tools developed at Caltech and Univ. of Florida. We intend to saturate three NLR 10GE waves: Sunnyvale to Pittsburgh, LA to Pittsburgh and Chicago to Pittsburgh. These links carry traffic between SLAC, Caltech and other partner Grid Service sites including UKlight, UERJ, FNAL, AARnet. More details may be found at: http://ultralight.caltech.edu/sc2004/

National Center for Data Mining SC04 Bandwidth Challenge

Contact person:

Connelly, Shirley Ann, UIC/NCDM

Author(s):

Robert Grossman, Univ. of Illinois at Chicago

Keywords:

Data intensive

Abstract:

We are developing an open source middleware called Open DMIX (Data Mining, data Integration and data eXploration) based upon high performance web services and new network transport protocols for exploring and analyzing remote and distributed data. Our SC04 Bandwidth Challenge Application will be focused on three related metrics: * Protocol advantages. We will show symmetric high speed data transfer in both * directions in and out of the booth. We will also visually show the * intra-fairness and TCP-friendliness properties of UDT. * End to end performance. We will show that we can scale bandwidth intensive * applications by moving data between memories and/or disks on both side to * achieve high end to end performance for data intensive applications. * Scaling web services for data analysis and data mining. Many distributed * data intensive applications will be built using web services. We will * demonstrate specialized web services designed for analyzing and mining * distributed data. For our Bandwidth Challenge Demonstration, we will transport multiple terabytes of astronomical data bewtween Chicago (USA), Amsterdam (Netherlands), Kingston (Canada), Geneva (Switzerland), Tokyo (Japan), and Pittsburg (US). We will also use high performance web services to perform distributed analysis of this data. Participants The following institutions and organizations will be participating: * University of Illinois at Chicago: Robert L. Grossman, Yunhong Gu, David * Hanley, Xinwei Hong, Michal Sabala. * Argonne National Laboratory, William * Allcock * John Hopkins University: Alex Szalay * Northwestern University: * Joe Mambretti * University of Amsterdam: Cees de Laat, Freek Dijkstra, Hans * Blom * SURFNet: Dennis Paus

The third generation Data Reservoir

Contact person:

Hiraki, Kei, University of Tokyo

Author(s):

Kei Hiraki, University of Tokyo
Mary Inaba, University of Tokyo
Makoto Nakamura, University of Tokyo
Junji Tamatsukuri, University of Tokyo
Ryutaro Kurusu, Fujitsu Computer Technologies
Masakazu Sakamoto, Fujitsu Computer Technologies
Yuki Furukawa, Fujitsu Computer Technologies
Yukichi Ikuta, Fujitsu Computer Technologies
Nao Aoshima, University of Tokyo

Keywords:

Small number of sites • New technology • High latency

Abstract:

Data Reservoir is a filesystem and operating system transparent data sharing system that share data in low level layer of storage using very high bandwidth high-latency Long Fat pipe Network. The objective of Data Reservoir is to construct a general, non-project-specific, file sharing facility to support data intensive scientific projects, spread in distant locations but connected each other by high-speed internet. The first generation Data Reservoir (http://www.sc-2002.org/paperpdfs/pap.pap327.pdf) showed the ability to continuous efficient use of available bandwidth of LFN using iSCSI protocol at BWC2002. The second generation Data Reservoir adopts transmission rate control in TCP and load balancing among parallel TCP streams to improve usable bandwidth at BWC2003. It showed importance of inter-layer coordination of parallel TCP for stable and efficient use of LFN (http://www.sc-conference.org/sc2004/schedule/pdfs/pap254.pdf). The third generation Data Reservoir extends inter-layer coordination for easy-to use, compact and efficient realization of data sharing system, and has following features: (1) Transparency to various filesystems, (2) Improvement on TCP optimization methods for LFN (Transfer Rate Controlled TCP, DECP, IPG tuning), (3) Use of newly developed 10Gbps NIC to achieve wire-speed data processing, and (4) Compact and inexpensive realization of 10Gbps disk to disk transfer facility. At BWC2004, we will have two experiments. In the first experiment, we focus on utilization of dual-ported intelligent NIC hardware to fine-grain control of parallel TCP on LFN. In the second experiment, we shows software optimization techniques with off the shelf 10GbE NIC. We will show the effectiveness of our TCP optimization methods in 10Gbps network connections.

Showing bandwidth of CJK(China,Japan,Korea)

Contact person:

OKAMURA, Koji, Kyushu University

Author(s):

Koji OKAMURA, Kyushu University
Hang Ryeol NA, KISDI

Keywords:

Visualization Real world application Small number of sites

Abstract:

We have established high bandwidth Internet among China, Japan Korea and are developing the advanced applications using this advanced Internet. In this showing, we will introduce our activities using this Internet among these 3 Asian countries using practice applications such as advanced multimedia streaming tools i.e. HD over MPEG2. And we will show our Internet's bandwidth using measurement tools and visualized measurement data. Finally we will try to get high band width among China, Japan and Korea. Getting more than 100Mbps even through 3 counties is surely interesting from many views, Internet operations, Traffic Engineering and Political issues. We will display the recent history of our Internet, how we get high bandwidth Internet among our countries, CJK.

Effective Rapid Remte File System for Supercomputer user who are not a network expert

Contact person:

FUJITA, Naoyuki, JAXA

Author(s):

Naoyuki FUJITA, JAXA
Hirofumi OHKAWA, JAXA

Keywords:

Real world application • Data intensive

Abstract:

When a High Performance Computing server becomes huge system, the system makes huge simulation data that is almost impossible to copy to remote computer over the Internet. Even if copy operation is possible, we have to manage multiple replicas to keep data coherency. One solution to solve this situation is "remote file system". Most famous remote file system should be Network File System. And many remote file system or remote storage system are researched and developed. These system works effectively under a certain circumstance, for example, trusted-network, low I/O throughput application, and so on. Our organization research field, which is Computational Fluid Dynamics, needs high speed and large I/O to do a numerical simulation, for example, we need 273 Mega bytes per second I/O speed and 30 Giga Bytes per file storage capacity. We already realized this requirement for local user on our HPC-system, but not yet realized for the remote user. And there are some other applications that need high speed, large I/O, and remote access file system. Space observation, disaster prevention simulation, and many real-time simulation applications are the example. In our Bandwidth Challenge on SC2004, we are going to benchmark a throughput of the "Shared Rapid File System on Ethernet" over the Pacific ocean network link. This file system's utmost speciality is that there is no userside TCP/IP-tuing. The file system already benchnmarked 56-90 Mega bits per second throughput on Fast-Ethernet and 608-616 Mega bit per second on Gigabit-Ethernet without TCP/IP tuning on Japanese domestic network.

Copyright © 1998-2004 SCinet. All rights reserved. This page last updated November 29, 2004.