Subject: I-D Action: draft-xiang-alto-exascale-network-optimization-02.txt

From: [email-masked]
Date: Tue, 27 Jun 2017 07:55:26 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.

        Title           : Resource Orchestration for Multi-Domain Data Analytics
        Authors         : Qiao Xiang
                          Harvey Newman
                          Greg Bernstein
                          Azher Mughal
                          Justas Balcas
                          Jingxuan Jensen Zhang
                          Haizhou Du
                          Y. Richard Yang
	Filename        : draft-xiang-alto-exascale-network-optimization-02.txt
	Pages           : 22
	Date            : 2017-06-27

Abstract:
   Data-intensive analytics is entering the era of multi-domain,
   geographically-distributed, collaborative computing, where different
   organizations contribute various resources to collaboratively
   collect, share and analyze extremely large amounts of data.  Examples
   of this paradigm include the Compact Muon Solenoid (CMS) and A
   Toroidal LHC ApparatuS (ATLAS) experiments of the Large Hadron
   Collider (LHC) program.  Massive datasets continue to be acquired,
   simulated, processed and analyzed by globally distributed science
   networks in these collaborations, and the volume of this data is
   growing exponentially.  Applications that manage and analyze such
   massive data volumes can benefit substantially from the information
   about networking, computing and storage resources from each member
   sites, and more directly from network-resident services that optimize
   and load balance resource usage among multiple data transfer and
   analytic requests, and achieve a better utilization of multi-
   resources in clusters.  Hence this new paradigm calls for a framework
   to manage a large set of distributively owned heterogeneous
   resources, with the fundamental objective of efficient resource
   utilization, following the autonomy and privacy of resource owners.

   The Application-Layer Traffic Optimization (ALTO) protocol can
   provide via extensions the network information about different
   clusters/sites, to both users and proactive network management
   services where applicable, with the goal of improving both
   application performance and network resource utilization.  However,
   it has been verified in both science networks and commercial data
   center networks that network resource in many cases is not the
   bottleneck preventing the efficiency of large dataset transfer and
   data-intensive analytics.  To achieve a greater overall efficiency of
   the science programs' workflows information about different
   resources, such as computing, storage and networking, should be
   provided to data intensive applications simultaneously.

   In this document, we propose that it is feasible to use existing ALTO
   services to provides not only network information, but also
   information about computation and storage resources in data analytics
   networks.  We introduce a uniform resource orchestration framework
   (Unicorn), which achieves an efficient multi-resource allocation to
   support low-latency dataset transfer and data intensive analytics for
   collaborative computing.  Unicorn provides simple APIs for users to
   submit and manage dataset transfer and analytic requests and to
   monitor the status of each request, along with fine-grained local and
   global network and site state information in real-time.  It collects
   cluster information from multiple ALTO services utilizing topology
   extensions and leverages emerging SDN control capabilities to
   orchestrate the resource allocation for dataset transfers and
   analytic tasks, leading to improved transfer and analytic latency as
   well as more efficient utilization of multi-resources in clusters/
   sites.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-xiang-alto-exascale-network-optimization/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-xiang-alto-exascale-network-optimization-02
https://datatracker.ietf.org/doc/html/draft-xiang-alto-exascale-network-optimization-02

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-xiang-alto-exascale-network-optimization-02

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/