ESnet

ESnet

Utilization of Transatlantic Circuits by European LHC Tier 2 Accesses of US LHC Tier 1 Centers February, 2010 Supporting Advanced Scientific Computing Research Basic Energy Sciences Biological and Environmental Research Fusion Energy Sciences High Energy Physics Nuclear Physics William E Johnston Senior Scientist, Energy Sciences Network Lawrence Berkeley National Lab [email protected], www.es.net Managing Transatlantic Capacity Used by non-OPN LHC The Tier 2 accesses across the Atlantic (in both directions) utilize a mix of planned and ad hoc capacity as opposed to the carefully managed capacity of the OPN GANT provides transatlantic capacity for general IP traffic, as does NSF / IRNC (?), and several European countries ESnet partnered with USHLCnet some time ago to provide some capacity specifically to support European Tier 2 accesses of US Tier 1 centers What ESnet is seeing is LHC traffic coming from Europe on almost every available path, including once or twice over the commercial peerings Managing Transatlantic Capacity Used by non-OPN LHC An analysis of the traffic from ESnets point of view seems essential in order to understand - whether the current planned capacity for this traffic is adequate and - how it will need to evolve The current exercise (deliberately a bottom up view) is to - determine the traffic patterns, - see if they are optimal, and - establish a baseline for capacity prediction when the LHC starts producing physics data Most of the European Tier 2 to US Tier 1 traffic is currently Layer 3 traffic, so focus on that - What data is available to determine traffic patterns and how can it be used? What is presented here is definitely a work in progress

Layer 3 Paths Related to LHC traffic to ESnet LHC OPN To CERN USLHC Ch ica go Bo s Starlight USLHC MAN LAN Clev. FNAL Aof Phil C NY n to n A UK 5G BNL DE Wash. DC FNAL MAX

NL FR xG CH AT ORNL Nashville Atlanta Notes: All paths are 10Gb/s unless noted The two ESnet transatlantic paths are carried on virtual circuits provided by USLHCnet to Amsterdam where oGANT carries the 3G circuit to Vienna where it advertises ESnet routes oUSLHCNet carries the 2G circuit to CERN where CERN advertises ESnet routes internally ESnet IP core ESnet Science Data Network core (N X 10G) ESnet SDN core, NLR links (backup paths) Other international LHC OPN (USLHCNet) Metro ring link International GANT Logical Connectivity for ESnet, GANT, and USLHCnet ESnet GANT ESnet IP ESnet GANT 32 AofA, New York MAN LAN Amsterdam London

ESnet GANT Fermilab 5G Paris xG ESnet IP ESnet CERN Washington MAX CERN (internal) ESnet GANT Vienna ESnet GANT Frankfurt What are Top Level Traffic Patterns? ESnet GANT ESnet IP ESnet GANT 32 AofA, New York MAN LAN London ESnet GANT

Fermilab Who comes to me? ESnet IP Washington MAX 5G Paris xG ESnet CERN CERN (internal) Amsterdam How do I get to Fermilab? ESnet GANT Vienna ESnet GANT Frankfurt Who Comes to Fermilab? FNAL as source, top 75 flows, past month Base Date = 2009-12-28 160 120 Mb/s, average 80 40 0 sites per Ian Bird

Kharkov, Ukraine Nordunet ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 97 2 1 2 8 9 6 78 6 7 6 6 S (3 68 0 8 0 1 6 1 1 1 5 9 (8 8 5 59 3 5 6 1 9 2 7 5 4 7 8 9 3 6 1 1 4 8 77 6 (4 6 5 9 0 7 2 3 5 0 1 5 05 (2 6 93 0 4 6 2 2 9 6 9 0 7 2 5 1 9 2 5 0 5 2 6 4 3 6 97 0 7 8 4 9 3 8 7 5 68 3 6 47 2 0 0 5 0 1 6 7 4 2 1 7 (8 1 1 0 6 (3 8 (4 3 9 1 6 (2 7 2 21 4 (7 (7 (7 ET ( IS ( AY IP ( (6 (2 S ( -AS H ( (6 (6 (1 3 ( (3 2 (2 (2 5 ERS (39 (5 (2 J (2 ELL (1 (1 6 (3 5 (2 (1 (1 2 (1 0 (2 (2 (1 1 (2 9 (2 (2 (36 (2 (8 (2 3 S ( EN (6 IUC -AS (1 ET t (3 AP BIL T L-AS AN DIR TEW FN- sian NET ET-A ON ITC NET ORE RG N2 P HT ITEP A TG GF J HU TISK ET- RN CCN AS AS T-AS cao SU AS -SU T-AS N PI NET SU Q1 TER -AS AP ET-A NCR EDU U NL ede DN Ne B R M EE S J E A D us EL N ET W C -C BU I IG M U D N O R IC- P T- NE da -M U- EP NE OW P N EP O-M -G NA IER S- -N U -A ER N R -G R B ON NC S ERD VIS AM CR HO R N EP C V K I SI un P KN IH ER BR ID /H D I O RE ON L -A GS ET ND U AL U A H U IT I N O S F R R I R I N L C B A A A N

H R R H A T M S G P T S X K N R P D Y M N N O JI YA F CS V UL ER UC DES FUM LHC Known -O V I European T2 UN FNAL as source, top 75 flows, past month Base Date = 2010-02-28 160 120 Mb/s, average 80 40 0 Kharkov, Ukraine

sites Ioannina, Greece ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 1 2 37 8 6 7 3 2 3 5 6 4 1 9 6 9 2 21 7 6 6 6 7 7 1 1 8 9 8 0 0 3 1 7 5 3 8 3 7 5 1 4 (88 5 9 S (3 0 7 6 0 3 1 2 5 5 4 4 7 5 6 59 3 0 5 2 (7 3 2 1 8 1 9 6 9 7 0 1 0 1 8 2 (4 3 2 0 1 6 3 8 1 7 05 2 1 7 2 R (1 T (7 4 57 (5 7 (3 3 (1 7 (7 8 (6 1 (3 2 2 5 7 IS (7 y (7 (2 6 3 (7 P (6 (1 1 S (2 (1 8 (2 6 (28 (3 3 -AS S (1 AY (2 9 S (1 S (1 S (2 (1 7 36 6 (6 3 H (5 (1 9 1 0 0 -AS 1 0 4 (8 5 3 52 (74 (2 5 (6 8 1 9 7 -AS 1 8 4 (1 9 (45 (8 5 (2 5 (32 ( LT R NE ( U L 3 AS AS RE et ( IR la ET P -I L -A ET U ET AS N -A W AS -A -A -A G ( ET C N S ( N T ( OI S ( P SK an ( L ( de B ET -J et BI SGA J A S-P K J H VE ET NL- CO EEN MA ED -Sac L N IN2 DFN ET-N NET ON -MS P N K I- ETO NET ATE ET- GO NET NIA BUR GQ1 CN W IT RCC -A TO NE U T-A AS-A TI ssi P OP BN -TW Re T-BK K N NET EEN R U G U O R A BE A P E A U P A L E UN U IS N S- C IO H E KF NC ON T-G SIN ICA SB- GI M O- RD S Ru I G A NE UL HE F KN HIN TT K I NET V AH CE NI S I RF -AG A AD R/ NC ND H C VI R -H A HO NE C L A R R G ER A U C S M S ST K N U E

R D P A S I C V Y A N A N J U LHC S Y N -O C X CP W UC IA UM DE OF ER HE Unknown VND I I UN European How do they get to Fermilab on the US side? JANET (786) (UK) aofa-sdn1 JINR/HEPNET (2875) star-cr1 REDIRIS (766)

(Spain) aofa-sdn1 RADIO-MSU (2683) star-cr1 DFN (680) (Germany) aofa-sdn1 FR-RENATER (2200) aofa-sdn1 Russian (6801) star-cr1 PIONIER-AS (8501) aofa-sdn1 BELNET (2611) (Belgian) aofa-sdn1 EENet (3221) aofa-sdn1 SWITCH (559) (Switzerland) aofa-sdn1 ASGARR (137) aofa-sdn1 FUNETAS (1741) aofa-sdn1 DESY-HAMBURG (1754) aofa-sdn1

(Germany) CEA-Saclay (777) aofa-sdn1 IN2P3 (789) (France) aofa-sdn1 SURFNET-NL (1103) aofa-sdn1 ITEP (2148) (Russia) star-cr1 ACONET (1853) aofa-sdn1 NDGF (Nordunet) 39590 (Nordic) aofa-sdn1 KFKI-AS (3314) aofa-sdn1 RCCN (1930) (Portugal) aofa-sdn1 NERDCNET (6356) star-cr1 NSCKIPT-AS (35296) (Ukraine) eqx-chi-rt1 (ouch)

UOI (8581) aofa-sdn1 IHEP-SU (2643) star-cr1 Rede (1916) aofa-sdn1 GRIDPNPI (29493) (Russia) aofa-sdn1 ULAKNET (8517) aofa-sdn1 How do they get to Fermilab on the US side? The traffic through star-cr1 is on an IRNC (?) or country provided path or GLORIAD path Not all of FNALs traffic is LHC, though most of it is (e.g. FNAL is also the data center for the Sloan Digital Sky Survey) Top Level Traffic Patterns Looking Glass to traceroute from both ends ESnet GANT UK ESnet IP Belgian 32 AofA, New York MAN LAN Amsterdam ESnet GANT How do I

get to Fermilab? London ESnet GANT Fermilab Who comes to me? 5G Paris xG Vienna Austria ESnet ESnet IP Washington MAX ESnet GANT Switzerland CERN CERN (internal) Estonia Spain Germany ESnet GANT Frankfurt ESnet GEANT peering in Amsterdam 4.0G

Out In ESnet GEANT peering in Vienna 1.4G ESnet GEANT peering in Frankfurt 1.0G By-Site Traffic: Europe -> FNAL (Mb/s) By-Site Traffic: FNAL-> Europe The pattern of substantially more Tier 1 (FNAL) to Tier 2 (Europe) traffic compared to T2 to T1 is at least approximately consistent with the European Tier 2s getting data from the US Tier 1s Next Steps Sort by-site traffic by path Disambiguate LHC site traffic in European ASNs Decide what is useful to publish monthly - Engage Internet2 to publish similar data for their network Would expect to see the complimentary pattern US Tier 2s pulling data from European Tier 1s - Probably need to also involve NLR and the new IRNC links as some of the US Tier2 connect directly to the New York exchange point (MAN LAN) and if GEANT peers with them, Internet2 would not see the traffic END of Slides ESnet KAREN/REANNZ ODN Japan Telecom America NLR-Packetnet Internet2 Korea (Kreonet2) SEA T SDSC UCSD Physics BE

CH TE GA LASV LNV L LAN NSTEC SNLA M UN Office Of Science Sponsored (22) NNSA Sponsored (13+) Joint Sponsored (4) Other Sponsored (NSF LIGO, NOAA) Laboratory Sponsored (6) ESnet core hubs R&E networks ORAU OSTI PA NT EX ELPA ~45 end user sites commercial peering points Allied Signal DOE GTN NNSA SH

NA ARM NOAA S US HO An accurate map is located at http://www.es.net/pub/maps/current.pdf Specific R&E network peers Other R&E peering points Geography is only representational OX PPPL GFDL PU Physics Equinix MA XG Int NLR PoP er ne t2 SRS ATLA ALBU CUDI (S. America) NS KA KCP NETL WA

SH YUCCA MT D EN V oP GP PAIX-PA Equinix, etc. DOE x ni ui Eq NASA Ames IAR C NREL ES AM CH IC CL EV SUNN IU oP GP R F Lab DC

Offices OR NL NL LL LL SN LOSA SUNN FN AL ANL ICCN Wav e Pac Internet2 Salt Lake LVK ST BO MIT/ PSFC BNL AOF A NEW Y Inte rne t2 L IN BOIS

NSF/IRNC funded CA*net4 SL CHI- ht t lig e ar CN St LH R L US N O LIG AU (USLHCnet: DOE+CERN funded) KAREN / REANNZ Transpac2 Internet2 Korea (kreonet2) SINGAREN Japan (SINet) ODN Japan Telecom America PNNL GANT - France, Germany, Italy, UK, etc CERN/LHCOPN PacWave AU NERSC JGI LBNL SLAC SNV1 ESnet Vienna peering with

GANT (via USLHCNet circuit) SINet (Japan) Russia (BINP) CA*net4 France MREN GLORIAD StarTap (Russia, China) Taiwan (TANet2, Korea (Kreonet2 ASCGNet) USHLCNet to GANT Japan (SINet) Australia (AARNet) Canada (CA*net4 Taiwan (TANet2) Singaren Transpac2 CUDI JLAB AMPATH CLARA (S. America) International peers (10 Gb/s) 10-20-30 Gb/s SDN core (I2, NLR) 10Gb/s IP core MAN rings (Nx10 Gb/s) Lab supplied links OC12 / GigEthernet OC3 (155 Mb/s) 45 Mb/s and less USLHCNet 1/2010 GEANT International

Recently Viewed Presentations

  • Polityka Open Access w Europie i w Stanach

    Polityka Open Access w Europie i w Stanach

    As Peter Suber wrote in the SPARC Open Access Newsletter, issue #141, January 2, 2010: "The European Science Foundation (ESF) and the EuroHORCs plan to issue a joint OA mandate. The ESF represents 80 member organizations in 30 European countries.
  • Soil - Effingham County Schools / Overview

    Soil - Effingham County Schools / Overview

    Soil and Glass Analysis The Becke Line The Becke line is a "halo" that can be seen on the inside of the glass on the left, indicating that the glass has a higher refractive index than the liquid medium.
  • Powerpoint Jeopardy

    Powerpoint Jeopardy

    POWERPOINT JEOPARDY Subject: Jeopardy Template Description: www.edtechnetwork.com Keywords: Jeopardy Powerpoint Template Educational Technology Category: Jeopardy Template Last modified by: maneiros Company: Educational Technology Network
  • Choosing Wisely Task ForceHypoglycemia Safety Initiative (HSI ...

    Choosing Wisely Task ForceHypoglycemia Safety Initiative (HSI ...

    Choosing Wisely:VA Hypoglycemia Safety Initiative (HSI) Ultimate Goal: To foster shared decision making between clinicians and Veterans that is informed by the best available evidence and reduce unnecessary care.
  • Part IV. Computational Interlude AstroBEAR

    Part IV. Computational Interlude AstroBEAR

    ; HPC = Multi-D, multi-physics, full non-linearity. Resolution in space (and time) still an issue **Laboratory Studies: macro-scopic volumes of plasma brought to astrophysical condition. Similarity allows relevance across scales
  • PMG Slides - Harvard University

    PMG Slides - Harvard University

    Eric Prebys, Program Director US LHC Accelerator Research Program (LARP) Ultimate tribute Sept 10, 2008 10/24/2008
  • Children Learn Through Play

    Children Learn Through Play

    Children Learn Through Play Circle Time How to organize their thoughts as they talk about their experiences - they learn how to tell a story with a beginning, middle, and end. Creative Appreciation Music Appreciation/Creative Movement The children learn the...
  • Core Competency 5: Subpopulations of HIV/HCV Co-infected Persons

    Core Competency 5: Subpopulations of HIV/HCV Co-infected Persons

    Kaplan-Meier curve showing survival by age, stratified by human immunodeficiency virus and smoking status for all study subjects. People who smoke are represented by the lowest line on the graph and have a dramatically reduced life span when compared to...