NORDUnet is a regional R&E network that serves the five Nordic countries in Europe: Denmark, Finland, Iceland, Norway, and Sweden. Each country has a national R&E network (NREN), serving scientists, educators, students, and their institutions in the country. Since the 1980s, these five NRENs have worked together to connect their network infrastructures to each other, and to jointly connect to the rest of the world. The five Nordic NRENs formed NORDUnet as a company to provide regional and global connectivity. Working together as a region has allowed the Nordic countries to have first-class inter-connections, and to create global connectivity that each of the Nordic NRENs would not have been able to build alone.
In 2013, NORDUnet was faced with a serious challenge from the continued traffic growth, also on its resources across the North Atlantic Ocean. NORDUnet had several trans-Atlantic 10 Gbit/s links, and the cost of continuing to add more was prohibitive. We knew we had to move to 100 Gbit/s, but that brought three significant challenges:
1. We had to convince the market players to offer clear channel 100 Gbit/s trans-Atlantic links,
2. We had to persuade them to offer such links at a reasonable price, and
3. We would ideally need at least three 100 Gbit/s links for resilience.
These were quite steep hills to climb in 2013 as there was no market for transoceanic 100G yet, and still if we would succeed from a technical point of view, we were afraid that the cost would be prohibitive.
This was an excellent opportunity for joining forces, and trying to create a partnership. The Advanced North Atlantic (ANA) Collaboration was formed, besides NORDUnet consisting of Internet2 (USA), SURFnet (The Netherlands), CANARIE (Canada), ESnet (USA), and GÉANT (Europe). Together, these partners managed to convince subsea cable operators to offer 100 Gbit/s at a reasonable price, and to acquire 100 Gbit/s links on geographically diverse subsea cable systems. Right now we have three such links, and this system, that is in full production today, is called ANA-300G.
These three separate links across the North Atlantic Ocean had to be connected on both sides. The landing points had to be interconnected with terrestrial links to form a full system, so in case of failures there were easy routes to the other transoceanic links. And, of course, the system had to be connected to the collaborators’ networks on both sides of the North Atlantic, in a way that allowed all partners to connect and use the full system, and have the benefits of full resilience.
To meet these requirements, a number of existing and newly created Open Exchange Points (OXPs) were used. Today ANA-300G touches down at:
• MAN LAN in New York City, NY, USA
• WIX in Washington, DC, USA
• Montreal OXP in Montreal, Canada
• NetherLight in Amsterdam, The Netherlands
• GÉANT Open in London, UK
• GÉANT Open in Paris, France
OXPs are key components of the ANA-300G system. All of ANA-300G’s trans-Atlantic links terminate on OXPs, and all OXPs on a continent are interconnected with links provided by the partners, sometimes dedicated links, sometimes part of an existing link with enough headroom. All ANA collaborators use ANA-300G through one or more OXPs. OXPs are instrumental for the collaboration; the collaboration happens at the OXPs and this is where partners use the shared resource.
Further resilience is added to the system, through the mutual back-up agreement between ANA-300G and ESnet (who runs their own 340 Gbit/s worth of bandwidth across the North Atlantic Ocean). The mutual back-up agreement enables the collaborators to keep connectivity even when multiple cable systems would be down due to outages, e.g. fiber cuts. Technically speaking, also for this purpose, OXPs play a major role in this set-up.
At the basis of operations of ANA-300G, the OXPs again play an important role. As the transoceanic links are directly connected to the OXPs, avoiding adding expensive equipment such as routers in the path, OXP operators manage the links on behalf of the ANA Collaborators. The entire system is operated in a federated manner, with the operators of the OXPs collaborating as joint operations units for the system. In this way, OXPs have allowed the ANA-300G collaborators to build a cost-effective shared system for both trans-Atlantic capacity and continental network access, sharing network resources as well as operations. This was only possible by working together.
During the past 10 years, NORDUnet has focused its network strategy on Open Exchange Points. All connectivity to regional networks, in Europe or globally, is created through the major OXPs. OXPs are used as hubs in the network, and OXPs are the places where we expect to connect with other R&E networks, with the networks of a region, and increasingly also with industrial partners and cloud providers. NORDUnet also run an OXP in Helsinki, Finland called NOX-HEL, that functions as a gateway to the East.
Many of the OXPs that we connect to today are starting to play a role in the works on the Global Network Architecture (GNA). For more information on GNA, please visit: https://gna-re.net/