Multi-Site Layer 2 Cloud Networks with Local-Site Internet Breakout

Multi-Site Layer 2 Cloud Networks with Local-Site Internet Breakout

Introduction

Big Network provides Software and Hardware that makes it fast and simple to deploy distributed multi-site Layer 2 networks, along with Local Internet Breakout functionality. By default, Local Internet Breakout relies upon two key functions: Network Address Translation (NAT) and Dynamic Host Configuration Protocol (DHCP) service. When combined with a Cloud Network for Layer 2 SD-WAN, Local Internet Breakout can be configured in few different forms:
  1. Hub and Spoke: Internet Access originates at the Hub, all NAT and DHCP functions happen at the Hub, and the Cloud Network provides a logical Layer 2 extension to the Spokes of the Network.
  2. Distributed Functions: Internet Access at each site is locally provided; however, the Layer 2 domain spans all sites.
This article will focus on the later use case.

Design Diagram / Application Concept

Multi-Site Layer 2 Cloud Network

The Layer 2 domain will use 172.16.0.0/20 as its addressing range.
Assuming a /24 (256 hosts per location), this design allows 14 locations to be connected, with two /24s reserved for other Cloud Network functions. Subnetting looks like the following:
  1. 172.16.0.0/24: Site #1
  2. 172.16.1.0/24: Site #2
  3. 172.16.2.0/24: Site #3
  4. 172.16.3.0/24: Site #4
  5. 172.16.4.0/24: Site #5
  6. 172.16.5.0/24: Site #6
  7. 172.16.6.0/24: Site #7
  8. 172.16.7.0/24: Site #8
  9. 172.16.8.0/24: Site #9
  10. 172.16.9.0/24: Site #10
  11. 172.16.10.0/24: Site #11
  12. 172.16.11.0/24: Site #12
  13. 172.16.12.0/24: Site #13
  14. 172.16.13.0/24: Site #14
  15. 172.16.14.0/24: RESERVED
  16. 172.16.15.0/24: RESERVED

Design Implementation and Configuration

All hosts attached to the Cloud Network Layer 2 domain will receive IP addresses from 172.16.0.0/20, however, DHCP servers / pools, and Internet Breakout NAT functions will occur at the site level.

Step #1: Cloud Network Configuration

Create a “Cloud Network”. A Cloud Network is a collection of devices that share a common Layer 2 network. Just think of it as your virtual LAN in the cloud.  Devices joined to a Cloud Network have complete access to each other.

Navigate to Cloud Networks

  • Click on “Create Network”; a wizard will appear to collect details about your Cloud Network.

  • Provide a Name and Description for your Cloud Network

  • Click “Next”

  • Define an IP Address Pool - Since this is an L2 network there is no need to define the IP pool just select “Do not assign address pool”.

  • Click “Next”

  • Define access to the Cloud Network - You may leave this blank since we only want Edge devices to join the network.

  • Click “Next”.

  • Now finalize

    • To proceed, click “Create”

    • To proceed, click “Network”

Step #2: Advanced Cloud Network Configuration

By default, Cloud Networks are L2 bridges / domains. By default, Edge Pro and Edge Lite DHCP servers are bridged into attached Cloud Network L2 domains, so in this use case, we must prevent DHCP from "leaking" between our sites. To do this, we use a Cloud Network Flow Rule:
  1. Navigate to your Cloud Network
  2. Go to "Advanced Settings"
  3. Scroll down to Flow Rules:
  4. Install the following flow rule:
drop sport 67;
drop dport 67;
drop sport 68;
drop dport 68;
  1. Click "Save"
  2. Click the “Pending Changes” gear to review configuration changes.
  3. Click “Apply” to Apply the configuration changes.
  4. Click “Back”.

Step #3: Site #1 Edge Configuration

Connecting physical ports on your Edge Device to Cloud Networks creates the final “gateway” connection needed to extend your physical network to a Cloud Network. 

  • Navigate to “Edge Devices”

  • Select “Configure” next to the Edge Pro / Lite you want to use.

  • Select “Networks” to see configured Networks on the device.

  • Select “Edit Config”

  • Select “Create Local Network”

    • Provide the Network a Name.

    • Provide the IP range for the "entire" L2 domain - in this example - 172.16.0.0/20

    • Click “Confirm”

  • Expand the newly created network; Select Local Services:

    • Add an Internet Breakout Service - The Service IP will be 172.16.0.1

    • Add a DHCP Service:

      • Choose a pool suitable for this site - i.e. the "subnet" is 172.16.0.0/24, so 172.16.0.20-250 is reasonable. Adjust as you like.

      • Use Elipsis for More Settings

        • The Service IP will be 172.16.0.2

        • The Gateway IP will be 172.16.0.1

      • Click Apply

  • Select “Connection”

  • In “Connected Cloud Network” select the Cloud Network you defined in Step #1.

  • In “Connected LAN Interfaces” select the physical ports you wish to use with your existing network.

  • Click “Validate” to validate the configuration.

  • Click the “Pending Changes” gear to review configuration changes.

  • Click “Apply” to Apply the configuration changes.

  • Click “Back”.

  • Click “Config History” to observe the configuration being applied to Edge Devices. When the new configuration’s checkbox turns from Blue to Green, you know the new configuration is applied.

Step #4: Site #2 Edge Configuration

Connecting physical ports on your Edge Device to Cloud Networks creates the final “gateway” connection needed to extend your physical network to a Cloud Network. 

  • Navigate to “Edge Devices”

  • Select “Configure” next to the Edge Pro / Lite you want to use.

  • Select “Networks” to see configured Networks on the device.

  • Select “Edit Config”

  • Select “Create Local Network”

    • Provide the Network a Name.

    • Provide the IP range for the "entire" L2 domain - in this example - 172.16.0.0/20

    • Click “Confirm”

  • Expand the newly created network; Select Local Services:

    • Add an Internet Breakout Service - The Service IP will be 172.16.1.1 <-- IMPORTANT: Note the change to the 3rd octet for Site #2

    • Add a DHCP Service:

    • Choose a pool suitable for this site - i.e. the "subnet" is 172.16.1.0/24, so 172.16.1.20-250 is reasonable. Adjust as you like. <-- IMPORTANT: Note the change to the 3rd octet for Site #2

    • Use Elipsis for More Settings:

      • The Service IP will be 172.16.1.2 <-- IMPORTANT: Note the change to the 3rd octet for Site #2

      • The Gateway IP will be 172.16.1.1 <-- IMPORTANT: Note the change to the 3rd octet for Site #2

      • Click Apply

  • Select “Connection”

  • In “Connected Cloud Network” select the Cloud Network you defined in Step #1.

  • In “Connected LAN Interfaces” select the physical ports you wish to use with your existing network.

  • Click “Validate” to validate the configuration.

  • Click the “Pending Changes” gear to review configuration changes.

  • Click “Apply” to Apply the configuration changes.

  • Click “Back”.

  • Click “Config History” to observe the configuration being applied to Edge Devices. When the new configuration’s checkbox turns from Blue to Green, you know the new configuration is applied.


Step #5: Other Site Configurations

  1. Repeat Step #4 for each site, using the subnetting plan listed above. Adjusting the IP prefixes / pools accordingly.

Conclusion and Result

As a result of this implementation:
  1. DHCP Service from one Edge Device to others, has been blocked from traversing the Cloud Network using Flow Rules.
  2. Devices at Site #1 will have access to all other devices number in 172.16.0.0/20 via the Cloud Network, at any location. Devices at Site #1, trying to access resources OUTSIDE of 172.16.0.0/20 will follow their default route to the Edge Device, running a NAT function, at 172.16.0.1, to their local WAN connections, using the specified WAN Priority for that Edge Device.
  3. Devices at Site #2 will have access to all other devices number in 172.16.0.0/20 via the Cloud Network, at any location. Devices at Site #2, trying to access resources OUTSIDE of 172.16.0.0/20 will follow their default route to the Edge Device, running a NAT function, at 172.16.1.1, to their local WAN connections, using the specified WAN Priority for that Edge Device.
  4. Devices at Site #N will have access to all other devices number in 172.16.0.0/20 via the Cloud Network, at any location. Devices at Site #N, trying to access resources OUTSIDE of 172.16.0.0/20 will follow their default route to the Edge Device, use the corresponding NAT function to their local WAN connections, using the specified WAN Priority for that Edge Device.