Objective

Configure 2 WAN Edge routers (IOS-XE) with multiple interfaces (WAN + OSPF) using a single playbook, leveraging:

• Grouped hosts in inventory

• Per-host variables stored in host_vars

• Ansible loop to iterate through interfaces

Topology

Project Layout

ansible/

├── cisco-config.yml # Main playbook

├── inventory/

│ └── hosts # Inventory file listing the IOS-XE devices

├── host_vars/

│ ├── 10.1.1.181.yml # Host-specific variables for router 10.1.1.181

│ └── 10.1.1.184.yml # Host-specific variables for router 10.1.1.184

Inventory Structure

We maintain a structured inventory and variable layout:

# inventory/hosts
[ios-xe]
10.1.1.181
10.1.1.184

Per-host variables:

📁 host_vars/10.1.1.181.yml

interfaces:
  - { wan_interface: "GigabitEthernet1", wan_description: "WAN1", wan_ip: "192.168.1.1", wan_mask: "255.255.255.0" }
  - { wan_interface: "GigabitEthernet2", wan_description: "OSPF-INTERFACE", wan_ip: "192.168.20.1", wan_mask: "255.255.255.0" }

📁 host_vars/10.1.1.184.yml

interfaces:
  - { wan_interface: "GigabitEthernet1", wan_description: "WAN1", wan_ip: "192.168.2.1", wan_mask: "255.255.255.0" }
  - { wan_interface: "GigabitEthernet2", wan_description: "OSPF-INTERFACE", wan_ip: "192.168.20.2", wan_mask: "255.255.255.0" }

The Ansible Playbook

cisco-config.yml:

---
- name: Configure 2 WAN Edges connected and OSPF
  hosts: ios-xe
  gather_facts: no
  connection: network_cli

  tasks:
    - name: Interface Configurations
      cisco.ios.ios_config:
        lines:
          - description {{ item.wan_description }}
          - no shutdown
          - ip address {{ item.wan_ip }} {{ item.wan_mask }}
        parents: interface {{ item.wan_interface }}
      loop: "{{ interfaces }}"

Run the Playbook

#ansible-playbook -i inventory/hosts cisco-config.yml

Output Highlights

PLAY RECAP *********************************************************************

10.1.1.181 : ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

10.1.1.184 : ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Each router receives the interface configuration defined in its own host_vars YAML file.

Running configuration of 10.1.1.181

interface GigabitEthernet1
 description WAN1
 ip address 192.168.1.1 255.255.255.0
 negotiation auto
!
interface GigabitEthernet2
 description OSPF-INTERFACE
 ip address 192.168.20.1 255.255.255.0
 negotiation auto
!

Running configuration of 10.1.1.184

interface GigabitEthernet1
 description WAN1
 ip address 192.168.1.1 255.255.255.0
 negotiation auto
!
interface GigabitEthernet2
 description OSPF-INTERFACE
 ip address 192.168.20.1 255.255.255.0
 negotiation auto
!

This is achieved through a loop, ensuring scalability and flexibility when configuring multiple routers or interfaces.

Tips

• Use network_cli for Cisco IOS devices, and ensure SSH + privilege access is enabled.

• Store variables per device in host_vars/ for modularity.

• Validate results with show running-config or collect configs using ios_command.

Feel free to clone, adapt, and expand this example to suit your network automation workflows. Automation doesn't have to be complex — start small, and grow from there!

🖥️ Topology

[ R1 ]---[ R2 ]---[ R3 ]---[ R4 ]
   |                         |
   |-------------------------|

• R1: PCC (client), Source

• R4: Destination

• R2 & R3: Transit

• Loopbacks:

  • R1 = 1.1.1.1/32, SID 16001

  • R2 = 2.2.2.2/32, SID 16002

  • R3 = 3.3.3.3/32, SID 16003

  • R4 = 4.4.4.4/32, SID 16004

🔧 Common OSPF & SR Config (on all routers)

hostname R1
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
!
interface Gig0/0
 ip address 10.0.12.1 255.255.255.0
 mpls ip
 ip ospf 1 area 0
!
router ospf 1
 router-id 1.1.1.1
 segment-routing mpls
!
mpls traffic-eng
!
segment-routing
 mpls
  global-block 16000 23999
  connected-prefix-sid-map
   address 1.1.1.1/32 absolute-sid 16001

Change IPs and SIDs accordingly for R2–R4.

🧠 On PCE Node (e.g., R2)

pce
 segment-routing
  peer 1.1.1.1
   source-address 2.2.2.2
   lsr-id 2.2.2.2
   capability pcc
   capability pce
!
pcc-address-family ipv4
 stateful-client
!
pce-address-family ipv4
 source-address 2.2.2.2

🧭 On PCC Node (R1)

pce
 segment-routing
  pcc
   peer 2.2.2.2
    source-address 1.1.1.1

🚀 Tunnel (SR-TE Policy on R1 to R4)

interface Tunnel100
 ip unnumbered Loopback0
 tunnel mode mpls traffic-eng
 tunnel destination 4.4.4.4
 tunnel mpls traffic-eng path-option 10 dynamic
 tunnel mpls traffic-eng autoroute announce
 segment-routing mpls

📋 Verification

• show pce peer

• show pce lsp

• show segment-routing traffic-eng policy

• show mpls forwarding-table

More details

     [R1 - PCC]
        |
     (IGP + SR)
        |
     [R2 - PCE]
      /      \
     /        \
[R3]----(IGP)----[R4]

Roles:

• R1 (PCC): Requests a path from the PCE.

• R2 (PCE): Computes the best SR-TE path based on constraints (e.g., low latency, bandwidth).

• R3 & R4: Intermediate and destination routers.

Traffic Flow:

1. R1 uses IGP (e.g., OSPF or IS-IS) with Segment Routing enabled to advertise and receive labels (SIDs).

2. R1, as PCC, sends a path computation request (PCEP) to R2.

3. R2, as PCE, computes a TE path using available topology and policy constraints.

4. R1 receives the path and installs it as an SR-TE policy.

5. Traffic follows the SR policy instead of IGP’s default shortest path.

Benefits of SR-TE with PCE:

• Centralized path computation (PCE).

• Dynamic re-optimization when network state changes.

• SLA-aware routing (e.g., avoid congested links or meet latency targets).

• Supports automation via NETCONF/YANG or CLI.

The configuration to include bandwidth, affinity (link color), and explicit path constraints in Cisco IOS-XR.

🧱 Example Network Setup

R1 -- R2 -- R3 -- R4

R1 = PCC (headend)
R2 = PCE (central controller)
R4 = destination
Links: SR-MPLS enabled, OSPF/ISIS as IGP

1️⃣ Add Affinity (Link Color) to Interfaces (on R2, R3)

Suppose you want traffic to use only "green" links.

interface GigabitEthernet0/0/0/1
 description Link to R3 (green)
 segment-routing
  forwarding
  affinity 0x2

Here:

• affinity 0x2 marks the link as "green".

• Affinity is a 32-bit bitmask — you define meanings separately.

2️⃣ Define Affinity Mapping (on PCE or router where policy is configured)

segment-routing
 traffic-eng
  affinity-map green 0x2

3️⃣ Define SR-TE Policy with Constraints (on R1 — PCC)

segment-routing
 traffic-eng
  policy SRTE-R1-R4
   color 100
   end-point ipv4 10.0.0.4
   candidate-paths
    preference 100
     constraints
      affinity include green
      bandwidth 1000000  ! in Kbps (1 Gbps)
     explicit segment-list R1-R2-R3-R4

4️⃣ Define Explicit Segment List (Optional for hard path pinning)

segment-routing
 traffic-eng
  segment-list R1-R2-R3-R4
   index 10
    mpls label 16002  ! R2 SID
   index 20
    mpls label 16003  ! R3 SID
   index 30
    mpls label 16004  ! R4 SID

✅ What This Does:

• Bandwidth: Allocates bandwidth-aware paths via the PCE.

• Affinity: Ensures only “green” tagged links are used.

• Explicit path: You can pin the SR path manually using segment SIDs.

Here is the full configuration for a Cisco SR-TE setup using a PCE-PCC model across four routers (R1 to R4). R1 is the PCC, R2 is the PCE, and R3/R4 are transit/destination nodes.

• ```plaintext ! ! R1 Configuration (PCC) ! hostname R1 ! interface Loopback0 ip address 1.1.1.1 255.255.255.255 ! interface GigabitEthernet0/0 ip address 10.0.12.1 255.255.255.0 mpls ip ip ospf 1 area 0 ! interface GigabitEthernet0/1 ip address 10.0.14.1 255.255.255.0 mpls ip ip ospf 1 area 0 ! router ospf 1 router-id 1.1.1.1 network 10.0.12.0 0.0.0.255 area 0 network 10.0.14.0 0.0.0.255 area 0 network 1.1.1.1 0.0.0.0 area 0 segment-routing mpls ! mpls traffic-eng ! segment-routing mpls global-block 16000 23999 connected-prefix-sid-map address 1.1.1.1/32 absolute-sid 16001 ! pce segment-routing pcc peer 2.2.2.2 source-address 1.1.1.1 ! interface Tunnel100 ip unnumbered Loopback0 tunnel mode mpls traffic-eng tunnel destination 4.4.4.4 tunnel mpls traffic-eng path-option 10 dynamic tunnel mpls traffic-eng autoroute announce segment-routing mpls

! ! R2 Configuration (PCE) ! hostname R2 ! interface Loopback0 ip address 2.2.2.2 255.255.255.255 ! interface GigabitEthernet0/0 ip address 10.0.12.2 255.255.255.0 mpls ip ip ospf 1 area 0 ! interface GigabitEthernet0/1 ip address 10.0.23.1 255.255.255.0 mpls ip ip ospf 1 area 0 ! router ospf 1 router-id 2.2.2.2 network 10.0.12.0 0.0.0.255 area 0 network 10.0.23.0 0.0.0.255 area 0 network 2.2.2.2 0.0.0.0 area 0 segment-routing mpls ! mpls traffic-eng ! segment-routing mpls global-block 16000 23999 connected-prefix-sid-map address 2.2.2.2/32 absolute-sid 16002 ! pce segment-routing peer 1.1.1.1 source-address 2.2.2.2 lsr-id 2.2.2.2 capability pcc capability pce ! pcc-address-family ipv4 stateful-client ! pce-address-family ipv4 source-address 2.2.2.2

! ! R3 Configuration (Transit) ! hostname R3 ! interface Loopback0 ip address 3.3.3.3 255.255.255.255 ! interface GigabitEthernet0/0 ip address 10.0.23.2 255.255.255.0 mpls ip ip ospf 1 area 0 ! interface GigabitEthernet0/1 ip address 10.0.34.1 255.255.255.0 mpls ip ip ospf 1 area 0 ! router ospf 1 router-id 3.3.3.3 network 10.0.23.0 0.0.0.255 area 0 network 10.0.34.0 0.0.0.255 area 0 network 3.3.3.3 0.0.0.0 area 0 segment-routing mpls ! mpls traffic-eng ! segment-routing mpls global-block 16000 23999 connected-prefix-sid-map address 3.3.3.3/32 absolute-sid 16003

! ! R4 Configuration (Destination) ! hostname R4 ! interface Loopback0 ip address 4.4.4.4 255.255.255.255 ! interface GigabitEthernet0/0 ip address 10.0.34.2 255.255.255.0 mpls ip ip ospf 1 area 0 ! interface GigabitEthernet0/1 ip address 10.0.14.2 255.255.255.0 mpls ip ip ospf 1 area 0 ! router ospf 1 router-id 4.4.4.4 network 10.0.34.0 0.0.0.255 area 0 network 10.0.14.0 0.0.0.255 area 0 network 4.4.4.4 0.0.0.0 area 0 segment-routing mpls ! mpls traffic-eng ! segment-routing mpls global-block 16000 23999 connected-prefix-sid-map address 4.4.4.4/32 absolute-sid 16004 ```

🔁 IGP + Fast Reroute (FRR)

• IGP convergence usually takes tens to hundreds of milliseconds—even up to a few seconds depending on the topology and tuning.

• During this time, traffic may drop. FRR solves this by precomputing a backup path.

• In Cisco’s world, Loop-Free Alternates (LFA) or TI-LFA (Topology Independent LFA) are used.

• If the primary next-hop fails, traffic is immediately redirected to the backup (pre-installed in forwarding plane), before IGP converges.

• Once convergence finishes, the IGP installs the new best path as usual.

Key Point: FRR is a local repair mechanism to protect traffic during the IGP convergence window.

🚦 IGP + SR-TE (Segment Routing - Traffic Engineering)

• SR-TE allows explicit path steering based on segments (SID list) rather than relying only on the IGP’s shortest path.

• While IGP still computes SPT, SR-TE policies can override that with custom paths for SLA, bandwidth, or avoidance.

• Even after IGP convergence, the SR-TE policy takes precedence for flows that match its criteria.

• If a failure occurs on an SR path, FRR can be used within SR-MPLS to protect it during convergence (via TI-LFA).

Key Point: SR-TE allows more flexible post-convergence routing choices beyond IGP's best path.

While IGP still computes SPT, SR-TE policies can override that with custom paths

✅ IGP (e.g., OSPF/IS-IS):

• Shortest Path Tree (SPT): IGP calculates the shortest path based on metrics (like cost).

• Static and Dynamic: It works well, but has limited control over how traffic flows.

🚧 Limitation in IGP:

• Cannot control:

  • Latency

  • Bandwidth requirements

  • Path avoidance (e.g., avoid congested or untrusted nodes)

  • Traffic differentiation (e.g., voice vs. bulk data)

🧭 Segment Routing - Traffic Engineering (SR-TE):

SR-TE builds on top of IGP, but introduces explicit path control via Segment Identifiers (SIDs). You can program traffic to follow a specific route even if it’s not the shortest path.

🎯 Use Cases:

🛠️ How SR-TE Overrides IGP:

You define a policy with:

• Source and destination

• Constraints (e.g., "path must avoid R2" or "minimum available bandwidth: 100 Mbps")

• A Segment List (i.e., list of router/node/adjacency SIDs to reach destination)

Then:

• The router installs this policy into the RIB and FIB

• Traffic matching that policy is forwarded according to the SID list, not the IGP’s SPT

🔄 If IGP recalculates (e.g., topology changes), the SR-TE path remains unchanged unless it becomes invalid, at which point a new SR-TE path can be computed or failover triggered.

📦 Use Case: Low-Latency Path for Voice Traffic

🌐 Network Topology:

        [R1]
        /   \
     10ms   50ms
     /         \
   [R2]——30ms——[R3]——10ms——[R4]

• You need to send voice traffic from R1 to R4.

• Voice requires <40ms latency.

🧮 What IGP Would Do (e.g., OSPF):

• OSPF sees R1–R3–R4 (50ms + 10ms = 60ms) as the shortest path by cost/metric.

• So it installs that path in the RIB.

• Problem: That path exceeds your latency SLA.

🛠️ What SR-TE Allows:

• You define a policy:

  • Source: R1

  • Destination: R4

  • Constraint: "Only allow paths with total latency < 40ms"

• SR-TE evaluates paths:

  • R1–R2–R3–R4 → 10 + 30 + 10 = 50ms ❌

  • R1–R2–R4 (via a new link) → 10 + 15 = 25ms ✅

• If this 25ms path exists (maybe R2–R4 has a direct link), SR-TE installs a segment list:

    R1 → R2 (Node SID) → R4 (Node SID)
  

• Voice traffic follows this explicit path, even though it’s not the IGP's shortest.

🧠 Summary:

Sample configuration of Segment Routing with Traffic Engineering (SR-TE) using Cisco IOS-XR.

This example assumes:

• IS-IS is used as the IGP.

• Segment Routing is enabled.

• An SR-TE policy is created to prefer a custom path with specific SLAs (e.g., low latency).

🔧 Step 1: Enable Segment Routing

bashCopyEditrouter isis CORE
 address-family ipv4 unicast
  segment-routing mpls
 !

🗺️ Step 2: Assign Segment IDs to Loopbacks

bashCopyEditinterface Loopback0
 ipv4 address 1.1.1.1 255.255.255.255
!
router isis CORE
 net 49.0001.0000.0000.0001.00
 address-family ipv4 unicast
  mpls traffic-eng router-id Loopback0
  segment-routing mpls
   connected-prefix-sid-map
    address 1.1.1.1/32 sid 16001

Repeat for other routers, adjusting the IP and SID.

📡 Step 3: Enable MPLS Traffic Engineering

bashCopyEditmpls traffic-eng
!
interface GigabitEthernet0/0/0/0
 mpls traffic-eng tunnels

🚧 Step 4: Create SR-TE Policy

bashCopyEditsegment-routing
 traffic-eng
  policy VOICE-POLICY
   color 123
   end-point ipv4 4.4.4.4
   candidate-paths
    preference 100
     explicit segment-list VOICE-PATH
      name VOICE-PATH
       index 10
        segment 16002
       index 20
        segment 16004

🧭 Step 5: Forward Traffic Using the Policy

bashCopyEditrouter static
 address-family ipv4 unicast
  4.4.4.4/32  segment-routing policy VOICE-POLICY

🔍 Notes

• segment 16002 and 16004 are SIDs representing routers R2 and R4.

• This policy manually forces the traffic to take R1 → R2 → R4 instead of the IGP shortest path.

• If you use a PCE, you can also compute the path dynamically based on constraints like latency or bandwidth.

Cisco IOS XE example for configuring Segment Routing with SR-TE and PCE-based dynamic policies.

🧱 Prerequisites

• IOS XE with SR-MPLS and PCE support (e.g., on ASR 1000, Catalyst 8000).

• You have a functioning IGP (like OSPF or IS-IS).

• Devices support SR-PCE (Path Computation Element).

1️⃣ Enable Segment Routing and MPLS TE

bashCopyEditrouter ospf 1
 router-id 1.1.1.1
 segment-routing mpls
 mpls traffic-eng
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
 ip ospf 1 area 0
!
interface GigabitEthernet0/0/0
 ip ospf 1 area 0
 mpls ip

2️⃣ Enable SRGB and SID on Loopback

bashCopyEditsegment-routing
 mpls
  global-block 16000 23999
  connected-prefix-sid-map
   address 1.1.1.1/32 absolute-sid 16001

Repeat on each router with a unique loopback and SID.

3️⃣ Configure PCE (on PCE node)

bashCopyEditpce
 segment-routing
  peer 2.2.2.2
   source-address 1.1.1.1
   lsr-id 1.1.1.1
   capability pce
   capability pcc

4️⃣ Enable PCC (on client router)

bashCopyEditpce
 segment-routing
  pcc
   peer 1.1.1.1
    source-address 2.2.2.2

5️⃣ Configure SR-TE Policy with Dynamic Path (computed by PCE)

bashCopyEditinterface Tunnel10
 ip unnumbered Loopback0
 tunnel mode mpls traffic-eng
 tunnel destination 4.4.4.4
 tunnel mpls traffic-eng path-option 10 dynamic
 tunnel mpls traffic-eng autoroute announce
 segment-routing mpls

🔁 Summary

• PCE computes the best path dynamically (e.g., lowest latency).

• SR-TE tunnel forwards traffic based on this path.

• You don't need to define explicit segment lists—PCE handles it.

In today's cloud-driven world, ensuring high availability and optimal performance of applications is crucial. Microsoft Azure provides robust load balancing solutions that efficiently distribute incoming network traffic across multiple resources. Two primary services in this domain are Azure Load Balancer and Azure Application Gateway. While both aim to enhance application scalability and reliability, they operate at different layers of the OSI model and cater to distinct scenarios.

Azure Load Balancer: Layer 4 Traffic Distribution

Azure Load Balancer operates at Layer 4 (Transport Layer) of the OSI model, focusing on transport-level traffic distribution. It directs incoming network traffic based on source and destination IP addresses and ports without inspecting the packet content. This makes it suitable for high-performance, low-latency distribution of TCP and UDP traffic.

Key Features:

• High Throughput and Low Latency – Designed for high-bandwidth applications, ensuring minimal delay in traffic distribution.

• Health Probes – Continuously monitors the health of backend instances, rerouting traffic away from unhealthy nodes.

• Outbound Connectivity – Provides outbound connectivity for virtual machines without public IPs by translating their private IPs to public IPs.

• Flexible Deployment – Supports public load balancing (for external traffic) and internal load balancing (for internal Azure Virtual Network communication).

Use Case Example:

A company hosts a multiplayer gaming platform on Azure Virtual Machines. To handle thousands of concurrent connections, Azure Load Balancer distributes incoming UDP traffic across multiple game servers, optimizing resource utilization and ensuring a seamless gaming experience.

Azure Application Gateway: Layer 7 Traffic Management

Azure Application Gateway operates at Layer 7 (Application Layer) of the OSI model, enabling advanced traffic routing based on HTTP request attributes. Unlike Azure Load Balancer, which only manages transport-level traffic, Application Gateway can inspect packet content and make intelligent routing decisions.

Key Features:

• Web Application Firewall (WAF) – Provides protection against web vulnerabilities like SQL injection and cross-site scripting.

• SSL Termination (SSL Offloading) – Offloads SSL/TLS encryption and decryption from backend servers, enhancing performance.

• End-to-End SSL – Ensures complete encryption and decryption of traffic between users and backend servers.

• URL-Based Routing – Routes requests to different backend pools based on URL paths, supporting microservices architectures.

• Cookie-Based Session Affinity – Ensures subsequent requests from the same user session are routed to the same backend instance.

How Azure Application Gateway Works

Application Gateway sits between users and backend virtual machines, using Application Request Routing (ARR) to forward requests to appropriate services. It requires a private or public IP address to handle incoming HTTP/HTTPS traffic and direct it to configured endpoints.

Use Case Example:

A financial services company hosts its online portal on Azure. Using Application Gateway, the company routes requests to different microservices based on URL paths (e.g., /accounts for account management and /transactions for payments). Additionally, SSL termination at the gateway enhances security while reducing the load on backend servers.

Comparing Azure Load Balancer vs. Application Gateway

When to Choose Azure Load Balancer vs. Application Gateway

• Choose Azure Load Balancer if you need high-performance, transport-layer load balancing for TCP and UDP traffic without content-based routing.

• Choose Azure Application Gateway if you need advanced Layer 7 capabilities such as SSL offloading, URL-based routing, and security with WAF.

• Combine Both – For comprehensive load balancing, you can use Azure Load Balancer for non-HTTP traffic and Application Gateway for web traffic.

Conclusion

Understanding the distinctions between Azure Load Balancer and Azure Application Gateway is essential for designing resilient and efficient cloud applications. By aligning the choice of load balancing service with application requirements, organizations can enhance performance, ensure high availability, and improve security.

What is Azure Private Link?

Azure Private Link allows private access to Azure services, third-party SaaS applications, or custom-built solutions within an organization’s virtual network. This ensures that traffic remains on the Azure backbone, eliminating exposure to the public internet.

What are Azure Private Endpoints?

An Azure Private Endpoint is a network interface that connects privately to Azure services. It allows communication between resources within a Virtual Network (VNet) and Azure services over a private IP.

Key Benefits:

• Enhanced Security: Eliminates the need for public IPs and restricts access to specific VNets.

• Improved Performance: Traffic remains within Azure’s private network.

• Granular Access Control: Integrates with Azure role-based access control (RBAC).

Real-Life Use Cases of Azure Private Link

1. Secure Access to Azure SQL Database

• A company hosts its production database in Azure SQL, and developers need secure access from their virtual network.

• By setting up a Private Endpoint, traffic between the application and database is routed securely over the Azure backbone.

2. Private Access to Azure Storage (Blob & Files)

• Organizations storing sensitive data in Azure Storage can use Private Endpoints to ensure that only authorized VNets can access the storage account.

3. Multi-Tenant SaaS Application Hosting

• A software vendor offering a cloud-based HR solution can expose their service via Azure Private Link Service.

• Customers connect via Private Endpoints, ensuring their data never traverses the public internet.

4. Hybrid Cloud Connectivity (On-Prem to Azure)

• A company using ExpressRoute or VPN can leverage Private Endpoints to access Azure services securely from their on-premises network.

How to Create a Private Endpoint using Azure CLI

# Variables
RESOURCE_GROUP="MyResourceGroup"
VNET_NAME="MyVNet"
SUBNET_NAME="MySubnet"
PRIVATE_ENDPOINT_NAME="MyPrivateEndpoint"
STORAGE_ACCOUNT_NAME="mystorageaccount"

# Create Private Endpoint
az network private-endpoint create \
  --resource-group $RESOURCE_GROUP \
  --name $PRIVATE_ENDPOINT_NAME \
  --vnet-name $VNET_NAME \
  --subnet $SUBNET_NAME \
  --private-connection-resource-id "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.Storage/storageAccounts/$STORAGE_ACCOUNT_NAME" \
  --group-id "blob"

How to Create a Private Endpoint using PowerShell

# Variables
$resourceGroup = "MyResourceGroup"
$vnetName = "MyVNet"
$subnetName = "MySubnet"
$privateEndpointName = "MyPrivateEndpoint"
$storageAccountName = "mystorageaccount"

# Create Private Endpoint
New-AzPrivateEndpoint -ResourceGroupName $resourceGroup -Name $privateEndpointName \
    -Location "East US" -Subnet (Get-AzVirtualNetworkSubnetConfig -Name $subnetName -VirtualNetwork (Get-AzVirtualNetwork -ResourceGroupName $resourceGroup -Name $vnetName)) \
    -PrivateLinkServiceConnection (New-AzPrivateLinkServiceConnection -Name "MyPLSConnection" -PrivateLinkServiceId (Get-AzStorageAccount -ResourceGroupName $resourceGroup -Name $storageAccountName).Id -GroupId "blob")

Best Practices for Deploying Private Endpoints

• Use a dedicated subnet for private endpoints to avoid IP conflicts.

• Enable private DNS integration to ensure seamless name resolution.

• Block public network access to services after configuring Private Link.

• Plan subnet capacity: Ensure enough IP addresses are available for scaling.

Conclusion

Azure Private Link and Private Endpoints provide a robust solution for securing access to Azure services. By routing traffic over the Azure backbone, organizations can enhance security, performance, and compliance. Whether connecting to Azure SQL, Storage, or a custom SaaS application, Private Link ensures a seamless and private network experience.

What are your thoughts on using Azure Private Link in your cloud infrastructure? Let’s discuss in the comments!

When working with Azure networking, user-defined routes (UDRs) play a crucial role in controlling traffic flow within a virtual network. By default, Azure provides system routes that automatically handle routing between subnets, virtual networks, and the internet. However, there are scenarios where we need to override these system routes to meet specific business or security requirements. This is where UDRs come in.

Each subnet in Azure can have zero or one associated route table, and a route table can be linked to multiple subnets. The key takeaway is that a subnet cannot be associated with multiple route tables simultaneously.

Next Hop Types in Azure Route Tables

When defining routes, we specify a next hop type, which determines where the traffic will be forwarded. The available next hop types include:

• Virtual Appliance – Used when directing traffic to a third-party firewall or routing appliance running on a virtual machine.

• Virtual Network Gateway – Used to send traffic to an on-premises network via a VPN or ExpressRoute connection.

• Virtual Network – For communication within a virtual network, including peered networks.

• Internet – Sends traffic directly to the internet, useful for scenarios where security policies do not require forced tunneling.

• None – Prevents traffic from exiting the virtual network, enforcing strict network isolation.

Forced Tunneling

One of the key concepts for the Azure exam is forced tunneling. This is used when organizations require all internet-bound traffic to be routed through an on-premises firewall for security inspection. Forced tunneling is configured via UDRs to redirect internet traffic to a site-to-site VPN. Note that forced tunneling cannot be configured via the Azure portal and must be set up using Azure PowerShell. It also requires a route-based VPN gateway (policy-based gateways are not supported).

Step-by-Step: Configuring User-Defined Routes

To implement UDRs in Azure, follow these steps:

1. Create a Route Table

   • Navigate to Route Tables in the Azure portal.

   • Click Create, select the appropriate subscription and resource group.

   • Define a name and region (e.g., LL-Demo-RT-01).

   • Decide whether to propagate gateway routes from on-premises networks.

   • Click Review + Create and deploy the route table.

2. Add Routes to the Route Table

   • Open the created route table and go to Routes.

   • Click Add route, specify a name, destination prefix, and next hop type.

   • For example, to direct traffic to a virtual network gateway:

     • Destination: 10.0.2.0/23

     • Next hop type: Virtual network gateway

   • Click Add.

3. Associate the Route Table with a Subnet

   • Navigate to Subnets inside the route table.

   • Click Associate, select the target VNet and subnet.

   • Confirm the association.

4. Testing the Routing Configuration

   • Deploy two virtual machines in different subnets within the virtual network.

   • Use PowerShell to test connectivity:

       Test-NetConnection -ComputerName 10.0.4.4 -InformationLevel Detailed -DiagnoseRouting
     

   • If the test returns True, the route is working correctly!

Troubleshooting UDRs in Azure

Sometimes, UDR configurations may not work as expected. Here are some troubleshooting steps:

1. Check Effective Routes

• Navigate to the virtual machine’s network interface.

• Click on Effective routes to see all system and user-defined routes applied to the VM.

• Ensure the correct route is listed.

2. Use Azure Network Watcher

• Go to Azure Network Watcher and use Next Hop to verify routing behavior.

• Check whether traffic is correctly directed to the intended next hop.

3. Common Misconfigurations

• Subnet Not Associated: Ensure the route table is properly associated with the correct subnet.

• Overlapping Routes: If multiple routes apply, the most specific route takes precedence. Avoid conflicts.

• Incorrect Next Hop Type: Ensure you have selected the correct next hop based on your architecture.

• VPN Gateway Configuration: If forced tunneling is configured, verify the default site is set correctly.

Conclusion

User-defined routes (UDRs) are a powerful tool in Azure networking, allowing us to control traffic flow and implement security policies such as forced tunneling. Understanding how to design, implement, and troubleshoot UDRs is essential for both real-world deployments and Azure certifications.

Stay tuned for more updates on my Azure networking journey as I continue preparing for my AZ-700 certification!

🚀 Let’s connect! If you're also studying for Azure certifications or working on cloud networking, feel free to share your thoughts in the comments!

As a Network Engineering Team Leader, I’ve always focused on ensuring high availability, performance, and security for critical infrastructure. However, as networks grow in complexity, technical expertise alone isn’t enough. We need structured processes to manage incidents, changes, and service improvements effectively. That’s where ITIL 4 comes in.

At first, I thought ITIL was mainly for service desks and IT operations, but after implementing some of its practices, I realized how much it can improve network service management. In this post, I’ll share how ITIL 4 principles can enhance the efficiency and effectiveness of network operations.

Why ITIL 4 Matters for Network Engineers

Many network engineering teams operate in a reactive mode—fixing issues as they arise, handling changes without structured workflows, and struggling to align network priorities with business goals. ITIL 4 helps by providing a framework for managing IT services in a way that is structured, repeatable, and value-driven.

By applying ITIL 4 principles, network teams can:

• Improve incident resolution times by following standardized troubleshooting and escalation processes.

• Minimize network outages through structured change management.

• Analyze and eliminate recurring problems instead of applying temporary fixes.

• Ensure alignment with business goals by managing service expectations and tracking key metrics.

Key ITIL 4 Concepts for Network Engineering

1. Incident Management – Faster and More Efficient Troubleshooting

Scenario: Your team receives a high-priority alert about an MPLS circuit failure affecting a critical business application.

Without ITIL: Engineers troubleshoot on an ad-hoc basis, communication is inconsistent, and escalations are unclear, leading to delays in resolution.

With ITIL: A structured incident management process ensures that incidents are logged, prioritized, and escalated following a clear workflow. Engineers follow predefined steps to diagnose the issue, document their findings, and ensure timely resolution while minimizing service impact.

2. Change Enablement – Reducing Risks in Network Upgrades

Scenario: Your team plans to upgrade the SD-WAN firmware across multiple sites.

Without ITIL: Changes are made without proper testing, leading to unplanned outages and rollback issues.

With ITIL: Every network change follows a structured process:

• Change requests are documented, including risk assessment and rollback plans.

• Approval processes ensure stakeholders are informed before deployment.

• Post-implementation reviews help the team learn from each change.

This approach reduces unexpected failures and improves overall stability.

3. Problem Management – Addressing Root Causes Instead of Symptoms

Scenario: Your team repeatedly resolves latency issues in a particular data center, but the problem keeps coming back.

Without ITIL: Engineers fix each instance of the issue but never identify the root cause.

With ITIL: Problem Management helps analyze incident trends, perform root cause analysis, and implement permanent fixes. Instead of continuously addressing the same issue, the team identifies misconfigured QoS policies and applies a long-term solution.

4. Service Level Management – Aligning Network Performance with Business Needs

Scenario: The company expects 99.99% uptime for key applications, but the network team isn’t tracking SLA compliance.

With ITIL: Service Level Agreements (SLAs) are clearly defined, and the team tracks:

• MTTR (Mean Time to Repair)

• Incident resolution times

• Network performance metrics

This ensures that the network team’s work directly supports business objectives and customer expectations.

5. Continual Improvement – Enhancing Network Operations Over Time

ITIL 4 emphasizes ongoing improvements. Network teams should:

• Review past incidents to refine troubleshooting processes.

• Analyze network performance data to optimize configurations.

• Automate repetitive tasks to improve efficiency.

By making continual improvements a priority, teams can increase productivity and reduce manual effort over time.

The Business Impact of ITIL 4 in Network Engineering

Implementing ITIL 4 in a network engineering team leads to several benefits:

• Faster resolution of network issues, improving service reliability.

• Fewer outages and disruptions, thanks to better change control.

• More strategic decision-making, based on data-driven problem management.

• Stronger collaboration between technical teams and business units.

Final Thoughts

ITIL 4 isn’t just for service desks—it’s a powerful framework that can transform how network engineering teams operate. By adopting structured processes for incident management, change enablement, and continual improvement, teams can move from being reactive to proactively delivering value to the business.

If you’re in networking, cloud, or IT operations, have you explored ITIL 4 in your work? I’d love to hear how it has impacted your approach to network service management!

#ITIL4 #NetworkEngineering #ServiceManagement #ProcessImprovement #ITLeadership

This article focuses on both the real-world necessity for vPC and the technical breakdown I learned during my studies.

Why vPC is Essential in Modern Data Centers

High Availability and Redundancy for Servers

In a modern data center, high availability is a top priority. Servers with multiple NICs (Network Interface Cards) require redundant paths to ensure uninterrupted service in case one switch fails. With vPC, a server can connect to two different Cisco Nexus switches using a single logical link (EtherChannel), so that even if one switch fails, the server remains connected.

Improved Bandwidth Utilization in Spine-Leaf Topologies

In leaf-spine architectures, Top-of-Rack (ToR) switches (leaf switches) connect to multiple spine switches using vPC, ensuring no bandwidth is wasted. Without vPC, redundant links would be blocked by Spanning Tree Protocol (STP). With vPC, all links remain active, improving the bandwidth efficiency between layers.

Multi-Homed Devices for Fault Tolerance

Devices like firewalls, load balancers, or storage systems often need multi-homing. By connecting to two different Nexus switches using vPC, these devices ensure redundancy and load balancing across both links. If one switch fails, the device still has an active connection through the other switch.

Simplifying Layer 2 Extensions Across Data Centers

vPC enables Layer 2 stretch between two data centers, allowing servers in different locations to be on the same VLAN. This is particularly useful in disaster recovery scenarios where seamless migration of workloads across data centers is required.

vPC Core Concepts

Once I understood the importance of vPC in the real world, I began studying how the technology operates at a technical level. Understanding vPC's architecture helped me configure and troubleshoot it more effectively.

vPC Domain

A vPC domain is the logical grouping of two Cisco Nexus switches. Both switches in the domain act as vPC peers, sharing a common control plane. One switch is designated as the primary, and the other as secondary.

vPC Peer Link

The vPC Peer Link is a crucial high-bandwidth link (typically 10Gbps or more) between the two switches in a vPC domain. This link is responsible for synchronizing data, such as MAC address tables, VLAN information, and traffic forwarding decisions, between the peers.

vPC Peer Keepalive Link

The vPC Peer Keepalive Link is a separate, out-of-band communication link between the two peers. This link ensures that both peers can detect the health status of each other and prevent split-brain scenarios. This keepalive communication often runs over the management interface.

vPC Member Ports

The vPC member ports are the actual interfaces on each Nexus switch that connect to the downstream devices (such as servers, firewalls, or another switch). These ports are part of the same EtherChannel that spans both switches.

Consistency Parameters

Consistency checks are performed between vPC peers to ensure that the configuration (such as VLANs, STP settings, and MTU size) is synchronized. If the configurations are inconsistent, vPC might fail, or traffic might not flow properly.

Lab Practice: Configuring vPC on Cisco Nexus Switches

Now that we’ve covered the need for vPC and how it works, let’s dive into a hands-on lab configuration. I spent quite some time practicing these setups, and they helped me solidify the theoretical knowledge with real-world application.

Step-by-Step Configuration

Step 1: Configure vPC Domain

In my lab, I configured the vPC domain on both Nexus switches. The domain allows the two switches to operate as one logical entity. The peer keepalive link is configured to maintain out-of-band communication.

! On both Nexus switches 
!
vpc domain 10
 peer-keepalive destination 192.168.1.2 source 192.168.1.1
!
# Nexus 1 to Nexus 2 The peer keepalive link is crucial for ensuring
 both switches can detect the operational state of each other.

Step 2: Configure vPC Peer Link

Next, I configured the vPC Peer Link, which is the high-bandwidth link used to synchronize forwarding state between the two Nexus switches. I used a Port Channel to aggregate multiple physical links into a single logical link for redundancy and increased bandwidth.

! On both Nexus switches 
interface Ethernet1/1
 switchport mode trunk
 channel-group 1 mode active
 no shutdown
!
interface port-channel1
 switchport mode trunk
 vpc peer-link
!

Step 3: Configure vPC Member Ports

Once the peer link was established, I configured the vPC member ports. These are the ports that connect to my downstream server, providing redundancy and load balancing across the two Nexus switches.

! On both Nexus switches 
interface Ethernet1/2
 switchport mode trunk
 channel-group 10 mode active
 no shutdown
!
interface port-channel10
 switchport mode trunk
 vpc 10
!

The Port Channel is configured to span both switches, providing the downstream server with a single logical link across two switches.

! On both Nexus switches 

interface Ethernet1/2
 switchport mode trunk
 channel-group 10 mode active
 no shutdown
!
interface port-channel10
 switchport mode trunk
 vpc 10 
!

The Port Channel is configured to span both switches, providing the downstream server with a single logical link across two switches.

Verify Configuration

Once the configuration was complete, I verified the status of the vPC domain using the following commands:

show vpc brief 
show vpc consistency-parameters

These commands helped me check for any misconfigurations or inconsistencies between the switches, ensuring that the vPC domain was functioning correctly.

Best Practices and Lessons Learned

Throughout my experience learning and configuring vPC, I picked up several important best practices that are critical for successfully deploying vPC in a production environment:

• Always use identical configurations on both Nexus switches to avoid vPC inconsistencies.

• Separate Peer Keepalive and Peer Links across different interfaces to avoid single points of failure.

• Regularly check vPC consistency parameters to prevent configuration drift that could bring the vPC down.

Summary

In conclusion, vPC is an essential technology in the Cisco Data Center environment, offering increased resilience, improved traffic load balancing, and better bandwidth utilization.

Studying it for the CCNP Data Center DCCOR (350-601) exam was a challenging yet rewarding experience, especially when applying it to real-world scenarios and hands-on labs. I hope this post gives you a clear roadmap to mastering vPC, whether you're studying for an exam or preparing for a live network deployment.

In this article, we’ll explore:

• Understanding the role of certificates in SD-WAN onboarding

• Common symptoms of certificate errors

• How to troubleshoot time synchronization issues between vManager and WAN Edge devices

• Steps to fix the issue and prevent it from recurring

1. Understanding the Role of Certificates in Cisco SD-WAN

In Cisco Catalyst SD-WAN, secure communication between the control plane (vManager, Controller, and Validator) and the WAN Edge devices (routers) is critical. This is achieved through Public Key Infrastructure (PKI)-based certificates. Each WAN Edge device must have a valid certificate, which is signed by vManager, to authenticate itself and join the network securely.

The certificate exchange process ensures that:

• WAN Edge devices are trusted and can participate in the SD-WAN fabric.

• The integrity and confidentiality of the data flowing between network nodes are protected.

When onboarding a WAN Edge device, it communicates with vManager to retrieve and install the signed certificate. However, if there is a mismatch in system time between the devices, this exchange can fail, leading to errors.

2. Common Symptoms of WAN Edge Certificate Errors

One of the primary indicators of a certificate signing issue is when vManager fails to onboard the WAN Edge device. You may see error messages in the vManager dashboard or logs such as:

• "Certificate Installed Failed"

• "Certificate is invalid or expired"

These issues often arise when the time on the WAN Edge device is not properly synchronized with the vManager server. PKI systems rely heavily on time synchronization to validate certificates, especially regarding expiration dates and validity periods. Even a slight time difference can lead to certificate rejection.

Check detailed logs via vManager shell to see insightful errors

Manager-20-9# vshell

Manager-20-9:~$ tail -f /var/log/nms/vmanage-syslog.log

---SNIP---

local6.INFO : 09-Oct-2024 02:11:34,762 UTC INFO  [] [VmanageSyslogLogger] (Thread-239) ||

      vedge-cloud: {"logid":"8598f1c7-0f4c-4f02-a5e6-96bd48de43e1","entry_time":1728439894761,"statcycletime":1728439894761,"logmodule":"vedge-cloud","logfeature":"vedge-cloud","loguser":"system","logusersrcip":"90.90.90.1",

      "logmessage":"Certificate Installation failed on vEdge cloud by vManage-b5657745-df80-4669-97ae-6a7b96890f3e-C8K-57282144-9F31-3AF3-799E-F33FDEFEF744","logdeviceid":"10.0.1.1",

            "auditdetails":["Failed to install certificate signed by vmanage <signerVmanageUUID: b5657745-df80-4669-97ae-6a7b96890f3e>

            on vedge <uuid: C8K-57282144-9F31-3AF3-799E-F33FDEFEF744, systemIP: 10.0.1.1>.

      Vedge cloud clock <1728439894000 ms> and vmanage clock <1728462976000 ms> are off by 23082000 ms"],"logprocessid":"436903f6-cefd-4c1e-8fd0-9f2578c5ea8c"}

---SNIP---

3. Troubleshooting Time Synchronization Between vManager and WAN Edge

To resolve the certificate error, it is essential to investigate whether time synchronization between the WAN Edge device and vManager is out of sync.

Step 1: Check Time on vManager and WAN Edge

Start by verifying the current time on both the WAN Edge device and the vManager server. You can check the time on these devices using the following commands:

For vManager:

show clock

For WAN Edge:

show clock

If there is a noticeable difference in time between the two devices, it will likely cause certificate validation failures.

Step 2: Verify NTP Configuration

Time synchronization in network environments is typically managed through the Network Time Protocol (NTP). To ensure the time is in sync, both the vManager server and WAN Edge device should be correctly configured to synchronize with the same NTP server.

For vManager NTP status:

Manager-20-9# show ntp associations

IDX  ASSOCID  STATUS  CONF  REACHABILITY  AUTH  CONDITION  LAST EVENT   COUNT

-------------------------------------------------------------------------------

1    15607    8013    yes   no            none  reject     unreachable  1

2    15608    961a    yes   yes           none  sys.peer   sys_peer     1

Manager-20-9# show ntp peer



INDEX  REMOTE  REFID   ST  TYPE  WHEN  POLL  REACH  DELAY   OFFSET  JITTER

--------------------------------------------------------------------------------------------

1    127.127.1.0  .LOCL.    5   l   105m  64   0    0.000   +0.000  0.000

2   *103.130.217.41  162.159.200.1  4   u     174   256   377    11.435  -0.179  0.300

For WAN Edge NTP status:

S1001-SD1#show ntp associations

  address         ref clock       st   when   poll reach  delay  offset   disp

 ~11.10.1.254    115.165.161.155  3      6     64     3  2.000  36.000 1939.2

 * sys.peer, # selected, + candidate, - outlyer, x falseticker, ~ configured

Ensure that both devices point to the same NTP server and that the server itself is functioning properly.

Step 3: Check for Time Drift

Time drift is another common issue, especially if devices have been running for a long period without synchronization. This can occur if the NTP configuration is incorrect or if the devices lose connectivity to the NTP server. You can manually update the time on the devices temporarily while investigating the root cause of the NTP issue.

Step 4: Correct Time Settings

If you notice a time mismatch, correct the settings by configuring NTP on both the vManager and the WAN Edge device. Below is an example of how to configure NTP on vManager and WAN Edge devices:

For vManager NTP configuration:

system server 1.vn.pool.ntp.orgvpn 512 version 4 exit !

For WAN Edge NTP configuration:

! ntp server 11.10.1.254 prefer !

Choosing your local NTP server or Regional Internet NTP server.

Note, don’t forget to allow ntp service on the sdwan tunnel interface that communicates with ntp server (if you are using public or outside ntp server).

sdwan interface GigabitEthernet2 tunnel-interface allow-service ntp ! !

Step 5: Restart Certificate Process

Once the time is synchronized correctly across vManager and WAN Edge, restart the certificate signing process via vManager. This will allow vManager to reissue a valid certificate and complete the onboarding.

Via vManager GUI:

Navigate to Configuration > Certificates > Select target WAN Edge by Chassis ID > Renew Device CSR

4. Preventing Future Time Sync Issues

After fixing the issue, it’s essential to implement preventive measures to avoid future time synchronization problems:

• Regular Monitoring of NTP Status: Ensure that NTP configurations on both vManager and WAN Edge devices are regularly monitored for status and health.

• Automate Alerts for Time Drift: Set up alerts in vManager to notify administrators if there is a significant time drift between the WAN Edge and vManager.

• Redundant NTP Servers: Configure multiple NTP servers to ensure time synchronization even if one NTP server goes down.

• Periodic Clock Verification: Perform routine checks on clock settings, particularly after maintenance or software updates, to ensure accurate time synchronization.

Summary

Time synchronization plays a vital role in network management, but it’s easy to overlook, especially in Cisco Catalyst SD-WAN environments. If your WAN Edge device is having trouble with certificate signing during onboarding, it’s a good idea to check the time settings on both vManager and the device. A difference in time between them can cause certificate validation to fail and interrupt the onboarding process.

By understanding common issues like time drift or incorrect NTP configurations, and following best practices, you can troubleshoot and fix these problems quickly. Keeping an eye on time settings across your SD-WAN network will help maintain smooth operations and prevent certificate errors down the road.

In this article, we will learn how the Centralized Control Policy works through a demonstration and guidelines.

Hub and Spoke is a common topology worldwide. By leveraging the separated service VPNs feature in Cisco SD-WAN, multiple types of topologies can be deployed per VPN.

Figure-1. Overview example of demonstration

Referring to Figure-1, this article will describe two examples of Hub-Spoke topology using Centralized Control Policy. Note that, in this example, I configure the tunnel group-id to achieve the public color tunnels separated from private color tunnels.

Figure-2. Hub-Spoke topology establishment

Figure 2 shows that data plane tunnels will not be established between the Spoke sites. By using the Centralized Control Policy (Figure 3), I will allow only the Hub's TLOCs to communicate with the Spoke sites. This way, the Spokes cannot learn each other's TLOC information.

Figure-3. Centralized control policy is applied to Controllers.

Using this topology example, I will demonstrate two scenarios below:

In Scenario 1, Spoke sites (site 102, site 103) will establish data plane tunnels (IPSec) with the Hub site (site 101), and the Spokes will not communicate with each other.

In Scenario 2, similar to Scenario 1 excepting the Spokes will be able to communicate with each other through the Hub site.

Before go to details, let me explain how I naming the router hostname / ip address / system ip, etc.

• Site ID: 101, 102, 103

• System IP: 101.101.101.x (3 octets will use site id and last octet will the router number at site, for example, site 101 have two WAN Edges, so I will use system ip as 101.101.101.1 and 101.101.101.2)

• Hostname: WAN Edge Router = WER-1011 (101 = site-id, 1 = router number at site)

• LAN segments: 10.101.1.0/24 (101 = site-id), the site id will be the 2nd octet.

• Service VPN: I use single service VPN in this article (VPN 1000)

Scenario 1: Hub-Spoke tunnels without Spoke-Spoke communication

Preview

By default, Controllers (vSmart) will receive and advertise all of TLOC routes to all site (101, 102, 103) using OMP.

Let’s check the WER-1021 (site 102) and WER-1031 (site 103) OMP TLOC tables by default as below

At the Site Id column, we can see Controller advertises fully TLOCs to these sites.
From site 102, WER receives TLOCs from site 101 and 103, so the full-mesh tunnels (among sites) are established.

How about OMP routes table? Let check below

The WER-1021 (site 102) receives the OMP LAN routes originated from site 101, and 102. Similar to WER-1021, the WER-1031 (site 103) receives the OMP LAN routes from site 101 and site 102.

Policy Implementation

By using these below steps, we can implement the Centralized Policy:

Step 1: Define the list of interest. (site list, VPN list, etc)

!
 lists
  site-list hub-sites-list
   site-id 101 
  !
  site-list spoke-sites-list
   site-id 102 
   site-id 103 
  !
  vpn-list VPN1000
   vpn 1000 
  !
 !
!

Step 2: Create control policy using structure match-action.

The sequence will be read from smallest to largest, and each sequence will include the match (condition) and action phases (do something if condition is matched).

viptela-policy:policy
 control-policy HUB-SPOKE-cp
    sequence 10
     match route
      site-list hub-sites-list
      vpn-list VPN1000
     !
     action accept
     !
    !
    sequence 20
     match tloc
      site-list hub-sites-list
     !
     action accept
     !
    !
  default-action reject
 !

In sequence 10, the “match route” condition means all of OMP routes originated from hub-sites-list (site 101) AND these routes belong to VPN 1000. Action accept shows that the Controller will allow matched routes to be advertised to the applied target sites. (at step 3).

In sequence 20, the “match tloc” condition means all the TLOC routes originated from hub-sites-list (site 101) will be accepted to be advertised to target sites.

Default-action reject, it means all of routes which do not match any condition will be rejected to be advertised to target sites.

Step 3: Apply policy to the list of sites (Spokes)

Apply policy to the list of sites (Spokes) with outbound direction on Controller. The centralized control policy will be pushed to Controller (vSmart) after activating, all of in / out TLOC/OMP routes will go through these control policy before distributing to sites in fabric.

apply-policy
 site-list spoke-sites-list
  control-policy HUB-SPOKE-cp out
 !
!

In summary, the OMP routes and TLOC routes from HUB site (101) will be advertised to Spokes (site 102, 103). And OMP/TLOC routes from site 102 will be rejected to advertise to site 103 and vice versa.

Verify

The TLOC route table on Spokes site will not receive any Spoke's TLOC anymore.

The OMP routes table on WER-1021 will not display the LAN routes originated from Spokes (site 103) any more and vice versa.

Let’s verify the result by traceroute from WER-1021 (spoke) to 10.101.1.1 (Hub’s LAN).
The below output show reachability to Hub site from Spoke site.

Let’s try to check connection from Spoke (102) to Spoke (103) as below Figure.

Great! The result is unreachable as our expectation.

The Scenario 2 will be released soon.

Firstly, control policies operate separately from data plane policies as they don't directly affect data plane operations. Instead, they influence the routing information upon which the data plane relies, thereby impacting traffic forwarding.

Control policies can modify, filter, summarize, or restrict the advertisement of routing prefixes or TLOCs. Consequently, a WAN Edge device will construct its forwarding plane based on the altered control plane information.

Figure 1-1. Cisco WAN Edge Packet Forwarding Order of Operation

When a packet traverses a WAN Edge router, it follows a carefully orchestrated sequence of steps:

1. IP Destination Lookup: The router begins by examining the destination IP address of the packet. It consults its routing table to determine the most appropriate next-hop for forwarding the packet. This lookup is crucial as it guides all subsequent forwarding decisions.

2. Ingress Interface ACL: Next, the packet encounters any ACLs configured on the ingress interface. These ACLs, often part of localized policies, dictate whether the packet should be allowed, denied, or subjected to specific actions like traffic shaping or Quality of Service (QoS) markings. If the packet is denied by the ACL, it is dropped immediately and not processed further.

3. Application-Aware Routing: Following the routing decision based on the routing table, the packet undergoes evaluation against an Application-Aware Routing policy. This policy is designed to optimize traffic based on application-specific requirements. However, it's important to note that Application-Aware Routing can only differentiate between paths of equal cost in the routing table. If multiple next-hop addresses for a destination don't have equal-cost paths, the Application-Aware Routing policy has no impact, and the packet follows the most preferred path based solely on the routing table.

4. Centralized Data Policy: Once Application-Aware Routing is applied, the packet's journey may be further influenced by centralized data policies. These policies have the authority to override decisions made by Application-Aware Routing, providing additional control over traffic forwarding.

5. Routing and Forwarding: With the routing decisions made, the router determines the appropriate output interfaces for the packet's continued journey.

6. Security Policy: If configured, the packet is subjected to various security measures in a predefined order. These may include firewall rules, intrusion prevention systems (IPS), URL filtering, and advanced malware protection. Each security measure is applied according to its configured rules and policies.

7. Encapsulation and Encryption: As the packet prepares to traverse the fabric, it undergoes encapsulation and encryption processes to ensure secure transmission across the network. This includes adding VPN labels and applying tunnel encapsulation as necessary.

8. Egress Interface ACL: Finally, the packet encounters any ACLs configured on the egress interface before leaving the router. Similar to the ingress ACLs, these ACLs may permit, deny, or manipulate the packet's contents based on predefined rules. Any modifications or denials applied here take effect before the packet is forwarded out of the router toward its final destination.

How Cisco SD-WAN Data Plane works?

**Get the Cisco SD-WAN Zero-to-One ebook

In Cisco SD-WAN, you can set up configurations in two main ways. The network administrator can either put in the settings by typing commands directly using the command line (CLI) as usual.

Another way is by using a visual interface called Manager GUI - it's like using a point-and-click system to set things up. Using vManage GUI is better because it's less likely to make mistakes and can recover from problems automatically.

When you're setting up configurations for WAN Edge devices or controllers using the Manager GUI, a network administrator follows a process. They use something called a "device template" which is like a blueprint for how the device should be set up. This device template can be based on two types:

• CLI (command line)

• Feature template.

  When making a CLI template, all the setup instructions must be included in the template, not just parts of it. On the other hand, feature templates are like building blocks, where each block is a specific feature like a piece of a puzzle. These feature templates define what you want the device to do, like setting up how it handles data, manages connections, and more. This could include things like routing rules, how different parts connect, and a protocol for managing the network.

You can reuse these feature templates in different device templates, which makes things more flexible and easier to manage. This flexibility is why feature templates are recommended. They're not limited to a specific type of device either. The person setting up the network only needs to think about what they want to achieve with the configuration.

When Manager applies the configuration to a specific device, whether it's a Cisco IOS-based device or a Viptela OS device, it knows how to put in the right instructions in a way that the device understands.

Device templates

Device templates serve as a collection of feature templates designed for specific types of devices. This means that you might have several device templates tailored to the same hardware model, depending on factors like the device's location, connectivity choices, or its designated role in the network. It's important to note that a device template cannot be used across different device types. However, feature templates can be shared across various device types.

A device template comprises four primary components or groups:

1. Basic Information: This section covers crucial items like System, Logging, AAA, BFD, and OMP feature templates.

2. Transport and Management VPN: Here, you'll find templates for configuring VPN 0 and VPN 512. This includes settings like underlay routing protocols and interface configurations.

3. Service VPN: This section is dedicated to configurations for service VPNs or LAN-facing templates. It's where you'll set up parameters for BGP, OSPF, and interfaces.

4. Additional Templates: In this section, you'll encounter templates for local policies, security policies, SNMP configurations, and more.

Feature Template

Feature templates enhance the flexibility of configuration options significantly.

For instance, they allow you to define variables for configuration parameters. This smartly reduces the number of templates needed for your deployment while maintaining modularity.

To illustrate, consider a scenario with MPLS transports using various physical interface numbers like Gi0/0, Gi0/1, and so forth. Initially, you might consider creating separate feature templates for each interface with different IP addresses, resulting in multiple templates.

However, by utilizing variables for both the physical interface and IP address choices, the administrator can streamline this into a single feature template that remains applicable across all device templates. This approach streamlines the configuration process and ensures consistency throughout.

In templates, you can define three types of values:

1. Default: These are factory default values that can't be changed. For instance, default BFD timers.

2. Global: Values set here are applied wherever this configuration is used. For example, SNMP community strings are applied globally to all devices using this template. The advantage is that updating the template's global option later automatically updates all relevant device templates.

3. Device Specific: These values are set using user-defined variables. For instance, setting interface names. These values are defined when attaching the device template to a specific device.

   Some template options might not have all three types, depending on the configuration. For instance, a BGP AS number won't have a default value.

There are numerous feature templates to configure, including:

• System: Basic system info like System IP, Site ID, and Hostname.

• BFD: Adjust BFD timers and app-route multipliers for transports/colors.

• OMP: Change graceful restart timers or control redistribution into OMP.

• Security: Modify IPsec settings like anti-replay, authentication, and encryption.

• VPN: Define service VPNs, routing protocol redistribution, or static routing.

• BGP/OSPF: Configure BGP/OSPF in VPNs or VRFs.

• VPN Interface: Define interfaces in service VPNs/VRFs with options like IP Address, QoS, ACLs, and NAT.

Feature templates are used in device templates. After creating a device template, you apply it to a specific device/group. Remember, device templates are specific to a device type. If feature templates have variables, values must be filled when attaching the device template. Successful configuration syntax checks are done in vManage before pushing to the device.

Variables can be populated within the vManage workflow or via a CSV file. The latter is useful for provisioning multiple devices simultaneously.

Note.

If the device loses control plane connectivity to vManage during configuration push, it initiates a 5-minute rollback timer. Failing reconnection within this time, it rolls back to the last-known good configuration. The network admin is notified of the sync issue and can address it accordingly.

Get the Cisco SD-WAN Zero-to-One ebook

Step-by-Step Configuration Template Creation

CLI Template

Using the CLI template is the most manual way you need to create the maintain the WAN Edge devices configuration using Command Lines.

But in some specific situations, such as testing some configuration parts and functions, you need to change a small piece of configuration frequently to check the result, the CLI template is good to go.

In the real-world business, the individual device CLI templates for each WAN Edge are usually used, because it will minimize the maintenance impacts when you adjust the configuration template for one of the WAN Edges.

Or just simply that you are a Network Admin guy, who loves the Command Lines.

From vManage > Configuration > Templates > Create Template > CLI Template

Enter the required fields Device Model, Template Name, and Descriptions.

Then you can upload your existing CLI configuration file using “Select a File”, or even you can manually enter your configuration CLI there.

From here, maybe you are thinking that it’s tough, how can we remember the CLI structure and syntax if we don’t have the existing configuration file?

No worries, we can use the function “Load Running config from reachable device”.

From the drop-down list, select the reachable WAN Edge, and it will load the full running-config of the selected WAN Edge here.

Tips: You can also SSH or Console to your running WAN Edge, and copy the running-configuration using the output of the command “show sdwan running-config” and Paste here.

Next, adjust the configuration like system-ip, site-id, hostname, or WAN IP Address, to match your target WAN Edge. And Click “Add” to create the CLI Template.

CLI Template with Variables

In some cases, we would like to create a single CLI template that can be reusable and attached to more than one WAN Edge.

Let’s think, the system-ip, WAN IP Address, etc. are unique and could not be duplicated between WAN Edges.

Thus, how can we use the same CLI template for many WAN Edges?

The Variables can be used to input the Values into the CLI template when attaching to devices.

Thus, while attaching CLI template to WAN Edges, the Variables require the Value inputted from Network Admin.

It means for each WAN Edge you can input different values and still use the same CLI template.

Feature Template

On the other hand, there are templates that define the configuration settings for specific SD-WAN features, such as Quality of Service (QoS) or Firewall rules.

By using feature templates, network administrators can quickly configure and deploy these features across multiple devices, ensuring consistency in the configuration and optimizing network performance.

One of the different points between CLI and Feature template when creating the Device template from the Feature template, “Device Role” is required to select two options:

• SDWAN Edge: Almost the WAN Edge model will be set in this role as a normal cEdge.

• Service Node: Until now, only CSR1kv and C8000v (new version of CSR1kv) supported this role. For more information about Service Node, check here.

In this post, I will go with SDWAN Edge mode with the C1111X-8P device.

The remaining parts are Template Name and Descriptions will be the same as CLI Template.

The above attachment and Figure 10 shows that there are some Main parts of the configuration

• Basic Information

• Transport & Management VPN

• Service VPN

• Cellular

• Additional Templates

• Switchport

Note, these parts is for the C1111X-8P model, and it can be different when you choose another device model.

Look at Figure 11, you can see the list of Feature Templates which are used to build your Device template like a Block Puzzle game.

For example, in Basic Information Part, we have Cisco System, Cisco Loggin, Cisco NTP, Cisco AAA, Cisco BFD, etc. I will call those Basic Information child parts.

Moreover, when you create the new Device Template from the Feature template, all of the required parts will be added using Factory_Default_xxx Feature Templates which are predefined by Cisco.

You can check their contents and use them in your Device Template. Note that the predefined template will be named “Factory_Default_xxxx” or “Default_xxxx“.

In this post, I gonna show you how to establish the very first simple Device template, so let’s go to define some basic simple Feature Templates.

vManage > Configuration > Templates > Feature > Add Template >*Select Device Model > Select Template*

As I mentioned above Basic Information is like the child configuration part of the Device Template, and Figure 12 show that Cisco AAA, Cisco BFD, etc. are child template of Basic information part.

For Example, I will create my own custom Cisco AAA Feature Template, the structure will be as below:

Device Template > Basic Information > Cisco AAA

In the Cisco AAA Feature templates, you can easily see there are other child parts inside like LOCAL, RADIUS, TACACS, Authentication and Authorization Order, etc.

In this device template, I would like to add a new local user to the target WAN Edge, so I will create a Feature Template Cisco AAA and add the user as below.l

Once the feature template Cisco AAA is created, let's go back and create Device Template from Feature Template as mentioned in above Figure 10.

I think from here, you can understand the concept of Feature Templates clearly.

Note that you can use a Feature Template to put into many Device Templates with the same device model selected.

Feature Template with Variables

Same as CLI Template, Feature Template can use the Variable to be flexible and attached to many WAN Edges with the same template.

I will demonstrate with the example that I will create the VPN 0 Feature Template to match the DUAL-WAN requirement.

I will input the fixed value VPN-ID = 0 because the requirement is to create VPN 0.

Go next with IPv4 Route to set the default route with dual next-hop (Dual WAN).

You can enter your fixed value of next-hop IP Addresses there, but our target is to use this Feature Template for many WAN Edges on another site.

So we can set these as Variables and will input the specific value when attaching the template to devices.

Conclusion

Even if you use the Device template from CLI Template or Feature Template, the same concept and structure are applied. Let's consider the requirements, and balance the convenience and safety maintenance when you choose the type of templates.

Hope the posts help you more clearly understand how to use the Cisco SDWAN Templates.

Connect me

In the world of modern networking, the ability to connect disparate networks securely and efficiently is crucial for businesses and organizations. One technology that has proven to be invaluable in achieving this is the Generic Routing Encapsulation (GRE) tunnel. GRE provides a flexible and scalable method for encapsulating packets and transporting them over an intermediate network. In this article, we will delve into the details of GRE tunnels, exploring how they work, their benefits, and various use cases in today's networking landscape.

What is a GRE Tunnel?

A GRE tunnel is a virtual point-to-point connection that encapsulates packets from one network protocol within another. Essentially, GRE creates a private, secure, and isolated path between two endpoints over a public or untrusted network, like the Internet. It achieves this by encapsulating the original packets inside GRE headers, allowing them to traverse the intermediate network while keeping their original routing information intact.

How GRE Tunnels Work

A GRE tunnel consists of two endpoints: the source and the destination. When a packet is sent from the source endpoint, it is encapsulated with a GRE header, which includes information such as the destination IP address and the original packet type. The packet is then routed through the intermediate network, like any other IP packet, until it reaches the destination endpoint. At the destination, the GRE header is stripped off, and the original packet is forwarded to its final destination based on its original routing information.

Benefits of GRE Tunnels

• Encapsulation Flexibility: GRE is protocol-agnostic, meaning it can encapsulate packets from various network protocols. This versatility allows for the seamless integration of different networks and devices.

• Enhanced Security: By creating a private tunnel, GRE adds an extra layer of security, shielding the original packet's content from potential eavesdroppers and unauthorized access.

• Isolation and Routing Independence: GRE tunnels allow networks to communicate with each other as if they were directly connected, even if they use different routing protocols. This independence makes it easier to connect networks with diverse configurations.

• Overcoming NAT Limitations: GRE tunnels can bypass Network Address Translation (NAT) devices, enabling communication between private IP networks across the internet.

Use Cases of GRE Tunnels

• Site-to-Site Connectivity: GRE tunnels are widely used for connecting remote offices and branch locations to the main corporate network, enabling secure communication over the Internet.

• VPN Solutions: GRE can be combined with encryption protocols to create Virtual Private Networks (VPNs), ensuring privacy and confidentiality in data transmission.

• IPv6 Transition: During the transition from IPv4 to IPv6, GRE tunnels can help establish connectivity between IPv4-only and IPv6-only networks, facilitating a smooth migration.

• Multicast over Unicast Networks: GRE tunnels can be utilized to transmit multicast traffic over networks that only support unicast traffic, expanding the reach of multicast applications.

Lab Example

As the above topology, R1 and R3 play the role of Edge Routers which will form site-to-site GRE tunnel.

The tunnel IP Address segment will be 192.168.10.0/24, and R1, R3 will form the EIGRP adjacency on this tunnel segment. Note that in this demonstration I use RIPv2 for underlay reachability (172.16.0.0).

Using EIGRP adjacency to advertise the LAN sides network 10.1.1.0/24 and 10.3.3.0/24 via tunnel IP next-hops.

R1 Configuration
!
interface Tunnel0
 ip address 192.168.10.1 255.255.255.0
 shutdown
!
tunnel source Ethernet0/0
 tunnel destination 172.16.31.2
!
interface Ethernet0/0
 ip address 172.16.11.2 255.255.255.252
!
interface Ethernet0/3
 ip address 10.1.1.254 255.255.255.0
!
router eigrp 100
 network 10.1.1.0 0.0.0.255
 network 192.168.10.0
!
router rip
 version 2
 network 172.16.0.0
 no auto-summary
!

R2 Configuration
!
interface Ethernet0/0
 ip address 172.16.11.1 255.255.255.252
!
interface Ethernet0/1
 ip address 172.16.31.1 255.255.255.252
!
router rip
 version 2
 network 172.16.0.0
 no auto-summary
!

R3 Configuration
!
interface Tunnel0
 ip address 192.168.10.3 255.255.255.0
 tunnel source Ethernet0/1
 tunnel destination 172.16.11.2
!
interface Ethernet0/1
 ip address 172.16.31.2 255.255.255.252
!
interface Ethernet0/3
 ip address 10.3.3.254 255.255.255.0
!
router eigrp 100
 network 10.3.3.0 0.0.0.255
 network 192.168.10.0
!
router rip
 version 2
 network 172.16.0.0
 no auto-summary
!

Verification

VPC1 traceroute to VPC2
VPC1> trace 10.3.3.3 trace to 10.3.3.3, 8 hops max, 
 1 10.1.1.254 0.428 ms 0.415 ms 0.300 ms
 2 192.168.10.3 1.132 ms 0.876 ms 1.028 ms
 3 *10.3.3.3 2.424 ms

The traffic goes through the Tunnel IP address (192.168.10.3) as the next-hop.

The below output shows the detailed Tunnel's interface information:

• Tunnel interface status

• Tunnel IP address, Tunnel Source

• Tunnel MTU, Bandwidth, Delay, Keepalive

• Tunnel Encapsulation type

R1#show interfaces Tunnel 0

Tunnel0 is up, line protocol is up 
  Hardware is Tunnel
  Internet address is 192.168.10.1/24
  MTU 17916 bytes, BW 100 Kbit/sec, DLY 50000 usec, 
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation TUNNEL, loopback not set
  Keepalive not set
  Tunnel source 172.16.11.2 (Ethernet0/0), destination 172.16.31.2
   Tunnel Subblocks:
      src-track:
         Tunnel0 source tracking subblock associated with Ethernet0/0
          Set of tunnels with source Ethernet0/0, 1 member (includes iterators), on interface <OK>
  Tunnel protocol/transport GRE/IP
    Key disabled, sequencing disabled
    Checksumming of packets disabled
  Tunnel TTL 255, Fast tunneling enabled
  Tunnel transport MTU 1476 bytes
  Tunnel transmit bandwidth 8000 (kbps)
  Tunnel receive bandwidth 8000 (kbps)
   ...

For your reference, the below output is the EIGRP neighbor status and routing table on R1.

R1#show ip eigrp neighbors 
EIGRP-IPv4 Neighbors for AS(100)
H   Address        Interface       Hold Uptime   SRTT   RTO  Q  Seq
                                    (sec)         (ms)       Cnt Num
0   192.168.10.3     Tu0           14 01:38:33   19     1470  0  3

R1#show ip route
...

Gateway of last resort is not set
...
D        10.3.3.0/24 [90/26905600] via 192.168.10.3, 01:39:44, Tunnel0
      172.16.0.0/16 is variably subnetted, 3 subnets, 2 masks
...
R        172.16.31.0/30 [120/1] via 172.16.11.1, 00:00:20, Ethernet0/0
      192.168.10.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.10.0/24 is directly connected, Tunnel0
L        192.168.10.1/32 is directly connected, Tunnel0

Conclusion

As networks continue to grow in complexity, the need for robust, secure, and scalable connectivity solutions becomes paramount. GRE tunnels provide a powerful tool for achieving these objectives, enabling organizations to create secure, private connections across diverse networks. With their flexibility, security features, and versatility, GRE tunnels remain an indispensable part of modern networking, facilitating seamless communication between geographically dispersed entities. As technology advances, GRE tunnels will continue to play a pivotal role in enabling the interconnectivity that drives today's digital world.

In a traditional network, the data plane is responsible for moving packets of data from one place to another. This is often done using the public internet or private wide area networks (WANs) like DMVPN, MPLS, or point-to-point connections. These technologies use overlays to encapsulate and secure the data packets.

However, as WANs grow larger, the existing technologies face challenges in scaling, especially when it comes to securing control and data planes. This requires a lot of processing power to handle tasks like key exchanges and routing updates.

To ensure security in the data plane, IPsec encryption and its tools are commonly used. However, when networks rely heavily on IPsec for security, scalability becomes a concern. The processing power required for key exchanges grows exponentially with the number of nodes in the WAN.

For example, a network with 100 nodes would need 10,000 key exchanges (n2), and each device would have to maintain 999 keys (n–1).

Read more: Cisco SD-WAN vSmart Controllers

Cisco SD-WAN also utilizes IPsec for securing the data plane, but it has made modifications to support larger deployments. It uses a centralized controller (vSmart) to distribute keys and routing information, allowing for scalability without requiring each WAN Edge device to negotiate keys with all other nodes in the network.

Furthermore, networks face scalability challenges when they need to support segmentation or different topologies per network segment. Traditional methods like MPLS L3VPN and DMVPN can be complex and require experienced network engineers to implement and troubleshoot.

In the Cisco SD-WAN solution, segmentation is implemented natively and doesn't require advanced experience to set up and support. It allows for different topologies per network segment, such as a full mesh for corporate users and a hub and spoke for specific devices.

TLOC Color

The Color attribute of the TLOC route is crucial in identifying the transport being used. Each transport should ideally have a different color assigned to it. The Color attribute allows policies to be constructed to influence how the data plane is built within the SD-WAN network.

There are 22 predefined colors available to choose from, and they also define whether the underlying transport is private or public. This distinction determines the IP address that should be used when establishing a data plane tunnel to a remote site.

By default, WAN Edge devices will attempt to build data plane tunnels to every other site using every available color.

Read more: Cisco SDWAN TLOC?

When setting up the IPsec data plane, routers in the network will automatically try to establish connections with all other routers in the fabric, creating full mesh connectivity.

For example, if cEdge-1 has the color "biz-internet", and "private1" and another router has the color "public-internet," and "private1" they will still establish an IPsec tunnel if they have IP connectivity between them.

This happens by default. If two routers have IP connectivity, regardless of their assigned "color" (a designation used in the network), they will build an IPsec tunnel between them.

One scenario where this decision becomes important is when your private WAN (MPLS) doesn't have direct IP connectivity to the internet. In such cases, you wouldn't want MPLS-connected routers to attempt to establish connections with internet-connected routers. (refer to the below figure)

Depending on the country or region, you may want full mesh tunnels between routers. However, when connecting different countries or regions (like the US and UK), it may be preferable to use point-to-point tunnels specifically between hub sites.

If there is no IP connectivity between TLOCs or if the design doesn't allow data connectivity between certain colors, there are two options available.

One option is to advertise the restrict attribute with the TLOC, which tells other devices in the network not to try to establish connectivity with the restricted color. The other option is to configure tunnel groups, which serve the same purpose.

Color Restrict

Let's focus on the "restrict" keyword first. It is an attribute within the TLOC route that can be set to either on or off for each site. In the example provided, the TLOC route uses the color "biz-internet" and has the "restrict" attribute set to 1.

This means that the device will only form tunnels with other TLOCs advertising the same color. If the restrict attribute were set to 0, the color would be unrestricted and able to form tunnels with other colors.

In the given scenario, if "restrict" is not set, each device would end up with four data plane tunnels, as IPsec tunnels would be established across all colors.

To enable "restrict", use the below command lines:

vpn 0
 interface ge0/0
  tunnel-interface
   color public-internet restrict

The output below verifies the TLOC "restrict" enabled.

---------------------------------------------------
tloc entries for 10.10.10.12
                 biz-internet
                 ipsec
---------------------------------------------------
            RECEIVED FROM:                   
peer            0.0.0.0
status          C,Red,R
loss-reason     not set
lost-to-peer    not set
lost-to-path-id not set
    Attributes:
     attribute-type    installed
     encap-key         not set
     encap-proto       0
     encap-spi         258
     encap-auth        sha1-hmac,ah-sha1-hmac
     encap-encrypt     aes256
     public-ip         79.12.12.12
     public-port       12346
     private-ip        79.12.12.12
     private-port      12346
     public-ip         ::
     public-port       0
     private-ip        ::
     private-port      0
     bfd-status        up
     domain-id         not set
     site-id           10
     overlay-id        not set
     preference        0
     tag               not set
     stale             not set
     weight            1
     version           3
    gen-id             0x8000000b
     carrier           default
     restrict          1            <<<<<<<<<<--## RESTRICT enabled
     on-demand          0
     groups            [ 0 ]

Tunnel Group

Another way to control data plane connectivity is by utilizing tunnel groups. When you use tunnel groups, only tunnels that have matching tunnel groups or no tunnel group defined at all will establish data plane connectivity, regardless of their color. It is advisable to have tunnel groups defined for all sites if you decide to implement this approach.

A common scenario where tunnel groups come into play is when a data center has two physical connections to the same MPLS provider, but the branch sites have only one physical connection each. However, the network design requires establishing connectivity across both physical interfaces in the data center.

To achieve this, the tunnel group is advertised as an attribute in the TLOC route, as shown in the above illustration. The tunnel group value can range from 0 to 4294967295, allowing for flexibility in setting up the desired configurations.

Data Plane Encryption

Up until now, we've been discussing various concepts related to the control plane's role in establishing the data plane. Similar to other overlay technologies, Cisco SD-WAN achieves encryption and authentication using IPsec. However, the scale of this process is handled differently, especially concerning key exchange.

In the traditional approach, the Internet Key Exchange (IKE) protocol manages key exchange. In the first phase of IKE, two peers negotiate encryption, authentication, hashing, and other techniques to establish a secure channel for the second phase of the IPsec tunnel. The second phase of IKE sets up a tunnel for transmitting user data. During this phase, several elements are negotiated, such as the encapsulation protocol (Authentication Header or Encapsulation Security Protocol), encryption algorithm, authentication type, and tunnel lifetime.

Cisco SD-WAN supports the following methods for key exchange:

• Authentication: Ensures that the communicating endpoints are valid and authentic, using 2048-bit keys with RSA encryption. The SD-WAN solution supports Encapsulation Security Payload (ESP) and Authentication Header (AH) for sender authentication.

• Encryption: Utilizes the AES protocol with a 256-bit key length for data encryption.

• Integrity: Verifies that data traffic traversed the network without tampering. This is achieved using the Galois Counter Mode (GCM) variant of AES-256, which has a built-in hashing mechanism for data integrity. Additionally, Anti-Replay Protection is enabled to safeguard against duplication attacks.

As the network grows larger, this negotiation process can become a scalability concern. Even after the initial negotiation, the devices must continue tracking the tunnel state, which consumes CPU cycles.

To address this issue, Cisco SD-WAN implements these negotiations within the control plane. The WAN Edge already has a tunnel established to the control plane with its encryption, authentication, and integrity. This infrastructure is leveraged for data plane negotiations. Each WAN Edge generates an AES-256 bit key (per transport) for encryption and integrity.

This key is then advertised to vSmart, along with the corresponding TLOCs, in an OMP update. These route advertisements are propagated throughout the network. Remote WAN Edges use this information to build IPsec tunnels between themselves.

This model of key distribution eliminates the need for individual negotiations as in IKE, reducing the burden on the system. Additionally, to enhance security, WAN Edges regenerate their keys every 24 hours, with the flexibility to adjust the rekey timer based on specific requirements. Importantly, renegotiating keys does not disrupt existing traffic, as this process occurs in parallel with the existing tunnels.

Encryption with Pairwise

This approach adds an extra layer of security by avoiding the use of the same key across all devices in the network fabric for encryption and decryption.

Pairwise encryption works by creating specific key pairs between two WAN Edges. When encrypting and decrypting data between WAN Edge 1 and WAN Edge 2, a unique key pair is generated for this specific pair of devices. Similarly, a different key pair is used for traffic between WAN Edge 1 and WAN Edge 3.

In this way, each WAN Edge has its own unique set of keys for communication with different peers.

The public key is the only piece of information exchanged over OMP (Overlay Management Protocol).

The advantage of this method is that security-conscious customers need not worry about the private key being exchanged as well. The key exchange process still occurs through the vSmart controller, and unique pairs are generated for each transport.

In summary, the pairwise encryption key model ensures greater security by creating unique key pairs for communication between WAN Edges, providing an additional layer of protection against potential vulnerabilities that could arise from using the same key across all devices in the fabric.

Get the Cisco SD-WAN Zero-to-One ebook

Hey there!

I'm Nam, an enthusiastic Network Engineer. After spending years working with Cloud Computing, I've decided to deep dive into Microsoft Azure and I'm really excited about it! Today, I want to share some essential tips and things that will be super helpful for you as you start your own Azure learning adventure. Let's get started!

Azure Resource Manager

Azure Resource Manager (ARM) is a technology platform from Microsoft that puts all the pieces of Azure together in an organized way. It brings together Azure's resource providers, resources (like virtual machines and storage), and resource groups to create a well-structured cloud platform.

With ARM, Azure services are available through subscriptions, and resource types can be used in resource groups. It also allows easy access to resource APIs, which are like doors to interact with the portal and other client applications. ARM makes sure only authorized users can access these resources, keeping everything secure.

To manage your cloud resources, ARM provides simple deployment and management tools like the Azure portal, Azure PowerShell, and the command-line interface (CLI). These tools make it easy to work with your cloud service.

Azure Resources

In Azure, everything is called a "resource." For example, storage accounts, virtual machines (VMs), network interfaces, public IP addresses, etc are all considered resources. These resources are managed through Azure Resource Manager (ARM), which works based on two main concepts: resource providers and resource consumers. (Refer above illustration)

Resource providers

Resource providers are like services that offer different types of resources. They act as containers for grouping similar resources together. For instance, a VM resource type is provided by a resource provider called "Microsoft.Compute/virtualMachines."

By using Resource Explorer via Azure Portal, you can easily see resources in detail. (Providers/Type/API version/contents)

Each resource provider has its own version of the resources it offers, which is identified by release dates. To use a specific resource, the corresponding resource provider must be available in the subscription. Not all resource providers are automatically available, so they might need to be registered separately for each subscription.

Resource groups

Resource groups are like containers for grouping multiple resource instances together. They serve as units of deployment and have a unique name within a subscription. Resources can be provisioned in different Azure regions but still belong to the same resource group.

Resource groups provide additional services like metadata services (such as tagging for categorization), policy-based management, role-based access control (RBAC), and protection against accidental deletions or updates.

Resources

Resources are instances created from resource types. Each resource instance has a name and type that make it unique globally or within a resource group. Resources inherit security and access configurations from their parent resource group, but these settings can be overridden for each resource. Resources can also be locked to prevent certain operations or access.

For example, Virtual Machine, Virtual Network, Network Interface, App Service, etc.

ARM Features

1. RBAC: Azure Active Directory (Azure AD) authenticates users and assigns roles to control access to resources, resource groups, and subscriptions based on permissions defined in roles.

2. Tags: Tags are name-value pairs used to add additional information and categorization to resources for better organization and management.

3. Policies: Custom policies are rules and conventions that define access control for resources and resource groups. They work alongside RBAC.

4. Locks: Subscriptions, resource groups, and resources can be locked to prevent accidental changes.

5. Multi-region: Resources can be provisioned in different Azure regions while still belonging to the same resource group.

6. Idempotent: This ensures that resource deployment is consistent and predictable, no matter how many times it's executed.

7. Extensible: ARM allows the platform to be extended with new resource providers and resource types.

Conclusion

In summary, ARM simplifies resource management in Azure by grouping resources under resource providers, organizing them into resource groups, and providing useful features like RBAC, tags, policies, locks, and more.

In the Cisco SD-WAN solution, the Overlay Management Protocol (OMP) serves as the routing protocol. However, OMP goes beyond just routing and provides several essential services within the control plane:

• Facilitation of network communication: OMP enables data plane connectivity between sites in the SD-WAN fabric, including service chaining and multi-VPN topology information.

• Distribution of data plane security information: OMP handles the distribution of encryption keys, ensuring secure communication within the fabric.

• Best-path selection and routing policy advertisement: OMP determines the optimal paths for data traffic and communicates routing policies across the network.

  Read more

  [Part 6] Cisco SDWAN - vSmart Controller

  [Part 7] Cisco SDWAN - Control Plane Operations - OMP

WAN Edge routers will establish the OMP peering session with all of vSmart by default. The OMP update is protected via DTLS or TLS sessions.

The below example shows the OMP peers on WAN Edges

vEdge11 (Viptela OS) "show omp peers"

vEdge11# show omp peers

R -> routes received
I -> routes installed
S -> routes sent

                 DOMAIN   OVERLAY  SITE
PEER       TYPE    ID      ID       ID   STATE  UPTIME     R/I/S
---------------------------------------------------------------
9.9.9.30   vsmart  1       1        99    up   0:10:16:52   2/2/2

cEdge41 (IOS-XE) "show sdwan omp peers"

cEdge41#show sdwan omp peers

R -> routes received
I -> routes installed
S -> routes sent


                 DOMAIN  OVERLAY  SITE
PEER      TYPE   ID     ID        ID    STATE  UPTIME      R/I/S
---------------------------------------------------------------
9.9.9.30 vsmart  1      1         99    up    6:03:33:18    1/1/2

The below output shows the OMP peers from vSmart to WAN Edges.

vSmart1# show omp peers

R -> routes received
I -> routes installed
S -> routes sent

                  DOMAIN  OVERLAY SITE
PEER       TYPE   ID      ID      ID    STATE  UPTIME      R/I/S
--------------------------------------------------------------
1.1.1.11  vedge   1       1       10     up   0:10:20:15   2/0/2
1.1.1.40  vedge   1       1       40     up   6:08:41:00   2/0/1

OMP in the Cisco SD-WAN solution is responsible for advertising different types of routes between the vSmart controllers and WAN Edge routers. These routes include:

• OMP routes (vRoutes): These routes represent network prefixes that enable connectivity services for data centers, branch offices, or any other endpoint within the SD-WAN fabric.

• TLOC Routes: TLOCs serve as identifiers that associate an OMP route with a physical location. They are the only IP addresses that are known and reachable within the underlay network.

• Service routes: Service routes identify network services within the SD-WAN overlay. These routes indicate the physical location of services such as firewalls, IPS, IDS, or any other device capable of processing network traffic. Service information is advertised through service routes and OMP routes.

1. OMP Routes (vRoutes)

In an SD-WAN setup, each WAN Edge device located at a site communicates with the central vSmart controllers by sharing information about the routes it can handle. These updates are similar to the way traditional routing updates work. They provide information about which network addresses (prefixes) the WAN Edge device can reach.

The Overlay Management Protocol (OMP) is responsible for sending these route updates to the vSmart controllers. OMP can advertise different types of routes, including routes that are directly connected to the WAN Edge device, static routes that are manually configured, and routes learned from traditional routing protocols like OSPF, EIGRP, and BGP.

In addition to the reachability information, several attributes are also shared when advertising routes. These attributes include:

• TLOC (Transport Locator)

• Origin

• Originator

• Preference

• Service

• Site ID

• Tag

TLOC (Transport Location)

The TLOC identifier represents the next hop for the OMP route. It is similar to the BGP_NEXT_HOP attribute in BGP routing.

Within the TLOC, there are three values:

• System IP Address: This is like a unique identifier for the WAN Edge device. It doesn't necessarily need to be a routable IP address but must be unique across all WAN Edges. The system IP address helps identify the original advertising WAN Edge device.

• Color: This attribute is further explored in the data plane section. It is used to mark a specific WAN connection and can later be utilized to influence policies and topology construction.

• Encapsulation Type: This value indicates the type of encapsulation used for the data plane tunnel, such as IPsec or GRE. It is advertised to specify the type of tunneling protocol being employed.

Origin

The origin attribute specifies the source of the route. As the route is advertised in the routing domain, the original source of the route is included in the update.

The source may indicate an identifier like BGP, OSPF, EIGRP, Connected, or Static, along with the protocol's original metric.

The origin attribute is also considered in the best-path selection for OMP routes. Similar to other attributes, it can be configured to influence how this information is used in policies and routing decisions.

Originator

The Originator attribute identifies the original source from which the route was learned. It is represented by the system IP address of the device that advertised the route. The network administrator can use this attribute to create policies that consider the route's origin.

Preference

The Preference attribute, sometimes called OMP Preference (not to be confused with TLOC Preference), can be modified to influence the criteria for selecting the best path for a route. Routes with higher preference values are preferred over those with lower values. This attribute operates similarly to LOCAL_PREF in BGP (Border Gateway Protocol).

Tips: With alargeromp preference value, thehigherpriority the prefix gets

Service

The Cisco SD-WAN solution supports service insertion, such as firewalls. If a service is associated with a route, it is indicated in the Service attribute. Further details about service routes will be discussed later in the chapter.

Site ID

The Site ID attribute is similar to a BGP autonomous system number (ASN). It is used for policy orchestration and to influence routing decisions. Each site in the network should have a unique Site ID. If there are multiple devices at a site, they should share the same Site ID to prevent routing loops.

Tag

The Tag attribute is optional and transitive. It can be applied to a route by an OMP peer and used in policies. However, when redistributing to or from OMP, the tag is not carried. This attribute functions similarly to a route tag in traditional routing protocols.

VPN

The VPN attribute identifies the specific VPN or VRF (Virtual Routing and Forwarding) from which the route was advertised. VPN tags allow the use of overlapping subnets as long as they belong to different VPNs or VRFs. This enables logical segmentation of the network into multiple data paths and separate routing instances per VPN or VRF. Note that in the Cisco SD-WAN solution, VPNs and VRFs are used interchangeably.

Read more: Cisco SDWAN VPN Segmentation or VRF?

Example of OMP Routes

The below example is the output of the OMP Route which include mentioned attributes and much other information.

vEdge11# show omp routes vpn 100

---------------------------------------------------
omp route entries for vpn 100 route 192.168.10.0/24
---------------------------------------------------
RECEIVED FROM:
peer            0.0.0.0
path-id         69
label           1002
status          C,Red,R
loss-reason     not set
lost-to-peer    not set
lost-to-path-id not set
    Attributes:
     originator       1.1.1.11
     type             installed
     tloc             1.1.1.11, public-internet, ipsec
     ultimate-tloc    not set
     domain-id        not set
     overlay-id        1
     site-id          10
     preference       not set
     tag              not set
     origin-proto     connected
     origin-metric    0
     as-path          not set
     unknown-attr-len not set

ADVERTISED TO:
peer    9.9.9.30

Above 192.168.10.0/24 is a directly connected network belonging to the internal network of vEdge11, so look at the Attributes originator above, it is the system-ip of vEdge11 (1.1.1.11)

Also, at the ADVERTISED TO: peer9.9.9.30 (vSmart system-ip), it shows this prefix has been distributed to vSmart1 mentioned above.

Regarding the status of prefix 192.168.10.0/24 as output above, (C, Red, R):

• C: chosen, it means the prefix chosen as the best path

• Red: redistributed, means the prefix is redistributed from the IGP protocol, connected, ... into OMP. (In this case, look at the origin-proto, the value is connected, and it shows the origin protocol for this prefix is directly connected)

• R: resolved, means tloc-nexthop available and reachability. (valid)

---------------------------------------------------
omp route entries for vpn 100 route 192.168.40.0/24
---------------------------------------------------

RECEIVED FROM:
peer            9.9.9.30
path-id         291
label           1002
status          C,I,R
loss-reason     not set
lost-to-peer    not set
lost-to-path-id not set
    Attributes:
     originator       1.1.1.40
     type             installed
     tloc             1.1.1.40, public-internet, ipsec
     ultimate-tloc    not set
     domain-id        not set
     overlay-id        1
     site-id          40
     preference       not set
     tag              not set
     origin-proto     connected
     origin-metric    0
     as-path          not set
     unknown-attr-len not set

Above 192.168.40.0/24 is the network prefix that vEdge11 received from vSmart (9.9.9.30), and you can see the originator is 1.1.1.40, the system-ip of cEdge41. Based on it, you can know this prefix comes from the service side of cEdge41.

In addition, the prefix 192.168.40.0/24 contains other attributes such as site-id 40 which tells us that cEdge41's site is 40, and TLOC information is mentioned in the above section. (system-ip, color, encapsulation type).

Regarding the status of prefix 192.168.40.0/24 as output above, (C, I, R):

• I: installed, it means the prefix has been installed into the routing table.

2. TLOC Route

The TLOC (Transport Location Identifier) routes play a crucial role in identifying the physical location of a device within the transport network. These routes provide addressing that is routable in the underlying network infrastructure and serve as the endpoints for the data plane tunnels.

Simply, it is called the "Next-Hop of OMP Routes" and "WAN Edge's Site-to-Site VPN endpoint".

By using the TLOC with system IP addresses, the SD-WAN solution can maintain consistent and easily identifiable endpoints for data plane tunnels, even if IP addresses change dynamically.

A TLOC consists of three attributes explained above

• System IP address

• Transport color

• Encapsulation type

If a WAN Edge device has multiple transports or interfaces, a separate TLOC route will be advertised for each interface to ensure proper routing and connectivity.

TLOC Color

The Color attribute of the TLOC route is crucial in identifying the transport being used. Each transport should ideally have a different color assigned to it. The Color attribute allows policies to be constructed to influence how the data plane is built within the SD-WAN network.

There are 22 predefined colors available to choose from, and they also define whether the underlying transport is private or public. This distinction determines the IP address that should be used when establishing a data plane tunnel to a remote site.

Note that the default color is not clarified in public or private color.

If no color is explicitly defined, the color "default" is used

By default, WAN Edge devices will attempt to build data plane tunnels to every other site using every available color. (refer to the below Figure)

However, this may lead to inefficient routing, such as MPLS sites attempting to build tunnels to public Internet sites unintentionally.

To control this behavior, the "restrict" command and/or tunnel groups can be used. These mechanisms allow for fine-tuning of the data plane connectivity and routing decisions. Further details about this will be discussed in the data plane articles later.

When a TLOC route is advertised, it contains the following information:

• TLOC private address: This attribute represents the private IP address derived from the physical interface of the WAN Edge device.

• TLOC public address: The WAN Edge device receives notification via STUN (Session Traversal Utilities for NAT) that it may be behind a NAT (Network Address Translation) device. This attribute contains the publicly routable or outside IP address assigned to the WAN Edge, enabling data plane connectivity across a NAT boundary. If the public and private addresses match in a TLOC route, it indicates that the device is not behind a NAT.

• Color: This attribute corresponds to the defined color of the transport being used. The list of colors is mentioned in the above table. If no color is explicitly defined, the color "default" is used.

• Encapsulation type: This attribute specifies the type of tunnel encapsulation used for the data plane. The available options are IPsec and GRE. Both sides of the tunnel must match in terms of the encapsulation type for data plane connectivity.

• Preference: Similar to OMP Preference, this attribute allows the network administrator to indicate a preference for one TLOC over another when comparing the same OMP route.

• Site ID: This value identifies the originator of the TLOC route and is used to control how data plane tunnels are established.

• Tag: Similar to route tags and OMP tags, this attribute allows the definition of a value that can control how prefixes are exchanged and influence the flow of traffic.

• Weight: This attribute functions similarly to BGP Weight. It is a path selection method, locally significant, and indicates a preference for one path over another. A higher weight value is preferred over a lower one.

These attributes provide detailed information and control over the TLOC routes, allowing for efficient data plane connectivity and routing decisions within the SD-WAN network.

3. Service Routes

Service routes are used to advertise specific services within the SD-WAN overlay network. Service chaining policies can then be applied to direct data traffic through one or more of these services before reaching its original destination.

Service chaining is beneficial when certain data traffic needs to pass through additional security or optimization services, such as firewalls, load balancers, or IDPs (Intrusion Detection and Prevention systems). These services can be applied on a per-VPN basis, allowing for granular control and flexibility.

For example, network services will be deployed according to the Hub-Spokes scheme, and services are connected at the central site (Hub) so that traffic from other sites is rerouted to the network service site (like Firewall) and then continues to forward to the original destination.

To enable service chaining in the overlay network, the following workflow is typically followed:

• The network administrator defines the service using a feature template. This template outlines the configuration settings and parameters for the specific service.

• WAN Edge routers within the network advertise the availability of these services to the central vSmart controllers. Multiple WAN Edge routers can advertise the same service for redundancy purposes.

• In addition to service advertisements, WAN Edge routers also advertise their OMP (Overlay Management Protocol) and TLOC routes to provide information about reachability and routing.

• The network administrator then applies a policy that specifies which traffic should flow through the advertised service(s). This policy ensures that the identified traffic is processed by the designated service(s) before being forwarded to its final destination.

It's important to note that for service chaining to work, devices providing the services must be Layer 2 adjacent to the WAN Edge device.

This means there should be no intermediate hops between the WAN Edge device and the service device. Layer 2 adjacency can be achieved using IPsec or GRE tunnels, which facilitate the required connectivity.

In order to offer a service within the SD-WAN network, the site serving as the hub or service provider will advertise a service route using a Subsequent Address Family Identifier (SAFI) in the OMP (Overlay Management Protocol) Network Layer Reachability Information (NLRI). This information is then communicated to the vSmart controller and propagated to the WAN Edge devices. The service route update contains the following details:

• VPN ID: This attribute specifies the VPN (Virtual Private Network) to which the service applies. It associates the service with a specific VPN within the SD-WAN network.

• Service ID: The Service ID defines the type of service being advertised. There are seven predefined service types available:

  • FW: Represents a firewall service (mapped to svc-id 1). (above example)

  • IDS: Represents an intrusion detection system service (mapped to svc-id 2).

  • IDP: Represents an Identity Provider service (mapped to svc-id 3).

  • netsvc1, netsvc2, netsvc3, and netsvc4: These service types are reserved for custom services and correspond to the service values of svc-id 4, svc-id 5, svc-id 6, and svc-id 7, respectively.

• Label: OMP routes that require traffic to flow through this service will have their Label field replaced with this specific label. It helps in identifying the service associated with the route.

• Originator ID: This attribute represents the system IP address of the node or device advertising the service. It identifies the source of the service advertisement.

• TLOC: The Transport Location address specifies where the service is located within the network. It identifies the physical location or endpoint associated with the service.

• Path ID: The Path ID serves as an identifier for the OMP path. It helps in distinguishing and tracking different paths within the SD-WAN network.

Configure Service Routes

Following the above example diagram, the service route will be configured on vEdge3 at site 3.

## vEdge3 ###
vpn 10
  service FW address 3.3.3.3
!

To enforce and drive the traffic between site 1 (LAN-A) and site 2 (LAN-B) through the Firewall (Services) at site 3, the Centralized Policy has to be done via vSmart.

policy
  lists 
    site-list BRANCHES
      site-id 1, 2
  !
  control-policy FIREWALL-SERVICE
    sequence 10
      match route
        site-id 3
      action accept
        set service FW vpn 10
    default-action accept
  !
apply-policy
  site-list BRANCHES control-policy FIREWALL-SERVICE out
!

Tips:

VPN 10 associates the Firewall service with a specific VPN10 within the fabric.

Verify the Service Routes

In order to verify the service route is configured correctly, use "show omp services" on Viptela OS or "show sdwan omp services" on IOS-XE.

vEdge3# show omp services

ADDRESS                                            PATH
FAMILY   VPN    SERVICE   ORIGINATOR   FROM PEER   ID    LABEL  STATUS
--------------------------------------------------------------------
ipv4     10     FW        3.3.3.31      9.9.9.30    52   1006   C,I,R

WRAP-UP

Through the article, we learn about Overlay Management Protocol that is used in the Cisco SDWAN Control Plane.

There are three kinds of routes in OMP:

• OMP Routes, like BGP prefixes.

• TLOC Routes provide addressing that is routable in the underlying network infrastructure and serve as the endpoints for the data plane tunnels.

• Service Routes are used to advertise specific services and the policies can then be applied to direct data traffic through one or more of these services before reaching its original destination.

My name is Nam who loves to talk and share knowledge related to Networking, Automation, and so on. More about me: www.nam-nguyen.me

Hope you enjoy the blog and don't forget to join the Tech-Learner-Hub to get more and more valuable content.

Get the Cisco SD-WAN Zero-to-One ebook

As discussed in the previous part "Cisco SDWAN Planes", The control plane in Cisco SD-WAN is responsible for establishing and maintaining logical connectivity and intelligence across the network. It encompasses the control plane protocols and functions that enable the exchange of routing information and the orchestration of data traffic flow.

In this article, we will be on the SD-WAN control plane and how OMP facilitates building the control plane.

Control Plane Operations

In the Cisco SD-WAN solution, the Overlay Management Protocol (OMP) is used to manage control plane operations. OMP enables a secure and scalable framework that works across various types of transport, including private options like MPLS, Layer 2 VPNs, and point-to-point networks, as well as public connectivity methods like the Internet and LTE.

The vSmart controller is responsible for handling the control plane. It ensures a scalable control plane infrastructure and distributes policy information to the WAN Edges.

To better understand its role, the vSmart controller can be compared to a BGP route reflector. It receives routing and topology information from the clients, performs calculations based on configured policies to determine the best paths, and then advertises these results to the WAN Edges, which act as route reflector clients.

In traditional networks, the control plane's primary focus was on managing how data flows within the network. This involved receiving routing updates, performing operations to determine the best paths, and using this information to populate forwarding tables.

However, configuring security using these protocols was often complex and time-consuming. It typically required manual effort and often resulted in network downtime during the transition to security mechanisms.

Security is a fundamental aspect of the Cisco SD-WAN solution. To ensure secure communication, the control plane tunnels in the SD-WAN overlay are encrypted and authenticated using Datagram Transport Layer Security (DTLS) or Transport Layer Security (TLS). In the SD-WAN overlay, all personas including vBond, vSmart, WAN Edges, and vManage maintain DTLS/TLS connections.

This ensures that all routing updates are validated and trusted to prevent the processing of any malicious routing information.

These connections are established using SSL certificates. Each component in the network authenticates the other end and creates a one-way tunnel. During the negotiation process, devices validate that the received certificate is signed by a trusted root Certificate Authority (CA) and has a valid serial number with a matching organization name. This ensures the authenticity and integrity of the communication. You can refer to the above illustration of a tunnel between a WAN Edge and vSmart controller.

The default protocol for communication is DTLS (Datagram Transport Layer Security).

DTLS communication takes place over UDP port 12346. It is recommended to keep this port open for communication between vBond and all WAN Edges.
Additionally, TLS (Transport Layer Security) is also supported if specific requirements demand it. It's important to note that TLS operates using the TCP protocol and is therefore stateful.

The vSmart and vManage components are deployed as virtual machines capable of supporting multiple cores, up to a maximum of eight cores. Each core is associated with a base port. When inbound DTLS/TLS connections are established, they initially target port 12346.
You can refer to the below figure of a DTLS Tunnel Authentication between vSmart and vBond.

Once the control plane tunnels are established, various protocols can utilize these secure sessions. In addition to OMP (Overlay Management Protocol), protocols like Simple Network Management Protocol (SNMP) and Netconf can also leverage these secure channels. By utilizing the established DTLS/TLS tunnels, we no longer need to worry about the individual security implementations of these protocols or any vulnerabilities they may have.

Overlay Management Protocol (OMP)

In the Cisco SD-WAN solution, the Overlay Management Protocol (OMP) serves as the routing protocol. However, OMP goes beyond just routing and provides several essential services within the control plane:

• Facilitation of network communication: OMP enables data plane connectivity between sites in the SD-WAN fabric, including service chaining and multi-VPN topology information.

• Distribution of data plane security information: OMP handles the distribution of encryption keys, ensuring secure communication within the fabric.

• Best-path selection and routing policy advertisement: OMP determines the optimal paths for data traffic and communicates routing policies across the network.

Read more: Cisco SDWAN vSmart Controllers

OMP is enabled by default in the SD-WAN solution and does not require explicit activation. As components in the fabric become aware of their control elements, they automatically establish control connections. This allows for reachability and orchestration of the network topology.

OMP is designed to interact with legacy routing protocols, including static routes and traditional interior gateway protocols such as OSPF, BGP, and EIGRP. However, unlike traditional IGPs, OMP peering occurs only between the WAN Edges and the vSmart controller(s). This peering model resembles the operation of a BGP route reflector within an Internal Border Gateway Protocol (IBGP) domain. This approach is beneficial for scalability, as it reduces CPU load on data plane devices by minimizing excessive routing updates and best-path recalculations.

OMP Graceful Restart

OMP in the Cisco SD-WAN solution also supports graceful restart functionality. Graceful restart allows WAN Edges to cache forwarding information in case they lose connectivity to the vSmart controllers. In such situations, the WAN Edge will continue using the last received routing information to maintain proper forwarding.

By default, graceful restart is enabled on both vSmart controllers and WAN Edge routers, with a default timer set to 12 hours. This timer can be adjusted within a range of 1 second to 7 days.

It's important to ensure that a valid IPsec encryption key is available during the entire graceful restart period. Otherwise, there is a risk of data plane tunnels being terminated when the graceful timer expires. To prevent IPsec rekey while OMP is down, it is recommended to set the IPsec rekey timer to twice the value of the graceful restart timer as a best practice.

Note:

The configuration of the graceful restart timer can be done through vManage using a CLI template or an OMP feature template. Further details on feature templates will be discussed in the next parts.

When a peering session with the vSmart controller becomes unavailable, the WAN Edge continuously tries to re-establish the connection. However, if the WAN Edge is reloaded, the cached information is lost.

In such cases, the WAN Edge will need to establish a new OMP session with the vSmart controller and receive updated forwarding information before it can resume forwarding traffic on the SD-WAN fabric.

Cisco SDWAN Type of Routes Overview

OMP in the Cisco SD-WAN solution is responsible for advertising different types of routes between the vSmart controllers and WAN Edge routers. These routes include:

1. OMP routes (vRoutes): These routes represent network prefixes that enable connectivity services for data centers, branch offices, or any other endpoint within the SD-WAN fabric.

2. Transport locations (TLOCs): TLOCs serve as identifiers that associate an OMP route with a physical location. They are the only IP addresses that are known and reachable within the underlying network.

3. Service routes: Service routes identify network services within the SD-WAN overlay. These routes indicate the physical location of services such as firewalls, IPS, IDS, or any other device capable of processing network traffic. Service information is advertised through service routes and OMP routes.

My name is Nam who loves to talk and share knowledge related to Networking, Automation, and so on. More about me: www.nam-nguyen.me

Hope you enjoy the blog and don't forget to join the Tech-Learner-Hub to get more and more valuable content.

Get the Cisco SD-WAN Zero-to-One ebook

In the world of software-defined wide-area networking (SD-WAN), the control plane plays a crucial role in managing network intelligence and ensuring seamless connectivity. One of the key components responsible for control plane functionality is the vSmart controller.

In this article, we will delve into the details of vSmart and its role in the Cisco SD-WAN fabric, exploring its capabilities, benefits, and operational mechanisms.

The Brain of the SD-WAN Fabric

vSmart acts as the central intelligence hub of the SD-WAN fabric, providing control plane services to orchestrate network operations. Its highly scalable architecture allows it to handle up to 5,400 connections per vSmart server, making it suitable for large-scale deployments.

With vSmart, organizations can efficiently implement control plane policies, centralized data policies, service chaining, and VPN topologies, while ensuring robust security and encryption through key management.

Simplifying Network Operations

By separating the control plane from the data and management planes, the Cisco SD-WAN solution achieves greater scalability and simplifies network operations.

Unlike traditional routing protocols, if you look at traditional link states routing protocols such as OSPF and IS-IS, each router knows about the state of the whole network and calculates its own routing table based on the link state database information.

This can be very CPU intensive and offers only a limited, autonomous view of the network. vSmart enables comprehensive network state visibility for efficient path calculations and reduced complexity. This ensures optimal routing decisions across the network.

vSmart leverages the Overlay Management Protocol (OMP) to communicate and manage network information. OMP goes beyond routing and encompasses key management, configuration updates, and more. It runs between vSmart and the WAN Edges within a secure tunnel. Policies built through the management plane are distributed to vSmart via NETCONF, and vSmart disseminates these policies to the WAN Edges through OMP updates.

OMP is the main exchange control information protocol using in Cisco SDWAN control plane.
The separated detailed article about OMP will come later.

Intelligent Routing and Topology Management

Similar to a BGP route reflector in iBGP, vSmart receives routing information from each WAN Edge and applies policies before advertising the information to other WAN Edges. vSmart defines different topologies per VPN, allowing flexible modification of routes and data plane construction.

This centralization of control enables efficient management of the network's routing and topology, ensuring optimized performance and reduced complexity.

The Cisco SDWAN Policies will be explain detail in the next parts.

Enhanced Security and Key Management

The control plane is responsible for fabric encryption, and vSmart plays a crucial role in ensuring secure communication. In legacy WAN technologies, each device computed its own encryption keys and distributed them using protocols like ISAKMP/IKE.

In Cisco SD-WAN, key exchange and distribution are centralized in vSmart.

WAN Edges compute transport-specific keys, which are then distributed and rekeyed by vSmart. This approach increases scalability and simplifies key management.

High Availability and Redundancy

To ensure network stability, it is recommended to deploy at least two geographically dispersed vSmart controllers. These controllers should have identical policy configurations to avoid routing issues and potential traffic blackholing.

vSmart controllers maintain a full mesh of OMP sessions among themselves, allowing synchronization of control and routing information. In case of vSmart controller failure, control connections from WAN Edges are dynamically rebalanced across the remaining controllers.

Key Takeaways

The vSmart controller in Cisco SD-WAN acts as the brain of the network, orchestrating control plane operations, optimizing routing decisions, and ensuring secure communication. By leveraging the power of OMP and centralized control, organizations can achieve scalable, intelligent, and highly available networks.

vSmart Controller

• The brain of the Cisco SDWAN Network

• Centralized Routing, Topology control (like iBGP reflector)

• Centralized encryption key distribution

• Recommend to deploy with geographical redundancy.

My name is Nam who loves to talk and share knowledge related to Networking, Automation, and so on. More about me: www.nam-nguyen.me

Hope you enjoy the blog and don't forget to join the Tech-Learner-Hub to get more and more valuable content.

Get the Cisco SD-WAN Zero-to-One ebook

In a software-defined wide area network (SD-WAN) architecture, the vBond controller plays a critical role in orchestrating the connectivity and authentication between various elements within the SD-WAN fabric.

Understanding the purpose and functionality of the vBond controller is essential to grasp the inner workings of Cisco SD-WAN. In this article, we will delve into the intricacies of vBond, its role within the SD-WAN fabric, and provide a detailed explanation of how it operates.

What is vBond Controller?

At its core, the vBond controller is a centralized orchestration component within the Cisco SD-WAN architecture.

It acts as the initial point of contact for SD-WAN devices, facilitating secure authentication and bootstrapping of the network elements.

The vBond controller establishes secure tunnels (DTLS/TLS) with each SD-WAN device in the fabric, ensuring secure communication and enabling the establishment of control and data plane connections.

Read more: Cisco SDWAN Planes

Role of vBond Controller

The vBond controller fulfills several vital roles within the SD-WAN fabric, including:

• Authentication and Identity Management

• NAT-T Support

• Device Bootstrapping

• Overlay Network Establishment

Authentication and Identity Management

vBond authenticates SD-WAN devices and ensures their identity before allowing them to join the SD-WAN fabric. It verifies device certificates, authorizes device enrollment, and securely distributes control plane information.

When an SD-WAN device boots up, it initiates contact with the vBond controller. The vBond controller verifies the device's identity, checks its certificate, and authorizes its enrollment into the SD-WAN fabric.

Once authenticated, the vBond controller and the SD-WAN device establish a secure DTLS (Datagram Transport Layer Security) tunnel. This secure tunnel is used for control plane communication between the vBond controller and the SD-WAN device.

NAT-T Support

In Cisco SD-WAN, the vBond controller plays a crucial role in handling Network Address Translation (NAT) within the fabric.

When a WAN Edge router connects to the vBond controller, it includes its real IP address (Private IP Address) and Port to the exchange information. If the router is behind a NAT device, the NAT device will translate the source IP and possibly the source port of the packet.

Since the message still contains the WAN Edge’s real IP Address and Port, the vBond controller is aware of this translation and sends a message back to the WAN Edge router, notifying it that it is behind a NAT.

The WAN Edge router updates its OMP TLOC route with this information and sends it to the vSmart controller. This information is then shared with other WAN Edge routers in the overlay network.

OMP - Overlay Management Protocol, which is used to exchange routing and encryption key information in Cisco SDWAN between vSmarts and WAN Edges.

No worry, the detailed explanation will come in later posts.

By exchanging this NAT information, all WAN Edge routers in the fabric can adapt their data plane accordingly. They will use the correct IP and port values to establish communication, even when they are behind a NAT device.

WAN Edges devices receive other Edge's information (Public IP, Private IP, NAT type, etc.), and use this information to establish a secure connection with each other (IPSec).

That's why they need to know the peer WAN Edge is behind the NAT device or not.

Device Bootstrapping

Device bootstrapping is a crucial aspect of the vBond controller's role in the SD-WAN fabric. During the bootstrapping process, the vBond controller delivers essential configuration parameters and establishes the initial connectivity for SD-WAN devices.

This process enables the devices to join the SD-WAN fabric and participate in the overlay network.

It simplifies the deployment process of SD-WAN devices and automates the provisioning of devices by eliminating the need for manual configuration.

This reduces the potential for human error and streamlines the overall deployment process, saving time and effort for network administrators.

The detailed onboarding WAN Edges with vBond bootstraps will be talked about in later specific posts.

Overlay Network Establishment

Through control plane messaging, vBond establishes secure control plane tunnels with SD-WAN devices.

Through the secure tunnel, the vBond controller provides critical control plane information to the SD-WAN device, including the IP addresses of other control plane elements such as vSmart controllers and vManage.

Read more: Cisco SDWAN vManage and Cluster

Key Takeaway

The vBond controller serves as a fundamental component of the Cisco SD-WAN fabric, enabling secure authentication, bootstrapping, and orchestration of SD-WAN devices.

My name is Nam who loves to talk and share knowledge related to Networking, Automation, and so on. More about me: www.nam-nguyen.me

Hope you enjoy the blog and don't forget to join the Tech-Learner-Hub to get more and more valuable content.

Get the Cisco SD-WAN Zero-to-One ebook

In today's interconnected digital landscape, secure access to resources and protected data is crucial. OAuth and OAuth 2.0 have emerged as industry-standard protocols for enabling secure authentication and authorization across various applications and services. In this blog post, we will explore what OAuth is, delve into the workings of OAuth 2.0, and discuss its key use cases with real-world examples.

What is OAuth?

OAuth, which stands for "Open Authorization," is an open standard protocol that allows users to grant limited access to their protected resources (such as data or services) to other applications without revealing their credentials. 1.2 Key Concepts in OAuth:

• Resource Owner: The user who owns the protected resource and grants access to it.

• Client: The application or service requesting access to the protected resource on behalf of the user.

• Resource Server: The server hosting the protected resource.

• Authorization Server: The server responsible for authenticating the user and issuing access tokens.

What is OAuth?

OAuth 2.0 is an evolution of the OAuth protocol and is widely adopted due to its simplicity and enhanced security features. 2.2 OAuth 2.0 Roles and Flow:

• Client Registration: The client application registers with the authorization server to obtain client credentials.

• Authorization Code Flow: The most common flow involves the exchange of an authorization code for an access token.

• Implicit Grant Flow: A simplified flow for browser-based or mobile applications.

• Client Credentials Flow: Suitable for machine-to-machine communication.

• Refresh Token Flow: Enables the renewal of access tokens without requiring re-authorization.

• Scopes and Permissions: Define the level of access granted to the client application.

• Access Token Usage: How to access tokens are utilized to access protected resources.

Use Cases

Single Sign-On (SSO)

• Simplify the user experience by utilizing OAuth 2.0 for seamless authentication across multiple applications.

• For instance, users can authenticate themselves through a social media account and gain access to various integrated applications without the need to provide separate login credentials for each application.

Third-Party Application Integration

• Enable seamless integration between different platforms by leveraging OAuth 2.0.

• For instance, a fitness tracking app can securely access workout data from a health service provider's platform using OAuth 2.0 authentication and authorization mechanisms. This allows users to track their fitness activities in a centralized app without the need to manually enter data from multiple sources.

API Authorization and Access Control

• Empower third-party applications to interact with user data on a cloud storage platform securely.

• With OAuth 2.0, specific permissions can be granted to third-party applications, ensuring controlled access to user data.

• For example, a cloud storage platform may allow a document editing application to access and modify files while restricting access to sensitive user information.

Mobile Application Authentication

• Enhance the security of mobile applications by implementing OAuth 2.0 for authentication.

• For instance, a mobile banking app can utilize OAuth 2.0 to authenticate users securely, granting them access to their account information without the need to store sensitive credentials on the device. This ensures that users' financial data remains protected even if their mobile device is lost or stolen

These are just a few examples of how OAuth 2.0 is utilized in various scenarios to enhance security, streamline user experiences, and enable secure integration between different applications and platforms.

OAuth 2.0's flexibility and robustness make it a powerful protocol for ensuring secure authentication and authorization in today's interconnected digital landscape.

Conclusion

OAuth and OAuth 2.0 play a significant role in enhancing security and simplifying the authorization process for users, applications, and services in today's interconnected world. By understanding the fundamentals of OAuth, the workings of OAuth 2.0, and exploring its use cases with real-world examples, we can leverage these protocols to create secure and seamless user experiences while maintaining control over access to valuable resources.

So, whether you're a developer, a system administrator, or an end-user, embracing OAuth and OAuth 2.0 can empower you with robust security measures and convenient authorization mechanisms.