Author: Tayo Ogunseyinde, Consultant Systems Engineer (EMEA)
This article describes an advanced use case. It assumes basic knowledge of the Versa architecture and is intended for use by experienced Versa Network engineers.
As enterprises transition from traditional MPLS networks to modern SDWAN overlay networks, the requirement to integrate with specialist third party security tools remains. SDWAN introduces a further dimension due to its inherent segmentation of the overlay from the underlay. SDWAN must be able to receive signals from external sources to protect both the overlay and /or underlay from attack.
Versa Secure SDWAN can achieve Remote Triggered Black Hole (RTBH) in the following ways.
This article explores the latter option using the BGP control plane to automate the remote triggering of a prefix black hole.
In the example above, prefix 111.111.111.1/32 is identified by an external system (‘External Trigger’) for blackholing. The prefix is then advertised via BGP to ‘Trigger Branch’. Trigger Branch onward advertises the prefix to all other ‘Remote Branches’. As a consequence, any traffic destined to 111.111.111.1/32 is instantly dropped by all remote branches in the network.
Blackholing on the remote branch is achieved by drawing the triggered prefix into a separate virtual routing instance (SINK VR) where the traffic is discarded. The concept can be leveraged to onramp traffic from the LAN-VR into a separate VR to be scrubbed or treated before returning to the LAN. The onramp use case is out of scope of this article.
Note: This article assumes a good understanding of the customer’s own network in the design and implementation of the external trigger feed. The details of such external trigger feed is out of scope of this article.
A virtual routing instance (SINK-VR) is created on each remote branch participating in the prefix black holing. The LAN-VR (CCTV-LAN in the example above) is the enterprise routing table. The route target from the CCTV-LAN is imported into both the SINK-VR and the CCTV-LAN-VR at the remote branch. Thus, routes in the CCTV-LAN-VR are imported into the Sink-VR. Further filtering is applied to the sink VR to ensure that only triggered routes are installed into the SINK-VR routing table. This will save system resources from duplicating all routes into the SINK-VR.
A paired tvi (ptvi) is created between the SINK-VR and the CCTV-LAN-VR. eBGP is enabled over the ptvi. The triggered prefix is learnt via a BGP update with a specific community tag (110 in this example). Routes matching this community are advertised with a preferred metric into the CCTV-LAN-VR over the ptvi.
Traffic once in the SINK-VR is dropped via security policy.
Workflow > Infrastructure > Organizations > Routing Instances
VPN must be set to true. This step auto generates the route distinguisher and route targets for each routing instance. In the example above, target:5L:5 belongs to CCTV-LAN-VR and target:7L:7 belong to the SINK-VR
This article describes the simplest approach using the workflow template to autogenerate this configuration. Manually creating a routing-instance with a split tunnel is a multistep activity with several config touchpoints.
Configuration > Objects > WAN Networks
This is created as a WAN Network (instead of a LAN Network) since the workflow wizard for ptvi (split tunnel) can only be created between the WAN and LAN.
An unused dummy interface should be used in the workflow to enable the creation of this WAN Routing Instance.
This step auto generates configuration for the ptvi and ebgp.
RemoteBranch > Virtual Routers > Sink-VR
Remote Branch > Virtual Routers > SINK-VR > BGP (Instance id) > Peer Policy > SINK ROUTES.
Virtual Routers > SINK-VR > BGP (Instance id) > Peer Group > ST-Group (e.g. Copy_of_TO_CCTV-LAN) > Advanced
Traffic may be discarded using SDWAN policies, QoS policies or null routes. All of these are viable alternatives. The benefit of using the stateful firewall to achieve this includes the visibility in analytics and potential flexibility in defining match criteria for the traffic based on Zone, application etc.
Remote Branch > Virtual Routers CCTV-LAN-VR > Redistribution Policy
The routing instance redistribution policy is used because the community tag must be carried via the control-VR to the remote branches.
Next-hop for prefix is SDWAN in the CCTV-LAN directly towards a remote gateway.
A BGP feed with the community 110 is injected from the external source and distributed to the remote branches. The Remote branches will then prefer next-hop via the SINK VR where the traffic is blackholed by Security policy.
Session output for showing traffic punted into the SINK VR and dropped by policy.
Remotely triggered Black Hole (RTBH) is not a new concept, it is traditionally used within a single routing table on traditional routers. The approach described in this article is suited for the inherent internal segmentation of the Versa architecture. This article has demonstrated the remote trigger of a prefix under attack and instantly causes all nodes in the network to back hole the specific prefix only.
This article describes an advanced use case. It assumes basic knowledge of the Versa architecture and is intended for use by experienced Versa Network engineers.
Overview
Networks often have a requirement to respond in a swift and coordinated way to security threats by blackholing the attack traffic. External systems may be used to analyze and detect these threats, and signal to the network to enforce some mitigation action. One type of action may be to blackhole or sink attack traffic destined to some prefix in the network.As enterprises transition from traditional MPLS networks to modern SDWAN overlay networks, the requirement to integrate with specialist third party security tools remains. SDWAN introduces a further dimension due to its inherent segmentation of the overlay from the underlay. SDWAN must be able to receive signals from external sources to protect both the overlay and /or underlay from attack.
Versa Secure SDWAN can achieve Remote Triggered Black Hole (RTBH) in the following ways.
- Using the Versa Director to push configuration updates to the remote branches. This is suited for scenarios where there is an orchestration layer API feed to push configuration on demand. Alternatively, tools such as the Versa VSync tool can be used to distribute threat feed from external sources to the Versa SDWAN appliances.
- A trigger based on a control plane update feed. This is ideal where the feed comes from traditional network components using networking protocols such as BGP. BGP is used as a signaling protocol to all edge routers. It can identify specific prefixes under attack and signal the type of mitigation that should be enforced.
This article explores the latter option using the BGP control plane to automate the remote triggering of a prefix black hole.
Solution Architecture
A security threat feed is triggered from an external source via BGP towards a trigger branch. A trigger branch is a VOS branch that receives the BGP trigger and initiates the route distribution to the remote branches. The prefix black holing action occurs on the remote branch.
In the example above, prefix 111.111.111.1/32 is identified by an external system (‘External Trigger’) for blackholing. The prefix is then advertised via BGP to ‘Trigger Branch’. Trigger Branch onward advertises the prefix to all other ‘Remote Branches’. As a consequence, any traffic destined to 111.111.111.1/32 is instantly dropped by all remote branches in the network.
Blackholing on the remote branch is achieved by drawing the triggered prefix into a separate virtual routing instance (SINK VR) where the traffic is discarded. The concept can be leveraged to onramp traffic from the LAN-VR into a separate VR to be scrubbed or treated before returning to the LAN. The onramp use case is out of scope of this article.
Note: This article assumes a good understanding of the customer’s own network in the design and implementation of the external trigger feed. The details of such external trigger feed is out of scope of this article.
A virtual routing instance (SINK-VR) is created on each remote branch participating in the prefix black holing. The LAN-VR (CCTV-LAN in the example above) is the enterprise routing table. The route target from the CCTV-LAN is imported into both the SINK-VR and the CCTV-LAN-VR at the remote branch. Thus, routes in the CCTV-LAN-VR are imported into the Sink-VR. Further filtering is applied to the sink VR to ensure that only triggered routes are installed into the SINK-VR routing table. This will save system resources from duplicating all routes into the SINK-VR.
A paired tvi (ptvi) is created between the SINK-VR and the CCTV-LAN-VR. eBGP is enabled over the ptvi. The triggered prefix is learnt via a BGP update with a specific community tag (110 in this example). Routes matching this community are advertised with a preferred metric into the CCTV-LAN-VR over the ptvi.
Traffic once in the SINK-VR is dropped via security policy.
Organization Configurations
Step 1: Create new Org
Workflow > Infrastructure > Organizations > Routing Instances
VPN must be set to true. This step auto generates the route distinguisher and route targets for each routing instance. In the example above, target:5L:5 belongs to CCTV-LAN-VR and target:7L:7 belong to the SINK-VR
Remote Branch Configuration
This article describes the simplest approach using the workflow template to autogenerate this configuration. Manually creating a routing-instance with a split tunnel is a multistep activity with several config touchpoints.
Step 1: Create SINK-VR as a WAN network.
Configuration > Objects > WAN Networks
This is created as a WAN Network (instead of a LAN Network) since the workflow wizard for ptvi (split tunnel) can only be created between the WAN and LAN.
Step 2: Add SINK VR as a WAN Network.
An unused dummy interface should be used in the workflow to enable the creation of this WAN Routing Instance.
Step3: Add SINK VR as a split tunnel to CCTV-LAN
Note DIA and Gateway should be unchecked.
This step auto generates configuration for the ptvi and ebgp.
Step 4: Import CCTV-LAN Route Target into the SINK-VR
RemoteBranch > Virtual Routers > Sink-VR
- Set Instance Type as Virtual Routing Forwarding Instance
- Set Control-VR
- Add VRF Import Target for the CCTV-LAN-VR (target:5L:5)
Step 5 : Match community in BGP Peer Group
The Trigger branch appends a community (0:110) to BGP updates for interesting prefixes. This is matched by the below term on the remote branch.
Remote Branch > Virtual Routers > SINK-VR > BGP (Instance id) > Peer Policy > SINK ROUTES.
- Add Term to match Trigger community (0:110)
Step 6: Add Export policy to Peer Group
The policy created in Step 5 above should be added as an export policy to the Split Tunnel Peer Group
Virtual Routers > SINK-VR > BGP (Instance id) > Peer Group > ST-Group (e.g. Copy_of_TO_CCTV-LAN) > Advanced
- Remove Import policy
- Set Export Policy to SINK-ROUTES
Step 7: Add Firewall Rule to Drop Traffic
This step adds a security rule to drop traffic arriving from the CCTV-LAN-VR into the SINK-VR. Services > Next Gen Firewall > Security > Policies
Traffic may be discarded using SDWAN policies, QoS policies or null routes. All of these are viable alternatives. The benefit of using the stateful firewall to achieve this includes the visibility in analytics and potential flexibility in defining match criteria for the traffic based on Zone, application etc.
Trigger Branch Configuration
The trigger branch receives the BGP feed from the external source. RTBH as described in this solution is triggered by a specified BGP community. This community may be set directly by the external source alternatively; this may be set by the trigger branch by policy.Step1: Match Protocol BGP and set Community
Remote Branch > Virtual Routers CCTV-LAN-VR > Redistribution Policy
- Add Rule to Default Policy.
- Match BGP Routes
The routing instance redistribution policy is used because the community tag must be carried via the control-VR to the remote branches.
Best Practice Principles
The following principles are considerations in deploying this solution.- On the Remote Branch – Filter SDWAN routes installed into SINK-VR routing table to prefixes with the trigger community tag.
- On the Trigger Branch – Add safety term to BGP Peer policy to block learning default route from the external source. This prevents an accident or misconfigured external source from causing a network wide blackhole of the default route.
Solution Verification
Before Attack
User traffic flows from the ingress remote branch to the destination host behind an egress branch.
Next-hop for prefix is SDWAN in the CCTV-LAN directly towards a remote gateway.
admin@sdlanbr1-cli> show route routing-instance CCTV-LAN-VR Routes for Routing instance : CCTV-LAN-VR AFI: ipv4 SAFI: unicast Codes: E1 - OSPF external type 1, E2 - OSPF external type 2 IA - inter area, iA - intra area, L1 - IS-IS level-1, L2 - IS-IS level-2 N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 RTI - Learnt from another routing-instance + - Active Route Prot Type Dest Address/Mask Next-hop Age Interface name ---- ---- ----------------- -------- --- -------------- conn N/A +10.10.10.0/24 0.0.0.0 1d21h33m vni-0/3.0 local N/A +10.10.10.1/32 0.0.0.0 1d21h33m directly connected BGP N/A +10.10.20.0/24 10.254.2.78 15:57:26 Indirect BGP N/A +111.111.111.1/32 10.254.2.78 15:57:26 Indirect conn N/A +169.254.0.2/31 0.0.0.0 1d21h33m tvi-0/603.0 local N/A +169.254.0.3/32 0.0.0.0 1d21h33m directly connected conn N/A +169.254.100.0/30 0.0.0.0 1d21h33m tvi-1/101.0 local N/A +169.254.100.2/32 0.0.0.0 1d21h33m directly connected Routes for Routing instance : CCTV-LAN-VR AFI: ipv6 SAFI: unicast [ok][2022-10-25 01:30:37] admin@sdlanbr1-cli>
Ongoing Attack
Once an attack is detected by an external source, it advertises the affected prefix into the SDWAN network via the Trigger Branch.
A BGP feed with the community 110 is injected from the external source and distributed to the remote branches. The Remote branches will then prefer next-hop via the SINK VR where the traffic is blackholed by Security policy.
admin@sdlanbr1-cli> show route routing-instance CCTV-LAN-VR Routes for Routing instance : CCTV-LAN-VR AFI: ipv4 SAFI: unicast Codes: E1 - OSPF external type 1, E2 - OSPF external type 2 IA - inter area, iA - intra area, L1 - IS-IS level-1, L2 - IS-IS level-2 N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 RTI - Learnt from another routing-instance + - Active Route Prot Type Dest Address/Mask Next-hop Age Interface name ---- ---- ----------------- -------- --- -------------- conn N/A +10.10.10.0/24 0.0.0.0 1d21h40m vni-0/3.0 local N/A +10.10.10.1/32 0.0.0.0 1d21h40m directly connected BGP N/A +10.10.20.0/24 10.254.2.78 16:04:19 Indirect BGP N/A 111.111.111.1/32 10.254.2.78 00:00:02 Indirect BGP N/A +111.111.111.1/32 169.254.100.1 00:00:02 tvi-1/101.0 conn N/A +169.254.0.2/31 0.0.0.0 1d21h40m tvi-0/603.0 local N/A +169.254.0.3/32 0.0.0.0 1d21h40m directly connected conn N/A +169.254.100.0/30 0.0.0.0 1d21h40m tvi-1/101.0 local N/A +169.254.100.2/32 0.0.0.0 1d21h40m directly connected Routes for Routing instance : CCTV-LAN-VR AFI: ipv6 SAFI: unicast [ok][2022-10-25 01:37:31] admin@sdlanbr1-cli>Route advertised from the SINK VR causes the next hop to be preferred. Notice the trigger community tag (0:110)
admin@sdlanbr1-cli> show route table ipv4.unicast routing-instance CCTV-LAN-VR receive-protocol bgp detail
Routes for Routing instance : CCTV-LAN-VR AFI: ipv4 SAFI: unicast
Routing entry for 111.111.111.1/32
Peer Address : 169.254.100.1
Next-hop : 169.254.100.1
Local preference : 110
AS path :
Origin : Egp
MED : 0
Community : 0:110 8015:0
ExtCommunity : N/A
Preference : Default
Weight : 0
admin@sdlanbr1-cli>
Session output for showing traffic punted into the SINK VR and dropped by policy.
admin@sdlanbr1-cli> show orgs org GNS-Org sessions sdwan extensive | select destination-ip 111.111.111.1 sessions sdwan extensive 0 2 78 source-ip 10.10.10.10 destination-ip 111.111.111.1 source-port 10 destination-port 10 protocol 1 natted No sdwan Yes application - forward-pkt-count 8 forward-byte-count 672 reverse-pkt-count 0 reverse-byte-count 0 dropped-forward-pkt-count 8 dropped-forward-byte-count 672 dropped-reverse-pkt-count 0 dropped-reverse-byte-count 0 session-age 00:00:07 idle-for 00:00:07 idle-timeout 10 drop-module policy pbf-enabled false forward-egress-vrf Sink-VR reverse-egress-vrf Sink-VR session-provider-zone 0 forward-offload false reverse-offload false forward-ingress-interface tvi-1/100.0 forward-egress-interface dtvi-0/88 reverse-ingress-interface n/a reverse-egress-interface tvi-1/100.0 forward-fc fc_be reverse-fc fc_be forward-plp low reverse-plp low external-service-chaining false is-child No parent-sess-id 0 device "" rx-wan-ckt - tx-wan-ckt INTERNET-WAN:DIRECT-INTERNET tx-branch l2br2 pbf-wan-ackt-enc (E) forward-ingress-ckt tvi-1/100.0 forward-egress-branch l2br2 forward-egress-ckt INTERNET-WAN:DIRECT-INTERNET reverse-ingress-ckt - reverse-egress-ckt tvi-1/100.0
