Remotely Triggered Blackhole Using BGP

What are your feelings
Author: Tayo Ogunseyinde, Consultant Systems Engineer (EMEA)

This article describes an advanced use case. It assumes basic knowledge of the Versa architecture and is intended for use by experienced Versa Network engineers.

Overview #

Networks often have a requirement to respond in a swift and coordinated way to security threats by blackholing the attack traffic. External systems may be used to analyze and detect these threats, and signal to the network to enforce some mitigation action. One type of action may be to blackhole or sink attack traffic destined to some prefix in the network.

As enterprises transition from traditional MPLS networks to modern SDWAN overlay networks, the requirement to integrate with specialist third party security tools remains. SDWAN introduces a further dimension due to its inherent segmentation of the overlay from the underlay. SDWAN must be able to receive signals from external sources to protect both the overlay and /or underlay from attack.

Versa Secure SDWAN can achieve Remote Triggered Black Hole (RTBH) in the following ways.
  1. Using the Versa Director to push configuration updates to the remote branches. This is suited for scenarios where there is an orchestration layer API feed to push configuration on demand. Alternatively, tools such as the Versa VSync tool can be used to distribute threat feed from external sources to the Versa SDWAN appliances.
  2. A trigger based on a control plane update feed. This is ideal where the feed comes from traditional network components using networking protocols such as BGP. BGP is used as a signaling protocol to all edge routers. It can identify specific prefixes under attack and signal the type of mitigation that should be enforced.

This article explores the latter option using the BGP control plane to automate the remote triggering of a prefix black hole.

Solution Architecture #

A security threat feed is triggered from an external source via BGP towards a trigger branch. A trigger branch is a VOS branch that receives the BGP trigger and initiates the route distribution to the remote branches. The prefix black holing action occurs on the remote branch. In the example above, prefix is identified by an external system (‘External Trigger’) for blackholing. The prefix is then advertised via BGP to ‘Trigger Branch’. Trigger Branch onward advertises the prefix to all other ‘Remote Branches’. As a consequence, any traffic destined to is instantly dropped by all remote branches in the network.

Blackholing on the remote branch is achieved by drawing the triggered prefix into a separate virtual routing instance (SINK VR) where the traffic is discarded. The concept can be leveraged to onramp traffic from the LAN-VR into a separate VR to be scrubbed or treated before returning to the LAN. The onramp use case is out of scope of this article.

Note: This article assumes a good understanding of the customer’s own network in the design and implementation of the external trigger feed. The details of such external trigger feed is out of scope of this article. A virtual routing instance (SINK-VR) is created on each remote branch participating in the prefix black holing. The LAN-VR (CCTV-LAN in the example above) is the enterprise routing table. The route target from the CCTV-LAN is imported into both the SINK-VR and the CCTV-LAN-VR at the remote branch. Thus, routes in the CCTV-LAN-VR are imported into the Sink-VR. Further filtering is applied to the sink VR to ensure that only triggered routes are installed into the SINK-VR routing table. This will save system resources from duplicating all routes into the SINK-VR.

A paired tvi (ptvi) is created between the SINK-VR and the CCTV-LAN-VR. eBGP is enabled over the ptvi. The triggered prefix is learnt via a BGP update with a specific community tag (110 in this example). Routes matching this community are advertised with a preferred metric into the CCTV-LAN-VR over the ptvi.

Traffic once in the SINK-VR is dropped via security policy.

Organization Configurations #

Step 1: Create new Org #

Workflow > Infrastructure > Organizations > Routing Instances VPN must be set to true. This step auto generates the route distinguisher and route targets for each routing instance. In the example above, target:5L:5 belongs to CCTV-LAN-VR and target:7L:7 belong to the SINK-VR

Remote Branch Configuration #

This article describes the simplest approach using the workflow template to autogenerate this configuration. Manually creating a routing-instance with a split tunnel is a multistep activity with several config touchpoints.

Step 1: Create SINK-VR as a WAN network. #

Configuration > Objects > WAN Networks This is created as a WAN Network (instead of a LAN Network) since the workflow wizard for ptvi (split tunnel) can only be created between the WAN and LAN.

Step 2: Add SINK VR as a WAN Network. #

An unused dummy interface should be used in the workflow to enable the creation of this WAN Routing Instance.

Step3: Add SINK VR as a split tunnel to CCTV-LAN #

  Note DIA and Gateway should be unchecked. This step auto generates configuration for the ptvi and ebgp.

Step 4: Import CCTV-LAN Route Target into the SINK-VR #

RemoteBranch > Virtual Routers > Sink-VR
  • Set Instance Type as Virtual Routing Forwarding Instance
  • Set Control-VR
  • Add VRF Import Target for the CCTV-LAN-VR (target:5L:5)
Route targets, route distinguisher and the MPLS transport routing instance are auto created by the organization workflow.

Step 5 : Match community in BGP Peer Group #

The Trigger branch appends a community (0:110) to BGP updates for interesting prefixes. This is matched by the below term on the remote branch.
Remote Branch > Virtual Routers > SINK-VR > BGP (Instance id) > Peer Policy > SINK ROUTES.
  • Add Term to match Trigger community (0:110)

Step 6: Add Export policy to Peer Group #

The policy created in Step 5 above should be added as an export policy to the Split Tunnel Peer Group Virtual Routers > SINK-VR > BGP (Instance id) > Peer Group > ST-Group (e.g. Copy_of_TO_CCTV-LAN) > Advanced
  • Remove Import policy
  • Set Export Policy to SINK-ROUTES

Step 7: Add Firewall Rule to Drop Traffic #

This step adds a security rule to drop traffic arriving from the CCTV-LAN-VR into the SINK-VR. Services > Next Gen Firewall > Security > Policies Traffic may be discarded using SDWAN policies, QoS policies or null routes. All of these are viable alternatives. The benefit of using the stateful firewall to achieve this includes the visibility in analytics and potential flexibility in defining match criteria for the traffic based on Zone, application etc.

Trigger Branch Configuration #

The trigger branch receives the BGP feed from the external source. RTBH as described in this solution is triggered by a specified BGP community. This community may be set directly by the external source alternatively; this may be set by the trigger branch by policy.

Step1: Match Protocol BGP and set Community #

Remote Branch > Virtual Routers CCTV-LAN-VR > Redistribution Policy
  • Add Rule to Default Policy.
  • Match BGP Routes
Set Community on received BGP routes. The routing instance redistribution policy is used because the community tag must be carried via the control-VR to the remote branches.

Best Practice Principles #

The following principles are considerations in deploying this solution.
  • On the Remote Branch – Filter SDWAN routes installed into SINK-VR routing table to prefixes with the trigger community tag.
  • On the Trigger Branch – Add safety term to BGP Peer policy to block learning default route from the external source. This prevents an accident or misconfigured external source from causing a network wide blackhole of the default route.

Solution Verification #

Before Attack #

User traffic flows from the ingress remote branch to the destination host behind an egress branch. Next-hop for prefix is SDWAN in the CCTV-LAN directly towards a remote gateway.
admin@sdlanbr1-cli> show route routing-instance CCTV-LAN-VR
Routes for Routing instance : CCTV-LAN-VR  AFI: ipv4  SAFI: unicast
Codes: E1 - OSPF external type 1, E2 - OSPF external type 2
IA - inter area, iA - intra area,
L1 - IS-IS level-1, L2 - IS-IS level-2
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
RTI - Learnt from another routing-instance
+ - Active Route

Prot   Type  Dest Address/Mask   Next-hop        Age      Interface name
----   ----  -----------------   --------        ---      --------------
conn   N/A  +         1d21h33m vni-0/3.0
local  N/A  +         1d21h33m directly connected
BGP    N/A  +     15:57:26 Indirect
BGP    N/A  +     15:57:26 Indirect
conn   N/A  +         1d21h33m tvi-0/603.0
local  N/A  +         1d21h33m directly connected
conn   N/A  +         1d21h33m tvi-1/101.0
local  N/A  +         1d21h33m directly connected

Routes for Routing instance : CCTV-LAN-VR  AFI: ipv6  SAFI: unicast

[ok][2022-10-25 01:30:37]

Ongoing Attack #

Once an attack is detected by an external source, it advertises the affected prefix into the SDWAN network via the Trigger Branch. A BGP feed with the community 110 is injected from the external source and distributed to the remote branches. The Remote branches will then prefer next-hop via the SINK VR where the traffic is blackholed by Security policy.
admin@sdlanbr1-cli> show route routing-instance CCTV-LAN-VR

Routes for Routing instance : CCTV-LAN-VR  AFI: ipv4  SAFI: unicast
Codes: E1 - OSPF external type 1, E2 - OSPF external type 2
IA - inter area, iA - intra area,
L1 - IS-IS level-1, L2 - IS-IS level-2
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
RTI - Learnt from another routing-instance
+ - Active Route

Prot   Type  Dest Address/Mask   Next-hop        Age      Interface name
----   ----  -----------------   --------        ---      --------------
conn   N/A  +         1d21h40m vni-0/3.0
local  N/A  +         1d21h40m directly connected
BGP    N/A  +     16:04:19 Indirect
BGP    N/A     00:00:02 Indirect
BGP    N/A  +   00:00:02 tvi-1/101.0
conn   N/A  +         1d21h40m tvi-0/603.0
local  N/A  +         1d21h40m directly connected
conn   N/A  +         1d21h40m tvi-1/101.0
local  N/A  +         1d21h40m directly connected

Routes for Routing instance : CCTV-LAN-VR  AFI: ipv6  SAFI: unicast

[ok][2022-10-25 01:37:31]
Route advertised from the SINK VR causes the next hop to be preferred. Notice the trigger community tag (0:110)
admin@sdlanbr1-cli> show route table ipv4.unicast routing-instance CCTV-LAN-VR receive-protocol bgp detail

Routes for Routing instance : CCTV-LAN-VR  AFI: ipv4  SAFI: unicast

Routing entry for
     Peer Address     :
     Next-hop         :
     Local preference : 110
     AS path          :
     Origin           : Egp
     MED              : 0
     Community        : 0:110 8015:0
     ExtCommunity     : N/A
     Preference       : Default
     Weight           : 0
Session output for showing traffic punted into the SINK VR and dropped by policy.
admin@sdlanbr1-cli> show orgs org GNS-Org sessions sdwan extensive | select destination-ip
sessions sdwan extensive 0 2 78
 source-port                10
 destination-port           10
 protocol                   1
 natted                     No
 sdwan                      Yes
 application                -
 forward-pkt-count          8
 forward-byte-count         672
 reverse-pkt-count          0
 reverse-byte-count         0
 dropped-forward-pkt-count  8
 dropped-forward-byte-count 672
 dropped-reverse-pkt-count  0
 dropped-reverse-byte-count 0
 session-age                00:00:07
 idle-for                   00:00:07
 idle-timeout               10
 drop-module                policy
 pbf-enabled                false
 forward-egress-vrf         Sink-VR
 reverse-egress-vrf         Sink-VR
 session-provider-zone      0
 forward-offload            false
 reverse-offload            false
 forward-ingress-interface  tvi-1/100.0
 forward-egress-interface   dtvi-0/88
 reverse-ingress-interface  n/a
 reverse-egress-interface   tvi-1/100.0
 forward-fc                 fc_be
 reverse-fc                 fc_be
 forward-plp                low
 reverse-plp                low
 external-service-chaining  false
 is-child                   No
 parent-sess-id             0
 device                     ""
 rx-wan-ckt                 -
 tx-wan-ckt                 INTERNET-WAN:DIRECT-INTERNET
 tx-branch                  l2br2
 pbf-wan-ackt-enc           (E)
 forward-ingress-ckt        tvi-1/100.0
 forward-egress-branch      l2br2
 forward-egress-ckt         INTERNET-WAN:DIRECT-INTERNET
 reverse-ingress-ckt        -
 reverse-egress-ckt         tvi-1/100.0

Summary #

Remotely triggered Black Hole (RTBH) is not a new concept, it is traditionally used within a single routing table on traditional routers. The approach described in this article is suited for the inherent internal segmentation of the Versa architecture. This article has demonstrated the remote trigger of a prefix under attack and instantly causes all nodes in the network to back hole the specific prefix only.