BGP is the protocol that runs the internet. It is also the protocol that runs some very small private networks, and a growing number of self-hosted ECMP-and-anycast setups. I wanted to understand it more deeply so I set it up at home between two routers, an OPNsense box and a mikrotik. The experiment was educational. My answer to “is BGP worth it at home” is “usually no, but”.

The setup

  • OPNsense on x86 with FRR plugin (AS 65010)
  • mikrotik hEX with RouterOS BGP (AS 65020)
  • Both advertise a different /24 worth of homelab subnets
  • eBGP session over a direct link between them
  • Multi-path disabled for simplicity

The FRR config on OPNsense:

router bgp 65010
 bgp router-id 10.99.0.1
 neighbor 10.99.0.2 remote-as 65020
 neighbor 10.99.0.2 description mikrotik
 !
 address-family ipv4 unicast
  network 192.168.30.0/24
  neighbor 10.99.0.2 activate
  neighbor 10.99.0.2 route-map HOMELAB-IN in
  neighbor 10.99.0.2 route-map HOMELAB-OUT out
 exit-address-family
!
ip prefix-list ALLOWED seq 5 permit 192.168.0.0/16 le 24
route-map HOMELAB-IN permit 10
 match ip address prefix-list ALLOWED
route-map HOMELAB-OUT permit 10
 match ip address prefix-list ALLOWED

The prefix list plus route maps are there because I wanted to practice. In a proper production setup you absolutely want strict inbound filtering; at home, on a link between two boxes you both own, it is belt and suspenders.

Getting the session up

First session attempt failed. vtysh in FRR:

OPNsense# show bgp summary
# Neighbor       V    AS  MsgRcvd  MsgSent  TblVer InQ OutQ Up/Down State/PfxRcd
# 10.99.0.2     4  65020      0       0       0   0    0  never    Connect

Idle/Active/Connect is BGP’s way of saying “I have not established the session”. Standard checklist:

  • TCP 179 reachable. tcpdump -i any port 179 on the mikrotik side showed packets coming in but something was missing.
  • AS numbers match. OPNsense has remote-as 65020, mikrotik has my AS 65020 local and remote 65010. Swap.

The mikrotik side:

/routing bgp connection
add name=opnsense remote.address=10.99.0.1 remote.as=65010 \
  local.address=10.99.0.2 local.role=ebgp templates=default

Once AS numbers were right, the session came up in 3 seconds. Routes exchanged.

What I learned

  1. The routing table updates are fast and smooth. I blocked the direct link; the cross-subnet routes vanished from the other router within a second. Reopening the link brought them back. OSPF would have done the same thing.

  2. The policy language is rich. Route maps let me inject communities, adjust local preference, set next-hop on receipt, and filter based on ASPath. Most of this is overkill for a homelab but genuinely expressive.

  3. The failure modes are legible. show bgp neighbors, show bgp, show ip route bgp give you a precise answer for any question about state.

Was it worth it

For my homelab specifically, no. I did not have a real traffic engineering need. OSPF (which I was already running) would have done the same failover in this topology. The BGP setup added:

  • An extra protocol to understand.
  • Two route policies to maintain.
  • A new surface for misconfiguration.

In exchange I got:

  • Stronger filtering (useful if the other router was not mine).
  • Community-based policy (useful at larger scale).
  • Learning.

The learning was the point. If that is your goal, do it. If your goal is “make my home network better”, stick with OSPF or static.

Where BGP does earn its keep in small networks

  • Self-hosting with anycast. If you have a Hetzner VM and a DO VM both running a service, and you want DNS-based load balancing with faster failover, BGP to a hosted service like Vultr’s BGP-capable cloud or a 6connect setup can make sense. This is a real pattern, not common, but real.

  • Multi-site VPN with overlapping subnets. OSPF does not natively carry next-hop information that lets you pick paths based on community or tag. BGP does. If you have three sites with specific routing policies, BGP is honestly cleaner than a mess of OSPF areas.

  • Kubernetes with Cilium or MetalLB in BGP mode. This is the big real reason a lot of homelab-ish environments run BGP. Your LoadBalancer services advertise their IPs from your nodes, and your upstream router picks them up. I set this up in a separate experiment and ended up keeping it because Cilium’s IPAM becomes nicer.

The Cilium/BGP note

Here is a chunk of the FRR config I settled on for the Cilium case:

router bgp 65010
 neighbor CILIUM peer-group
 neighbor CILIUM remote-as 65010
 neighbor 10.20.30.11 peer-group CILIUM
 neighbor 10.20.30.12 peer-group CILIUM
 neighbor 10.20.30.13 peer-group CILIUM
 !
 address-family ipv4 unicast
  neighbor CILIUM activate
  neighbor CILIUM soft-reconfiguration inbound
  neighbor CILIUM route-map CILIUM-IN in
 exit-address-family
!
route-map CILIUM-IN permit 10
 match ip address prefix-list LB-RANGE
ip prefix-list LB-RANGE seq 5 permit 10.40.0.0/16 le 32

Cilium advertises specific /32s to my router, which installs them with next-hop as the node. The router does ECMP across nodes and forwards LB traffic. This is genuinely useful.

Reflection

BGP is a very elegant tool in the right hands. It is also a hammer that can bruise if you are not careful. At home the default answer is “no”. If you are doing anycast, multi-site routing policies, or a CNI that wants to advertise service IPs, the answer shifts to “yes, and you will learn a lot”.

Related: see my post on testing routers with Linux network namespaces for the safe way to break things before you deploy them.