My book: 'Running IPv6' by Iljitsch van Beijnum BGPexpert My book: 'BGP' by Iljitsch van Beijnum

Home · BGP Expert Test · What is BGP? · BGP Vendors · Links · Archives · Books · My BGP Book

BGP (advertisement)

Table of contents (for this page):

inet⁶ consult

If you could use some help with BGP, have a look at my business web site:

BGP routing courses

There are currently no training courses planned.

Interdomain Routing & IPv6 News

  • Search for:
  • Hunting down the stuck BGP routes (posted 2021-04-22)

    Ben Cox (Benjojo) has an interesting post about stuck BGP routes and a flaw in many BGP implementations where they hang when their neighbor stops accepting data over TCP: Hunting down the stuck BGP routes

    A stuck BGP route means that a prefix was advertised at some point, and then it's withdrawn but the withdrawal somehow gets lost somewhere, so part of the internet still sees the withdrawn route.

    There doesn't seem to be an obvious way to get these prefixes unstuck, although I would try advertising them again and withdrawing them again. The AS in the middle where the prefixes are stuck should probably clear (reset) BGP sessions until the problem is fixed, although I would be tempted to just reboot the whole router just to be safe.

    The hanging issue happens when a router gets behind on processing incoming BGP updates. At some point its TCP receive buffer fills up, so it sets its TCP window size to 0. Then the buffer on the sending BGP neighbor starts to fill up with updates and keepalives. When that buffer is full, the sending BGP process can't write data to the TCP socket anymore... and bad things start to happen.

    Ben feels that the BGP spec needs to be updated to address this issue. However, this is not a matter of interoperation that needs to be standardized, but simply an implementation issue that each vendor can fix on their own, in my opinion.

  • No joke: running BGP on a $100 home router / Wi-Fi access point (posted 2021-04-01)

    For some time, I've been hearing about Mikrotek routers, which couple being quite capable with being affordable. But I never got my hands on one. I'm now in the process of upgrading my home network, and learned about the Mikrotik hAP ac³. The ac³ defies easy classification, but I think it's mostly a home router and/or Wi-Fi access point. I paid € 95 and shipping, and I believe it's available in the US for about $100.

    I was somewhat disappointed to learn that "5 gigabit ports" doesn't mean ports that are capable of 5 gigabit, but 5 ports that just ordinary 1 Gbps Ethernet. Initially it seemed the box didn't support IPv6, but it turns you have to enable that under "packages" and then reboot. (Not shutdown.)

    However, I wasn't disappointed to learn that the ac³ supports RIP, OSPF and BGP, both for IPv4 and IPv6. So I made a configuration on the ac³ that fits with the lab network that I use for my training courses. Initially it didn't seem to work, but then after a reboot of the ac³ the BGP sessions came up. Pretty cool!

    The filtering is quite different from what I'm used to, so I'll need a bit more time to figure that out.

    The hardest part was setting up VLAN IP interfaces. For some reason I needed to use the command line to create the VLANs:

    /interface vlan add interface=bridge name=vlan1401 vlan-id=1401
    /interface ethernet switch vlan add ports=ether3,switch1-cpu independent-learning=no switch=switch1 vlan-id=1401

    After that, I could use the web interface to add IP addresses. I also set up BGP through the web interface, but the command line export of that looks like this:

    > /routing bgp export
    # apr/01/2021 20:50:37 by RouterOS 6.48.1
    # software id = C3HH-K6NW
    # model = RBD53iG-5HacD2HnD
    # serial number = E7290D96999E
    /routing bgp instance
    set default disabled=yes
    add as=65082 client-to-client-reflection=no name=Router82 router-id=
    /routing bgp network
    add network= synchronize=no
    /routing bgp peer
    add address-families=ip,ipv6 hold-time=1m30s instance=Router82 name=ISP30 out-filter=out-as-filter
    remote-address= remote-as=65030
    add instance=Router82 name=ISP40 remote-address= remote-as=65040

    Initially it seems the Mikrotik won't show you any BGP info, but with a bit more digging you'll get there:

    Way back in 1996 the global BGP table was 30,000 prefixes and that just about fit in the 16 MB that was the maximum a Cisco 2501 router would take. The ac³ has 128 MB storage (using less than half of that) and 256 MB RAM, which is probably enough for 200k - 500k prefixes but not a full table. And of course the ac³ is not an obvious choice for such a task in the first place.

  • When the BGP table hits 1 million prefixes, will history repeat itself? (posted 2021-03-23)

    On the APNIC blog, Danny Pinto asks What will happen when the routing table hits 1024k? Back in 2014, the IPv4 BGP table reached 512k, a common limit in many routers at the time, and some bad things happened. See my post BGP table hitting 512k limit in older routers. And pretty much the same thing happened in 2008, when the BGP table hit 256k.

    Interestingly, reaching the 1024k limit was predicted for 2019, but now in March 2019, we're still well below 900k: BGP table growth has leveled off from an average 10% per year in the 2010s to 5% in 2020. My prediction is that we won't reach 1024k IPv4 prefixes in the global BGP table until 2024.

    However, asking when we'll reach 1024k IPv4 prefixes is probably asking the wrong question. (But, do read the APNIC blog article for a few relevant pointers.) Back in 2014, there were less than 20k IPv6 prefixes in BGP, and many networks didn't run IPv6 yet. Well, actually, there are 71k networks (autonomous systems) advertising one or more IPv4 prefixes, but only 22k networks announcing one or more IPv6 prefixes.

    Still, many networks run IPv4 and IPv6 side-by-side, which means 874k IPv4 prefixes + 114k IPv6 prefixes = 988k total. Add to that internal prefixes, which can add up to many thousands in larger networks, and I'm pretty sure many networks are already over the 1024k limit today, or will be soon.

    So I'm sure a few networks will be caught unprepared and run into trouble when the IPv4 table goes over 1024k, but to a much lesser degree than in 2008 and 2014.

    In any event, when buying a new router, check if it can handle 2 million total prefixes in the main routing table (RIB) and the forwarding information base (FIB). Also read the fine print, sometimes IPv6 prefixes can count double, and it's relatively common that the 2M limit applies to a combination of firewall rules as well as IPv4 and IPv6 prefixes.

    There is usually a separate limit for the BGP RIB, which needs to be much larger, because unlike the main RIB and the FIB, the BGP RIB keeps multiple copies of every prefix: one for each BGP neighbor. So for the BGP RIB you'll want something like 8 million. However, the BGP RIB is usually only limited by available RAM.

  • → The dark side of BGP community attributes (posted 2020-12-07)

    An article I wrote for the Noction blog looking at possible attacks using the BGP community attribute.

    A while ago, RIPE Labs published the two-part article BGP Communities – A Weapon for the Internet. That may have been a bit of a shock for those of us making good use of BGP community attributes as an important tool in our BGP arsenal.


    This community-based attack is definitely something we need to be prepared for and defend against. But does this warrant considering BGP communities “a weapon for the internet”? That seems a bit extreme.


    Treat your BGP communities with respect, you don’t want to encounter their dark side.

    Read the whole article

Archive of all articles - RSS feed

My Books: "BGP" and "Running IPv6"

On this page you can find more information about my book "BGP". Or you can jump immediately to chapter 6, "Traffic Engineering", (approx. 150kB) that O'Reilly has put online as a sample chapter. Information about the Japanese translation can be found here.

More information about my second book, "Running IPv6", is available here.

BGP Security

BGP has some security holes. This sounds very bad, and of course it isn't good, but don't be overly alarmed. There are basically two problems: sessions can be hijacked, and it is possible to inject incorrect information into the BGP tables for someone who can either hijack a session or someone who has a legitimate BGP session.

Session hijacking is hard to do for someone who can't see the TCP sequence number for the TCP session the BGP protocol runs over, and if there are good anti-spoofing filters it is even impossible. And of course using the TCP MD5 password option (RFC 2385) makes all of this nearly impossible even for someone who can sniff the BGP traffic.

Nearly all ISPs filter BGP information from customers, so in most cases it isn't possible to successfully inject false information. However, filtering on peering sessions between ISPs isn't as widespread, although some networks do this. A rogue ISP could do some real damage here.

There are now two efforts underway to better secure BGP:

  • Secure BGP (S-BGP) is developed by Bolt, Beranek and Newman (BBN). It has been around for several years and there is a proof-of-concept implementation. S-BGP tries to secure all aspects of the BGP protocol, and subsequently needs several signature checks for each BGP update, making the protocol relatively heavy-weight. You can see my earlier rants on S-BGP at the top of this page. Note that I'm not as anti-S-BGP as I used to be any more, although I still think implementing the protocol will be expensive because routers will need lots of extra memory (up to four times as much) and CPU power (possibly dedicated crypto hardware) and this aspect deserves some serious attention.

    Secure BGP (S-BGP) index at BBN.

  • Secure Origin BGP (soBGP) has surfaced fairly recently and hails from Cisco. There are no implementations so far. soBGP mainly focusses on securing the relationship between prefixes and the source AS number, and doesn't need as many computationally expensive checks as S-BGP. However, the protocol can easily be expanded to perform more checks.

    draft-ng-sobgp-bgp-extensions-00.txt (main soBGP draft)
    draft-white-sobgp-bgp-extensions-00.txt (deployment considerations)

    (If the links don't work, the drafts have expired; you'll have to use a search engine to find them.)

There is now also a different approach to increasing BGP security using an "Interdomain Routing Validation" service that works independent from the BGP protocol itself. See what I wrote about this in interdomain routing news on this site, or jump immediately to the Working Around BGP: An Incremental Approach to Improving Security and Accuracy of Interdomain Routing paper.

The IETF RPSEC (routing protocol security) working group is active in this area.

What is is a website dedicated to Internet routing issues. What we want is for packets to find their way from one end of the globe to another, and make the jobs of the people that make this happen a little easier.

Your host is Iljitsch van Beijnum. Feedback, comments, link requests... everything is welcome. You can read more about me here or email me at iljitsch@bgpexpert. or follow iljitsch on Twitter.

Ok, but what is BGP?

Have a look at the "what is BGP" page. There is also a list of BGP and interdomain routing terms on this page.

BGP and Multihoming

If you are not an ISP, your main reason to be interested in BGP will probably be to multihome. By connecting to two or more ISPs at the same time, you are "multihomed" and you no longer have to depend on a single ISP for your network connectivity.

This sounds simple enough, but as always, there is a catch. For regular customers, it's the Internet Service Provider who makes sure the rest of the Internet knows where packets have to be sent to reach their customer. If you are multihomed, you can't let your ISP do this, because then you would have to depend on a single ISP again. This is where the BGP protocol comes in: this is the protocol used to carry this information from ISP to ISP. By announcing reachability information for your network to two ISPs, you can make sure everybody still knows how to reach you if one of those ISPs has an outage.

Want to know more? Read A Look at Multihoming and BGP, an article about multihoming I wrote for the O'Reilly Network.

For those of you interested in multihoming in IPv6 (which is pretty much impossible at the moment), have a look at the "IPv6 multihoming solutions" page.

Are you a BGP expert? Take the test to find out!

These questions are somewhat Cisco-centric. We now also have another set of questions and answers for self-study purposes.

You are visiting over IPv4. Your address is