[ixpmanager] arouteserver vs IXP Manager

[Barry O'Donovan (INEX)]

Hi Richard,

so obviously I'm biased 😉

But besides being biased, unless you want to create more work for 
yourself and miss out on various integration's and new features, the 
answer is simple: if you're using IXP Manager, use IXP Manager's route 
server provisioning unless you have a really really good reason not to 
that negates all of the following.

NB: none of this is to denigrate Arouteserver - my argument here is a 
basic if/then: if you're using IXP Manager then you're best placed to 
use IXP Manager's route server provisioning.

The feature comparison is pretty much like for like and it's ultimately 
not a IXP Manager vs Arouteserver issue but a Bird configuration issue. 
I'll look at your specific points towards the end but I'll start with 
integration.

IXP Manager is tightly coupled with its own Bird configuration and 
provisioning pipeline. It also has proven stability since 2007. Some 
specific existing features include:

- dedicated per-member view on their blocked prefixes across all routers 
at an IXP configured via IXP Manager (route servers, AS112 and route 
collector) - shown in realtime on a single page with the reason for each 
block.
- integrated looking glass for all servers
- router's managed from within IXP Manager (/router/list) which feeds 
into live status views of the routers, Nagios automation for monitoring 
(routers themselves and individual sessions - all templates available on 
IXP Manager)
- extensive documentation, on-list experience, talks and tutorial videos

One new feature that is in master and pending release is route server 
prefix filtering from within IXP Manager. I.e. replaces the member 
having to set communities on outbound advertisements and filter inbound 
advertisements to make basic routing policy changes like "don't peer 
with X over route servers" or "prepend x times to y over route servers". 
This also requires a quicker deploy and sync system with feedback to IXP 
Manager on last sync and any issues (which can feed into 
Nagios/monitoring). See the GPF 2023 presentation at 
https://www.ixpmanager.org/support/talks

You won't get these with Arouteserver / or non IXP Manager route server 
configs.

We do plan to include OpenBGPD in this also so you'll be able to have 
two different daemons all fully integrated. I don't have timelines for 
this except I know it won't be in the next four months.

On the specific questions:

 > arouteserver:
 >    - Unclear whether IXP Manager treats a valid ROA as an IRR pass.

It does with one important step. See #6 and #7 at 
https://docs.ixpmanager.org/features/route-servers/#configuration-generation-features 


While the origin ASN might be correct to pass a ROA test, it does not 
tell us if a specific member, via an AS-SET, is defined as being able to 
originate that origin ASN. So it's:

- AS origin in AS-SET for IRRDB
- RPKI ROA -> if pass we're done, else:
- IRRDB prefix check.

 > I'm told this helps with stub networks who are missing IRR:
 > 
https://arouteserver.readthedocs.io/en/latest/CONFIG.html#use-rpki-roas-as-if-they-were-route-objects 


This is equivalent functionality to Arouteserver as per that link "whose 
origin ASN is already authorized by a client’s AS-SET but whose prefix 
is not".


 > - Supports PeeringDB "never via route server", which IXP Manager 
seemingly does not:
 > https://github.com/inex/IXP-Manager/issues/798

Yes and no. We do support it:

https://docs.ixpmanager.org/features/routers/#filtering-known-transit-networks

We do not take it from PeeringDB.

Having looked, admittedly some time ago, at the number of networks with 
this set in PeeringDB, my inclination is that many networks do not know 
what this means and I felt it would be dangerous to pull this from 
PeeringDB in its current form. Especially as it would propagate to up to 
200 IXPs.

 >    - Support blackhole (RTBH) community. We wanted to add this
 >      at MICE, but it is unclear how much participants will actually
 >      care. This does have complexities with integrating with IRR/RPKI
 >      filtering. For example, you really want to ignore the max-prefix on
 >      the ROA, I'd think.

Not currently supported. Of 200+ IXPs using IXP Manager, I'm not sure I 
recall anyone ever asking for it.

 >    - It supports some features that I'm not sure how much people will
 >      care about:
 >      - RTT-based tagging/filtering.

I know a couple IXPs that have this on their own systems but I'm not 
sure they are used to any extent in reality. Same as above - of 200+ 
IXPs using IXP Manager, I'm not sure I recall anyone ever asking for it.

 >      - ADD-PATH capability (RFC7911)
 >      - BGP roles (RFC9234)
 >      - BGP graceful shutdown

All Bird configs - we'll do a general template review when we look at 
OpenBGPD as our goal will be parity of features and general review of 
the templates.

 > IXP Manager:
 >    - All in one. One less piece to worry about.

100%

 >    - Built in looking glass which will show why routes were rejected.
 >      This is really important for us, so participants have a way to know
 >      what is being filtered and why. I am not not sure how the looking
 >      glass functionality would need to be built with arouteserver.

The built in one-page showing all routers is a very popular feature for 
our members at INEX and a critical debugging tool for our operations team.

Hope that helps,
  - Barry






> Richard Laager via ixpmanager <mailto:ixpmanager at inex.ie>
> 27 April 2023 at 03:16
> MICE (in Minneapolis, MN USA) is using IXP Manager. We are looking to 
> replace our hand-configured route servers with automated ones, using 
> IXP Manager as the source of truth about participants.
>
> Notably, we are NOT doing IRR/RPKI filtering today. A big goal of the 
> project is to start doing that. A particular pain point will be people 
> whose routes will start being filtered.
>
> We are intending on using BIRD 2.x on Ubuntu 22.04.
>
> I am comparing arouteserver (which can accept a EuroIX export from IXP 
> Manager) and IXP Manager's direct configuration (which I found out 
> about from Barry O'Donovan's excellent videos on YouTube).
>
> What are the advantages of using IXP Manager directly?
>
> arouteserver features are listed here:
> https://arouteserver.readthedocs.io/en/latest/#features
>
> At this point, I'm thinking the trade-offs are:
>
> arouteserver:
>   - Unclear whether IXP Manager treats a valid ROA as an IRR pass.
>     I'm told this helps with stub networks who are missing IRR:
>
> https://arouteserver.readthedocs.io/en/latest/CONFIG.html#use-rpki-roas-as-if-they-were-route-objects 
>
>   - Supports PeeringDB "never via route server", which IXP
>     Manager seemingly does not:
>     https://github.com/inex/IXP-Manager/issues/798v
>   - Support blackhole (RTBH) community. We wanted to add this
>     at MICE, but it is unclear how much participants will actually
>     care. This does have complexities with integrating with IRR/RPKI
>     filtering. For example, you really want to ignore the max-prefix on
>     the ROA, I'd think.
>   - It supports some features that I'm not sure how much people will
>     care about:
>     - RTT-based tagging/filtering.
>     - ADD-PATH capability (RFC7911)
>     - BGP roles (RFC9234)
>     - BGP graceful shutdown
>
> IXP Manager:
>   - All in one. One less piece to worry about.
>   - Built in looking glass which will show why routes were rejected.
>     This is really important for us, so participants have a way to know
>     what is being filtered and why. I am not not sure how the looking
>     glass functionality would need to be built with arouteserver.
>

-- 

Kind regards,
Barry O'Donovan
INEX Operations

https://www.inex.ie/support/
+353 1 531 3339


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.inex.ie/pipermail/ixpmanager/attachments/20230427/4fe25d46/attachment-0001.htm>