[ixpmanager] RADB WHOIS Rate Limiting

[Richard Laager]

I seem to be getting rate limited by RADB with update-asn-db / 
update-prefix-db. It will get partway through and then die. Doubly 
unfortunate is the fact that IXP Manager doesn't keep track of where it 
was in the cache run, so the next time it starts over again at the top. 
As a result, networks farther down probably never get updated.

Are other people running into this?


https://www.radb.net/support/informational/query.html says:

> Please note: The RADb WHOIS service is rate-limited. For large 
> queries, we recommend using the RADb API
>
> Programmatic access is considered abuse of the service.

Isn't that exactly what IXP Manager / bgpq3 / bgpq4 are doing???


Later, it says:

> If you plan on making a large number of queries please invoke a 
> persistent TCP/IP session. This is done by telnetting directly to 
> whois.radb.net and issuing the !! command. This will spare our server 
> having to establish and teardown connections for every query.

bgpq4, at least, does that. But IXP Manager is still separately invoking 
bgpq4 for each AS when running update-asn-db / update-prefix-db.

-- 
Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.inex.ie/pipermail/ixpmanager/attachments/20230607/41083571/attachment.htm>