Andrei Robachevsky. APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia. 1 New Version of the...

1Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net

New Version of the RIPE Database

Andrei Robachevsky

RIPE NCC

<[email protected]>

Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net

2

Outline

• Current status of the RIPE Database

• New database software

• Migration timeline

• More information


3

RIPE Database Status

• Contains• IP allocations/assignments• Domain registry• Routing registry

• 3.7 Million objects• 80% person, 10% inetnum, 0.65% route

• 6,700 updates/day• 770,000 queries/day (9 queries/s)

• 38% IP addresses, 1% IP prefixes


4

aut-num0,11%

domain10,43%

inetnum9,87%

person78,62%

role0,11%

route0,66%

as-macro0,04%

mntner0,15%

Other1,09%

Distribution by object type(February 2001)


5

0

5.000.000

10.000.000

15.000.000

20.000.000

25.000.0009/sec

Queries =~ 9/sec average


6

% of queries by object type(February 2001)

IP43%

domains27%

prefixes1%

other29% domains

IP

prefixes

other


7

0100.000200.000300.000400.000500.000600.000700.000800.000900.000

1.000.000

Updates 21/min -> 5/min


8

RIPE Database

• Whois service

• http://www.ripe.net/ripencc/pub-services/db/

• Database Consistency Project

• http://www.ripe.net/ripencc/pub-services/db/state/

• Routing Registry Consistency Check• http://www.ripe.net/ripencc/pub-services/db/rrcc/


9

What’s wrong with current version?

It’s good old software, but...

• RIPE-181 for routing policy description

• Lack of IRR security

• Poor scalability

• Performance limits

• Hard to maintain


10

New version of the RIPE Database

• Supports RPSL (RFC2622)• Extended syntax• New objects and attributes

• Supports RPSS (RFC2725)• New authorization rules

• Supports RAToolset• RtConfig -protocol bird

• Code is completely rewritten


11

RPSL Support

• Extended syntax rules apply to all object types• end of line comments• line continuation• order of attributes

• New objects• as-set (as-macro), route-set (community)• peering-set, filter-set, rtr-set

• New attributes• member-of• mbrs-by-ref

person: Test Person Objectsource: TESTnic-hdl: TP-TEST # nic handleaddress: Nobody knows where he lives…+remarks: be prepared to parse one


12

RPSS support

• New object• as-block

• New attributes• mnt-routes: <mnt_name> [ rpsl list of prefixes | ANY]• referral-by: <mnt_name>• auth-override: YYYYMMDD

• New authorization rules• route creation• aut-num• hierarchical names


13

RAToolset Support

• New queries• -l <ip range>• -x <ip range>• -K

• RtConfig -protocol bird• Patch is available

• to parse RIPE-style comments (%)• ftp://ftp.ripe.net/ripe/dbase/software/RAToolSet/


14

New software

• Mainly in C, multithreaded• RDBMS as a back-end

• MySQL, transaction support

• In-memory radix tree for IP lookups• also more and less specific lookups for reverse delegation

domains

• MIME and GPG support • correct PGP keys are also accepted

• Automatic access control• separate accounting for public and contact data


15

Server architectureE

-mai

l

RDBMS

Core Server

Update FE

Update FE

RDBMS

Mirror ServerNRTM clients

Queuerules

Messagequeues

Syntax checks,acks, notifications

qu

eries


16

What’s different ?

• Extended object syntax• Modified objects• New attributes• New objects• New query flags

person: Test Person Objectsource: TESTnic-hdl: TP-TEST # nic handleaddress: Nobody knows where he lives…+remarks: be prepared to parse one

Modified objects:mntnerrouteaut-numas-set (was: as-macro)route-set (was: community)inet-rtrinetnum

New objects:as-blockrtr-setpeering-setfilter-set

New attributes:member-ofmbrs-by-refmnt-routesreferral-byauth-override

New query flags:-l <ip range>-x <ip range>-K-d-q sources [<source>]-q version

Access control:%ERROR:202: access control limit reached % You have reached the limit of returned contact information objects. % This connection will be terminated now. % Continued attempts to return excessive amounts of contact % information will result in permanent denial of service.

• New access control• New database format• New version of the mirroring protocol

RDBMS (MySQL):CREATE TABLE mntner ( thread_id int(11) DEFAULT '0' NOT NULL, object_id int(10) unsigned DEFAULT '0' NOT NULL, mntner varchar(80) DEFAULT '' NOT NULL, dummy tinyint(4) DEFAULT '0' NOT NULL, PRIMARY KEY (object_id));

New NRTM protocol:

was:UPD = (ADD + DEL)

will be:UPD = ADD


17

Who will be affected ?

• Query users• new query flags

• Update users• new syntax rules• new authorization rules

• Scripts• new object format and syntax• new/modified objects and attributes• access control

• NRTM clients• new software• new version of the mirroring protocol


18

Transition timeline - Updates

Updates in RIPE-181to <[email protected]>

Updates in RPSLto <[email protected]>

Updates in RPE-181to <[email protected]>RIPE181

RPSL

Production

Prototype/Compatibility

TEST

Updates in RIPE-181to <[email protected]>




Proposed dates: X=23 April Y=14 May Z=15 October


19

Transition timeline - Queries

Querying RIPE DB in RIPE-181at whois.ripe.net :43

Querying RIPE DB in RPSL at rpsl.ripe.net :43

Additional flags available

Querying RIPE DB in RPSLat whois.ripe.net : 43

Additional flags available

RIPE-181v2.x

RPSLv3.0

Production

Prototype

Proposed date: X=23 April


20

Transition timeline - NRTM

Mirroring RIPE DB in RIPE-181at whois.ripe.net :43

Mirroring RIPE DB in RPSL at rpsl.ripe.net :4444

Mirroring RIPE DB in RPSLat whois.ripe.net : 4444

RIPE181v2.x

RPSLv3.0

Production

Prototype

Proposed date: X=23 April


21

Project Status

• Version 3.0ß2 has been released• Core server functionality is complete• Infrastructure is under development• Testing is in progress• Portability issues are on our list

• Solaris, Linux, FreeBSD, UnixWare(?), ...• Thanks to everyone who helps make it more portable

• Special thanks to George Michaelson!


22

Prototype servers

• Near real-time mirror of the RIPE Database• whois -h rpsl.ripe.net• contains live RIPE Database in RPSL format

• Test server for submissions• mail <[email protected]>• whois -h rpsl.ripe.net -p 4343

• NRTM• rpsl.ripe.net, port 4444• please contact <[email protected]>


23

More Information

• RIPE-181 to RPSL Migration page• http://www.ripe.net/rpsl

• Documentation• Transition to the RIPE DB v3.0• Whois Queries in the RIPE DB v3.0• Updates in the RIPE DB v3.0• Error codes in the RIPE DB v3.0

• Software• New whois client

ftp://ftp.ripe.net/ripe/dbase/reimp/whoisRIP-1.0.tar.gz• Server software v3.0

http://www.ripe.net/ripencc/pub-services/db/reimp/latestbeta.html


24

Questions?

25Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net

New Version of the RIPE Database

Andrei Robachevsky

RIPE NCC

<[email protected]>


26

New objects

• peering-set• filter-set• rtr-set• as-block


27

New attributes

• RPSL:• member-of, mbrs-by-ref

• RPS-auth:• mnt-routes: <mnt_name> [ rpsl list of prefixes | ANY]• referral-by: <mnt_name>• auth-override: YYYYMMDD


28

Modifications to all objects

• Line continuation• Attribute order is relevant• Support for end of line comments• Handling of empty attributes• Legend:

holes: [optional] [multiple] automatically translated member-of: [optional] [multiple] newcross-nfy: [optional] [multiple] preservedcommunity: [optional] [multiple] deprecated


29

Modified objects

• mntner objectmntner: [mandatory] [single] [primary/look-up key]descr: [mandatory] [multiple] [ ]admin-c: [mandatory] [multiple] [inverse key]tech-c: [optional] [multiple] [inverse key]upd-to: [mandatory] [multiple] [inverse key]mnt-nfy: [optional] [multiple] [inverse key]auth: [mandatory] [multiple] [ ]remarks: [optional] [multiple] [ ]notify: [optional] [multiple] [inverse key]mnt-by: [mandatory] [multiple] [inverse key]auth-override: [optional] [single] [ ] *** RPS auth ***referral-by: [mandatory] [single] [inverse key] *** RPS auth ***changed: [mandatory] [multiple] [ ]source: [mandatory] [single] [ ]


30

Modified objects

• route object

route: [mandatory] [single] [primary/look-up key]descr: [mandatory] [multiple] [ ]origin: [mandatory] [single] [primary/inverse key]holes: [optional] [multiple] [ ] *** hole in RIPE 181 ***withdrawn: [optional] [single] [ ] comm-list: [optional] [multiple] [ ] advisory: [optional] [multiple] [ ] member-of: [optional] [multiple] [inverse key] *** RPSL ***inject: [optional] [multiple] [ ] *** RPSL ***aggr-mtd: [optional] [single] [ ] *** RPSL ***aggr-bndry: [optional] [single] [ ] *** RPSL ***export-comps:[optional] [single] [ ] *** RPSL ***components: [optional] [single] [ ] *** RPSL ***cross-nfy: [optional] [multiple] [inverse key]community: [optional] [multiple] [ ]mnt-lower: [optional] [multiple] [inverse key] *** RPS auth ***mnt-routes: [optional] [multiple] [inverse key] *** RPS auth ***mnt-by: [mandatory] [multiple] [inverse key]changed: [mandatory] [multiple] [ ]source: [mandatory] [single] [ ]


31

Modified objects

• autnum objectaut-num: [mandatory] [single] [primary/look-up key]as-name: [mandatory] [single]descr: [mandatory] [multiple]as-in: [optional] [multiple] [ ] as-out: [optional] [multiple] [ ] interas-in: [optional] [multiple] [ ] interas-out: [optional] [multiple] [ ] as-exclude: [optional] [multiple] [ ] member-of: [optional] [multiple] [inverse key] *** New in RPSL *** import: [optional] [multiple] *** as-in in RIPE 181 ***export: [optional] [multiple] *** as-out in RIPE 181 ***default: [optional] [multiple]remarks: [optional] [multiple]admin-c: [mandatory] [multiple] [inverse key]tech-c: [mandatory] [multiple] [inverse key]cross-mnt: [optional] [multiple] [inverse key]cross-nfy: [optional] [multiple] [inverse key]notify: [optional] [multiple] [inverse key]mnt-lower: [optional] [multiple] [inverse key] *** RPS auth ***mnt-routes: [optional] [multiple] [inverse key] *** RPS auth ***mnt-by: [mandatory] [multiple] [inverse key]changed: [mandatory] [multiple]source: [mandatory] [single]


32

Modified objects

• as-set (previously as- macro)as-set: [mandatory] [single] [primary/look-up key] *** as-macro in RIPE 181 ***descr: [mandatory] [multiple]members: [optional] [multiple] *** as-list in RIPE 181 ***mbrs-by-ref: [optional] [multiple] [inverse key] *** New in RPSL *** remarks: [optional] [multiple]tech-c: [mandatory] [multiple] [inverse key]admin-c: [mandatory] [multiple] [inverse key]notify: [optional] [multiple] [inverse key]mnt-by: [mandatory] [multiple] [inverse key]changed: [mandatory] [multiple]source: [mandatory] [single]


33

Modified objects

• route-set (previously community)route-set: [mandatory] [single] [primary/look-up key] *** community in RIPE 181 ***descr: [mandatory] [multiple]members: [optional] [multiple] *** New in RPSL ***mbrs-by-ref: [optional] [multiple] [inverse key] *** New in RPSL ***remarks: [optional] [multiple]tech-c: [mandatory] [multiple] [inverse key]admin-c: [mandatory] [multiple] [inverse key]notify: [optional] [multiple] [inverse key]mnt-by: [mandatory] [multiple] [inverse key]changed: [mandatory] [multiple]source: [mandatory] [single]


34

Modified objects

• inet-rtrinet-rtr: [mandatory] [single] [primary/look-up key]descr: [mandatory] [multiple]alias: [optional] [multiple] *** New in RPSL ***local-as: [mandatory] [single] [inverse key] *** localas in RIPE 181 ***ifaddr: [mandatory] [multiple] [look-up key]peer: [optional] [multiple]member-of: [optional] [multiple] [inverse key] *** New in RPSL ***remarks: [optional] [multiple]admin-c: [mandatory] [multiple] [inverse key]tech-c: [mandatory] [multiple] [inverse key]notify: [optional] [multiple] [inverse key]mnt-by: [mandatory] [multiple] [inverse key]changed: [mandatory] [multiple]source: [mandatory] [single]


35

Modified objects

• inetnuminetnum: [mandatory] [single] [primary/look-up key]netname: [mandatory] [single] [lookup key]descr: [mandatory] [multiple] [ ]country: [mandatory] [multiple] [ ]admin-c: [mandatory] [multiple] [inverse key]tech-c: [mandatory] [multiple] [inverse key]rev-srv: [optional] [multiple] [inverse key]status: [generated] [single] [ ]remarks: [optional] [multiple] [ ]notify: [optional] [multiple] [inverse key]mnt-by: [mandatory] [multiple] [inverse key]mnt-lower: [optional] [multiple] [inverse key]mnt-routes: [optional] [single] [inverse key] *** RPS auth *** changed: [mandatory] [multiple] [ ]source: [mandatory] [single] [ ]


36

New object: peering-set

• Peering-set

peering-set: [mandatory] [single] [primary/look-up key]descr: [mandatory] [multiple]peering: [mandatory] [multiple]remarks: [optional] [multiple]tech-c: [mandatory] [multiple] [inverse key]admin-c: [mandatory] [multiple] [inverse key]notify: [optional] [multiple] [inverse key]mnt-by: [mandatory] [multiple] [inverse key]changed: [mandatory] [multiple]source: [mandatory] [single]

<=


37

New object: filter-set

• defines a set of routes that are matched by its filter

filter-set: [mandatory] [single] [primary/look-up key]descr: [mandatory] [multiple]filter: [mandatory] [single]remarks: [optional] [multiple]tech-c: [mandatory] [multiple] [inverse key]admin-c: [mandatory] [multiple] [inverse key]notify: [optional] [multiple] [inverse key]mnt-by: [mandatory] [multiple] [inverse key]changed: [mandatory] [multiple]source: [mandatory] [single]

<=


38

New object: rtr-set

• defines a set of routers specified by inet-rtr names, ipv4_addresses or other rtr-set names

rtr-set: [mandatory] [single] [primary/look-up key]descr: [mandatory] [multiple]members: [optional] [multiple]mbrs-by-ref: [optional] [multiple]remarks: [optional] [multiple]tech-c: [mandatory] [multiple] [inverse key]admin-c: [mandatory] [multiple] [inverse key]notify: [optional] [multiple] [inverse key]mnt-by: [mandatory] [multiple] [inverse key]changed: [mandatory] [multiple]source: [mandatory] [single]

<=


39

New object: as-block

• Defines a range of AS numbers delegated to a given repository

as-block: [mandatory] [single] [primary/look-up key]descr: [optional] [multiple]remarks: [optional] [multiple]tech-c: [mandatory] [multiple] [inverse key]admin-c: [mandatory] [multiple] [inverse key]notify: [optional] [multiple] [inverse key]mnt-lower: [optional] [multiple] [inverse key]mnt-by: [mandatory] [multiple] [inverse key]changed: [mandatory] [multiple]source: [mandatory] [single]

<=


40

Queries

• New queries• -l <ip range>• -x <ip range>• -K• -d• -q sources [<source>]• -q version

• Inverse queries• Other differences


41

-l <ip range>

• One level less specific• Does not return the exact match• Returns the smallest IP range that is bigger than

the supplied range and that fully contains it• whois -r -Tin 193.0.0.0/23• whois -r -Tin -l 193.0.0.0/23• whois -r -Tin -L 193.0.0.0/23


42

-x <ip range>

• Exact match• If no matching object is found nothing is returned• whois -r -Tin 193.0.2.0/24• whois -r -Tin -x 193.0.2.0/24


43

-K

• Only primary keys are returned• Exception is a set object, where the members

attribute is also returned• Does not apply to person and role objects• whois -Trt -K -M 193.0.0.0/16• whois -K -imo RS-HEPNET• whois -K AS-WORLD


44

-d (proposed)

• Triggers inclusion of in-addr.arpa and ip6.int domain objects in the result of IP lookup

• More/less specific lookups are possible• whois -r -d 193.0.2.0• whois -d -Tdn -K -M 193.0.0.0/20


45

Accounting and Access Control

• Access to “public” and “contact” data is accounted differently

• Is based on number of objects returned• limit = f(max_limit1, query_rate)

• when limit is hit - the query is aborted and limit =0• limit recovers in time • # of times the limit may be hit before permanent denial

• Trusted proxies: accounting is based on client’s IP


46

Authorization of route creationroute: 10.1.0.0/16mnt-by: M2-MNT...

inetnum: 10.1.0.0 - 10.1.255.255mnt-by: M1-MNT...

aut-num: AS65000mnt-by: M3-MNT...

route: 10.1.1.0/24origin: AS65000mnt-by: M4-MNT...

mntner: M1-MNTauth:...





47

Membership of set objects

route-set: RS-FOOmbrs-by-ref: MNT-FOOBAR...

route: 193.0.0.0/22origin: AS3333member-of: RS-FOOmnt-by: MNT-FOOBAR...

route: 192.168.0.0/24origin: AS3333member-of: RS-FOOmnt-by: OTHER-MNT...

as-set: AS-BARmembers: AS3333mbrs-by-ref: MNT-FOOBAR...

aut-num: AS3333...

aut-num: AS3267member-of: AS-BARmnt-by: MNT-FOOBAR...

Andrei Robachevsky. APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia. 1 New Version of the...

Documents

Transcript of Andrei Robachevsky. APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia. 1 New Version of the...