OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.
-
Upload
theresa-rose -
Category
Documents
-
view
216 -
download
2
Transcript of OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.
![Page 1: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/1.jpg)
OpenAFS in a multihomed universe
Jeffrey Altman
Derrick Brashear
![Page 2: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/2.jpg)
Topics
This is really 2 discussions:• Configuring and using multi-homed clients.
• Configuring and using multi-homed servers. It’s important to understand how we got
here. Some issues are still “on the table”.
![Page 3: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/3.jpg)
Background
When AFS development began, multi-homed hosts were uncommon.
Assumptions made early did not accommodate multi-homed hosts.
As commercial (Transarc) customers began to require multi-homed support, changes were made over time to parts of AFS.
• Some offered better functionality than others.
• Full integration wasn’t always possible.
![Page 4: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/4.jpg)
Architecture (abridged)
VLserver• Used to discover volume locations.
• Uses ubik distributed database technology to provide current copies of data, provided a fully-replicated service.
Fileserver• Registers its address with the VLserver.
• Used to provide data to clients.
• Must track clients to provide callback notifications.
Client• Uses VLserver to discover data location, then fetch it from
fileservers on behalf of users.
![Page 5: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/5.jpg)
Issues: VLserver
How do clients find the VLserver in the first place?• Hardcoded address in CellServDB.
• IBM never solved this; you had to maintain copies of the file if you needed to address some or all database servers by different addresses on different groups of clients.
![Page 6: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/6.jpg)
Solutions: VLserver
AFSDB
• Support for AFSDB DNS records added by OpenAFS volunteers.
• Use of split-horizon DNS allows different information to be distributed to different groups of clients, but still be managed in few places.
![Page 7: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/7.jpg)
Issues: fileserver
Original VL RPCs returned a location which was server and partition.• Server meant “one IP address”.
If you had a different address space than the one in which the VLserver was providing answers, you were sad.
![Page 8: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/8.jpg)
Solutions: fileserver
UUID• Fileservers generate a “universally-unique
identifier” which can be used in place of a single address.
• Included in the data structure with the actual UUID number is a list of addresses the fileserver registered with the VLserver.
• Revised VL RPCs use UUID instead of address; New clients and servers use the new RPCs when available.
![Page 9: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/9.jpg)
Issues: client
The client itself can have more than one address.• Fileserver tracking of client by IP address only.
• A fixed client port of 7001 was assumed in some cases.
Worse, the client can have an alternate port!
![Page 10: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/10.jpg)
Solutions: client
Clients also get a UUID.• Fileserver callback interfaces changed to
support UUIDs rather than addresses.
• Port assumptions removed.
![Page 11: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/11.jpg)
11
Complications
It’s not that simple. NAT
• A client can be behind a NAT and not know its real address, precluding it from telling the server its actual address.
• A server can be behind a NAT! You may not want to advertise all of your
addresses!
11
![Page 12: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/12.jpg)
12
Enter NetInfo and NetRestrict
In order to only publish some addresses, or mask some addresses, a new interface was created, for both clients and servers.
NetInfo is a list of addresses to publish. NetRestrict is a list of addresses to
mask. Typically you use one or the other.
![Page 13: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/13.jpg)
13
NetInfo and NAT
Remember servers can also be behind NAT!• NetInfo supports fake addresses.
• The only gotcha here is that you need to know the “other” address of your server.• If it changes, you need to update NetInfo and restart.
For clients, it matters less.
13
![Page 14: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/14.jpg)
Client addresses
Who can you trust?•Clients behind a NAT will not know their “real”
external address.
•Advertised internal addresses may be non-routable.
Fileserver updated to use the calling address and not use addresses advertised to it without first verifying they are the same host.
![Page 15: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/15.jpg)
Forget your problems?
Just because a host has multiple addresses doesn’t mean you want to use them all.
Rx RPC library built on a single socket model.
• Allow socket to be bound to exactly one address. (“rxbind”)• Requires list of addresses as modified by NetInfo
and/or NetRestrict to be whittled to exactly one address to work.
![Page 16: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/16.jpg)
Or at least make them manageable
Clients support the notion of server preferences.
• Server preferences are by address, not UUID!
• Preference can be given to a particular address for a server.• Sadly, the defaults are based on classful networking,
which is not as useful as it could be.
![Page 17: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/17.jpg)
Except when you can’t
Ubik databases will have one primary address.• True multi-homing is not supported in the
internal ubik interfaces.
• Related, special powers are assigned always to the member of a ubik service pool with the lowest IP address.
![Page 18: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/18.jpg)
What’s still needed
Truly multi-homed Ubik.A way to change what addresses a
fileserver is advertising, on the fly.A way to discover what addresses those
should be other than manual configuration.Server preference support in clients which
is not based on classful networks.Multi-socket Rx.
• Or maybe just finish Rx TCP and IPv6 support.
![Page 19: OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear.](https://reader036.fdocuments.us/reader036/viewer/2022082711/56649ece5503460f94bdaf41/html5/thumbnails/19.jpg)
OpenAFS in a multi-homed universe
Questions?