DRAFT
1 | P a g e D R A F T
DRAFT SIS-DTN Green Book 20090304
Table of Contents
Revision History ....................................................................................................................................... 2
1 Introduction ....................................................................................................................................... 2
1.1 Purpose ....................................................................................................................................... 2
1.2 Scope .......................................................................................................................................... 2
1.3 Document Structure .................................................................................................................... 2
1.4 Conventions and Definitions ...................................................................................................... 2 1.5 References .................................................................................................................................. 2
2 Rationale ........................................................................................................................................... 3
2.1 Overview .................................................................................................................................... 3
2.2 Current Mission Architecture ..................................................................................................... 5 2.3 Recent Advances ........................................................................................................................ 5
2.4 Benefits of Networked Communications ................................................................................... 6 2.5 Future Missions .......................................................................................................................... 6
2.6 Next Steps In Networking .......................................................................................................... 6
3 Requirements and Desired Characteristics of a Space Internetworking Protocol ............................ 7
3.1 Requirements of a Space Internetworking Service .................................................................... 7 3.2 Desirable Characteristics ............................................................................................................ 9
4 Scenarios ........................................................................................................................................... 9
4.1 PDU Delivery in the Presence of Disruptions ............................................................................ 9 4.2 PDU Delivery in the Presence of Disruptions and Requiring Proactive Fragmentation ......... 10
4.3 PDU Delivery in the Presence of Disruptions and Requiring Reactive Fragmentation ........... 11
4.4 Prioritized PDU Delivery in the Presence of Disruptions and Requiring Reactive Fragmentation ..................................................................................................................................... 11
4.5 PDU Delivery in the Presence of Route Changes .................................................................... 12 5 Candidate Space Internetworking Technologies ............................................................................ 12
5.1 Custom Data Forwarding (Mars Exploration Rovers, Mars Phoenix Lander) ........................ 12
5.2 CCSDS Space Packets ............................................................................................................. 12
5.2.1 Features of Space Packets ................................................................................................. 13 5.3 CCSDS File Delivery Protocol (CFDP) ................................................................................... 13
5.3.1 CFDP Features .................................................................................................................. 13 5.4 The Internet Protocol (IPv4/IPv6) ............................................................................................ 14 5.5 Delay / Disruption Tolerant Networking ................................................................................. 15
5.5.1 Services RFC5050 Requires of Lower Layers .................................................................. 17 5.5.2 Naming of Bundle Protocol Endpoints ............................................................................. 18 5.5.3 Bundle Protocol Forwarding and Routing ........................................................................ 18 5.5.4 Bundle Protocol Network Management ............................................................................ 18
5.6 Conclusions .............................................................................................................................. 19
6 Use Cases / Services ....................................................................................................................... 19
6.1 Bundle Protocol PDU Delivery Service ................................................................................... 19 6.2 Implementing Existing High-Level Services Over The Bundle Protocol ................................ 19
6.2.1 File Transfer Service ......................................................................................................... 19
DRAFT
2 | P a g e D R A F T
6.2.2 Messaging Service ............................................................................................................ 20 6.3 New High-Level Services Requiring Specification ................................................................. 20
6.3.1 Space Packet / Encapsulation Packet Delivery Services .................................................. 21
6.3.2 Bitstream Delivery Services ............................................................................................. 22 6.3.3 Hardware Commanding Service ....................................................................................... 22
7 Notional Architecture and Transition Path ..................................................................................... 24 7.1 Transition Path ......................................................................................................................... 25
7.2 Interoperable Protocol Stacks at Interfaces .............................................................................. 25 7.3 Transitioning CFDP to run over DTN ...................................................................................... 25 7.4 Coexistence of DTN, IP, and Space Packets ............................................................................ 26 7.5 Initial DTN Operations ............................................................................................................. 26
Revision History Date Version Comments
0.2 First version distributed to WG 20081205 0.6 Attempted to address, or at least log, comments by Chris Taylor and Jane
Marquart. 20090406 0.7 Incorporated comments and included architectural pictures derived from
SISG discussions.
1 Introduction
1.1 Purpose
This document describes the rationale, scenarios, and use cases to be used to evaluate a proposed Delay / Disruption Tolerant Networking (DTN) protocol targeted at the space internetworking environment. A space internetworking architecture will allow different agencies to share extra-terrestrial (in space and at other planets) resources and to provide cross-support to one another, even if the end systems are not directly accessible from the Earth. A common space internetwork design will support interoperability and lower design costs. This in turn will allow resource sharing and the opportunity for greater science return and reduced mission risk.
1.2 Scope
This document will guide the evaluation / development of DTN and supporting protocols for space internetworking.
1.3 Document Structure
XXX
1.4 Conventions and Definitions
Internetworking – constructing a more far-reaching network by defining a protocol layer that supports end-to-end delivery of data across multiple, possibly heterogeneous data link layer technologies.
1.5 References
[CFDP] CCSDS 727.0-B-4 CCSDS File Delivery Protocol (CFDP). Blue Book. Issue 4.
DRAFT
3 | P a g e D R A F T
January 2007. [CFDP-Green] CCSDS 720.1-G-3 CCSDS File Delivery Protocol (CFDP) – Part 1: Introduction
and Overview, Green Book, Issue 3, April 2007. [MARS Green Book] CCSDS 740.0-G-1 Mars Mission Protocol Profiles--Purpose and Rationale.
Green Book. Issue 1. July 2008. [CCSDS SPP] CCSDS 133.0-B-1 Space Packet Protocol. Blue Book. Issue 1. September
2003. [CGR] Dynamic Routing for Delay-Tolerant Networking in Space Flight Operations, S.
Burleigh, SpaceOps 2008. [SISG Report] Recommendations on a Strategy for Space Internetworking, November 15, 2008. [RFC791] RFC791, Internet Protocol, J. Postel, September 1981. [RFC5050] RFC5050, Bundle Protocol Specification, K. Scott and S. Burleigh, November
2007. [RFC4838] RFC4838, Delay-Tolerant Networking Architecture, V. Cerf et. al., April 2007. [CBHE] Compressed Bundle Header Encoding (CBHE), draft-irtf-dtnrg-cbhe-01, Scott
Burleigh, March 5, 2009.
2 Rationale
2.1 Overview
Current mission communication architectures are essentially point-to-point between the mission control center and the spacecraft. Standardization of a suite of cross-support services on the ground has and is continuing to extend this model so that agencies can share resources such as ground stations for cross-support. This sharing is implemented by tunneling the space data link from the ground station across the terrestrial communications infrastructure to the control center, so that the ground station and terrestrial infrastructure becomes transparent to the space data link protocol. Said another way, the space data link protocol is terminated on board the spacecraft and in the control center. In some instances it is desirable to provide extra network ‘hops’ in space instead of using only a single data link between the mission control center and the spacecraft. Space relays can buffer data that cannot be transferred end-to-end due to visibility constraints, provide points for signal regeneration, switch data link layers to match the environment, and serve as decision points for data forwarding (routing). Communications from the Mars Exploration Rovers (MERs) Spirit and Opportunity are a good example of this, where relaying data to Earth via a Mars orbiter has:
• Increased the volume of data the missions have been able to return. • Decreased the power required to return that data • Enabled increased mission objectives (such as moving further due to increased power available
to the driving subsystem) These are direct results of using in-space relaying instead of direct-to-Earth data transfers from the rovers. Because the orbiting relays use different data link layers for Mars surface-to-orbit communications than for the Mars orbit-to-Earth links, they provide different data link services that are better suited to the local environments. For orbiter-to-Mars-surface communications, Prox-1 can be used in its reliable mode since round trip times are small and automatic repeat request (ARQ) is both feasible and efficient. Reliability ensures that important data is successfully transferred before moving on to less important data. This reliability can not be performed between the Mars surface and Earth due to the much longer round trip times. For the long-haul Mars-orbiter-to-Earth link, traditional telecommand and telemetry, including more powerful coding, are used.
DRAFT
4 | P a g e D R A F T
Typically a network-layer end-to-end data structure is used when data needs to be transferred across possibly heterogeneous links. This end-to-end data structure allows for logical addressing of the endpoints independent of the data link layer addresses and has some multiplexing mechanism to support higher-layer protocols. CCSDS currently has three recommended data structures that could serve as end-to-end network layer protocols: the CCSDS Space Packet Protocol, the Space Communications Protocol Specification – Network Protocol (SCPS-NP), and the Internet Protocols (IPv4/IPv6). For various reasons detailed below, we do not believe that any of these can form the basis of a general network-layer protocol for space communications. Section 5 discusses candidate networking technologies in more detail. Here we briefly review the following methods:
• Custom forwarding (application-layer forwarding) • CCSDS Space Packets as an internetworking packet format • CCSDS File Delivery Protocol (CFDP) • Internet Protocol (IP) • The Bundle Protocol from Delay / Disruption Tolerant Networking (DTN)
Custom forwarding, while sometimes expedient, is not a good basis for standardization and interoperability. The current custom forwarding solution at Mars, while a great leap forward, is inefficient and does not enable some of the benefits of internetworking. While CCSDS Space Packets have been proposed in the past as a networking layer for space, their use would rely on fields in the CCSDS data link headers for end-to-end addressing. This would make their use problematic if, for example, space agencies were to deploy non-CCSDS links such as WiMax on the Moon or other planets. CFDP extended procedures and store-and-forward overlay could both serve as a basis for internetworking, but are tied to a file transfer application. Both the Internet Protocols (IPv4/IPv6) and the Bundle Protocol (DTN) provide network-layer addressing of data independent of the data links used. The IP protocol suite (including IP and the associated protocols such as routing, domain name service, etc.) is very mature and well-understood terrestrially, but makes some assumptions that limit its utility as a fully general space internetworking protocol. Specifically, the bulk of experience with the IP suite assumes well-connected end-to-end paths, while mature terrestrial IP routing protocols assume relatively stable network topologies. Some other aspects of the IP suite, such as resolving Domain Name Service (DNS) names to IP addresses, assume low latencies as well as connectivity. Where these assumptions can be made to hold or where static tables can be used in place of network lookups, such as in near-Earth (and possibly out to Lunar) environments, the IP suite, including commonly-used IP-based applications, can be used. Of the protocols mentioned, the Bundle Protocol associated with Delay / Disruption Tolerant Networking seems best-suited for the widest range of space internetworking environments. Like IP, the Bundle protocol provides an internetwork-layer data unit with end-to-end addressing capabilities. Unlike IP, however, the Bundle Protocol does not assume continuous connectivity and specifically allows for in-network data storage such as might take place when Earth can transmit to an orbiter which then has to hold on to the data until the orbiter can relay the data to a landed asset. Finally, different mission requirements will probably drive development of parallel architectures, at least in parts of the design space, where some subset of the above mechanisms may all coexist.
5 | P a g e
2.2 Current Mission Architecture
In the 20th century, science and exploration spacecraft were built to communicate primarily with ground stations, with commands flowing from ground control center to spacecraftdata flowing from spacecraft to ground. There were few cases where a scommunicate directly with another spacecraft or with multiple control centers on the ground.situation is illustrated in Figure 1
Figure 1: Traditional point
This approach was successful and has supported many missions, but the data architecture that has evolved provides limited support for multi
1. CCSDS recommendations that allow for incompatible implementations.2. Data structures optimized for managed, point
The first of these can be addressed by modifying existing recommendations and/or by constructing ‘profiles’ that restrict the protocol options in particular mission groups such as Mars relay operations.The second problem requires a new protocol or suite of protocols to be developed that better support automated multi-hop forwarding of data.
2.3 Recent Advances
Experience with data relaying at Mars has demonstrated a number of advantages over traditional directto-Earth communications. These include:
• Increased science data return substantially increase data return from ~30Mb/sol achievable with the Directlink to ~100-200Mb/sol via the Mars Odyssey orbiter.
• Lower power required – The MER DTE links require roughly 5 Watt hours per Mb of data return, while the relay uses around 0.1 Watt homission concepts and increases the amount of energy available for science.
• Lower mass required – Relay operations require lower mass on landed elements which are typically more mass-constrained than orbiters.
• More communications opportunities opportunities that DTE links. This in turn supports complex inexecute multiple command/telemetry cycles per sol.
DRAFT
D R A F T
Current Mission Architecture
century, science and exploration spacecraft were built to communicate primarily with ground stations, with commands flowing from ground control center to spacecraftdata flowing from spacecraft to ground. There were few cases where a science spacecraft would communicate directly with another spacecraft or with multiple control centers on the ground.
1.
Traditional point-to-point space mission architecture.
and has supported many missions, but the data architecture that has evolved provides limited support for multi-hop networking for two reasons:
CCSDS recommendations that allow for incompatible implementations.Data structures optimized for managed, point-to-point communications.
The first of these can be addressed by modifying existing recommendations and/or by constructing ofiles’ that restrict the protocol options in particular mission groups such as Mars relay operations.
The second problem requires a new protocol or suite of protocols to be developed that better support hop forwarding of data.
xperience with data relaying at Mars has demonstrated a number of advantages over traditional directEarth communications. These include:
Increased science data return – The Mars Exploration Rovers have used data relaying to e data return from ~30Mb/sol achievable with the Direct
200Mb/sol via the Mars Odyssey orbiter. The MER DTE links require roughly 5 Watt hours per Mb of data
return, while the relay uses around 0.1 Watt hour per Mb. This enables small scoutmission concepts and increases the amount of energy available for science.
Relay operations require lower mass on landed elements which are constrained than orbiters.
communications opportunities – Relaying typically supports more communications opportunities that DTE links. This in turn supports complex in-situ missions that might want to execute multiple command/telemetry cycles per sol.
century, science and exploration spacecraft were built to communicate primarily with ground stations, with commands flowing from ground control center to spacecraft, and telemetry and
cience spacecraft would communicate directly with another spacecraft or with multiple control centers on the ground. This
point space mission architecture.
and has supported many missions, but the data architecture that has
CCSDS recommendations that allow for incompatible implementations. point communications.
The first of these can be addressed by modifying existing recommendations and/or by constructing ofiles’ that restrict the protocol options in particular mission groups such as Mars relay operations.
The second problem requires a new protocol or suite of protocols to be developed that better support
xperience with data relaying at Mars has demonstrated a number of advantages over traditional direct-
The Mars Exploration Rovers have used data relaying to e data return from ~30Mb/sol achievable with the Direct-to-Earth (DTE)
The MER DTE links require roughly 5 Watt hours per Mb of data ur per Mb. This enables small scout-class
mission concepts and increases the amount of energy available for science. Relay operations require lower mass on landed elements which are
Relaying typically supports more communications situ missions that might want to
DRAFT
6 | P a g e D R A F T
Current relay operations at Mars implement multi-hop relaying without true internetworking. There is no network-wide addressing scheme, no provision for different classes of data, and no true end-to-end network layer data unit. These deficiencies will inhibit operations as more elaborate missions involving orders-of-magnitude-more systems and communication links, as well as human crews, are developed.
2.4 Benefits of Networked Communications
The data relay benefits described above in the context of the MER missions are an example of benefits achievable within a single agency. Standardizing the relay (network-layer) protocols will enable the same types of cross-support in space that are currently possible on the ground, with the additional benefits of signal regeneration at the relays, switching data link layers to suit the local environment, and the ability to make routing decisions on board the relays. The Space Internetworking Strategy Group (SISG REF) chartered by the Inter-Agency Operations Advisory Group (IOAG) has come to similar conclusions when examining the current and future states of space internetworking. In particular, the SISG report makes the following recommendations:
• Recommendation R-1: There should be international agreement on how to do space-to-space interoperability and space-based infrastructure that supports space-to-space interoperability in a standard way.
• Recommendation R-2: In-space internetworking should be fully verified as feasible in long delay mission environments.
• Recommendation R-3: There should be international agreement on how to manage space-to-space or end-to-end interoperability.
• Recommendation R-4: There should be interoperable services for timing, positioning, management, etc., in addition to services for relaying data.
2.5 Future Missions
Planning is also underway for missions that envision multiple nodes that communicate not only between space and ground but also among systems in space. Managing the connectivity and data transfers among this increasing number of systems will become more and more difficult. The situation is reminiscent of the early days of telephones and switchboards. When the number of systems was sufficiently small, human circuit switching with operators in the loop was possible. As the number of users grew, the phone system had to evolve to automated switching systems that were fully computer controlled and software upgradeable. The future space communication architecture requires a similar shift from traditional circuit-switched space communication toward a more flexible network architecture for space communication.
2.6 Next Steps In Networking
By standardizing a multi-hop relay mechanism, CCSDS member agencies will lay one part of the technical foundation for interoperability and cross-support in space. By developing a set of DTN protocols to be provided in subsequent documents, common data handling functions can be implemented in standard and hopefully reusable software/hardware. Moving these capabilities into the infrastructure allows mission software to focus on mission-specific functions instead of ‘re-inventing the wheel’ with each mission when it comes to communications. Finally, a
DRAFT
7 | P a g e D R A F T
common set of protocols for space-based internetworking will enable inter-agency cross-support which should increase science data return and decrease mission risk. Relay operations will depend on interoperability at the lower layers of the communications stack such as the physical and data link layers for compatible frequencies, modulation schemes, coding, etc. Thus one recommendation of this document is to further specify the protocols and protocol options needed for interoperability of space data links. The internetworking capabilities should support fully networked interoperability as shown in the following figure.
Lander
Control
Center
Orbiter1
Control
Center
Orbiter2
Control
Center
Ground
Station 1
Ground
Station 2
Orbiter 1
Orbiter 2
Figure 2: Possible data paths in a cross-supported, networked architecture.
Given the above motivation, there are a number of protocols that could be used to support multi-hop internetworking, including the Internet Protocol suite (IP), the CCSDS File Delivery Protocol (CFDP), CCSDS Space Packets, and Delay / Disruption Tolerant Networking (DTN).
3 Requirements and Desired Characteristics of a Space
Internetworking Protocol This section attempts to define the set of requirements on a space internetworking protocol from the perspective of the applications that would use it.
3.1 Requirements of a Space Internetworking Service
Any space internetworking service must provide the following: An OSI Layer-3 (Network Layer) Service to Applications – The space internetworking protocol must provide for the addressed, end-to-end delivery of octet-aligned, user-defined protocol data units (PDUs). It must be possible to de-multiplex PDUs to a number of different upper-layer services,
DRAFT
8 | P a g e D R A F T
including specific applications (i.e. it must be possible to address a PDU to a specific application on board the spacecraft, not just to the spacecraft as a whole). Ability to handle arbitrarily-sized application-layer PDUs – The size of the application-layer data units transported by the space internetworking protocol should not be constrained by the underlying technologies used. End-to-end network PDU delivery in the presence of delays / disruptions – For space communication there may be multiple sources of delay and disruption, some planned and some not. Planned delays include light-time latencies that range from minutes to hours and beyond for deep-space communication. Disruptions include planned resource scheduling issues that restrict connectivity to certain windows, unplanned reallocation of resources that may interrupt communications, and unplanned disruptions due to environmental factors (e.g. multipath, solar activity). Thus any space internetworking protocol must be able to function in the presence of the following environmental constraints:
• Long delays – when even the data link round trip time (RTT) may be measured in minutes or hours.
• Temporary Network Partitioning –when there is no network path to the destination for some period of time.
• Half-duplex communication paths –when communication is one-way for some period of time. It is expected that if A can send information to B then there will be some time in the future when B can send information to A, but that this reverse path may not be disjoint in time from the forward path. Note that individual links may be completely simplex.
• Contacts that do not support entire network-layer PDUs – It may be the case that no single contact contains enough bandwidth to forward an entire network-layer PDU. In these cases, the space internetworking protocol must be capable of fragmenting the network layer PDU so that it can be forwarded in pieces and reassembling the PDU before presenting it to the next higher layer.
Data Accountability – A space internetworking protocol must provide mechanisms to ascertain where particular network PDUs are in the network and, if they have been discarded, the reasons for discarding them. Optional Reliability – Both reliable and unreliable data delivery mechanisms are needed for space communications. Some data such as low-priority cyclic telemetry is well-served by an unreliable delivery model. If a particular piece of data is lost, it is better to simply wait for the next sample than to waste resources retransmitting old (and presumably less useful) data. Alternately file transfers, messaging, and other applications will benefit from a common, standardized service that provides reliable data delivery. Providing reliable delivery as a network service frees applications from having to implement reliability and improves interoperability among applications needing reliability. Prioritized Data Delivery – Not all data should be treated equally by the network service. More important data should have a higher probability of being delivered, be delivered sooner, or both. A space internetworking service needs to provide mechanisms for applications to signal the importance of data and needs to provide ‘better’ service to the more important data. Data Link Layer Agility – The space internetworking protocol must be able to function over a wide
DRAFT
9 | P a g e D R A F T
range of data link layers, including at least CCSDS AOS, TC/TM, and Prox-1. The space internetworking protocol must be able to function over paths composed of heterogeneous data links. Compatibility with the terrestrial Internet – The space networking protocol must be transportable across the terrestrial Internet. Security – The space internetworking protocol SHOULD provide security mechanisms for:
• authenticated access to the network • integrity and confidentiality services for user data
Whether or not the various security mechanisms are invoked MUST be controllable by policy. Management – The space internetworking protocol SHOULD provide mechanisms for:
• network management • automated route construction and maintenance (dynamic routing)
3.2 Desirable Characteristics
The following are not necessarily requirements on the network service provided, but are highly desirable characteristics. Low Latency – The network service should impose minimal latency in addition to the physical transmission latency of the path when the path is connected end-to-end. Low Overhead – The network service should not impose more than a reasonable amount of per-packet overhead.
4 Scenarios This section presents a number of scenarios that define the required capabilities of a delay / disruption tolerant networking service..
4.1 PDU Delivery in the Presence of Disruptions
Reliable / Unreliable delivery of routed network PDUs in the presence of delays and disruptions where PDUs are not split among contacts. Delays here refer to link propagation delays such as might be encountered when communicating with Mars. ‘In the presence of disruptions’ means that links between nodes may be deactivated without warning. When no forward path exists, PDUs should be stored and transmission resumed when a new path becomes available. For this scenario we assume that an entire PDU fits onto a particular communications opportunity (contact) between two entities, thus PDUs are not split among different contacts. Figure 3 shows an example topology for a Mission Operations Center (MOC) communicating with a target spacecraft via a ground station and a number of in-space relays.
DRAFT
10 | P a g e D R A F T
MOC
G/S
Space
Relay1
Lander
Space
Relay2
Figure 3: Example topology for testing PDU delivery.
Figure 4 shows an example connectivity timeline for the topology of Figure 3. In Figure 4, double-headed arrows indicate bidirectional connectivity between the endpoints and the number inside the arrows indicates the (bidirectional) bandwidth of the connection period in arbitrary units. Single-headed arrows indicate simplex connectivity. The green triangles are the data objects to be transferred via DTN, with a size in the same units as the bandwidth arrows. The red line indicates the path of a particular piece of application data and its response.
MOC
G/S
Space
Relay1
MOC
G/S
Space
Relay1
DTN Message
stored at G/S
waiting for uplink
to SR1
5 5
7 7 7 7 7 7 7 7
7 7 7 7 7 7 7 7
Space
Relay2
Space
Relay2
TargetTarget
7 77
7
7
7
7 7
Figure 4: Example timeline for connectivity with disruptions.
The characteristic of Figure 4 that is relevant to motivating a DTN protocol is that the message must be stored at various points in the network waiting for a forward path to become available.
4.2 PDU Delivery in the Presence of Disruptions and Requiring Proactive
Fragmentation
Reliable / Unreliable delivery of routed network PDUs in the presence of disruptions where at some point PDUs *must* be split among more than 1 contact but the contact sizes are known ahead of time (fragmentation of network PDUs is required; proactive fragmentation at the transmitting end of a particular link is OK; this implies the ability to do 'in series and parallel' transmission since fragments may take different paths).
DRAFT
11 | P a g e D R A F T
Space
Relay2
G/S
Space
Relay1
MOCMOC
G/S
Space
Relay1
DTN Message
stored at G/S
waiting for uplink
to SR1
9
9 9 9 9 9 9 9
7 7 7 7 7 7 7
Space
Relay2
TargetTarget
77
7
7
7
7 7 7
9
7
2
7 9
7
2
7
2
7
Figure 5: Proactive DTN PDU fragmentation
Figure 5 illustrates a situation requiring proactive fragmentation, since there is no contact between G/S and any space relay large enough to accommodate the entire PDU (the PDU has size 9 and each of the individual contacts has size 7). Thus the original PDU of size 9 is split at the G/S into two pieces that are sent over different contacts to Relay1 and Relay2.
4.3 PDU Delivery in the Presence of Disruptions and Requiring Reactive
Fragmentation
Reliable / Unreliable delivery of routed network PDUs in the presence of disruptions where PDUs must be split among more than 1 contact and the contact sizes are not known ahead of time (reactive fragmentation is required, 'series¶llel' is implied). The exact bandwidth available during a contact may not be known in advance. For example, advanced data link layer protocols instantiate links and manage their data rates to try to maximize data transfer. Alternately, interference may unexpectedly degrade communications lowering the available bandwidth of a pass. In these cases, the exact bandwidth available will not be known ahead of time. Link-layer fragmentation and reassembly may not be sufficient to complete transfers that are in progress when the contact breaks, as the next reasonable transmission opportunity may be to a different next hop. Thus the space internetworking protocol must determine or infer the amount of data that has been successfully transferred at contact termination and fragment the network PDU into two parts. The first fragment contains the data that has been successfully transferred, and the second contains the untransferred part of the original PDU. This process is referred to as reactive fragmentation and is described in RFC4838. While reactive fragmentation can increase performance, there is no known way to provide hop-by-hop security services that cover the entire PDU when reactive fragmentation is used. In this case, security measures can only be verified once the PDU is reassembled.
4.4 Prioritized PDU Delivery in the Presence of Disruptions and Requiring
Reactive Fragmentation
This is intended to add non-preemptive PDU-level priority to the above scenarios. That is, whenever a
DRAFT
12 | P a g e D R A F T
PDU must be selected for transmission, a PDU is selected based on some notion of application-level priority of the data.
4.5 PDU Delivery in the Presence of Route Changes
It may be possible that the route a PDU needs to take to the destination will change while the PDU is en route. This would be the case, for example, if using the topology of Figure 3 it were believed that a PDU could be placed onto a contact between Space Relay 2 and Space Relay 3, only to discover later that Space Relay 3 was unavailable, or that the given PDU couldn’t be transmitted. In this case, the PDU should be re-routed along another path, which might be to Space Relay 4.
5 Candidate Space Internetworking Technologies This section examines in more detail the following candidate technologies for use as a space internetworking layer:
• Custom data forwarding • CCSDS Space Packets • CCSDS File Delivery Protocol (CFDP) • Internet Protocol (IPv4/IPv6) • The Bundle (DTN) Protocol
5.1 Custom Data Forwarding (Mars Exploration Rovers, Mars Phoenix Lander)
The mars Exploration Rovers and the Mars Phoenix lander currently use an ad-hoc mechanism to forward data between Earth and landed elements on the surface of Mars. The basic mechanism uses the Proximity-1 data link between landed assets and an orbiter (generally Mars Odyssey but Mars Global Surveyor and Mars Express have also been used), and the CCSDS Telecommand / Telemetry protocols between the orbiter and the Earth.
5.2 CCSDS Space Packets
CCSDS Space Packets [CCSDS 133.0-B-1] can be used to form the basis of a multi-hop data forwarding architecture as shown in Figure 6.
Figure 6: Figure 2-2 from the CCSDS 133.0-B-1, Space Packet Protocol
DRAFT
13 | P a g e D R A F T
5.2.1 Features of Space Packets
• Spacecraft ID / APID to identify an application. • Multi-hop data transfers using Logical Data Paths (LDPs)
In the space packet protocol, LDPs are defined by the application identifier (APID) and an optional APID qualifier such as the combination of the spacecraft identifier (SCID) and transfer frame version number. An LDP defines a unidirectional path from source to destination through the set of intermediate links. Issues:
• The APID qualifier is not part of the space packet protocol data structure; it is usually carried by a protocol or protocols of the underlying subnetworks. This is problematic if not all of the subnetworks in the path support compatible APID qualifiers.
• Because Space packets use a single SCID/APID pair, this pair needs to function as a path identifier in a multi-hop system. A second SCID/APID pair would be needed to identify the reverse path.
• If the APID qualifier is the master channel identifier (as is required for CCSDS TC/TM and AOS), then there is a further issue with addressing: does each frame for intermediate hops carry the SCID of the destination spacecraft or the next hop?
o If it carries the SCID of the destination, then possibly multiple spacecraft might receive copies of the frame. This might be the case if there were multiple orbiters around Mars and the frame were destined for a particular lander, e.g.
o If it carries the SCID of the intermediate destination (next hop), then APID space would need to be allocated for every destination application reachable through that next hop. This would be difficult given the limited APID space (11 bits).
o The Space Packet Protocol service interface allows the specification of a QoS requirement to be used to select the appropriate QoS in the subnetworks along the LDP. While this might work for the initial subnetwork, the QoS indication is not signaled in the Space Packet protocol itself and so it is unclear how QoS requirements can be applied to subsequent subnetworks.
5.3 CCSDS File Delivery Protocol (CFDP)
As a mechanism to provide multi-hop file transfers, the CCSDS developed the CCSDS File Delivery Protocol [CFDP]. CFDP provides for the optionally-reliable delivery of files across multiple hops in space. CFDP is designed to run over a Unitdata Transfer layer (UT layer) that mediates between the file transfer mechanisms of CFDP and an underlying data transport mechanism. For example, CFDP can use different UT layers to transfer CFDP blocks over User Datagram Protocol (UDP) packets or CCSDS Telemetry / Telecommand (TC/TM) links.
5.3.1 CFDP Features
The main features of CFDP are: • File transfer mechanism, including metadata associated with files. • Reliable / unreliable file transfer modes. • Multi-hop file transfer using CFDP extended procedures and/or store-and-forward overlay
(SFO) service (not globally implemented) Figure 7 shows an example of CFDP operation taken from the SISG report. Here applications build
DRAFT
14 | P a g e D R A F T
files that are then commanded to be sent to various destinations, in this case the relay orbiter or a landed element on the surface of Mars. In this example, a CCSDS TC data link is used to transfer files across the long-haul space link between the Earth and the orbiter, and the Proximity-1 data link layer is used between the orbiter and the landed asset.
Figure 7: Forward link example of Mars scenarios - End-To-End File Delivery
5.4 The Internet Protocol (IPv4/IPv6)
The Internet suite of protocols is ubiquitous in terrestrial networking. The main features of the Internet Protocol are:
• Unreliable delivery of datagrams with possible differentiated service. • IPv4 supports in-network fragmentation and reassembly of fragmented datagrams at the
destination; IPv6 requires fragmentation to take place at the source. • The IP suite comprises a mature set of protocols for security, network management, and
routing. • IPv4 implementations require a contemporaneous end-to-end path from source to destination in
order to deliver data. • Transport protocols (e.g. TCP, SCTP) can be used to provide reliability on top of IP services.
The largest issue with deploying IP is the assumption of an end-to-end path. If the network is partitioned so that there is no current path from one part of the network to another, IP datagrams that reach the partition boundary will have nowhere to go and will be discarded. The standard transport protocol used for reliability with IP, the Transport Control Protocol (TCP), also suffers moderate to severe performance degradation as round trip times and error/loss rates increase.
Cross Support GDS
ApplicationApplication
User GDS
FileFile
Transport Layer Security
TCP
IP
Data Link
Physical
SLE Forward
File Service
Transport Layer Security
TCP
IP
Data Link
Physical
SLE Forward
File Service
CFDP
Randomization
BCH Coding
CLTU Generation
TC PLOP
RF & Modulation
TC SDLP
CCSDS SPP
User Mars Asset Cross Support Orbiter
CFDP
Proximity-1 coding &
synchronization
Proximity-1 Physical
Proximity-1 coding &
synchronisation
Proximity-1 Physical
FileFile
CCSDS SPP
Prox-1 link
PKT
Prox-1 link
PKT
CFDP
Application
CCSDS SPP
Application
Derandomization
BCH Decoding
CLTU & bit
synchronization
RF & Modulation
TC SDLP
DRAFT
15 | P a g e D R A F T
It might seem attractive to write a custom implementation of the IP protocols that stored packets when no end-to-end path was available as a way to leverage the large body of work in the IP suite. Unfortunately, other protocols besides just the datagram forwarding depend on end-to-end connectivity and low delays. For example, the more mature routing protocols for IP networks exchange information in order to build a graph of the current connectivity and then to route datagrams on that graph. In disconnected environments, these protocols won’t function well. Reactive IP routing protocols for mobile ad-hoc networks typically need to probe to establish new paths, which could involve long delays before data could be transmitted. Thus the large body of supporting protocols for IP cannot be directly leveraged for space internetworking in environments that may contain disruptions and temporary network partitions.
5.5 Delay / Disruption Tolerant Networking
RFC5050 defines a Delay / Disruption Tolerant Networking protocol known also as the Bundle Protocol after the name given to RFC5050 network PDUs. RFC5050 defines the rules for formatting bundles for transmission between DTN nodes, and the requirements for processing and responding to administrative flags and messages. Figure 8 shows the format of RFC5050-compliant bundle headers.
Creation Stamp1
Version Flags
Blocklength
DestinationScheme
DestinationSSP
SourceScheme
SourceSSP
Report-toScheme
Report-toSSP
Custodianscheme
CustodianSSP
CreationStamp2
Lifetime
DictionaryLength
Dictionary
Fragment
Offset
Total
ADU length
Block
Type
Primary
Bundle
Block
Control
FlagsBlock Length
Payload
Bundle
Payload
Block
32 bits
Figure 8: Bundle Protocol Blocks
The main features of the Bundle Protocol are: • Flexible naming/addressing scheme.
• Support for fragmentation. Bundles may be split inside the network and reassembled later before being delivered to the destination(s).
• Time-to-live. Each bundle is assigned (by the source application) a ‘time-to-live’ that is meant to reflect the useful lifetime of the data. The time to live represents an actual time duration, not a network hop count, and is used to remove bundles from the system if they cannot be delivered in a timely manner.
• Optionally-reliable data transfer. DTN implements reliable data delivery by means of in-network checkpointing of bundle progress called custody transfer.
DRAFT
16 | P a g e D R A F T
• Per-Bundle Control Flags. Each bundle contains a set of flags that can trigger particular status reports about the bundle’s progress. These include:
o A bundle priority field that allows 3 levels of priority o Optional end-to-end acknowledgements of bundle delivery o Request reporting of bundle reception. o Request reporting of custody acceptance. o Request reporting of bundle forwarding. o Request reporting of bundle delivery o Request reporting of bundle deletion
The reports can be used to provide data accountability for bundles.
• Alternate ‘Report-To’ Addressing. The reports generated by a bundle may be directed to a different destination than the source. Reports may be directed towards destinations that are not generally reachable so that data accountability reports could be generated at nodes but would not be transmitted unless specific action were taken to retrieve the records.
• Extensibility. DTN protocol data units are composed of a variable number of ‘blocks’. Block types are identified by self-delimiting numerical values (SDNVs) so that expression is both efficient and highly extensible. Each block carries with it a set of flags identifying how nodes that do not understand the block should treat it (pass it unmodified, remove the block, discard the bundle, etc.). Thus additional capabilities such as “keep at most N of this type of cyclic telemetry value” can be implemented.
Figure 9 shows how DTN would operate in an end-to-end data transfer between a mission control center and a Mars surface asset. In the terrestrial Internet between the mission control center and the ground station, DTN can be deployed as an overlay network on top of TCP. Practically this means that DTN may only be present at the mission control center and the ground station, relying on IP to connect the two. At the ground station, DTN may store messages until the next contact period with the relay satellite. A custody transfer acknowledgement from the ground station would inform the control center that the messages had been successfully received and queued for transmission.
When transmitting messages across the space link, DTN would probably use a different set of data link and transport layer protocols than were used for the Internet connection. The figure shows DTN using the Licklider Transport Protocol (LTP), CCSDS Encapsulation Packets, and the CCSDS Advanced Orbiting Systems (AOS) data link. Again, messages marked for reliable delivery may be stored on the orbiter and acknowledgements sent to the ground station at the next opportunity. This way, if messages are lost in transit between the orbiter and the rover, they can be retransmitted from the orbiter instead of having to go back across the deep-space link.
Finally, the orbiter can use the Proximity-1 data link protocol to send messages to the rover.
DRAFT
17 | P a g e D R A F T
Application
DTN
TCP
IPv6
Ethernet
UTP
DTN (potential delay)
TCP
IPv6
ATM
DS-1
IPv6
Ethernet
UTP
Orbit-to-SurfaceTerrestrial Network
LTP
Application
DTN
GroundStation
Deep Space
DTN (Potential delay)
LTP
Mars RelaySatellite
IP Router
ATM
DS-1
Persistent StorageCT Custody Transfer Capability
Bundle PathCustody Acknowledgements
CT CTCT CT
MissionControl
MarsRover
LTP LTP
Encap
AOS
Ka-Band
Encap
AOS
Ka-Band
Encap
Prox-1
UHF
Encap
Prox-1
UHF
Figure 9: DTN used for end-to-end data transfer to a Mars rover.
If during a communications pass, some new command messages that were not transmitted to and stored at the ground station need to be sent, mission control can transmit the messages during the contact. Depending on the priorities of the various messages, the new messages from mission control might be transmitted before messages queued at the ground station, or they might be placed into a FIFO queue for transmission once all of the queued messages have been sent. It is immediately obvious from Figure 9 that DTN bears a striking resemblance to multi-hop CFDP.
5.5.1 Services RFC5050 Requires of Lower Layers
RFC5050 was designed to support a wide range of underlying network and data link technologies via the ‘convergence layer’ mechanism. Section 7.2 of RFC5050 lists the minimum requirements of a convergence layer as:
Each convergence layer protocol adapter is expected to provide the following services to the bundle protocol agent:
• sending a bundle to all bundle nodes in the minimum reception group of the endpoint identified by a specified endpoint ID that are reachable via the convergence layer protocol; and
• delivering to the bundle protocol agent a bundle that was sent by a remote bundle node via the convergence layer protocol.
One other requirement of RFC5050 is worth noting here, though it may be fulfilled by a lower-layer or a higher-layer protocol. To support bundle lifetimes as ‘wall-clock’ times-to-live, the bundle protocol requires loose time synchronization among nodes. The time-to-live field in the bundle header is used to remove bundles that remain in the communications system past their useful lifetimes, and applications are expected to set the lifetime long enough to allow delivery of bundles to their destinations. Because this delivery latency is not necessarily known ahead of time, and possibly not known at all by the application, we expect that applications will be liberal in setting their data timeout values. Thus setting a bundle’s timeout value at, say, a minute past the expected useful lifetime of the data is not unreasonable. This would allow for a clock skew of up to a minute among nodes in the
DRAFT
18 | P a g e D R A F T
DTN network delivering the bundle. There is ongoing work examining the possible benefits of redefining the bundle lifetime field as a ‘countdown timer’ instead of a delta from the bundle creation time. If such investigations prove useful, future extensions to the RFC5050 could adopt the new convention, removing the requirement for even loose time synchronization. CCSDS should coordinate with this work to determine its applicability to the space domain.
5.5.2 Naming of Bundle Protocol Endpoints
The Bundle Protocol allows for rich naming of endpoints via Uniform Resource Identifier (URI) syntax. This provides a great deal of power to support concepts such as intentional naming (identifying the characteristics of the endpoint rather than explicitly identifying the endpoint by address) that may not be needed in space communications. A less powerful but much more compact naming scheme has been proposed that identifies bundle protocol applications by the combination of a node number and a service number. These are akin to an IP address and port number in the IP protocols. This level of addressability will probably suit space applications for a long time to come, and has the added benefit of being highly compressible within the bundle protocol via Compressed Bundle Header Encoding [CBHE]. CBHE allows the node and service number of the various bundle protocol endpoint identifiers to be directly encoded in the integer offsets within the primary bundle block of Figure 8. This removes the overhead of a text representation of the URI and allows the dictionary to be completely removed from the primary bundle block. CBHE can reduce the size of the primary bundle block to as little as 27 bytes.
5.5.3 Bundle Protocol Forwarding and Routing
In the simplest case, bundle protocol routers can use static tables to choose next-hop addresses for bundles based on the bundles’ destination endpoint identifiers. These contents of these routing tables may be completely managed by the mission operations personnel on Earth. This amounts to a set of static, managed forwarding tables. A slightly more complex routing algorithm would allow bundle protocol routers to make decisions based on the destination endpoint identifier and the bundle’s time-to-live field. This approach was explored in [CGR] which takes as inputs a schedule of link connectivity and a set of bundles and attempts to schedule bundle transmissions to maximize the number of bundles delivered before they expire. Depending on need, more complex dynamic routing protocols akin to dynamic routing on Earth may be deployed. These will probably continue to differ from their Internet analogues in that the bundle versions will need to deal with scheduled connectivity.
5.5.4 Bundle Protocol Network Management
While mature network management protocols exist for the Internet Protocol, protocols and procedures to manage bundle protocol networks are still under development. Thus early missions will have to manually mange the network, much as links are manually managed now. As bundle protocol network management develops, it can be deployed into the network and the manual efforts can be scaled back.
DRAFT
19 | P a g e D R A F T
5.6 Conclusions
We believe that the bundle protocol as specified in RFC5050 is best-suited to support in-space relaying / internetworking. In particular, bundling supports the requirements of Section 3, including PDU delivery in the presence of possible network partitions and/or simplex links/paths, ability to address PDUs to particular applications, data accountability, reliability, and security. We acknowledge that the overall architecture for space internetworking will probably involve build-out of space packet-based services as well as IP-based services in addition to bundle-based services.
6 Use Cases / Services This section describes a number of use cases for a Delay / Disruption Tolerant space networking protocol based on bundling as described in RFC5050. These are example applications / application types that can be constructed using a DTN protocol that meets the requirements and scenarios above. The services are:
1. DTN PDU Delivery Service 2. File Transfer Service 3. Space Packet Delivery Service 4. Short Messaging Service 5. Hardware Commanding Service
The interest of the SIS-DTN working group is to define a DTN PDU delivery service that can support the other services and that can (potentially) be cross-supported at arbitrary points in the network. For the above list of services to take advantage of the increased connectivity and cross-support provided by DTN, each would need to be modified to use the DTN PDU Delivery service. It is worth noting that the end-to-end services above can be provided irrespective of the way those services are provisioned. For example, CFDP has a well-defined service interface for delivering files to CFDP endpoints across any communications system that supports the CFDP unitdata transfer primitives. While current CFDP implementations for space are being designed to run over Space Packets, a particular implementation could use DTN PDUs and still provide the same file delivery service. Section 6.2 outlines how CFDP could be modified to use DTN as its unitdata transfer layer, and Section 6.2.2 outlines how an end-to-end networked Space Packet Delivery Service could be layered on top of the DTN PDU Delivery Service.
6.1 Bundle Protocol PDU Delivery Service
The basic service provided by DTN is the delivery of DTN-Network-layer PDUs – the basic space internetworking service. No applications currently exist to take advantage of this service.
6.2 Implementing Existing High-Level Services Over The Bundle Protocol
This section briefly describes the implementation of common high-level services, file transfer and messaging, over DTN. Because these services are designed
6.2.1 File Transfer Service
File transfer is an application that is increasingly gaining acceptance in the space community, especially for the delivery of telemetry. Using the file transfer model, ‘files’ of telemetry information are created on board the spacecraft for transmission to the ground. The CCSDS File Delivery Protocol
20 | P a g e
(CFDP) was designed to meet this need in environments where the source and destination are connected by a single data link. CFDP’s procedures were designed to addressflexibility of the bundle protocol to deal withsuch as from a lander on Mars via one of two or more orbiters to Earth. While a new file transfer service could be built on top of the DTN PDU Delivery Service, more sense to re-factor the CCSDS File Delivery Protocol (CFDP) file delivery servicethe DTN PDU Delivery Service in networked environments.software that use the CFDP interface from the bundle protocol such as allow CFDP to address scenarios 4 and 5 [are sent along different paths to the destination. Using fragmentation and reassembly, DTN can easily implement CFDP Scenario 4 (reliable/unreliable end-to-end transfer via multiple waypoints in parallel) shown in containing the file to be transferred can be fragmented (proactively or reactively) at a network control center and the fragments forwarded over multiple paths to the destination. This is trivially extended to the case where there are multiple serial hops along one or the other of the paths.
6.2.2 Messaging Service
Many application and middleware protocols use a messageapplication layer PDUs are exchanged over the network. Spacecraft commanding, for example, could be implemented using a messaging model. While spacecraft operations may use a filewhere sequences of spacecraft commands are uplinked , checked, and executed as blocks, tlikely be cases where single commands are also depends on an underlying messagedelivery service.
6.3 New High-Level Services
The high-level services described above exist in some form or another over packetand their implementation to take advantage of bundling services is relatively straighthigh-level services described in this section
DRAFT
D R A F T
(CFDP) was designed to meet this need in environments where the source and destination are link. CFDP’s extended procedures and store-and-forward overlay
procedures were designed to address multi-hop communications paths, but lack the power and flexibility of the bundle protocol to deal with multiple, possibly changing, multisuch as from a lander on Mars via one of two or more orbiters to Earth.
While a new file transfer service could be built on top of the DTN PDU Delivery Service, factor the CCSDS File Delivery Protocol (CFDP) file delivery service
the DTN PDU Delivery Service in networked environments. This will allow existing applications and the CFDP interface to continue without modification while providing enhancements
from the bundle protocol such as multi-hop routing. Enhancing CFDP to run over BP will immediately allow CFDP to address scenarios 4 and 5 [CFDP-Green]: multi-hop file delivery where parts of the file are sent along different paths to the destination.
fragmentation and reassembly, DTN can easily implement CFDP Scenario 4 (reliable/unreliable end transfer via multiple waypoints in parallel) shown in Figure 10. A particular
containing the file to be transferred can be fragmented (proactively or reactively) at a network control center and the fragments forwarded over multiple paths to the destination. This is trivially extended to
are multiple serial hops along one or the other of the paths.
Figure 10: CFDP Scenario 4
Many application and middleware protocols use a message-based communications model, where changed over the network. Spacecraft commanding, for example, could
be implemented using a messaging model. While spacecraft operations may use a filewhere sequences of spacecraft commands are uplinked , checked, and executed as blocks, tlikely be cases where single commands are warranted. The Asynchronous Messaging Service (AMS) also depends on an underlying message-based service, which could be the proposed DTN PDU
Services Requiring Specification
level services described above exist in some form or another over packetand their implementation to take advantage of bundling services is relatively straight
level services described in this section do not currently exist and will require specification.
(CFDP) was designed to meet this need in environments where the source and destination are forward overlay
hop communications paths, but lack the power and multiple, possibly changing, multi-hop network paths
While a new file transfer service could be built on top of the DTN PDU Delivery Service, it makes factor the CCSDS File Delivery Protocol (CFDP) file delivery service to make use of
This will allow existing applications and o continue without modification while providing enhancements
routing. Enhancing CFDP to run over BP will immediately hop file delivery where parts of the file
fragmentation and reassembly, DTN can easily implement CFDP Scenario 4 (reliable/unreliable . A particular DTN PDU
containing the file to be transferred can be fragmented (proactively or reactively) at a network control center and the fragments forwarded over multiple paths to the destination. This is trivially extended to
are multiple serial hops along one or the other of the paths.
based communications model, where changed over the network. Spacecraft commanding, for example, could
be implemented using a messaging model. While spacecraft operations may use a file-oriented model where sequences of spacecraft commands are uplinked , checked, and executed as blocks, there will
The Asynchronous Messaging Service (AMS) , which could be the proposed DTN PDU
level services described above exist in some form or another over packet-based sublayers, and their implementation to take advantage of bundling services is relatively straight-forward. The
do not currently exist and will require specification.
DRAFT
21 | P a g e D R A F T
The first two services, packet delivery and bitstream delivery, are nominally transitional and temporary. These services allow existing packet- and bitstream-based applications to take advantage of the network delivery service by encapsulating series of packets / bytes in DTN bundles and tunneling them across the network infrastructure. Because bitstreams and space packets do not contain enough addressing information to identify DTN endpoints, each of these services would require an additional management interface. The third proposed standardized service is to support hardware commanding / essential telemetry. This should be a permanent service to support low-level commanding / telemetry to/from spacecraft where the higher-layer functions can not be assumed to be functioning correctly.
6.3.1 Space Packet / Encapsulation Packet Delivery Services
Many existing space applications use CCSDS Packets to organize and transfer data, and may want to continue to use Space Packets as application-layer data units even as they want to move to a multi-hop, internetworked environment. It may also be desirable to have a similar service capable of delivering CCSDS Encapsulation packets across multiple hops to an end system. To support these, standard DTN services can be developed to support carriage of CCSDS Space and Encapsulation Packets across DTN Networks. Figure 11 shows a Space Packet Proxy application resident in the host agency’s control center. This application receives packets transmitted over a to-be-defined CSTS Packet service and uses DTN to route the packets to the penultimate spacecraft (relay). There the peer packet proxy extracts the packets and presents them to the packet SAP of the last-hop data link layer.
PacketProxy
SpacePackets
DTN DTN
TargetSpacecraft /
Lander
Relay
Packet App
PacketProxy
DTN
GroundStation
CSTSPacket SAP
ControlCenter
ControlCenter
CSTSPacket SAP
SpacePackets
Packet App
Figure 11: Packet Transfer via DTN Tunnel Example
Alternately, the packet proxy could be implemented in the guest control center, proximate to the end system application as shown in Figure 14.
PacketProxy
SpacePackets
DTN DTN
TargetSpacecraft /
Lander
Relay
Packet App
DTN
GroundStation
ControlCenter
ControlCenter
Packet App
PacketProxy
DTN
Figure 12: Packet Transfer via DTN Tunnel Example
DRAFT
22 | P a g e D R A F T
The packet proxies just described could be used to support space packets or encapsulation packets. Not shown in the above figures is the management interface for controlling the packet tunnels. This interface will need to set up the tunnels, identifying the tunnel endpoint (e.g. the particular application and relay spacecraft at the end of the tunnel). Also, any information needed to invoke the last data link hop such as the protocol options to use, the time to instantiate it, etc. will need to be specified either as part of the packet tunnel interface or as a separate management exchange.
6.3.2 Bitstream Delivery Services
Figure 15 shows a bitstream application using a DTN proxy to communicate with a bitstream application on the target spacecraft.
BitstreamProxy
Bitstream
DTN DTN
TargetSpacecraft /
Lander
Relay
Bitstream / Frame App
DTN
GroundStation
ControlCenter
ControlCenter
Bitstream / Frame App
BitstreamProxy
DTN
Figure 13: Bitstream Transfer via DTN Tunnel Example
The bitstream proxy can be used to support existing bitstream applications and to support frame layer / bitstream-based hardware commanding.
6.3.3 Hardware Commanding Service
Spacecraft often have mechanisms to respond to very low-level commands in case higher spacecraft functions are not available or are not operating correctly. Typically such low-level commands are encoded in the data link layer frame headers or in packets or parts of packets that are placed at a fixed offset from the frame synchronization marker to facilitate command detection via a hardware correlator. This allows the radio itself to detect the commands without relying on higher protocol layers. Such commands are typically used to reboot the C&DH or to place the spacecraft in ‘safe mode’. Figure 14 shows a single data link layer frame with hardware commands embedded within the frame header itself (at offset A from the start of the synchronization marker). Figure 15 shows a data link layer frame with hardware commands embedded in a packet in the frame.
DRAFT
23 | P a g e D R A F T
Synchronization
Marker
Hardware Commanding
Bit Pattern In The Frame
Header
A
Packets
Figure 14: Hardware commands (light area) in the frame header at fixed offset from frame synchronization marker.
Synchronization
Marker
Hardware Commanding
Bit Pattern In Packet
B
Packets
Figure 15: Hardware commands (light area) embedded in a packet.
Because data link layer frames are generally not routed, commands can not propagate past a single data link layer hop. Similarly, an end-to-end Space Packet service might not guarantee the placement of packets within the frame sent to the destination. This might be difficult, for example, if the path contained both AOS or TC/TM and Proximity-1 data links, for example. Thus the above methods, while they work for single-hop communications, may not function in multi-hop, networked scenarios. The packet and bitstream services described above can be used to implement packet- and frame-based hardware commanding services. The disadvantage of using the packet and bitstream services is that they do not provide the full functionality of the bundle protocol. For example, the signaling mechanisms provided by DTN to indicate bundle progress are not available to the packet-, bitstream-, or frame-based application. Another solution is to provide a common, DTN-based application whose task is to process directives for sending data link layer hardware commands. Figure 16shows how a DTN-based hardware command application might function. The steps in the hardware commanding process are:
1. Hardware Commanding application (HC App) generates the hardware command for the target spacecraft. This may be in the form of a pre-formatted data link layer frame for transmission across the last link to the target spacecraft, or it may be just the information needed to generate such a frame at the Proximate Relay.
2. The HC application sends the hardware commanding data (frame or information) via DTN , addressed to the hardware commanding application at the Proximate Relay. Together with the hardware command, the sending HC App identifies the target spacecraft. This requires all other relays in the path to function properly.
DRAFT
24 | P a g e D R A F T
3. The DTN message makes its way to the Proximate Relay.
4. The HC application on the proximate relay consumes the HC application message and generates the frame containing the hardware command to transmit to the target spacecraft. The hardware commanding data might be in the frame header, a space packet within the frame, or some other well-defined location.
5. The frame containing the hardware commands is sent to the target spacecraft. We assume that neither the networking stack nor higher applications on the target spacecraft are functioning.
6. The hardware command is detected by the receiver and acted on by the target spacecraft.
HC App
DTN
HC App
DTN DTNDTN
MissionOperations
ProximateRelay
App
DTN
TargetSpacecraft
C&DH
1
23
4
56
Figure 16: Hardware commanding application example
7 Notional Architecture and Transition Path Figure 17 shows the end-to-end connectivity and inter-agency service access points described above. In the figure, Bitstreams, Space Packets, and Encapsulation packets are all tunneled across the space internetwork in bundle protocol PDUs.
Control Center
Ground StationOrbiterLander
App App
DTN-BP
Encap. Packets (in a file via FTP)
Space Packets (in a file via FTP)
DTN-BP
Prox-1Prox-1
Bundles
DTN-BP
Cross-Supported Interfaces to/from Lander• Bitstream (via bitstream tunnel and Prox-1 UDD SAP)
• Space Packets (via space packet tunnel and Prox-1 Packet SAP)
• Encapsulation packets (via encapsulation packet tunnel and Encap service)
• IP Packets over Encapsulation
• DTN Bundles over Encapsulation
Space Packet Tunnel
Encapsulation Packet Tunnel
AppBitstream Tunnel
Bits (in a file via FTP) / Bitstream
(in multiple files via FTP)
Prox-1 UDD (Bitstream) SAP
Prox-1 Packet SAP
Encapsulation Service
InternetSpaceLinks
DTN-BP
Cross-Supported Inter-Agency Services
CSTS Services
• Return All Frames
• Return VC Frames
• Forward CLTU
Special-purpose proxies supporting bitstream,
space packet, and encap packet tunnels.
Other AgencyControl Center
Could also connect
directly to ground
station without going
through Control Center
(controlled by policy)
CSTS
Services
DRAFT
25 | P a g e D R A F T
Figure 17: Architecture showing DTN Internetworking together with tunnels to support packet and bitstream delivery.
The figure also shows bundles sent directly from the guest control center to the host control center as first-class network data units and cross-support transfer service (CSTS) used to exchange frames between the guest control center to the host ground station.
7.1 Transition Path
Any space networking service will need to coexist with other services both at the data link and network layers. In particular, a Delay / Disruption Tolerant Networking service must be able to operate in parallel with CCSDS Space Packets and CCSDS Encapsulation Packets across CCSDS links, and should be able to operate in parallel with IP packets. The first stage in transitioning to an internetworked architecture is to establish specifications and conventions for interoperable space data links. Experience at Mars has shown that even with existing specifications, options chosen by different implementations may render the implementations non-interoperable or may reduce the efficiency of such interoperation. Additional specification or ‘profiling’ of existing specifications is needed to provide a suite of space data link services that are each capable of delivering delimited network-layer data units across space links. Such specifications / profiles are needed for at least: TC/TM, Proximity-1, and AOS. This is presumably work that should be undertaken by the CCSDS Space Link Services area, possibly within the Space Link Services working group.
7.2 Interoperable Protocol Stacks at Interfaces
Interoperable protocol stacks, from physical through the internetworking layer, are needed at the interface points between agencies. [Mars Green Book] has made significant progress in describing current configuration of the Proximity-1 protocol as the ‘local’ communications protocol for relay operations around Mars.
7.3 Transitioning CFDP to run over DTN
Figure 18 shows how CFDP could be migrated to use a DTN Protocol, including an intermediate stage that allows a CFDP implementations to communicate with both ‘old’ (non-DTN) and ‘new’ (DTN-based) implementations. This makes use of the layering internal to most CFDP implementations at the Underlying Transport Adaptor layer. Using this approach, CFDP implementations migrate from the configuration on the left to the one on the right. The part in the dotted oval on the right represents the ‘forward migration’ of the old architecture.
DRAFT
26 | P a g e D R A F T
Bundle Protocol
Delay / Disruption Tolerant Store-and-Forward, Routed Protocol With Optional
Custody Transfer
User Applications
CFDP
File System Functions
CFDP
File System Functions
UT Adapter
Convergence Layer Adapter CLA
AMS / RAMS Other
Messaging Functions
User Applications (e.g. Spacecraft Monitoring and Control, Science Applications…)
TC / TM
LTP
AOS
AOS
UT Adapter
UDP Encapsulation Packets
Encapsulation Packets
Encapsulation Packets
LTP
IP
Current CFDP: an integrated
application that handles files and
provides hop-by-hop reliability.
Encap.Packets
AOS
Unacknowledged Mode (no reliability)
UTA
Acknowledged Mode
Unacked Mode (no reliability)
Acknowledg
ed Mode
Future CFDP: a file transfer application
sitting atop a general infrastructure
providing routing and reliability. Figure 18: CFDP Evolution Path
This represents a completely seamless growth path for CFDP from the current implementation to one based on DTN.
7.4 Coexistence of DTN, IP, and Space Packets
The proposed DTN network service will need to coexist with other network layer protocols, including at least CCSDS Packets, and probably with Internet Protocols in parts of the network.
Mission Operations(Agency A)
TargetSpacecraft(Agency A)
GroundStation
(Agency B)
A C B
Encapsulation Packets
Containing IP Packets
Space Packets
Encapsulation Packets
Containing DTN Bundles
App
App
DTN
Relay Spacecraft(Agency C)
A B
B
IP Packets Routed by
Relay Network Layer
DTN Bundles Routed by
Relay DTN Layer
App
Space Packets
Delivered to
Application on Relay
AOS Data Link Prox-1 Data Link
AOS over
CSTS (SLE)
IPA
C
DTNB
App
IPA
C
AppApp AppApp
DTN
IP
DTN
IP
Some agencies may eventually support native
IP and DTN routing at ground stations (no CSTS
tunnel) in addition to CSTS services
C
Figure 19: Example showing coexistence of Space Packets, IP Packets and DTN PDUs on space links.
7.5 Initial DTN Operations
As described above, the complete suite of protocols to support the bundle protocol is still under development. This does not mean that the bundle protocol cannot be deployed immediately as a space internetworking layer. It merely means that those functions that are currently under development, in particular network management and routing, will need to be performed by hand until the supporting
DRAFT
27 | P a g e D R A F T
protocols are completed. The effort required to manually manage a multi-hop network based on the bundle protocol no different than if multi-hop networking based on CFDP, Space Packets, or the Internet Protocol were deployed, as none of these protocols has mature management or routing protocols suitable for a disconnected space environment. The advantage of deploying a network protocol is that it will be possible to take advantage of automated forwarding even if the routes have to be set up manually. This will allow mission operators to focus on what data they want and how they want that data to flow through the network, rather than on individual data transfers between spacecraft.
Top Related