SDTO_Plan_Photosynth

SDTO Plan_Photosynth of 16 1

PHOTOSYNTH

THREE-DIMENSIONAL MODELING

OF THE

INTERNATIONAL SPACE STATION

INTERIOR AND EXTERIOR

GARY H. KITMACHER

PRINCIPAL INVESTIGATOR

NATIONAL AERONAUTICS

AND

SPACE ADMINISTRATION

2008

SDTO Plan_Photosynth of 16

SDTO XXXX-U: PHOTOSYNTH THREE-DIMENSIONAL MODELING OF ISS INTERIOR AND

EXTERIOR

Sponsor and Principal Investigator:

Gary H. Kitmacher, Manager, ISS Program Education and Communications, External Integration Office, JSC-OX, 2814831059, [email protected]

Co-investigators:

NASA/Ames Research Center

Chris C. Kemp, ARC:D, 650.604.4822, [email protected]

NASA/Headquarters/Space Operations Mission Directorate

Ronald L. Ticker, NASA Headquarters, CJ0001, 202.358.2429, Ronald L. [email protected]

Other Participants:

NASA/Ames Research Center

James F. Williams, ARC:D, 650.604.6377 [email protected]

Live Labs, Microsoft

Adam Sheppard, Group Program Manager

Jonathan Dughi, PhotoSynth Program Manager

David Gedye, PhotoSynth Group Manager

1 Microsoft Way, Redmond, WA 98052, (425) 882-8080

University of California at San Diego

Increment/Stage/Flight Applicability: Procedures for photographing the Station interior and exterior will be available no earlier than March 24, 2008. SDTO execution is requested beginning in Stage 1J and continuing through ISS Assembly Complete.

Category: Utilization

Impact: This SDTO is intended to return and integrate high resolution imagery of the interior and exterior of all elements of the ISS, including those of all of the international partners to form a hi resolution, detailed, 3-dimensional virtual immersive simulator.

mailto:[email protected]




This SDTO presents no risk to the Russian segment or elements. It does not require performance by Russian or other IP crewmembers and does not impact their resources.

This SDTO is to be performed through both the Shuttle and Station Programs; joint agreements will be obtained and documented.

Standard ISS Program photographic equipment will be used. Photographs will be taken of all interior-accessible areas of the habitable ISS, including all international partner modules, and exterior views of all ISS elements, including international partners.

1.0 HYPOTHESIS

This SDTO will attempt to develop a high resolution, internet compatible, three-dimensional

model virtual immersive of the ISS interior and exterior, publicly accessible through the website:

http://www.nasa.gov/mission_pages/station/

The interior model, using actual imagery of the on-orbit space facility, will permit the close-up inspection of accessible areas of the ISS interior, with the user moving through each element, stopping at almost any location, and looking at specific interior details.

The exterior model will permit the movement and rotation of the ISS and will permit the user to look in closely at specific details of most of the ISS exterior.

Such a high resolution, highly detailed, 3-dimensional virtual model based upon the actual flight system does not yet exist. This SDTO will attempt to show that the model can be developed. A significant aspect of the hypothesis of this SDTO is to assess whether such a 3-dimensional digital model and virtual environment, based up actual facility imagery, is useful for technical as well as communications purposes.

2.0 PURPOSE/OBJECTIVE

The primary objective is to provide a digital environment based upon actual ISS imagery, which permits virtual immersion into the ISS environments (internal and external) for communications/educational purposes.

Based upon use of the ISS Reference Guide and the NASA ISS Interactive Website, in which this virtual digital environment will be eventually be incorporated, it is believed that this

simulation will have applications for mission training, operations and other technical purposes.

The goal of the on-orbit phase of this SDTO is to obtain the highest practical resolution overlapping images of the interior and exterior of the International Space Station that can be used to construct a 3-dimensional digital model. High resolution images can be easily down sampled to fit a wide range of uses, but the reverse is not true.

2.1 The interior modeling requires that crewmembers photograph the interior of the ISS.

Preliminary ground-based testing with flight-like program photographic hardware will identify the best strategy and photographic settings for imaging the interior of the ISS. Procedures will be developed in conjunction with the Photo/TV Operations personnel (JSC-DX) and Photo/TV


Operations personnel will assist in training crew members. Shuttle crewmembers will be trained in conduct of the SDTO at the same time as other photo training. Procedures and on-board training documentation for conduct of the SDTO will be provided to ISS crewmembers.

On-orbit, some staging of each module/local area may be required in order to remove any obscuring protrusions, small non-permanent hardware, or other clutter. Equipment that is normally left in place during long duration operations should be photographed from multiple orientations and will be included in the interior model. As new modules are added to the ISS, additional photography will be required after new racks/major equipment is in place. If new racks/equipment are installed later in the program, additional images may be required of the new equipment and of areas surrounding the installed new equipment, in order to be incorporated into the model.

- Care should be taken to keep ‘non permanent hardware’ or crewmembers out of the images.

Standard ISS Program photographic equipment will be used. Photographs will be taken of all interior-accessible areas of the habitable ISS, including all international partner modules, and exterior views of all ISS elements, including international partner elements.

- Care should be taken to overlap images on all sides. An overlap of ½ to 2/3 of each image, on all sides, is desired. Adequate imagery of the same area from multiple positions should permit the creation of stereoscopic views of all areas;

- Care should be taken to maintain good focus and even lighting;

- As many wide angle pictures of the interior, including all walls, floor, ceiling, end cone bulkheads, hatch areas, node vestibules, as possible are required;

- As many images of interior compartments (sleep compartments, waste management area, airlocks, recessed areas where no racks are present) walls, floor, ceiling as possible are required;

- Detailed close-ups of equipment that is permanent/semi-permanent or otherwise interesting hardware installations are required; in the case of protruding hardware, pictures should be taken of all sides.

2.1.1 Overview/Wide Angle Photography

Wide angle images are used to build a framework within PhotoSynth of a desired location, such as a module. a. Photos obtained during this step should use the widest possible angle. A wide-angle

lens is suggested but not required. b. Photos should be done in passes. Never take a single photo of a given area. The

absolute minimum is 3 photos of any specific location. The camera should be displaced several inches between pictures. Take as many as required to cover a 360-degree area. This should include ‘floor and ceiling’. It is important to capture junctions between a ‘wall’ and a ‘ceiling’. PhotoSynth can handle hundreds of photos.

c. Photos should overlap at least 50%. An overlap of 2/3 is ideal.


Figure 2.1.1-2. Examples

2.1.2 Detail Photography

In this step those items of interest/protruding hardware from 2.1.1 are to be photographed individually

a. Items cannot be more than 50% larger than another photograph of the same item. PhotoSynth won’t recognize items that are more than 50% larger or smaller.

b. Photograph an item in stages. First take a zoomed out photo. Then zoom in 50% and take another photo. From there you can zoom in repeatedly by 50% increments until the desired level of detail is reached.

Figure 2.1-1. A series of overlapping images are taken of interior walls, floor, ceiling, endcones.

.


c. Protruding items are to be photographed from multiple angles. Some of the same rules as step 1 should be followed:

o Photos should overlap by 2/3 as you move around the item of interest.

Figure 2.1.2-1. When there is a semi-permanent gap in the interior, overlapping images of the interior walls within the gap should be taken.


Figure 2.1.2-3. Examples of photographing a major piece of hardware.

2.2 Exterior Modeling

2.2 Exterior modeling requires a series of overlapping images, as many as are possible, of the ISS taken as the Shuttle performs a 360 degree fly-around. It is desirable to conduct fly-arounds of the ISS in multiple planes. For instance, the typical Shuttle fly-around is done

around the Truss-axis. It is desirable to conduct a fly-around with detailed photography of the long module (Node2-US Lab-Node 1-FGB-SM) axis.

Further, the highest resolution images practical are sought through for this SDTO.

Two different types of images should be taken. Framework images should show the entirety of the ISS in each view. Detail images should focus on specific features on the exterior of ISS.

2.2.1 ISS Exterior Framework Images

Most important in these images is

a. inclusion of the entirety of the ISS in each image

Figure 2.1.2-2. When a semi-permanent installed piece of hardware protrudes into the aisle (central volume) of the module, overlapping images of the protruding hardware should be taken from all sides.


b. as high a resolution as possible

c. excellent focus

d. a continuous series of images from as many positions around the ISS as possible

d. adequate lighting

Figure 2.2-1. ISS Exterior Framework Images are required from as many positions as possible at as high a resolution as possible throughout a 360 degree fly around. Each image should capture the entirety of the ISS. This drawing shows the nominal Shuttle fly-around.

Figure 2.2-2.It is desirable to take a series of ISS Exterior Framework Images during an off-nominal fly-around that would permit better side views of modules, end-views of truss, and features not captured during the nominal Shuttle fly-around.


As new modules are added to the ISS, additional photography will be required and can be integrated into the model.

2.2.2 Exterior Detail Photography

In order to be to examine external features close-up in the PhotoSynth models, high resolution, detailed close-ups of exterior features are to be photographed using telephoto lenses.

3.0 RATIONALE/JUSTIFICATION

In order for individuals to support the ISS, its operations, functions and goals, it is best if they

understand the ISS. A high fidelity virtual model based on images of the actual facility will permit individuals to be virtual astronauts, seeing many of the same features that the astronaut sees.

As evidenced by the overwhelmingly positive response to of the ISS Reference Guide and the ISS Interactive website by the Astronaut Office, Mission Operations Directorate Flight Directors, and the Mission Evaluation Room staff, adequately detailed images which illustrate specific details of the vehicle, and which are readily accessible, are useful for training and operational purposes.

Once developed, this type of virtual digital environment may be used on-line for on-orbit support of operations, configuration planning, stowage, maintenance, inspection, training, or as a feature at NASA Visitor Centers, museums, and other public venues around the country. Because the goal is the production of views which can be examined at life size or greater, it is the applications drive the requirement for the highest resolution images.

The following products are intended to be developed based on work conducted in orbit in support of this SDTO:

3.1 Evaluation of prototype use of the VR environment for in-flight maintenance, stowage, experiment, payload planning and/or crew operations/training. A high-resolution VR environment can provide a useful guide for users and operators of the ISS to understand how hardware is installed, where equipment is located and difficulties associated with accessibility, how equipment may be through the interior, transferred from outside the ISS to the interior; or installation locations on the exterior, where stowage, glove boxes, refrigerator/freezers, experiment racks, and other facilities are located; it may also provide users/operators with an understanding of size and resource constraints. This understanding can lead to better designed hardware or improved transfer or installation operations.

3.2 PhotoSynth 3-Dimensional model for use on the NASA ISS interactive website. The ISS Interactive Website is functional today with surrogate 360 tours of the interior and exterior. These have been widely acclaimed by the public and media and the website has won national

awards.

3.2 A life-sized virtual reality (VR) immersion environment, similar in concept to a "holodeck", via collaboration between NASA and the University of California at San Diego.


3.2.3 Modification of the life-sized virtual reality immersion environment described in 3.2 for application in a bus or trailer, so that it can be transported throughout the US. 3.2.4 Creation of a VR environment in a VR helmet similar to the current VirtueArts work with Lunar exploration for the Lunar Racing Championship game now under development. (http://www.virtuearts.com/) 3.2.5 Creation of an 'edutainment' high fidelity VR based education-gaming environment for consumer desktop applications. In this application, educational content will be added to the VR ISS (eg, Don Pettit's Saturday Morning Science) to provide additional learning opportunities in an engaging manner.

4.0 LAUNCH/RETURN VEHICLE SUPPORT

No unique hardware is required for this DTO. Photographic or other imagery equipment already in the manifest for ISS or Shuttle will be used. Imagery may be downlinked from orbit or returned via Shuttle.

5.0 GROUND SUPPORT REQUIREMENTS

5.1 Center(s) and Facilities required for Ground Test and Checkout:

No unique facilities are required for ground test or checkout.

5.2 Ground Support Equipment/Hardware:

No unique equipment is required for this DTO. PhotoSynth software, provided by Microsoft, will be used to integrate imagery into three-dimensional models. Staff at NASA JSC, Headquarters, Ames, and/or GSFC will compile the 3-D models and develop/maintain the NASA ISS Interactive website.

5.3 Ground Test Requirements and Procedures:

No unique ground test requirements or procedures are required.

5.4 Late Access/Early Destow/Launch Scrub:

No special late access or early retrieval is required.

5.5 Temporary Duty Support at NASA or International Centers:

In preparation of detailed crew procedures, access to training facilities at JSC Building 9, and use of ISS Program Photographic hardware will be required.

No other unique access requirements are envisioned. PI or others can be available for crew training or during mission operations, as required.

5.6 Ground Safety:

5.6.1 Shuttle:

No unique, hazardous or dangerous equipment will be used.

http://www.virtuearts.com/


5.6.2 IP Launch Vehicle:

No unique, hazardous or dangerous equipment will be used.

5.7 Special Ground Support Requirements:

No special ground support requirements are anticipated.

6.0 FLIGHT EQUIPMENT/HARDWARE AND MANIFEST

6.1 Standard Equipment/Hardware:

6.1.1 ISS:

All equipment to be used for conduct of this SDTO will be shared program hardware. Agreements have been worked out with JSC-DX for use of program hardware.

Item No. Qty Nomenclature

1 1 Digital Camera, Nikon D2XS

2 1 or more Photographic Strobe light, as required

3 1 Lenses, as required

6.1.2 Launch Vehicle:

Shared standard Shuttle hardware/equipment items (e.g., operational hardware or support equipment) shall be used in conduct of this SDTO.

Item No. Qty Nomenclature

1 1 Digital Camera, Nikon D2XS

2 1 or more Photographic Strobe light, as required

3 1 Lenses, as required

6.1.3 Consumables:

No consumables are required.

6.2 Unique Equipment/Hardware/Software:

6.2.1 No unique flight equipment, hardware or software is required. Special PhotoSynth software will be required only for use during ground-based image processing and model compilation.

No unique flight hardware/equipment is required.

6.2.2 Constraints or Special Requirements:

No unique flight hardware/equipment is required.

6.2.2 Consumables:

No non-standard items are required.

6.3 Physical Characteristics (Manifest Data):

No unique or specialized hardware is required.


6.4 Flight Safety:

No unique or specialized hardware or operations are envisioned and therefore no special safety assessment is planned.

7.0 LOCATION, DURATION, FREQUENCY, PLACARDS, AND CREWMEMBERS REQUIRED

7.1 Location:

This SDTO will be performed in every ISS module and open volume: FGB, Node 1, SM, US Lab, US Airlock, Pirs, Node 2, Columbus, ATV, and Kibo. In addition, photography may take place in Soyuz, Progress and Space Shuttle.

7.1.1 Priorities

The chief priority in photography of different elements is minimized crewtime. Photographs should be taken in order of priority to reduce time/crew workload. It is a high priority to develop PhotoSynth models of elements as they first arrive in orbit as this is the time of peak media and public interest, and the imagery may be of particular value for operational uses. However additional photographs will need to be taken later and additional models developed as major hardware/rack installations take place and the module interiors are reconfigured. It is also a high priority to take photographs and develop models of primary crew living and working areas.

7.1.2 Session Duration:

TBD crewtime will be required for staging of volumes prior to photography, and for actual photographic procedures. Time is dependent upon location, module size, and timing during the increment. For instance, it would be ideal to photograph modules when they have been cleared of major stowage blockages during times, such as when major new racks or facilities are installed. It is estimated that session time could range from 15 minutes for a small module/element that is clear of obscuring non-permanent blockages, to as long as 1-2 hours, if staging of a larger element is required.

7.2 Docked Duration:

Photography will be required during rendezvous mission phases, especially as the Shuttle undocks and performs fly-around maneuvers. The only requirements for photography of the ISS interior during Shuttle-docked mission phases, is in order to photograph the interior of the MPLM. Photography of the ISS interior may be taken by Shuttle crewmembers if they are available. It is recommended that photography of the Shuttle interior also be taken. It may be

taken at any time during a mission.

7.3 Number of Sessions:

It is anticipated that every open volume within ISS will require time for staging and photography. Volumes may need to be repeated as major configuration changes occur.

7.4 Crewmember(s) Required:

Crewmembers trained in use of photographic equipment are required to complete this SDTO. ISS and Shuttle crewmembers crewmembers may participate in this SDTO. ISS crewmembers are prime during long duration mission phases and Shuttle crewmembers are prime during proximity operations/fly-arounds.


8.0 ATTITUDE REQUIREMENTS OR IMPACTS

Exterior modeling requires a series of overlapping images of the ISS taken as the Shuttle performs a 360 degree fly-around. It is desirable to take photographs during a nominal fly-around of the ISS, which takes place in a plane perpendicular to the truss. It is also desirable to take photographs during an off-nominal fly-around which permits enhanced side views of modules and end-views of truss, such as in a plane perpendicular to the long module axis. Images should be taken from as many positions during the Shuttle fly-around as possible. See section 2.0 for a more complete description.

9.0 POWER

No special power requirements.

10.0 PROPULSIVE CONSUMABLES

No special or unique propulsive consumables are required.

11.0 COMMANDS

No special commanding activities.

12.0 DATA MANAGEMENT UPLINK/DOWNLINK

No special data measurement, recording or uplink/downlink requirements beyond typical mission photography.

13.0 IMAGERY REQUIREMENTS

Imagery requirements are specified in Section 2.0. Still electronic or still photographic imagery may be used for this SDTO. No external remote video, EVA, wireless video, or internal hand held video is required. Handheld High Definition video may be substituted for Hand Held Digital imagery if quality is sufficient. Imagery is not required to be down-linked. It is planned for all imagery to become publicly available.

14.0 TEST CONDITIONS/ACTIVITIES REQUIRED

14.1 Flight Requirements and Procedures:

The ISS or Shuttle crew is responsible for all mission photography to obtain the required imagery.

Attitude and Attitude Rate Constraints:

No unique attitude or attitude rate requirements on ISS.


14.2 List required specific attitudes and the rates required or allowed. Activity Schedule:

Activities internal to the ISS should be planned during periods when a minimum of ‘clutter’ is expected in the interior volumes of the ISS that are to be photographed. It is not required to remove all temporary stowed items. Clutter should be minimized only when it obscures major portions of the element from view.

Photography of the exterior of the ISS is required during Shuttle fly-around, post undocking.

New photography is required of effected areas, after major new equipment installations, including the berthing of new elements, or the installation of new racks.

List all activities required prior to launch and provide the facility where training is to occur.

Pre-crew Launch Activity Duration Schedule Performance Constraints

Training Facility

Crew Familiarization 0 minutes C.L. - 0 days Fam can be conducted during nominal photo/tv training

None

Crew Training 0 minutes C.L. - 0 days Training can be conducted during nominal photo/tv training

None

Other Pre-flight Activity 120 minutes C.L. - 60 days Ground-personnel surveys of element interiors using nominal photo hardware will identify selections of best hardware combinations of hardware and strategies for implementation

JSC Building 9 Mock-up Facility

Note: C.L. - Crew Launch

Identify the activities that must occur on-orbit and include requested launch vehicle (e.g., Shuttle, middeck locker, Progress, etc.) the SDTO is to be transported on and the on-

orbit location(s) in which the SDTO is to be performed (e.g., ISS module, Service Module (SM), Japanese Experiment Module (JEM), etc.).

In-flight Activity Duration Schedule Performance Constraints

Vehicle or Module

Launch Vehicle Ascent

Ascent activity 0 minutes N/A Performance constraints

(e.g., Shuttle or Progress)

On-Orbit Activity

Hardware setup and test 0 minutes 0 days prior to performing SDTO

Nominal photographic activity, per task list

(e.g., Node 1)

Other On-orbit Activity (enter additional lines as needed)

Para 7.1.2 0 days prior to performing SDTO

Performance constraints

(e.g., Node 1, SM, Airlock, etc.)

Entry


In-flight Activity Duration Schedule Performance Constraints

Vehicle or Module

Entry activity 0 minutes N/A Performance constraints

List any action required once the SDTO is returned.

Post-flight Activity Duration Schedule Performance Constraints

Location

Data collection 0 minutes Return plus 0 days

Nominal photographic activities

Image processing and model synthesis will take place at JSC, NASA Headquarters, and Ames Research Center

14.0 DELAY/TERMINATION REQUIREMENTS

Photography of the interior of ISS should be completed as crew-time and mission activities permit. Itemization on the crew task list is recommended.

Photography of ISS exterior should be conducted during all fly-arounds, post Shuttle undocking. It is important to continue photography on all Shuttle missions because the ISS configuration is changing and expanding.

15.0 POST-FLIGHT REQUIREMENTS

The PI will work with the ISS Imagery Group (JSC-OC) to obtain sufficient resolution of all required images. The PI and his support organizations will compile 3 Dimensional models. The PI and his support organization will develop products required to adapt the 3 Dimensional models for use on the internet.

It is anticipated that this SDTO will be conducted over the course of multiple Expeditions, and that imagery of new elements/modules will be performed as the new elements are integrated onto the ISS. As imagery is reviewed for incorporation into the 3 Dimensional models, if significant gaps or otherwise unusable imagery is identified, then requirements to repeat photography sessions may be identified.

16.0 POST-FLIGHT REPORTING REQUIREMENTS

Progress reports on the conduct of this SDTO will be provided to the SDTO Working Group

and MIOCB within 1 month of the return of the first set of images. A preliminary and final report will be delivered as needed. Other ISS and Shuttle program forums will be briefed as results warrant. Results will be publicly available through the NASA ISS Interactive Website.

Delivery of final report to the Mission Integration and Operations Control Board (MIOCB) is considered closure of the SDTO; the SDTO will be archived and removed from the SDTO Catalog with the next CR unless otherwise requested by the Investigator.

SDTO_Plan_Photosynth

Documents

Transcript of SDTO_Plan_Photosynth