Data flow

mission specific

msl

Several project documents describe the detailed functionality of key components of the data flow from instruments to the rover computer to Earth:

  • MSL Telecommunications Functional Design Description (MSL-376-1070, JPL D-31499)
  • MSL Downlink, Telemetry, and Data Management Functional Design Description (MSL-376-1072, JPL D-34201)
  • MSL Flight-Ground Interface Control Document (FGICD) Volume 1: Downlink, (MSL-232-0219, JPL D-27356)
  • MSL Ground Data System Functional Design Document (JPL D-36046, MSL 377-1437)

Each science instrument generates its own data products which are sent to the rover computer. The rover computer turns those products into telemetry data products which also contain ancillary rover data. For surface operations, the primary data path is for rover data products to be sent to an orbiter spacecraft (Odyssey or Mars Reconnaissance Orbiter) via UHF for later relay to the Deep Space Network (DSN) on Earth. An alternate path exists to send data directly to the DSN using the rover’s X-band High Gain Antenna. Then the Data Acquisition and Command Subsystem of the DSN send relay pass products to the MSL Mission Data Processing and Control Subsystem (MPCS) which builds telemetry data products and sends them to the OPGS (Operational Product Generation System) subsystem of the MSL Ground Data System.

The OPGS will generate EDR and RDR products for the engineering cameras (Navcams and Hazcams) and deliver those EDR and RDR archive volumes to the PDS.

The baseline process for science instrument data flow is for the OPGS to generate Experiment Data Records (EDRs) from the science instrument telemetry data and SPICE data, assemble those EDR archive volumes, and for the instrument teams to deliver them to the PDS. In a few cases (Mastcam, MAHLI, MARDI, and SAM), an instrument team (rather than the OPGS) will generate their own EDR products and deliver the EDR archive volumes to the PDS.

Reduced Data Records (RDRs) for the science instruments will be generated from EDRs, calibration data, and SPICE data by instrument teams, or by the OPGS if requested by an instrument Principal Investigator and successfully negotiated with the OPGS. The entity that generates the RDRs will also assemble them into archive volumes and deliver those volumes to the PDS.

The NAIF members of the MSL flight operations team will generate all SPICE kernels as well as maintain a web-based time conversion utility called chronos.

EDRs and RDRs will be generated according to designs specified in data product Software Interface Specification (SIS) documents that conform to PDS standards. Every MSL standard product will be described in a Data Product SIS. All MSL archive volumes will be assembled according to designs specified in Archive Volume SIS documents.

The Analyst’s Notebook (AN) will be a repository for MSL science data products archived by the PDS. The AN will contain additional archive materials such as documentation, software, calibration data, engineering data, and SPICE kernels.

Different processing levels are associated with data products. Most of the products are considered standard products, meaning that they are generated in a well-defined fashion on a regular schedule throughout the mission. In addition to the standard products, special products may be generated by some data suppliers as time and other resources permit on an ad hoc basis. Since special products are not part of the regular data processing, but rather are generated only as needed, they are not always described in SIS documents. Any special products that are completed and validated in time for a scheduled release to PDS may be delivered along with the standard products. PDS will continue to accept special products after the end of the mission as long as they are documented and validated according to PDS standards.

A data delivery may take the form of electronic transfer or delivery on physical media to the appropriate PDS Node. PDS personnel will work closely with science team members and OPGS to ensure a smooth transfer. When data products have been delivered to the PDS, they are regarded as publicly available. It is expected that the data will be promptly released to the public online through the PDS online distribution system, and through the Analyst's Notebook.

Online access will be the primary distribution method for MSL archives. The PDS is responsible for maintaining a long-term MSL archive and for delivering a copy of MSL archives to NSSDC according to agreements in place between the PDS and NSSDC.

MER

APXS

The APXS EDR data products generated by MIPL during operations are created collectively from: a) SSW data products b) SPICE kernels, and c) a meta-data database. They are created on the OSS and then deposited into MIPL's File Exchange Interface (FEI) for electronic distribution to remote via a secure subscription protocol. After a data validation period, the APXS EDR data products are collected with other science data and written to physical media for archiving with the Planetary Data System.

The size of the APXS EDR data file is 0.032 MB. The APXS EDR is generated 60 seconds after the data product for the EDR has been received by MIPL. The APXS data are reprocessed only if packets in the original downlink are not received. Partial files are created with missing data filled with zeroes. The APXS EDR is reprocessed after all data is retransmitted and received and the original version is overwritten.

Mössbauer

The MB EDR data products generated by MIPL during operations are created collectively from: a) SSW data products and b) SPICE kernels. They are created on the OSS and then deposited into MIPL's File Exchange Interface (FEI) for electronic distribution to remote sites via a secure subscription protocol. After a data validation period, the MB EDR data products are collected with other science data and written to physical media for archiving with the Planetary Data System.

The size of the Mössbauer Spectrometer EDR binary data file is 0.160 MB. The Mössbauer Spectrometer EDR is generated 60 seconds after the last packet SFDU for the EDR has been received by MIPL from the MER TDS. Mössbauer Spectrometer data are reprocessed only if packets in the original downlink are not received. Partial files are created with missing data filled with zeroes. The MB EDR is reprocessed after all data is retransmitted and received and the original version is overwritten and placed into FEI for distribution.

RAT

The RAT EDRs are generated by MIPL from telemetry data products from the MER Project SSW team, and SPICE kernels. The EDRs are created and stored on the MER Operations Storage Server (OSS) and then deposited into MIPL's File Exchange Interface (FEI) server for electronic distribution to remote sites via a secure subscription protocol. After a data validation period, the EDRs are collected with other science data and written to physical media for archiving with the Planetary Data System by MIPL.

The size of a RAT EDR is 2.5 MB. The total estimated volume of the RAT EDRs over the course of the MER mission is 15-20 MB. The RAT EDR is be generated 60 seconds after the data product for the EDR has been received by MIPL from the MER OSS. The RAT data are reprocessed only if packets in the original downlink are not received. The RAT EDR is reprocessed after all data is retransmitted and received and the original version is overwritten and placed into FEI for distribution.

Mini-TES

The Mini-TES EDRs generated by MIPL are created collectively from: telemetry data products produced by the MER SSW team The EDRs are created on the OSS and then deposited into MIPL's File Exchange Interface (FEI) server for electronic distribution to remote sites via a secure subscription protocol. After a data validation period, the EDR data are collected with other science data and written to physical media for archiving with the Planetary Data System by MIPL.

Camera Payload Instruments

The Operations EDR processing begins with the reconstruction of packetized telemetry data resident on the TDS by the System Software (SSW) program "mer_dp" into a binary data product and matching meta-data file. The data product and meta-data are written to the OSS and symbolic links to each are created in a repository directory under the OPGS substructure on the OSS, where they are ingested by MIPL's telemetry processor "mertelemproc" and processed with SPICE kernels provided by NAIF. The Operations EDR is generated 60 seconds after the binary data product and matching meta-data have been symbolically linked into the OPGS repository directory. This data flow is illustrated in the figure below.

The Science EDR produced is the Operations EDR processed further by utilizing Pancam and M.I. SOAS tools to generate exactly the same data format and label keywords (but not keyword values) as the Operations EDR. This data flow is illustrated in the figure below.

PHX

Phoenix standard data products are raw and derived products generated in "pipeline" fashion during mission operations using well-understood procedures. Each type of Phoenix standard products is described in a Data Product Software Interface Specification (SIS) document. Definitions of processing levels for science data products are found in Data Processing Level.

In addition to the standard products, special products may be generated by some data suppliers as time and other resources permit. Those special products that are completed and validated in time for a scheduled release to PDS are delivered along with the standard products. PDS will continue to accept special products after the end of the mission as long as they are documented and validated according to PDS standards.

Figure 1 shows the flow of science data products from the Phoenix downlink through mission operations and science team operations to their ultimate delivery to the Planetary Data System. Downlink telemetry is received at Mission Operations at JPL. The JPL Multi-Mission Image Processing Laboratory (MIPL) generates raw data products (EDRs) and some derived data products (RDRs) for the imaging instruments. EDR and RDR products for other instruments will be generated under the cognizance of the relevant Science Teams. Both MIPL and the Science Teams will transfer their EDR and RDR products to the Science Operations Center (SOC) at the University of Arizona to make them available to other Science Team members.

Complete archive volumes including data products, documentation, and ancillary materials are assembled, validated under Phoenix Project auspices, and then delivered to the designated PDS Nodes. PDS personnel will work closely with science team members to ensure a smooth transfer. Table 1 lists the elements that comprise the Phoenix archives. All Phoenix archive collections are assembled according to designs specified in Archive Volume Software Interface Specification (SIS) documents, which can be found under the "Resources" tab in "Data Set Documents".