Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.




SpacePy Example

This Jupyter Notebook demonstrates the reading and writing of CDF files using the SpacePy Python package.

Contents

  1. Prerequisites
  2. Import Packages
  3. Reading CDFs
  4. Writing CDFs

Prerequisites

Import Packages

import glob
import numpy as np
import datetime as dt

import spacepy
print('SpacePy Version \n{}'.format(spacepy.__version__))
from spacepy import pycdf
print('\nPyCDF Version \n{}'.format(pycdf.lib.version))
SpacePy Version 
0.6.0

PyCDF Version 
(3, 9, 0, b'0')

Reading CDFs

Load Local File

Please put CDF file in the same folder/location as script.

# CDF File
filename = glob.glob('mms*_edp_*.cdf')[0]
print('Filename \n{}'.format(filename))

# Read CDF
cdf = pycdf.CDF(filename)

# Read Type
print('\nRead Type \n{}'.format(type(cdf)))
Filename 
mms1_edp_fast_l2_scpot_20201029000000_v2.7.3.cdf

Read Type 
<class 'spacepy.pycdf.CDF'>

Global Metadata

# Global Summary Format = "Variable: Value"
# Prints in Alphabetical Order
print('Global Variables & Attributes \n{}'.format(cdf.attrs))
Global Variables & Attributes 
Acknowledgement: 
Calibration_file: mms1_edp_sdp_scpot_20160204_v0.0.0 [CDF_UCHAR]
Data_type: fast_l2_scpot [CDF_UCHAR]
Data_version: v2.7.3 [CDF_UCHAR]
Descriptor: EDP>Electric Double Probe [CDF_UCHAR]
Discipline: Space Physics>Magnetospheric Science [CDF_UCHAR]
File_naming_convention: source_descriptor_datatype_yyyyMMddHHmmss [CDF_UCHAR]
Generated_by: Y.Khotyaintsev and T.Nilsson, IRFU, using IRFU Matlab v1.15.1 while running at SDC. [CDF_UCHAR]
Generation_date: 20201031 [CDF_UCHAR]
HTTP_LINK: https://mms.gsfc.nasa.gov/ [CDF_UCHAR]
           http://mms.space.swri.edu/ [CDF_UCHAR]
Instrument_type: Electric Fields (space) [CDF_UCHAR]
LINK_TEXT: Magnetospheric Multiscale (MMS) mission home page [CDF_UCHAR]
           SMART package home page [CDF_UCHAR]
LINK_TITLE: At NASA GSFC [CDF_UCHAR]
            At SWRI [CDF_UCHAR]
Logical_file_id: mms1_edp_fast_l2_scpot_20201029000000_v2.7.3 [CDF_UCHAR]
Logical_source: mms1_edp_fast_l2_scpot [CDF_UCHAR]
Logical_source_description: MMS 1 dual probe scpot (fast), Spacecraft potential [CDF_UCHAR]
MODS: V.0. Initial release. [CDF_UCHAR]
      V.1. QL (v1.0.z), SCPOT (v1.0.z), L2A (v0.1.z) now uses ASPOC srvy l2 and DEFATT, if these are available. Brst QL uses intermediate L2A file from Fast mode for delta offsets. Bitmask changed to uint16 and Quality to uint8. [CDF_UCHAR]
      V.2. SCPOT (v2.0.z), L2A (v1.0.z) now uses variable names in accordance with new recommended standard for FIELDS, All products change shortening factor to 1.25 on SDP, offsets applied indicated by GlobalAttribute Calibration_file. [CDF_UCHAR]
      V.2. L2a (v2.0.z), QL (v1.6.z) now try to remove solar wind wake which previously left a clear sinusodial signal in the data. [CDF_UCHAR]
      V.3. L2a (v3.0.z) Slow Mode probe Gain set to 1.0 when orbital radius less than 5 RE (1.25 otherwise), L2pre (v2.0.z) DSL offsets removed from field is now included in the file as the Slow mode is dependent on scpot product (Fast/Brst is simply based on offset in Calibration_file). [CDF_UCHAR]
Mission_group: MMS [CDF_UCHAR]
PI_affiliation: SWRI, LASP, KTH [CDF_UCHAR]
PI_name: J.Burch, R.Ergun, P.Lindqvist. [CDF_UCHAR]
Parents: CDF>mms1_fields_hk_l1b_10e_20201029_v0.5.5 [CDF_UCHAR]
         CDF>mms1_fields_hk_l1b_105_20201029_v0.2.5 [CDF_UCHAR]
         CDF>mms1_edp_fast_l1b_dce_20201029_v1.4.3 [CDF_UCHAR]
         CDF>MMS1_DEFATT_2020302_2020303.V00 [CDF_UCHAR]
         CDF>MMS1_DEFATT_2020303_2020304.V00 [CDF_UCHAR]
         CDF>mms1_aspoc_srvy_l2_20201028_v2.0.3 [CDF_UCHAR]
         CDF>mms1_aspoc_srvy_l2_20201029_v2.0.3 [CDF_UCHAR]
Project: STP>Solar-Terrestrial Physics [CDF_UCHAR]
Rules_of_use: 
Skeleton_version: v0.0.7 [CDF_UCHAR]
Source_name: MMS1>MMS Satellite Number 1 [CDF_UCHAR]
TEXT: https://mms.gsfc.nasa.gov/ [CDF_UCHAR]
      The full name of PI affiliations: SWRI - Southwest Research Institute. LASP - Laboratory for Atmospheric and Space Physics. KTH - Kungliga Tekniska Hogskolan (Swedish Royal Institute of Technology).  [CDF_UCHAR]
      For detailed timing information, such as needed for cross spectral analysis, please consult the EDP Data Products Guide. [CDF_UCHAR]
Time_resolution: 
# Individual Global Variables
for variable in cdf.attrs:
    print('\n{}: {}'.format(variable,list(cdf.attrs[variable])))
Data_type: ['fast_l2_scpot']

Data_version: ['v2.7.3']

Descriptor: ['EDP>Electric Double Probe']

Discipline: ['Space Physics>Magnetospheric Science']

Calibration_file: ['mms1_edp_sdp_scpot_20160204_v0.0.0']

Generation_date: ['20201031']

Instrument_type: ['Electric Fields (space)']

Logical_file_id: ['mms1_edp_fast_l2_scpot_20201029000000_v2.7.3']

Logical_source: ['mms1_edp_fast_l2_scpot']

Logical_source_description: ['MMS 1 dual probe scpot (fast), Spacecraft potential']

File_naming_convention: ['source_descriptor_datatype_yyyyMMddHHmmss']

Mission_group: ['MMS']

PI_affiliation: ['SWRI, LASP, KTH']

PI_name: ['J.Burch, R.Ergun, P.Lindqvist.']

Project: ['STP>Solar-Terrestrial Physics']

Source_name: ['MMS1>MMS Satellite Number 1']

TEXT: ['https://mms.gsfc.nasa.gov/', 'The full name of PI affiliations: SWRI - Southwest Research Institute. LASP - Laboratory for Atmospheric and Space Physics. KTH - Kungliga Tekniska Hogskolan (Swedish Royal Institute of Technology). ', 'For detailed timing information, such as needed for cross spectral analysis, please consult the EDP Data Products Guide.']

HTTP_LINK: ['https://mms.gsfc.nasa.gov/', 'http://mms.space.swri.edu/']

LINK_TEXT: ['Magnetospheric Multiscale (MMS) mission home page', 'SMART package home page']

LINK_TITLE: ['At NASA GSFC', 'At SWRI']

MODS: ['V.0. Initial release.', 'V.1. QL (v1.0.z), SCPOT (v1.0.z), L2A (v0.1.z) now uses ASPOC srvy l2 and DEFATT, if these are available. Brst QL uses intermediate L2A file from Fast mode for delta offsets. Bitmask changed to uint16 and Quality to uint8.', 'V.2. SCPOT (v2.0.z), L2A (v1.0.z) now uses variable names in accordance with new recommended standard for FIELDS, All products change shortening factor to 1.25 on SDP, offsets applied indicated by GlobalAttribute Calibration_file.', 'V.2. L2a (v2.0.z), QL (v1.6.z) now try to remove solar wind wake which previously left a clear sinusodial signal in the data.', 'V.3. L2a (v3.0.z) Slow Mode probe Gain set to 1.0 when orbital radius less than 5 RE (1.25 otherwise), L2pre (v2.0.z) DSL offsets removed from field is now included in the file as the Slow mode is dependent on scpot product (Fast/Brst is simply based on offset in Calibration_file).']

Acknowledgement: []

Generated_by: ['Y.Khotyaintsev and T.Nilsson, IRFU, using IRFU Matlab v1.15.1 while running at SDC.']

Skeleton_version: ['v0.0.7']

Rules_of_use: []

Time_resolution: []

Parents: ['CDF>mms1_fields_hk_l1b_10e_20201029_v0.5.5', 'CDF>mms1_fields_hk_l1b_105_20201029_v0.2.5', 'CDF>mms1_edp_fast_l1b_dce_20201029_v1.4.3', 'CDF>MMS1_DEFATT_2020302_2020303.V00', 'CDF>MMS1_DEFATT_2020303_2020304.V00', 'CDF>mms1_aspoc_srvy_l2_20201028_v2.0.3', 'CDF>mms1_aspoc_srvy_l2_20201029_v2.0.3']

Data Variables

# Variable Summary Format = "Variable: Type [Shape]"
# Prints in Alphabetical Order
print('Variable Summary \n{}'.format(cdf))
Variable Summary 
mms1_edp_bitmask_fast_l2: CDF_UINT2 [1519680]
mms1_edp_dcv_fast_l2: CDF_FLOAT [1519680, 6]
mms1_edp_deltap_fast_l2: CDF_INT8 [] NRV
mms1_edp_epoch_fast_l2: CDF_TIME_TT2000 [1519680]
mms1_edp_label1_fast_l2: CDF_UCHAR*6 [6] NRV
mms1_edp_psp_fast_l2: CDF_FLOAT [1519680]
mms1_edp_quality_fast_l2: CDF_UINT1 [1519680]
mms1_edp_scpot_fast_l2: CDF_FLOAT [1519680]
# Individual Data Variables
print('Dictionary Keys\n{}'.format(cdf.keys()))

# Convert Dictionary Keys to List
variables = list(cdf.keys())    
print('\nVariables as List\n{}'.format(variables))
Dictionary Keys
KeysView(<CDF:
mms1_edp_bitmask_fast_l2: CDF_UINT2 [1519680]
mms1_edp_dcv_fast_l2: CDF_FLOAT [1519680, 6]
mms1_edp_deltap_fast_l2: CDF_INT8 [] NRV
mms1_edp_epoch_fast_l2: CDF_TIME_TT2000 [1519680]
mms1_edp_label1_fast_l2: CDF_UCHAR*6 [6] NRV
mms1_edp_psp_fast_l2: CDF_FLOAT [1519680]
mms1_edp_quality_fast_l2: CDF_UINT1 [1519680]
mms1_edp_scpot_fast_l2: CDF_FLOAT [1519680]
>)

Variables as List
['mms1_edp_epoch_fast_l2', 'mms1_edp_label1_fast_l2', 'mms1_edp_scpot_fast_l2', 'mms1_edp_psp_fast_l2', 'mms1_edp_dcv_fast_l2', 'mms1_edp_bitmask_fast_l2', 'mms1_edp_quality_fast_l2', 'mms1_edp_deltap_fast_l2']
# Access Variable Data

# Method #1: Via Variable Name
variable = 'mms1_edp_dcv_fast_l2'
data1 = cdf[variable][...]
print('{}\n{}\n{}'.format(variable,data1,data1.shape))

# Method #2: Via Index from Variable List
idx = 4  # From Variable List
data2 = cdf[idx][...]
print('\n\n{}\n{}\n{}'.format(variables[idx],data2,data2.shape))
mms1_edp_dcv_fast_l2
[[-3.2267735 -3.2671506 -3.1827219 -3.469033  -3.982991  -3.980059 ]
 [-3.2267735 -3.2671506 -3.1900637 -3.4745395 -3.986662  -3.9833632]
 [-3.2304444 -3.2708216 -3.1974058 -3.4800463 -3.990333  -3.9870343]
 ...
 [-2.4191623 -2.4760575 -2.37144   -2.635727  -3.2084186 -3.2003553]
 [-2.4228332 -2.4797285 -2.3751109 -2.6412332 -3.2120895 -3.2036598]
 [-2.4265041 -2.4852345 -2.3751109 -2.6430686 -3.2157605 -3.2073307]]
(1519680, 6)


mms1_edp_dcv_fast_l2
[[-3.2267735 -3.2671506 -3.1827219 -3.469033  -3.982991  -3.980059 ]
 [-3.2267735 -3.2671506 -3.1900637 -3.4745395 -3.986662  -3.9833632]
 [-3.2304444 -3.2708216 -3.1974058 -3.4800463 -3.990333  -3.9870343]
 ...
 [-2.4191623 -2.4760575 -2.37144   -2.635727  -3.2084186 -3.2003553]
 [-2.4228332 -2.4797285 -2.3751109 -2.6412332 -3.2120895 -3.2036598]
 [-2.4265041 -2.4852345 -2.3751109 -2.6430686 -3.2157605 -3.2073307]]
(1519680, 6)

Variable Metadata

for attribute in cdf[variable].attrs:
    print('\n{}: {}'.format(attribute,cdf[variable].attrs[attribute]))
CATDESC: Individual probes. P1=V1, P2=(V1-0.120*E12), P3=V3, P4=(V3-0.120*E34), P5=V5, P6=(V5-0.0292*E56)

DEPEND_0: mms1_edp_epoch_fast_l2

DISPLAY_TYPE: time_series

FIELDNAM: Probe to spacecraft potential individual probe

FILLVAL: -9.999999848243207e+30

FORMAT: F8.3

LABL_PTR_1: mms1_edp_label1_fast_l2

SI_CONVERSION: 1.0>V

TENSOR_ORDER: 0

UNITS: V

VALIDMIN: -120.0

VALIDMAX: 50.0

VAR_TYPE: data

Close File

cdf.close()
print(cdf)
Closed CDF mms1_edp_fast_l2_scpot_20201029000000_v2.7.3.cdf

Writing CDFs

Create Variables

# Epoch Variable
times = [dt.datetime(2020,3,10,12,tmp) for tmp in range(60)]
print('Epochs\n{}'.format(times))

# Data Variable
data = np.random.rand(len(times),4)
print('\nData\n{}'.format(data))
Epochs
[datetime.datetime(2020, 3, 10, 12, 0), datetime.datetime(2020, 3, 10, 12, 1), datetime.datetime(2020, 3, 10, 12, 2), datetime.datetime(2020, 3, 10, 12, 3), datetime.datetime(2020, 3, 10, 12, 4), datetime.datetime(2020, 3, 10, 12, 5), datetime.datetime(2020, 3, 10, 12, 6), datetime.datetime(2020, 3, 10, 12, 7), datetime.datetime(2020, 3, 10, 12, 8), datetime.datetime(2020, 3, 10, 12, 9), datetime.datetime(2020, 3, 10, 12, 10), datetime.datetime(2020, 3, 10, 12, 11), datetime.datetime(2020, 3, 10, 12, 12), datetime.datetime(2020, 3, 10, 12, 13), datetime.datetime(2020, 3, 10, 12, 14), datetime.datetime(2020, 3, 10, 12, 15), datetime.datetime(2020, 3, 10, 12, 16), datetime.datetime(2020, 3, 10, 12, 17), datetime.datetime(2020, 3, 10, 12, 18), datetime.datetime(2020, 3, 10, 12, 19), datetime.datetime(2020, 3, 10, 12, 20), datetime.datetime(2020, 3, 10, 12, 21), datetime.datetime(2020, 3, 10, 12, 22), datetime.datetime(2020, 3, 10, 12, 23), datetime.datetime(2020, 3, 10, 12, 24), datetime.datetime(2020, 3, 10, 12, 25), datetime.datetime(2020, 3, 10, 12, 26), datetime.datetime(2020, 3, 10, 12, 27), datetime.datetime(2020, 3, 10, 12, 28), datetime.datetime(2020, 3, 10, 12, 29), datetime.datetime(2020, 3, 10, 12, 30), datetime.datetime(2020, 3, 10, 12, 31), datetime.datetime(2020, 3, 10, 12, 32), datetime.datetime(2020, 3, 10, 12, 33), datetime.datetime(2020, 3, 10, 12, 34), datetime.datetime(2020, 3, 10, 12, 35), datetime.datetime(2020, 3, 10, 12, 36), datetime.datetime(2020, 3, 10, 12, 37), datetime.datetime(2020, 3, 10, 12, 38), datetime.datetime(2020, 3, 10, 12, 39), datetime.datetime(2020, 3, 10, 12, 40), datetime.datetime(2020, 3, 10, 12, 41), datetime.datetime(2020, 3, 10, 12, 42), datetime.datetime(2020, 3, 10, 12, 43), datetime.datetime(2020, 3, 10, 12, 44), datetime.datetime(2020, 3, 10, 12, 45), datetime.datetime(2020, 3, 10, 12, 46), datetime.datetime(2020, 3, 10, 12, 47), datetime.datetime(2020, 3, 10, 12, 48), datetime.datetime(2020, 3, 10, 12, 49), datetime.datetime(2020, 3, 10, 12, 50), datetime.datetime(2020, 3, 10, 12, 51), datetime.datetime(2020, 3, 10, 12, 52), datetime.datetime(2020, 3, 10, 12, 53), datetime.datetime(2020, 3, 10, 12, 54), datetime.datetime(2020, 3, 10, 12, 55), datetime.datetime(2020, 3, 10, 12, 56), datetime.datetime(2020, 3, 10, 12, 57), datetime.datetime(2020, 3, 10, 12, 58), datetime.datetime(2020, 3, 10, 12, 59)]

Data
[[0.67764576 0.85571295 0.03173693 0.26131419]
 [0.78664736 0.64587791 0.36980716 0.96838399]
 [0.85464338 0.61158033 0.83220973 0.23973256]
 [0.17912739 0.6048243  0.42231735 0.89710913]
 [0.06522457 0.25881726 0.64477787 0.9972318 ]
 [0.01665314 0.20294434 0.20295474 0.10264677]
 [0.70636241 0.6596243  0.42797708 0.30976239]
 [0.27722108 0.8219891  0.31791088 0.03977133]
 [0.60180759 0.30433475 0.81620102 0.38938249]
 [0.56175612 0.87493187 0.53690647 0.59474836]
 [0.13990132 0.1975923  0.47219399 0.40653043]
 [0.23591823 0.11021238 0.6961963  0.52934731]
 [0.28552751 0.59204015 0.63850044 0.61439997]
 [0.73843523 0.47130911 0.22896874 0.80689735]
 [0.30549819 0.95427645 0.74222561 0.14381271]
 [0.28254732 0.94118268 0.84283184 0.55424329]
 [0.05584889 0.14183651 0.37947609 0.48091815]
 [0.93224111 0.01542374 0.87099937 0.87945524]
 [0.4764418  0.8681995  0.65546709 0.41315925]
 [0.76189622 0.50902494 0.27409579 0.22405323]
 [0.12765449 0.82920201 0.14419951 0.06115871]
 [0.29733782 0.69456604 0.55334698 0.41235673]
 [0.17646638 0.01237249 0.09923137 0.11364385]
 [0.58168029 0.53976722 0.54629268 0.62806565]
 [0.951068   0.01122793 0.72359975 0.59522533]
 [0.43959393 0.45057321 0.61638073 0.91486734]
 [0.17839328 0.63936669 0.39216253 0.86952041]
 [0.30001719 0.85718177 0.07319884 0.56303606]
 [0.02040298 0.50162747 0.85373436 0.96361963]
 [0.70534585 0.09507381 0.62298307 0.06116245]
 [0.56798156 0.36173879 0.02965295 0.53271343]
 [0.3251583  0.92507999 0.76284124 0.63964303]
 [0.80341302 0.16719529 0.13240215 0.56789812]
 [0.23991446 0.68920963 0.31299911 0.17924399]
 [0.36666784 0.0030161  0.54197822 0.82417089]
 [0.40615318 0.04651165 0.23027346 0.94864824]
 [0.01932678 0.63081199 0.68014121 0.28777007]
 [0.89547304 0.98615466 0.90732291 0.5668571 ]
 [0.75354677 0.50086957 0.15283413 0.47968242]
 [0.48345765 0.35388262 0.35309715 0.20891603]
 [0.66854159 0.21233693 0.02258363 0.1820359 ]
 [0.33242611 0.10363061 0.63019587 0.08977313]
 [0.09125923 0.95446869 0.40750332 0.62709955]
 [0.55620298 0.52452063 0.73470279 0.3864934 ]
 [0.54415198 0.16016637 0.65256109 0.62275859]
 [0.06425221 0.55344958 0.07652476 0.27384328]
 [0.26895408 0.06339728 0.37056046 0.94762971]
 [0.48793109 0.04925775 0.9925787  0.43811252]
 [0.12955525 0.74507883 0.30686318 0.21577077]
 [0.68257691 0.32292273 0.34088038 0.11755354]
 [0.75352802 0.62992708 0.51134607 0.64454472]
 [0.0960222  0.77419849 0.47203525 0.44673113]
 [0.59213419 0.87665565 0.82158639 0.03048673]
 [0.00271452 0.08883383 0.48052256 0.13983402]
 [0.51682918 0.29267053 0.55013065 0.7686603 ]
 [0.08933111 0.80126642 0.70748334 0.95682608]
 [0.65494947 0.56784542 0.10536355 0.28668606]
 [0.07070622 0.04954588 0.85312975 0.88313842]
 [0.01127777 0.89374061 0.69614985 0.51059755]
 [0.57281359 0.47605322 0.36901902 0.08922841]]

Create Local CDF

cdf = pycdf.CDF('spacepy_example.cdf', masterpath='')

Write Variables

# Epoch Variable
cdf['Epoch'] = times

# Data Variable
cdf['Data'] = data

Add Metadata

# Add Global Metadata
cdf.attrs['Author'] = 'John Doe'
cdf.attrs['CreateDate'] = dt.datetime.now()

# Variable Metadata
cdf['Data'].attrs['UNITS'] = 'percent'

Close File

cdf.close()
print('Check for spacepy_example.cdf file!')
Check for spacepy_example.cdf file!