How to Use CDFs in Perl Quick Start Guide

1.Introduction

Common Data Format (CDF) is a conceptual data abstraction for storing, manipulating, and accessing multidimensional datasets. CDF is referred to as a data abstraction because the actual physical format in which datasets are stored is not discussed. Instead, the form of the datasets and the means (interface) by which they may be manipulated are described. The Perl-CDF package includes two interfaces: Internal Interface and Standard Interface. The Standard Interface only covers limited functions that deal mainly with the older rVariables and their attributes in the CDF. This interface is mirrored the original functions that are covered in the C’s Standard Interface. The Internal Interface, based on the C’s Internal Interface, provides a complete suite of CDF functionality. The Perl-CDF is based on both CDF internal interface and original standard interface (rVar-based). They all call CDFlib. The old standard interface is used to create a variable, if for a rVar. The internal interface can be used to create either zVar or rVar. Further information regarding the interfaces can be found in Section 3 below. The CDF home page contains examples, documentation, FAQs, and tools - https://cdf.gsfc.nasa.gov/. The CDF User Guide Perl Reference Manual are available at:https://cdf.gsfc.nasa.gov/html/cdf_docs.html.

2.Installation

In order to use either one or both interfaces from any Perl script, the search path for the Perl-CDF package must be set up properly. In addition, the Perl-CDF package needs to be imported as well prior to using the either CDF interface. There are two ways to define the search path for the Perl-CDF package. One way is to include the location of the PerlCDF package at the beginning of a Perl script. The following code illustrates how to define a Perl-CDF package that is installed under /home/cdf/PerlCDF32:

use strict; BEGIN { unshift @INC,‘/home/cdf/PerlCDF32/blib/arch’,

                                    '/home/cdf/PerlCDF32/blib/lib'; }

use CDF; # Import the CDF module - optional

The other way is to define the location of the Perl-CDF package at the command line when invoking the Perl script. The following command is equivalent to the above example:

perl -I/home/cdf/PerlCDF32/blib/arch -I/home/cdf/PerlCDF32/blib/lib

Since the Perl CDF interface uses the shared CDF library, the user has to tell the operating system where to find the shared library. For Linux, DEC Alpha/OSF1, Sun Solaris or SGI, the environment variable LD_LIBRARY_PATH must be set to point to the directory that contains the shared CDF library, libcdf.so. For example, if the shared CDF library is installed under /usr/local/share/cdf32/lib and you are using the C-shell, enter:

setenv LD_LIBRARY_PATH /usr/local/share/cdf32/lib

For HP-UX, the shared library is libcdf.sl. For IBM RS6000, the library is libcdf.o.

For BSD-based Mac OS X, the environment variable is DYLD_LIBRARY_PATH that must be set to point to the directory containing the shared library libcdf.dylib.

For Windows 9x/NT/2000/XP, similarly, set the PATH variable to point to the directory that contains dllcdf.dll.

Two Perl test scripts, testPerlCDFii.pl and testPerlCDFsi.pl, are provided in the distribution. Both use extensive Perl-CDF interface functions: testPerlCDFii.pl tests CDF’s Internal Interface functions while testPerlCDFsi.pl tests the Standard Interface functions. They can be used as sample scripts for development.

3.Standard Interface and Internal Interface

Standard Interface functions are easier to use and require a much shorter learning curve than the Internal Interface. It is not as efficient as the Internal Interface and can only create and maipulate rVariables, not zVariables. If you are not familiar with Internal Interface and need a very simple CDF in a short time, the use of the Standard Interface is recommended. However, the Internal Interface is strongly recommended since it is much more flexible and powerful than the Standard Interface. There are two types of variables (rVariable and zVariable) in CDF, and they can happily coexist in a CDF: Every rVariable in a CDF must have the same number of dimensions and dimension sizes while each zVariable can have its own dimensionality.

The Internal Interface consists of only one routine, CDFlib. CDFlib can be used to perform all possible operations on a CDF. In fact, all of the Standard Interface functions are implemented using the Internal Interface. CDFlib must be used to perform operations not possible with the Standard Interface functions. These operations would involve CDF features added after the Standard Interface functions had been defined (e.g., specifying a single-file format for a CDF, accessing zVariables, or specifying a pad value for an rVariable or zVariable). Note that CDFlib can also be used to perform certain operations more efficiently than with the Standard Interface functions.

Unlike other programming languages, Perl only has three basic data types: scalars, arrays of scalars and hashes of scalars. No other defined data types are needed for any of the Perl-CDF operation items. For Perl applications, all CDF items are referenced starting at zero. These include variable, attribute, and attribute entry numbers, record numbers, dimensions, and dimension indices. Note that both rVariables and zVariables are numbered starting at zero.

One of the following constants must be used when specifying a CDF data type for an attribute entry or variable: CDF_BYTE 1-byte, signed integer. CDF_CHAR 1-byte, signed character. CDF_INT1 1-byte, signed integer. CDF_UCHAR 1-byte, unsigned character. CDF_UINT1 1-byte, unsigned integer. CDF_INT2 2-byte, signed integer. CDF_UINT2 2-byte, unsigned integer. CDF_INT4 4-byte, signed integer. CDF_UINT4 4-byte, unsigned integer. CDF_INT8 8-byte, signed integer. CDF_REAL4 4-byte, floating point. CDF_FLOAT 4-byte, floating point. CDF_REAL8 8-byte, floating point. CDF_DOUBLE 8-byte, floating point. CDF_EPOCH 8-byte, floating point. CDF_EPOCH16 two 8-byte, floating point. CDF_TIME_TT2000 8-byte, signed integer.

CDF_CHAR and CDF_UCHAR are considered character data types. These are significant because only variables of these data types may have more than one element per value (where each element is a character).

The Quick Interface functions represent a set of easy to use functions that are based on Internal Interface to acquire information from the CDF library, a CDF file, its variable and variable data. Normally, a few basic calls to Internal Interface calls are needed to accomplish a function. These functions perform the basics for users so the data can be returned easily. Examples can be found in Section 5.

4.Epoch Utility Routines and Time Utility Routines

Several functions exist that compute, decompose, parse, and encode CDF_EPOCH and CDF_EPOCH16 values. These functions may be called by applications using the CDF_EPOCH and CDF_EPOCH16 data types and are included in the CDF library. Function prototypes for these functions may be found in the include file cdf.h. The Concepts chapter in the CDF User’s Guide (https://spdf.gsfc.nasa.gov/pub/software/cdf/doc/cdf381/cdf381ug.pdf) describes EPOCH values. The date/time components for CDF_EPOCH and CDF_EPOCH16 are UTC-based, without leap seconds. The CDF_EPOCH and CDF_EPOCH16 data types are used to store time values referenced from a particular epoch. For CDF that epoch values for CDF_EPOCH and CDF_EPOCH16 are 01-Jan-0000 00:00:00.000 and 01-Jan-000000:00:00.000.000.000.000, respectively.

Several functions exist that compute, decompose, parse, and encode CDF_TIME_TT2000 values. These functions may be called by applications using the CDF_TIME_TT2000 data type and are included in the CDF library. The Concepts chapter in the CDF User’s Guide describes CDF_TIME_TT2000 values. The date/time components for CDF_TIME_TT2000 are UTC-based, with leap seconds. The CDF_TIME_TT2000 data type is used to store time values referenced from J2000 (2000-01-01T12:00:00.000000000). For CDF, values in CDF_TIME_TT2000 are nanoseconds from J2000 with leap seconds included. TT2000 data can cover years between 1707 and 2292.

5.Examples

CDFgetLIBInfo

CDFgetLIBInfo returns the currrent library version, release and increment information, along with the leap second table information. The latest leap second added to the leap second table is returned. Both fields are presented in a Perl Hash object.

The following example shows how the current CDF library and the leap second table are used.

.
.
.
my $status; # Returned status code.
my %info; # Returned hash.
.
.
($status, %info) = CDF::CDFgetLIBInfo();
UserStatusHandler (“1.0”. $status) if ($status < CDF_OK);
print Dumper(\%info);
$VAR1 = {
 'LATEST_LEAPSECOND_IN_TABLE' => 20170101,
 'LIB_VERSION' => '3.8.0'
118
 };

CDFgetCDFInfo

CDFgetCDFInfo returns the information about a specified CDF. The information includes the name (if available), its format, majority, encoding, file version, the number of global and variable attributes, the number of rVariables and zVariables. It also has the time tag about the leap second table that this file is based on (only applicable if the CDF has TT2000 epoch data type variables). All fields are presented in a Perl Hash object.

The following example shows the information for a given CDF.

.
.
.
my $status; # Returned status code.
my %info; # Returned hash.
.
.
($status, %info) = CDF::CDFgetCDFInfo(“testfile”);
UserStatusHandler (“1.0”. $status) if ($status < CDF_OK);
print Dumper(\%info);
$VAR1 = {
 'FORMAT' => 'SINGLE',
 'NAME' => 'testfile',
 'BASED_LEAPSECOND_LAST_UPDATED' => 20170101,
 'NUMgATTRS' => 5,
 'MAJORITY' => 'ROW',
 'NUMrVARS' => 0,
 'ENCODING' => 'IBMPC',
 'CDF_VERSION' => '3.8.0',
 'NUMzVARS' => 22,
 'NUMvATTRS' => 7,
 };

CDFgetGlobalMetaData

CDFgetGlobalMetaData returns all or a list of selected global attributes from the specified CDF. All fields are presented in a Perl Hash object. All data of epoch data types will be encoded accordingly.

The following example shows all of the global attributes from a given CDF, testfile.

.
.
.
my $status; # Returned status code.
my %info; # Returned hash.
.
.
($status, %info) = CDF::CDFgetGlobalMetaData(“testfile”);
UserStatusHandler (“1.0”. $status) if ($status < CDF_OK);
print Dumper(\%info);

CDFgetVarInfo

CDFgetVarInfo returns specification about a given variable from the specified CDF. The information includes its name, data type, number of elements, number of dimensions, dimensional sizes and variances if its dimension > 0, record variance, and last written record number (as MaxRec). All fields are presented in a Perl Hash object.

The following example shows the information from variable: “Latitude” in a CDF: “testfile.”

.
.
my $id; # CDF identifier
my $status; # Returned status code.
my %info; # Returned hash.
.
.
$status = CDF::CDFlib(&OPEN_, &CDF_, “testfile”, \$id, &NULL_);
($status, %info) = CDF::CDFgetVarInfo($id, “Latitude”);
UserStatusHandler (“1.0”. $status) if ($status < CDF_OK);
print Dumper(\%info);
$VAR1 = {
 'VarName' => 'Latitude',
 'NumElems' => 1,
 'RecVary' => 'False',
 'NumDims' => 1,
 'DimVarys' => {
 '0' => 'True'
 },
 'DataType' => 'CDF_INT1',
 'DimSizes' => {
 '0' => 3
 },
 'MaxRec' => 0
 };

CDFgetVarAllData

CDFgetVarAllData returns all data for a given variable from the specified CDF in an array object. The variable can be entered as a name or number.

The following example reads all variable data, a total of 3 records, from variable: “Latitude1”, a 1-D dimension of 3 elements with type: CDF_INT2, in a CDF: “testfile.”

.
.
my $id; # CDF identifier
my $status; # Returned status code.
my @data; # Returned data.
.
$status = CDF::CDFlib(&OPEN_, &CDF_, “testfile”, \$id, &NULL_);
($status, @data) = CDF::CDFgetVarAllData($id, “Latitude”);
UserStatusHandler (“1.0”. $status) if ($status < CDF_OK);
print Dumper(\@data);
$VAR1 = [
 254,
 254,
 5,
 15,
 25,
 35,
 100,
 128,
 255
 ];
($status, @data) = CDF::CDFgetVarAllData($id, “Latitude”, 0, 1);
UserStatusHandler (“1.0”. $status) if ($status < CDF_OK);
print Dumper(\@data); $VAR1 = [
 [
 254,
 254,
 5
 ],
 [
123
 15,
 25,
 35
 ],
 [
 100,
 128,
 255
 ]
 ];
.
.

CDFgetVarMetaData

CDFgetVarMetaData returns all the variable attribute information for a given variable from the specified CDF. All fields are presented in a Perl Hash object. All data of epoch type will be encoded accordingly.

The following example shows the variable attributes from variable: “Latitude” in a given CDF.

.
.
my $id; # CDF identifier
my $status; # Returned status code.
my %info; # Returned hash.
.
124
.
($status, %info) = CDF::CDFgetVarMetaData($id, “Latitude”);
UserStatusHandler (“1.0”. $status) if ($status < CDF_OK);
print Dumper(\%info);
$VAR1 = {
 'FILLVAL' => 111,
 'validmin' => 20,
 'VALIDMAX' => 90
 };
.
.