On Gentry I did the following:
import netCDF3 print netCDF3.__doc__
In the following you will see
>>> import netCDF3
>>> ncfile = netCDF3.Dataset('test.nc', 'w')but for consistency with other netCDf interfaces, the above can also be coded as:
>>> from netCDF3 import Dataset as NetCDFFile
>>> ncfile = NetCDFFile('test.nc', 'w')Okay, here is the result of help(netCDF3):
Introduction
============
Python interface to the netCDF version 3 library. The API modelled after
U{Scientific.IO.NetCDF
<http://starship.python.net/~hinsen/ScientificPython>}, and should be
familiar to users of that module. Some new features not found in
Scientific.IO.NetCDF:
- ability to read multi-file netCDF Datasets, making variables
spanning multiple files appear as if they were in one file
(see L{MFDataset} for more details).
- support for masked arrays, automatic packing and unpacking of
packed integer data (see L{Variable.set_auto_maskandscale} for details).
- supports more complicated slicing (including numpy 'fancy indexing').
- includes convenience functions for converting to and from datetime
objects to numeric time values (see L{num2date} and L{date2num}), using
all the calendars in the CF standard.
- convenience functions for converting arrays of characters to arrays
of strings, and vice-versa (L{stringtochar} and L{chartostring}).
- can use numpy dtype objects to specify netCDF variable datatype.
Download
========
- U{Project page <http://code.google.com/p/netcdf4-python/>}.
- U{Subversion repository <http://code.google.com/p/netcdf4-python/source>}.
- U{Source tar.gz <http://code.google.com/p/netcdf4-python/downloads/list>}.
Requires
========
- numpy array module U{http://numpy.scipy.org}, version 1.0 or later.
- The netCDF-3 C library (version 3.6 or later), available at
U{ftp://ftp.unidata.ucar.edu/pub/netcdf}.
Install
=======
- install the requisite python modules and C libraries (see above).
Optionally, set the C{NETCDF3_DIR} environment variable to point to where the
netCDF version 3 library and headers are installed. If C{NETCDF3_DIR} is
not set, some standard locations will be searched.
- run 'python setup-nc3.py install'
- run some of the tests in the 'test3' directory.
Tutorial
========
1) Creating/Opening/Closing a netCDF file
-----------------------------------------
To create a netCDF file from python, you simply call the L{Dataset}
constructor. This is also the method used to open an existing netCDF
file. If the file is open for write access (C{w, r+} or C{a}), you may
write any type of data including new dimensions, variables and
attributes. netCDF files come in several flavors (C{NETCDF3_CLASSIC,
NETCDF3_64BIT, NETCDF4_CLASSIC}, and C{NETCDF4}). The first two flavors
are supported by version 3 of the netCDF library, and are supported
in this module. To read or write C{NETCDF4} and C{NETCDF4_CLASSIC}
files use the companion L{netCDF4} python module. The default format
C{NETCDF3_64BIT}. To see how a given file is formatted, you can examine the
C{file_format} L{Dataset} attribute. Closing the netCDF file is
accomplished via the L{close<Dataset.close>} method of the L{Dataset}
instance.
Here's an example:
>>> import netCDF3
>>> ncfile = netCDF3.Dataset('test.nc', 'w')
>>> print ncfile.file_format
NETCDF3_64BIT
>>>
>>> ncfile.close()
2) Dimensions in a netCDF file
------------------------------
netCDF defines the sizes of all variables in terms of dimensions, so
before any variables can be created the dimensions they use must be
created first. A special case, not often used in practice, is that of a
scalar variable, which has no dimensions. A dimension is created using
the L{createDimension<Dataset.createDimension>} method of a L{Dataset}
instance. A Python string is used to set the name of the
dimension, and an integer value is used to set the size. To create an
unlimited dimension (a dimension that can be appended to), the size
value is set to C{None}. netCDF 3 files can only have one unlimited
dimension, and it must be the first (leftmost) dimension of the variable.
>>> ncfile.createDimension('press', 10)
>>> ncfile.createDimension('time', None)
>>> ncfile.createDimension('lat', 73)
>>> ncfile.createDimension('lon', 144)
All of the L{Dimension} instances are stored in a python dictionary.
>>> print ncfile.dimensions
{'lat': <netCDF3.Dimension object at 0x24a5f7b0>,
'time': <netCDF3.Dimension object at 0x24a5f788>,
'lon': <netCDF3.Dimension object at 0x24a5f7d8>,
'press': <netCDF3.Dimension object at 0x24a5f760>}
>>>
Calling the python C{len} function with a L{Dimension} instance returns
the current size of that dimension. The
L{isunlimited<Dimension.isunlimited>} method of a L{Dimension} instance
can be used to determine if the dimensions is unlimited, or appendable.
>>> for dimname, dimobj in ncfile.dimensions.iteritems():
>>> print dimname, len(dimobj), dimobj.isunlimited()
lat 73 False
time 0 True
lon 144 False
press 10 False
>>>
L{Dimension} names can be changed using the
L{renameDimension<Dataset.renameDimension>} method of a L{Dataset} instance.
3) Variables in a netCDF file
-----------------------------
netCDF variables behave much like python multidimensional array objects
supplied by the U{numpy module <http://numpy.scipy.org>}. However,
unlike numpy arrays, netCDF3 variables can be appended to along one
'unlimited' dimension. To create a netCDF variable, use the
L{createVariable<Dataset.createVariable>} method of a L{Dataset}
instance. The L{createVariable<Dataset.createVariable>} method
has two mandatory arguments, the variable name (a Python string), and
the variable datatype. The variable's dimensions are given by a tuple
containing the dimension names (defined previously with
L{createDimension<Dataset.createDimension>}). To create a scalar
variable, simply leave out the dimensions keyword. The variable
primitive datatypes correspond to the dtype attribute of a numpy array.
You can specify the datatype as a numpy dtype object, or anything that
can be converted to a numpy dtype object. Valid datatype specifiers
include: C{'f4'} (32-bit floating point), C{'f8'} (64-bit floating
point), C{'i4'} (32-bit signed integer), C{'i2'} (16-bit signed
integer), C{'i1'} (8-bit signed integer), or C{'S1'} (single-character string)
The old Numeric single-character typecodes (C{'f'},C{'d'},C{'h'},
C{'s'},C{'b'},C{'B'},C{'c'},C{'i'},C{'l'}), corresponding to
(C{'f4'},C{'f8'},C{'i2'},C{'i2'},C{'i1'},C{'i1'},C{'S1'},C{'i4'},C{'i4'}),
will also work.
The dimensions themselves are usually also defined as variables, called
coordinate variables. The L{createVariable<Dataset.createVariable>}
method returns an instance of the L{Variable} class whose methods can be
used later to access and set variable data and attributes.
>>> times = ncfile.createVariable('time','f8',('time',))
>>> pressure = ncfile.createVariable('press','i4',('press',))
>>> latitudes = ncfile.createVariable('latitude','f4',('lat',))
>>> longitudes = ncfile.createVariable('longitude','f4',('lon',))
>>> # two dimensions unlimited.
>>> temp = ncfile.createVariable('temp','f4',('time','press','lat','lon',))
All of the variables in the L{Dataset} are stored in a
Python dictionary, in the same way as the dimensions:
>>> print ncfile.variables
{'temp': <netCDF3.Variable object at 0x24a61068>,
'pressure': <netCDF3.Variable object at 0.35f0f80>,
'longitude': <netCDF3.Variable object at 0x24a61030>,
'pressure': <netCDF3.Variable object at 0x24a610a0>,
'time': <netCDF3.Variable object at 02x45f0.4.58>,
'latitude': <netCDF3.Variable object at 0.3f0fb8>}
>>>
L{Variable} names can be changed using the
L{renameVariable<Dataset.renameVariable>} method of a L{Dataset}
instance.
4) Attributes in a netCDF file
------------------------------
There are two types of attributes in a netCDF file, global and variable.
Global attributes provide information about a dataset as a whole.
L{Variable} attributes provide information about
one of the variables in a dataset. Global attributes are set by assigning
values to L{Dataset} instance variables. L{Variable}
attributes are set by assigning values to L{Variable} instance
variables. Attributes can be strings, numbers or sequences. Returning to
our example,
>>> import time
>>> ncfile.description = 'bogus example script'
>>> ncfile.history = 'Created ' + time.ctime(time.time())
>>> ncfile.source = 'netCDF3 python module tutorial'
>>> latitudes.units = 'degrees north'
>>> longitudes.units = 'degrees east'
>>> pressure.units = 'hPa'
>>> temp.units = 'K'
>>> times.units = 'hours since 0001-01-01 00:00:00.0'
>>> times.calendar = 'gregorian'
The L{ncattrs<Dataset.ncattrs>} method of a L{Dataset} or
L{Variable} instance can be used to retrieve the names of all the netCDF
attributes. This method is provided as a convenience, since using the
built-in C{dir} Python function will return a bunch of private methods
and attributes that cannot (or should not) be modified by the user.
>>> for name in ncfile.ncattrs():
>>> print 'Global attr', name, '=', getattr(ncfile,name)
Global attr description = bogus example script
Global attr history = Created Mon Nov 7 10.30:56 2005
Global attr source = netCDF3 python module tutorial
The C{__dict__} attribute of a L{Dataset} or L{Variable}
instance provides all the netCDF attribute name/value pairs in a python
dictionary:
>>> print ncfile.__dict__
{'source': 'netCDF3 python module tutorial',
'description': 'bogus example script',
'history': 'Created Mon Nov 7 10.30:56 2005'}
Attributes can be deleted from a netCDF L{Dataset} or
L{Variable} using the python C{del} statement (i.e. C{del var.foo}
removes the attribute C{foo} the the variable C{var}).
6) Writing data to and retrieving data from a netCDF variable
-------------------------------------------------------------
Now that you have a netCDF L{Variable} instance, how do you put data
into it? You can just treat it like an array and assign data to a slice.
>>> import numpy
>>> latitudes[:] = numpy.arange(-90,91,2.5)
>>> pressure[:] = numpy.arange(1000,90,-100)
>>> print 'latitudes =\n',latitudes[:]
latitudes =
[-90. -87.5 -85. -82.5 -80. -77.5 -75. -72.5 -70. -67.5 -65. -62.5
-60. -57.5 -55. -52.5 -50. -47.5 -45. -42.5 -40. -37.5 -35. -32.5
-30. -27.5 -25. -22.5 -20. -17.5 -15. -12.5 -10. -7.5 -5. -2.5
0. 2.5 5. 7.5 10. 12.5 15. 17.5 20. 22.5 25. 27.5
30. 32.5 35. 37.5 40. 42.5 45. 47.5 50. 52.5 55. 57.5
60. 62.5 65. 67.5 70. 72.5 75. 77.5 80. 82.5 85. 87.5
90. ]
>>>
>>> print 'pressure levels =\n',pressure[:]
[1000 900 800 700 600 500 400 300 200 100]
>>>
Unlike numpy array objects, netCDF L{Variable} objects with unlimited
dimensions will grow along those dimensions if you assign data outside
the currently defined range of indices.
>>> # append along two unlimited dimensions by assigning to slice.
>>> nlats = len(ncfile.dimensions['lat'])
>>> nlons = len(ncfile.dimensions['lon'])
>>> nlevs = len(ncfile.dimensions['press'])
>>> print 'temp shape before adding data = ',temp.shape
temp shape before adding data = (0, 10, 73, 144)
>>>
>>> from numpy.random.mtrand import uniform
>>> temp[0:5,:,:,:] = uniform(size=(5,nlevs,nlats,nlons))
>>> print 'temp shape after adding data = ',temp.shape
temp shape after adding data = (5, 16, 73, 144)
>>>
Time coordinate values pose a special challenge to netCDF users. Most
metadata standards (such as CF and COARDS) specify that time should be
measure relative to a fixed date using a certain calendar, with units
specified like C{hours since YY:MM:DD hh-mm-ss}. These units can be
awkward to deal with, without a utility to convert the values to and
from calendar dates. The functione called L{num2date} and L{date2num} are
provided with this package to do just that. Here's an example of how they
can be used:
>>> # fill in times.
>>> from datetime import datetime, timedelta
>>> from netCDF3 import num2date, date2num
>>> dates = [datetime(2001,3,1)+n*timedelta(hours=12) for n in range(temp.shape[0])]
>>> times[:] = date2num(dates,units=times.units,calendar=times.calendar)
>>> print 'time values (in units %s): ' % times.units+'\n',times[:]
time values (in units hours since January 1, 0001):
[ 17533056. 17533068. 17533080. 17533092. 17533104.]
>>>
>>> dates = num2date(times[:],units=times.units,calendar=times.calendar)
>>> print 'dates corresponding to time values:\n',dates
dates corresponding to time values:
[2001-03-01 00:00:00 2001-03-01 12:00:00 2001-03-02 00:00:00
2001-03-02 12:00:00 2001-03-03 00:00:00]
>>>
L{num2date} converts numeric values of time in the specified C{units}
and C{calendar} to datetime objectecs, and L{date2num} does the reverse.
All the calendars currently defined in the U{CF metadata convention
<http://cf-pcmdi.llnl.gov/documents/cf-conventions/>} are supported.
All of the code in this tutorial is available in C{examples/tutorial-nc3.py},
Unit tests are in the C{test3} directory.
7) Reading data from a multi-file netCDF dataset.
-------------------------------------------------
If you want to read data from a variable that spans multiple netCDF files,
you can use the L{MFDataset} class to read the data as if it were
contained in a single file. Instead of using a single filename to create
a L{Dataset} instance, create a L{MFDataset} instance with either a list
of filenames, or a string with a wildcard (which is then converted to
a sorted list of files using the python glob module).
Variables in the list of files that share the same unlimited
dimension are aggregated together, and can be sliced across multiple
files. To illustrate this, let's first create a bunch of netCDF files with
the same variable (with the same unlimited dimension).
>>> for nfile in range(10):
>>> f = Dataset('mftest'+repr(nfile)+'.nc','w')
>>> f.createDimension('x',None)
>>> x = f.createVariable('x','i',('x',))
>>> x[0:10] = numpy.arange(nfile*10,10*(nfile+1))
>>> f.close()
Now read all the files back in at once with L{MFDataset}
>>> f = MFDataset('mftest*nc')
>>> print f.variables['x'][:]
[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49
50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74
75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
>>>
Note that MFDataset can only be used to read, not write, multi-file
datasets.
@contact: Jeffrey Whitaker <jeffrey.s.whitaker@noaa.gov>
@copyright: 2007 by Jeffrey Whitaker.
@license: Permission to use, copy, modify, and distribute this software and
its documentation for any purpose and without fee is hereby granted,
provided that the above copyright notice appear in all copies and that
both the copyright notice and this permission notice appear in
supporting documentation.
THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO
EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, INDIRECT OR
CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF
USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR
OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THIS SOFTWARE.