GeoTIFF/GDAL OPeNDAP Server Design
Introduction
This document is a design for a GDAL based OPeNDAP server primariliy aimed
at serving GeoTIFF files such as those available from the UMD Global
Land Cover Facility. However, an effort is made to try and handle the broadest
set of geospatial raster products possible using GDAL and to capture as much
supporting information as possible.
GeoTIFF Data Model
GeoTIFF is a flexible raster format with geospatial extensions.
TIFF Imaging Model
A TIFF file may contain a series of image directories. Each image
directory describes a raster image which may have one or more samples
(bands) along with a variety of auxilary information describing the color
model, compression mechanisms, and other descriptive information. Each
directory has the following core image properties:
- Width: the image/raster width in pixels.
- Height: the image/raster height in pixels.
- SamplesPerPixel: the number of samples (bands) at each pixel in an image.
A 24bit RGB image is represented as 3 samples (red, green and blue) for
instance.
- BitsPerSample: The number of bits of data for each sample value. This
is often 8 for common greyscale, paletted or RGB images but values of 1, 2, 4,
8, 16, 32 and even a few other odd values are fairly common.
- SampleFormat: indicates how the bits should be interpreted. Normally this
is "unsigned integer", but other possible values include signed integer,
complex integer, ieee floating point and complex ieee floating point.
- PhotometricInterpretation: Contains the colormodel of the image such as
RGB, Paletted, and Greyscale.
- ColorTable: A colortable to be applied if the image is paletted. This
is a list of RGB values and assumes the image is unsigned integer with the
values being indexes into the color table. Effectively speaking color
tables only occur for images with 16 BitsPerSample or less.
In theory different samples in an image can have different bitspersample but
in practice this is extremely uncommon, and unsupported by GDAL or
libtiff so we will ignore this issue.
As mentioned, a TIFF file can have multiple image directories. Each
image directory (IFD) describes one image. When a file contains multiple
images, each image can be independent with size, data type and photometric
interpretation unrelated to others in the same file. It is also common that
each directory represent a page in documented oriented files, or that some
directories are reduced resolution forms of other directories.
The full
TIFF 6 Specification is found on the Adobe web site.
GeoTIFF Georeferencing
The GeoTIFF specification is an extention to the TIFF format, adding tags
to describe how pixel locations can be related to some geospatial coordinate
system and describing that geospatial coordinate system. That is it allows
you to determine where the image lies in the real world.
The GeoTIFF specification and additional information can be found on
the GeoTIFF
page.
Georeferencing Transformation
The georeferencing transformation is how pixel/line locations on the
image are related to some geospatial coordinate system. There are three
approaches to defining this relationship in GeoTIFF:
- Tiepoint + PixelScale: A location for one point on the image (a tiepoint)
is provided, along with a dimension of a pixel in the x and y dimensions in
that coordinate system. This is suitable for defining images that have
a regular unrotated sampling in the coordinate system in question.
- GeoTransMatrix: This is a matrix defining an affine transformation between
image spaces and georeferenced space. In 2D is it equivelent to the ESRI
world file often used for simple georeferencing. Once again, the image must
have already been sampled regularly in the coordinate system in question, but
the image may be rotated and sheared.
- Multiple Tiepoints: In the most general case, the GeoTIFF file just
contains a series of tiepoint. That is a list of locations on the image
with associated geospatial coordinate system positions. When this method
is used, there is no explicit assumption of how the location of other pixels
can be computed though various means such as best-bit polynomials are often
used. This method is appropriate for images that are not regularly sampled
in the target coordinate system or those that might have no analytical
relationship with the target coordinate system (such as raw satellite images
without terrain or other corrections).
Most GeoTIFF files in use used the first georeferencing form though it is
not uncommon for "raw" GeoTIFF files to be provided in the 3rd form with
only approximate tiepoints at image corners.
Coordinate System Descriptions
The description of the coordinate system is loosely based on the
EPSG coordinate system data model.
EPSG maintains catalogs of published geographic and projected coordinate
system descriptions with associated code numbers. For instance, the
common geographic coordinate system "WGS84" has the EPSG id 4326. The
projected coordinate system "UTM zone 11 North / WGS84" has EPSG id 32611.
The GeoTIFF file can store the whole coordinate system definition by
reference to a particular GCS (geographic coordinate system) or
PCS (projected coordinate system) code from EPSG, or it can contain a set
of parameters defining a user provided coordinate system.
GDAL Data Model
The GDAL data model is fairly closely related to the GeoTIFF data model,
and is more completely described in the
GDAL Data Model
document. Here we will point out some key differences between the GeoTIFF
data model, and the GDAL data model that will have implications for our
design.
- The GDAL data model has a smaller set of pixel data types available than
are possible in TIFF. For instance, there is no explicit 1bit, 2bit, or 4bit
data type in GDAL. GDAL does include Byte, UInt16, Int16, UInt32, Int32,
Float32, Float64, ComplexInt16, ComplexInt32, ComplexFloat32, and
ComplexFloat64 data types. Typically underlying pixel types that don't
match one of these are promoted to the next larger type that fully expresses
possible values.
- The GDAL data model does not have a concept of multiple "image
directories" in a single dataset. However, it does have a concept of
"sub datasets" where a GDAL dataset can refer to sub-component images which
may even be part of the same file. At this time however the GDAL GeoTIFF
driver does not reference extra image directories this way.
- The GDAL data model includes some specific kinds of per-band (per-sample)
metadata that are not included in TIFF including pixel scaling information,
pixel "units", and a nodata value.
- The GDAL georeferencing model has two methods. An affine
geotransform (equivelent to the GeoTIFF GeoTransMatrix) that also handles
the simpler tiepoint+pixelscale case, and a list of GCPs approach. While
expressed slightly differently the GDAL and GeoTIFF georeferencing models
are essentially identical.
- GDAL describes coordinate systems with the OpenGIS Well Known Text
textual descriptions of coordinate systems. This approach is also built
on the EPSG model and is quite analygous to the GeoTIFF approach though
there are some discontinuities converting between GeoTIFF and WKT formats.
- GDAL includes an arbitrary list of name=value metadata items at the
image and band level. Some TIFF tags (such as the description tag) are
captured as specific image-wide metadata in the GDAL GeoTIFF driver.
- GDAL does support having different pixel data types for different
bands in a data and this does practically occur for some file formats
(such as PCIDSK).
For the most part the GDAL data model is a superset of the GeoTIFF data
model.
DDS Model Design
The core decision is to model each GDALRasterBand (of TIFF "sample") as
a Grid 2D object. The maps will be used to associate georeferencing
information with the grid. A simple 512x512 1 band 8bit image might have
a DDS that looks like this:
Dataset {
Grid {
ARRAY:
Byte band_1[easting = 512][northing = 512];
MAPS:
Float64 easting[easting = 512];
Float64 northing[northing = 512];
} band_1;
} utm.tif;
An RGBA file would have a grid for each band, and might look like this:
Dataset {
Grid {
ARRAY:
Byte band_1[easting = 256][northing = 200];
MAPS:
Float64 easting[easting = 256];
Float64 northing[northing = 200];
} band_1;
Grid {
ARRAY:
Byte band_2[easting = 256][northing = 200];
MAPS:
Float64 easting[easting = 256];
Float64 northing[northing = 200];
} band_2;
Grid {
ARRAY:
Byte band_3[easting = 256][northing = 200];
MAPS:
Float64 easting[easting = 256];
Float64 northing[northing = 200];
} band_3;
Grid {
ARRAY:
Byte band_4[easting = 256][northing = 200];
MAPS:
Float64 easting[easting = 256];
Float64 northing[northing = 200];
} band_4;
} rgba.tif;
Some points worth noting:
- Each band is given a generic name based on the band number (ie. band_1).
At this time there is no effort being made to name bands based on
photometric interpretation or any other information.
- The x/y map dimensions are called easting and northing.
For datasets in a geographic coordinate system these would hold the longitude
and latitude. For projected coordinate systems they would normally be the
easting and northing in the projected coordinate system. For ungeoreferenced
images the easting and northing would just be the pixel/line offsets of each
pixel (as implied by a unity geotransform).
- The size of the easting and northing arrays are of course the number
of pixels and lines in the raster.
- Complex pixel types will be represented as two grids with names
like "band_1_real" and "band_1_imaginary".
Outstanding Issues:
- Do we want to represent all subdatasets as additional sets of grid
when accessing the master dataset? I think this would desirable though it
would result in a fairly complex dds.
DAS Model Design
A variety of information will be transported via the DAS describing
the dataset.
Attributes {
GLOBAL {
Float64 Northernmost_Northing 71.1722;
Float64 Southernmost_Northing 4.8278;
Float64 Easternmost_Easting -27.8897;
Float64 Westernmost_Easting -112.11;
Float64 GeoTransform "71.1722 0.001 0.0 -112.11 0.0 -0.001";
String spatial_ref "GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SPHEROID[\"WGS 84\",6378137,298.257223563]],PRIMEM[\"Greenwich\",0],UNIT[\"degree\",0.0174532925199433]]";
Metadata {
String TIFFTAG_XRESOLUTION "400";
String TIFFTAG_YRESOLUTION "400";
String TIFFTAG_RESOLUTIONUNIT "2 (pixels/inch)";
}
}
band_1 {
String Description "...";
String
}
}
Dataset
There will be an object in the DAS named GLOBAL containing
attributes of the dataset as a whole.
It will have the following subitems:
- Northernmost_Northing: The latitude or northing of the north edge of the image.
- Southernmost_Northing: The latitude or northing of the south edge of the image.
- Easternmost_Easting: The longitude or easting of the east edge of the image.
- Westernmost_Easting: The longitude or easting of the west edge of the image.
- GeoTransform: The six parameters defining the affine transformation
between pixel/line space and georeferenced space if applicable. Stored
as a single string with values seperated by sapces. Note this
allows for rotated or sheared images. (optional)
- SpatialRef: The OpenGIS WKT description of the coordinate system.
If not provided it will be assumed that the coordinate system is WGS84.
(optional)
- Metadata: a container with a list of string attributes for
each available metadata item. The metadata item keyword name will be used
as the attribute name. Metadata values will always be strings. (optional)
- address GCPs
Note that the edge northing and easting values can be computed based
on the grid size and the geotransform. They are included primarily as
extra documentation that is easier to interprete by a user than the
GeoTransform.
Band
There will be an object in the DAS named after each band containing
attribute of the specific band.
It will have the following subitems:
An effort has been made to adhere to netCDF COORDS and/or CF conventions where
the appear to be applicable. Some further review in this area is still
called for. For instance, the CF convention includes (in appendix F)
naming conventions for describing projected coordinate systems that are
not currently being used.
Implementation Notes
- The server is being implemented as a CGI, though this may result in
some performance overhead if many small requests are made against a large
raster file, or a raster file stored in a format that is expensive to
open as it will need to be reopened for each das, dds or dods request.
In general there is relatively little overhead in opening GeoTIFF files
through GDAL.
- The code is being implemented in C++ modelled somewhat on the matlab
exampled used in the "how to write an OPeNDAP server" document.
- For now seperate mainlines are being prepared for the das, dds and
dods cgis with of course lots of shared code.
- The GDALRasterBand RasterIO() method will be used to extract
projected sub-windows requested by the client. This should generally result
in efficient access to small areas of a large image or
large subsampled areas of images with overviews pre-built.
- Custom "GDAL" versions of all datatypes except String and URL are
planned to ensure data can be returned in the closest available
OPeNDAP data type.
- Normal GDAL-style documentation is being included in the code, but not
Doxygen style documentation. As well, an HTML document will be written
describing how to build, install and configure the GDAL OPeNDAP server.
- The C++ source will be provided back to OPeNDAP Inc to include in
CVS. I would be happy to continue maintaining the code as needed if
CVS commit access for this subtree can be arranged. Alternatively,
I would be willing to include the server in the main GDAL source tree
if desired.