How are data files used with ArcGIS software are handled for digital preservation?

Question

What are the techniques that people are using to group, preserve, and maintain access to files used with ArcGIS software?

Answer 1

As no-one's answered this question so far, I'll try to have a first stab by giving a couple of resources that may be useful. Disclaimer: I've worked with some older versions of ArcGIS and its predecessors (Arc/Info, ArcView) in the past, but this is quite a while ago, so some of my knowledge on this may be outdated (e.g. I largely missed out on Geodatabases, which is apparently the native format of ArcGis these days).

A good general overview of the challenges of geospatial data (this also addresses some issues that are specific to the ESRI formats):

• Technology Watch Report Preserving Geospatial Data: http://www.dpconline.org/component/docman/doc_download/363-preserving-geospatial-data-by-guy-mcgarva-steve-morris-and-gred-greg-janee

Recent versions of ArcGIS mainly work with Geodatabases. I never worked with them myself, but here are some resources:

• ESRI Arc Geodatabase (LoC): http://www.digitalpreservation.gov/formats/fdd/fdd000293.shtml

• Archival Challenges Associated with the Esri Personal Geodatabase and File Geodatabase Formats: http://www.geomapp.net/docs/Geodatabase_Report_v1.0_final_20111206.pdf

Then the 'classic' formats: coverages, grids and shapefiles:

• ESRI ArcInfo Coverage (LoC): http://www.digitalpreservation.gov/formats/fdd/fdd000284.shtml

• ESRI Grid (LoC): http://www.digitalpreservation.gov/formats/fdd/fdd000281.shtml

• ESRI ArcInfo interchange file (E00) (LoC): http://www.digitalpreservation.gov/formats/fdd/fdd000291.shtml

• ESRI Shapefile (LoC): http://www.digitalpreservation.gov/formats/fdd/fdd000280.shtml

What I remember from working with Coverages and Grids is that it is absolutely vital that they should never be separated from the info directory in the folder in which they are located. The thing to watch for with Shapefiles is that they are actually made up of a multiple files (with different file extensions) that share a common prefix. So things will start breaking down when you separate them.

Most of the above formats are proprietary, and some of them are quite complex. So far my (slightly haphazard) 2 cents on this ...

 

Answer 2

Johan's answer is a good one.

In terms of packaging, the usual options apply for packaging tools (Zip, Tar, Bagit etc.). However, the Open Geospatial Consortium recently finalized the GeoPackage Encoding Standard. The GeoPackage Encoding Standard (http://www.opengeospatial.org/standards/geopackage) that:

“defines GeoPackages for exchange and GeoPackage SQLite Extensions for direct use of vector geospatial features and / or tile matrix sets of earth images and raster maps at various scales. Direct use means the ability to access and update data in a "native" storage format without intermediate format translations in an environment (e.g. through an API) that guarantees data model and data set integrity and identical access and update results in response to identical requests from different client applications. GeoPackages are interoperable across all enterprise and personal computing environments, and are particularly useful on mobile devices like cell phones and tablets in communications environments with limited connectivity and bandwidth.”

It’s not quite the same as a container wrapper. It’s an SQLite database schema. As they say (http://www.geopackage.org/):

“Install Spatialite – the premiere spatial extention to SQLite – and you get all the performance of a spatial database along with the convenience of a file-based data set that can be emailed, shared on a USB drive or burned to a DVD.”

It is being implemented in a variety of tools, but I’d need more information to determine its suitability as a preservation packaging format for geospatial data.