New Open File Formats for the Biological Imaging Community

Biological microscopy has always involved "imaging": images were initially hand drawn and with the advent of light-sensitive film, recorded and then reproduced on paper. These methods distorted the relationships between the signals they recorded (formally, they are "non-linear media"), making it difficult to use them for scientific measurements. However, the application of digital detectors to microscopy delivered "linear" measurements suitable for scientific use. This, combined with automation, spawned massive growth in the number and diversity of uses for digital imaging in basic and clinical research. Each platform produces many GBytes of data, usually in a closed, proprietary file format. These are powerful systems, but their full utility is limited by closed data and the difficulty of viewing and sharing large datasets on standard desktop computers.

The Open Microscopy Environment (OME) has built open software tools that enable access, analysis, viewing and sharing of this data. Initially built for light microscopy, we have successfully extended these tools to electron microscopy, high content screening (used for drug discovery in pharmaceutical research) and digital pathology. This proposal seeks to extend the file formats that OME supports and ensure there are specifications and software that meet the demands of the most advanced biological imaging modalities. All of OME's software and resources are open source, available on-line to anyone, and supported by a dedicated team that manages documentation and community outreach.

OME's OME-TIFF is well-adopted in the biological imaging community. In this proposal, we aim to use OME's strong standing in the community as a developer and supplier of specifications and software for biological imaging metadata and apply our expertise and tools to several new binary image format s that have appeared. The proposed formats compromise a range of different data storage strategies and thus will be useful on their own, and also provide templates for others to extend other formats.

We will deliver software that supports storage of OME metadata in:

1. the HDF5-based BigDataViewer (BDV) format, making OME metadata available for users of this format (to date, primarily LSFM and block-face scanning electron microscopy). In a related project, we will add OME metadata, in using OME's ScreenPlateWell specification to the HDF-based Cellh5 format, which is used in a small number of HCS applications. Besides delivering updates to the specified formats, the formats and software will also serve as templates for anyone wanting to store metadata in their own version of HDF5.

2. the KLB format, an open source binary format for LSFM that uses fast lossless compression. As image volumes grow, several applications will require ways of incorporating compression. The KLB format provides one of the more advanced methods of storing and and compressing large numbers of 2D planes.

3. DICOM-Sup145, a derivative of DICOM developed to support pyramidal multi-resolution whole slide imaging data. To date the specification does not include released software. Moreover the current specification includes no concepts around annotations, regions of interest, etc.

Our implementations will be delivered to the community in Java and C++ to ensure they have the maximal exposure and utility.

The rise of quantitative biology has driven the generation of ever increasing stores of experimental data that are the foundation for biological research and discovery. Unfortunately, full exploitation of these data still remains unrealised. Data generated on commercial platforms are not stored in easily accessible formats and the size and complexity of these data makes routine analysis and sharing difficult. Collaborations depend on data sharing, but the transfer of complex, large datasets (>100 Gbytes is routine) between scientists, labs and/or software tools limits what can be achieved and is ultimately a barrier to scientific discovery.

OME's goal is to provide interfaces that enable data exchange-- between different software tools and between geographically remote scientists. Currently, OME's Bio-Formats file translation library and OMERO data management platform enable:

-- access to >145 scientific image file formats;
-- management, analysis, and sharing of image data relevant to a diverse range of biological research topics;
-- the foundation for the first on-line image publication facilities.
- and relevant to this proposal, open formats for storing image metadata and binary data in an open form.

The major impact of this project will be the appearance of alternative open formats for biological imaging. This recognises the reality that a single file format can't cover the range of imaging modalities and experimental regimes across the wide range of biological imaging applications. We aim to initiate the development of several formats that support multi-resolution and TB-scale imaging.

More generally, OME's tools are used worldwide, in thousands of laboratories, across many different domains of biological research. OME's commitment to an open development process, where all planning, roadmapping, user support, and developed code are openly available has built an active community of users in academic, biotech and pharmaceutical research. Some simply use the software as is, but many see it as a platform upon which their own applications, defined by their research needs, can be built. OMERO is the foundation for PerkinElmer's Columbus data management system which now runs HCS data in most major pharmaceutical companies in the world. OMERO and Bio-Formats also power several on-line scientific image repositories, the largest of which is the JCB DataViewer (http://jcb-dataviewer.rupress.org). Thus, the impact of OME, and its future funding and activities enhance research and productivity in laboratories in the UK and around the world.