As images (chest x-rays, photographs of rashes, etc.) are loaded into OpenMRS, we will need tools to allow users to view and manipulate these images. We would like a clinician to be able to bring up one of the images in OpenMRS and easily manipulate the image to view details and potentially annotate the image. For example, a radiologist could bring up a patient's chest x-ray, zoom in to part of the image and adjust the image contrast to diagnose tuberculosis, and then add their impression to the database e.g., add an arrow to the image along with a comment like "this is consistent with tuberculosis."
The Google Maps Image Viewer module is based on Bmckown's Complex Obs Support system. The complex obs system handles storage and loading of image files. The Google Maps Image Viewer provides an alternative to the default image hander for viewing and editing images.
The image viewer is implemented as an OpenMRS module and AJAX-based web application using the Google Maps API. This API is well suited for zooming and navigating through very large images while keeping bandwidth usage to a minimum.
An intuitive user interface based on Google Maps enables easy navigation of large images. Zooming and panning works just like in Google Maps. The browser progressively downloads portions of the image (image tiles) as needed. Annotations are overlaid on the image, these can easily be read, edited, moved or deleted. It is also possible to choose not to display annotations.
The image viewer works with JPEG, GIF, PNG and possibly also BMP image files. Very large images can be used, it has been tested with a 12500x8750 pixel color image, which is 328 MB when decoded. Note that the fully decoded image will have to be stored in memory, so a large Java heap space is required.
The image viewer stores recently used images in memory on the server to speed up loading. The number of images to store can be set in the global property gmapsimageviewer.bufferSize. The Default is 2. When this setting has been changed, the module (or the server) will have to be restarted for the changes to take effect.
When scaling images, nearest neighbor scaling is used by default. This algorithm is fast, but does not provide the best image quality. Bilinear and bicubic scaling may be used instead, this can be set in the global property gmapsimageviewer.scalingQuality. Note that bilinear and bicubic scaling puts quite a bit more load on the server CPU.
The Google Maps API Terms of Service are potentially too restrictive, depending on how the clauses are interpreted. We have, however, gotten assurances from Google that the API can be used in this project.
The Google Maps Image Viewer module shows a medical image file as a map overlay in the map viewer. Annotations are shown as markers. Storing the full sets of image tiles for each zoom level is impractical, so the image tiles are instead generated from the original image on each request. This is handled by a servlet.
The picture above shows how a large image file is split into smaller image tiles. The numbers show the x,y coordinates for each tile. The tiles also have a third coordinate: The zoom level. This image is shown at zoom level 2.</div>
This picture shows how new tiles are created for zoom level 3. Each zoom level displays the image twice as high and twice as wide as the previous level. Four image tiles in this picture covers the same area as one tile in the previous picture.</div>
Start date: 1 June (GSoC 'Phase I')
Midterm date: 14 July
By this date, the basics should be ready: A working image viewer that loads images from the database. Annotation should also be ready.
*Second half (GSoC 'Phase II') *
Some things that may be added in the future.
Many image transformation types can be defined using the AffineTransform class in the Java API. These include translations, scales, flips, rotations, and shears. The image processing in this module is performed in the class ImageTileServlet, and is defined like so:
BufferedImage bufferedImageTile = new BufferedImage(IMAGE_TILE_SIZE, IMAGE_TILE_SIZE, colorFormat);
Graphics2D graphics = bufferedImageTile.createGraphics();
AffineTransform transform = new AffineTransform();
transform.translate(-IMAGE_TILE_SIZE * x, -IMAGE_TILE_SIZE * y);
graphics.drawImage(image, new AffineTransformOp(transform, interpolationType), 0, 0);
The defined AffineTransform performs two operations: Translate and scale. Additional image transformations may be added to the existing AffineTransform instance, please refer to the Java API documentation.
AffineTransform operations only manipulate the pixel coordinate space. In order to modify the pixels themselves, the class RescaleOp (also in the Java API) may be used. Example:
RescaleOp rescale = new RescaleOp(scaleFactor, offset, null);
The class name may be misleading: RescaleOp does not rescale the image in size. Its function is to rescale or offset the value of each pixel, i.e. modifying the colour value. The scaleFactor argument specifies a "contrast" adjustment, and the offset argument specifies a "brightness" adjustment. Again, please refer to the Java API.
New controls in the webapp may be required in order to manipulate settings for new features, the Google Maps API contains facilities to support this. Please refer to the documentation, especially the section on custom controls. The webapp is defined in the file imageViewerWindow.jsp.
If implementing additional image manipulation features, you may want to control these by passing additional HTTP GET parameters to the ImageTileServlet. This can be done in the function CustomGetTileUrl(a, b). This function contains a URL template that the webapp uses to generate URLs for each image tile.