Enzymind Labs

Dense Image Space (DIS) description

    This is a skeleton description of the construction and display geometry of a spatially dense array of images (dense image space, or DIS), which retains and utilizizes all overlapping image data. Superficially a DIS is similar to stitching multiple images into a panorama.

    With a panorama:

    1) Typically the viewpoint is identical for each component image. The viewpoints must identical for perfect "stitching" (coregistration and compositing) of photographs of 3-D subject matter.

    2) Overlapping image data is only used for coregistration of adjacent images, and duplicate information is discarded or blended.

    But with a DIS the goal is to:

    1) Acquire component images at an array of different closely space viewpoints, on a rectilinear or cylindrical grid (in one, two, or three spatial dimensions).

    2) Retain overlapping image data and allow display of several adjacent perspectives of the same spatial points.

    Conceptually, a panorama can be compared to standing at one spot and pivoting about your eye to look in a variety of directions; a panning operation in panorama display software, or visually scanning a large panoramic print, mimics the visual effect of this egocentric rotation. The analogous comparison for DIS navigation would be eye translation and eye pivoting about any subject point.

Figure 1.
a) Single panorama viewpoint covering a wide field of view.
b) Multiple DIS viewpoints covering a wide field of view. A panning operation corresponds to a viewpoint changes (three on left) without change in direction of gaze, while a pivot operation corresponds to viewpoint changes (three on right) with coordinated changes in direction of gaze such that the same subject point stays centered in the field of view.

    To achieve visually smooth translation and pivoting with DIS, a small image spacing (a high density image space) is required. But even a moderately dense image space can be used to mimic a smooth translation and also present discrete adjacent perspectives in a contiguous fashion. A DIS cannot be projected onto a single 2-D surface like a panorama. In a sense a DIS is a multi-perspective panorama; many panoramas from a variety of perspectives could be constructed from DIS components. Software image management and digital display, with large random access memory availability, is an ideal environment for navigating such a multi-dimensional image space.

    Stereo pair display of DIS data is naturally accomodated if the grid spacing (distance between image viewpoints) is chosen well. Stereo pairs can be constructed simply by always choosing horizontally adjacent pairs of viewpoints while maintaining the same subject points in the fields of view of the pair.

A simple dense image space; acquisition, image, and display geometry

    Conside a set of photographs, all taken in the same direction but from horizontally adjacent veiwpoints (Fig. 2). In general the fields of view of these images will overlap, so the images will contain several perspectives of the same object. If this array of images is very large, we would like a method for quickly and intuitively navigating the image space, with as much continuity is possible. To achieve this in software with a digital display, the image set will be conceptually arranged as a coregistered "stack" (Fig. 2. c) at the subject distance, and pan and pivot operations (Fig. 3) will allow navigation of all possible views.


Figure 2. A set of photographic images acquired from a linear grid of viewpoint locations forms one possible dense image space.
a) With a fractional overlap f = .8 at the subject background distance D, any central point will be covered by the field of view of 1/(1-f) = 5 images (five colored rectangles, with corresponding crosses indicating respective viewpoint grid locations).
b) Cross section of the angular fields of view of each image intersecting the subject background (gray). The lower mauve ball, although occluded in the central image, is contained within all five images, but the upper blue ball at a distance < D is not fully contained in the far left and right fields of view.
c) Cross section of all images, projected at the subject background distance D. The order of the stacking is arbitrary. At this projected distance (relative horizontal alignment) image points at the background distance will be vertically aligned (horizontally coregistered).
d) All images, spread out to show the relevant image differences. In the central image the blue ball occludes the mauve ball. The image to the immediate right of center (green frame) shows the perspective difference of this viewpoint. The image to the far right shows an increased perspective difference, and clips central foreground (blue ball) subject matter at a distance < D.

    In the one dimensional DIS example of Fig. 2, with fractional overlap f = .8 at the subject background distance D, every subject point at distance D will be covered by 1/(1-f) = 5 images. This allows five discrete rotational views of every central point. If this example was extended to two dimensions (f = .8 in both dimensions) every point would be covered by (1/(1-f))² = 25 images, allowing twenty-five discrete rotational views of every central point.

    The DIS display software will, by default, present the image with a viewpoint that is closest to perpendicular at the center of the viewport (display window with respect to the images). In addition the software will allow discrete pivoting of the viewpoint to available adjacent perspectives. The number of available adjacent perspectives will depend on the amount of overlap in the acquisitions. Any perspective mismatch between adjacent images displayed in the same viewport will be minimized by dynamically adjusting the relative display position of the images based on an estimated distance to subject near the boundary.

    For software and digital display purposes, all images that have coverage within or close to the current viewport are made available in main memory. An estimate of the subject distance is made from coregistration information (from the disparity of corresponding points within the viewport, centrally weighted) . This subject distance is used to co-align the images, as in Fig. 2.c. Generally, at native resolution, the viewport width/height will be a fraction (~1/2) of each image width/height, and the viewport will display the single image with a viewpoint that most closely corresponds to that viewport's pan position and desired pivot angle (Fig. 3).




Figure 3. Image selection and viewport display with pan and pivot operations. The available and displayed images are indicated with brown/green/red. The viewport (blue double stripes, square) is shown with respect to the field of view of each and all images.
a) Vertical and central perspective.
b) Right pivot with respect to a). The same subject matter is in the center of the field of view, but from an adjacent perspective.
c) Right pivot with respect to b). The viewport cannot be filled by the extreme right images field of view, so the remainder is filled with the adjacent perspective's content.
d-f) A central field of view pan operation, with respect to a-c), that does not change the perspective of the image, only the portion of the image within the viewport. This is similar to panning within an image editor.
g-h) A edge of field of view pan operation, with respect to d-e), that does change the perspective of the image.
i) No image is available at this extreme pan and pivot "edge" of DIS. A pivot from h) will not be allowed, and a pan from f) will result in the closest perspective available at that pan location.

    A panning operation that shifts the most closely corresponding viewpoint (e.g. Fig. 3, d/e to g/h respectively) will cause a viewpoint shift (an adjacent coregistered image to be displayed) during the pan operation, minimizing the visual disruption caused by the perspective change of the viewpoint shift. A pivot operation (e.g. Fig. 3, a/d/g to b/e/h) causes an immediate viewpoint shift. At extreme pivot angles, a composite of two or more images will be required to fill the viewport (e.g. Fig. 3. c), introducing a perspective mismatch across the boundary. The amount of this perspective mismatch will depend on the depth of the subject along the boundary and the viewpoint grid spacing and lens angular field of view geometry.

[[ insert Fig. 4. Stereo pair display of DIS ]]
    In the one dimensional DIS example of Fig. 2, with five discrete rotational views of every central point, four adjacent stereo pairs are available. If this example was extended to two dimensions, twenty (4x5) adjacent stereo pairs at this orientation would be available, and many more would be available at other orientations (20 more at a 90 degree rotation).
 

Attachment	Size
Pan_grid_geometry.jpg	19.89 KB
Grid_geometry_1.jpg	62.07 KB
Grid_geometry_2.jpg	233.5 KB