10/13/97

Draft

Issues and Ideas from

Z39.50 Interoperability Testbed Meeting, October 13, 1997

The following lists a set of issues, problems to be resolved, etc., identified at the Monday, October 13, 1997, meeting of the Z39.50 Interoperability Testbed Working Group. These will be discussed again on the Wednesday, October 13, 1997 meeting. This is Moen’s framing of the issues – clarifications and explication of issues and potential solutions are more than welcome.

1. Problems of Existing Database Structures and Integration of CIMI Profile Specifications for GRS

[to be revisited at the Wed., October 13, 1997 meeting]

A very important issue relates to the use of a CIMI compliant server with existing databases of information. The testbed demonstrated that not all databases are currently structured in a way that allows for the creation of the hierarchically structured GRS record defined in the Implementors Agreement. Another aspect of this issue relates to mapping between CIMI Use Attributes and CIMI Schema Elements and local database elements and structure. Based on the discussion on Monday, October 13, 1997, it seems we need to address these two separate problems.

For the issue of mapping, an approach of providing detailed guidance to implementors seems appropriate. This can be enabled by the crosswalk between CIMI Use Attributes and Elements being done by Kody Janney. Each attribute value and each schema element will be associated with one or more elements in other existing data standards such as Spectrum, etc. Clear semantics for each of the attribute values and elements are also essential. Such a mapping will provide implementors with a way to identify database elements/fields with a data standard or model they are familiar with, and that will point them to the appropriate Use Attribute value and Element to which such a database element/field can be mapped.

Associated with this work will be to ensure that there is a close mapping (one-to-one) between Use Attribute values and Schema Elements. Again, Kody’s mapping will likely show where such gaps currently exist.

The other problem related to integrating Z39.50 with existing databases is more complex. Databasix’s experience with the RKD database suggests that to provide CIMI Z39.50 access to some existing databases may require different "levels" of "conformance" to the CIMI Profile. In effect, we would be defining conformance levels that relate to different types of search and retrieval transactions to allow wider interoperability and use of the CIMI Profile with databases not sufficiently structured to easily build the complex GRS records as defined in the current Implementors Agreement.

One solution is to use the Dublin Core elements as a framework for resource discovery level search and retrieval. In this case, a "Level 1 conformance" with the CIMI Profile would require only that the server implementations support the Use Attributes defined for Dublin Core elements and support only that part of the CIMI GRS that uses tagSet-G and tagSet-M elements (those associated with Dublin Core elements). Such conformance would not include any elements beyond the first instance of a schemaIdentifier. Such a flat GRS record might be more amenable to creation with many existing databases. The tradeoff, however, would be that when multiple images and associated renditions are available, these would not be included by the server. But this level of conformance would allow a user to search across a potentially broader number of servers for the purpose of resource discovery, but the user would be severely constrained in getting more information about a particular resource through the Z39.50 implementation. The question is whether or not such a level of search and retrieval is desirable. Another question is what does a user do with the knowledge that a resource exists but is unable to get any more data about the resource (e.g., images).

To alert potential implementors of this issue, it might be necessary to state that when a database structure is not amenable to creating the hierarchically structured GRS records, the data might need to be restructured in a new database to make it "GRS ready" for their implementation.

Solution

1. Prepare explicit and detailed guidance to be included in the CIMI Profile to assist implementors in mapping local system access points/indexes to CIMI Use Attributes and in mapping local database fields to CIMI Schema Elements. This can be done through a fully articulated crosswalk between the CIMI Use Attributes and Schema Elements and existing data standards and data models.

2. Define several "levels of conformance" for the CIMI Profile to address the problems of integrating Z39.50 applications to existing database structures. "Levels of conformance" could be related to classes of search and retrieval transactions such as "cross-domain resource discovery" (coarse grained search and retrieval at the Dublin Core level), and "navigation" (e.g., using more fully the Collections Profile), and "enhanced search and retrieval" (finer-grained search and retrieval as currently provided for in the Implementors Agreement.

2. Insufficient Museum Access Points Represented

Several implementors have expressed some concern about the "bibliographic-oriented appearance" of the Implementors Agreement and the need to ensure that appropriate and sufficient access points for museum searching are represented in the Use Attributes.

The crosswalk being done by Kody should assist in assuring that the CIMI-1 Attribute Set carries the appropriate access points needed.

One suggestion for dealing with the crosswalk is not to look horizontally at the crosswalk but vertically – that is, organize the crosswalk in such a way to have at the top the attributes and elements common to all of the data standards. Need to recognize that the CIMI Profile probably can’t define sufficient attributes and elements to cover all local database elements. This may not be appropriate either. Therefore, need to draw a line between a "level of commonality" that the CIMI Profile can address and the unique local database attributes and elements that will not be included.

Another suggestion was to use "string values" for Use Attribute values for very specific indexes and fields. On the retrieval side, this can be accommodated with TagType 3, for locally defined elements.

[Discussion at Wednesday, October 13, 1997 meeting on crosswalk should inform the solution/guidance.]

3. Issue of Record Syntaxes Required -- SUTRS

One implementor suggested that SUTRS be included as an acceptable record syntax in the CIMI Profile. This would allow those systems with databases that are not amenable to use with the hierarchically structured CIMI GRS record to be interoperable.

[Need for discussion and agreement on this at the Wednesday, October 13, 1997 meeting.]

4. AMICO Project Recommendation on Image Sizes

One implementor suggested that the AMICO recommendation on image sizes be used in the CIMI Profile to identify standard sizes for images.

5. Harmonization of CIMI Profile and the Aquarelle Profile

These two profiles have diverged, especially related to the GRS record. The Aquarelle Profile uses the earlier relatively flat GRS record from the May draft of the Implementors Agreement . It’s important to consider how to harmonize these two profiles for longer-term interoperability. Some of the testbed implementors who are involved with the Aquarelle work will propose that the Aquarelle Profile move towards the CIMI Profile. But it may be too early for that decision to be made by the Aquarelle people.

Moen suggested that at the very least, the testbed implementors who are working with Aquarelle should flag anything in the CIMI Profile that is a point where divergence will create interoperability problems and point to some possible solutions for dealing with that. It may be a matter of timing. But it may also be a case of the installed base of Aquarelle implementations that may make the decision to move more towards the CIMI Profile more difficult.

Also, based on the Aquarelle implementation experience, if there are things that should be brought into the CIMI Profile to enhance harmonization, these should be identified and suggestions for inclusion in the revised Profile should be made.

6. Problems of Port Numbers Being Use in Implementations

At least one of the implementors suggested that the CIMI implementors agree to use a limited range of TCP/IP port numbers for their implementations. When an implementor is behind a firewall, using a limited range of port numbers eases the problems with system security personnel.

Solution

1. Agreement among the testbed participants to assign port numbers between 2100 –2110 for their Z39.50 servers. This becomes effective on November 1, 1997.

7. Size of Brief Records

A number of the implementors suggested that the b (brief) element set name does not serve the purpose of presenting the user with a "brief" record. Approximately 50 elements are currently defined in the b element set name (with some of those repeatable). One of the issues relates to "time to download" the result set of brief records when there are numerous records in the result set. Another issue relates to "bandwidth" requirements when there are one or more images (thumbnails) returned with the brief record. Should a smaller number of elements be defined for b?

A solution should be driven by requirements for what clients will need to process and prepare different views of the records. One clearly stated requirements was that when there is one or more images associated with the record, a thumbnail must be included. In some ways the brief record is really a client-side issue. The CIMI Profile needs to identify a minimum set of elements (possibly what is currently defined) such that clients can prepare one or more views of the record from the elements in the b element set. Arriving at that minimum set of elements requires agreement among the community as to necessary elements to process the different views.

Solution Paths

1. Clients should provide an option to "turn off" loading of the images in the brief record.

2. A minimum set of elements needs to be defined to address various implementation requirements to provide different views of the record built from a "base" CIMI record. Such views might include a "hit list" view (a couple of lines representing the record, such as title, author, etc.), "tombstone" view, and others. This is a client issue, however, and should be determined by the customer requirements in conjunction with a vendor.

8. Client and Server Behavior When No Thumbnail Exists

There is some concern about the current specifications for handling images. Some implementations will not have available a thumbnail of the image. The CIMI Java client simply gets the "first" listed image [(4,4)(4,14)(4,29)(5,28)(5,29)(5,30) …actualDO/mrObject/rendition/resource]. The client does "look" to see the size of the image, but simply goes and fetches it. Other clients may be developed that acts differently. The question is what sort of guidance should be in the CIMI Profile to address this issue. The Implementors Agreement currently defines appliedVariant information for the element resource – in terms both of a list of string values as well as in byte size.

Guidance

9. Expressing & Standardization of the Mappings

[Issue raised by Michael Selway – I need to get a clearer sense of this issue. Raise at Wednesday October 13, 1997 meeting.]

10. Schema Identifier in Presentation of Brief Record

Since the element schemaIdentifier is defined in the b record, the current client (Blue Angel’s Java client) displays this with the other elements. Most users are likely not to know what this refers to and probably should be a non-displayed element in the presentation of the record.

There may be other elements similar to this that are intended more for processing by the client to know something about the record rather than providing useful information to the user. Another such element might be typeOfDescriptiveRecord.

Guidance

11. Ordering and Grouping of Records in a Result Set

Based on the demonstration of multiple server/database searching that is possible with the CIMI Java client, there is a question of sorting and grouping result sets. The Java client is multi-threaded so that when records are being retrieved from multiple databases, they are simply interfiled as the client receives them. This is a client-side issue. Different clients may provide different functionality for sorting and grouping of records. One implication, however, is that the client will have to wait until all records are retrieved before performing any sorting and grouping functions. This may affect display time.

Guidance

12. Need for Use Attribute MuseumObjectID

Currently there is an element museumObjectID in the schema but there is not Use attribute defined to allow searching on such a database element.

Solution

1. Define a new Use Attribute for MuseumObjectID

13. Combination of Attributes and Mandatory Attributes

Appendix C identifies a set of priority attribute values for support by clients and servers as well as identifying legal combinations of attributes and values. Several implementors suggested that this needs to be revisited.

[to be discussed in more detail on Wed, 10/15/97]