Pittsburgh Post-Gazette

Specifications for a digital text/image archive system

submitted by Tim Rozgonyi, Assistant Technology Systems Editor,

Pittsburgh Post-Gazette

compiled by the Post-Gazette's image archive group 1997

The following list of specifications was compiled by a group of people working on the purchase of an image archive system for the Pittsburgh Post-Gazette. Efforts have been made to eliminate references to specific vendors, though there are still plenty of references to things that are Post-Gazette-specific (such as our Atex system). Nevertheless, this document should hopefully serve as a good starting point for any publisher looking to acquire a robust archiving system capable of handling text, images, graphics, full pages and other desired media. The following does not necessarily represent the final version of the list as submitted to the chosen vendor, nor is it intended to be a comprehensive blueprint for a publisher looking to purchase a system. If it helps you along in your process, that's great. If you have any questions, comments or suggested additions to this list, or if you would just like to talk archiving, please contact Tim Rozgonyi (412-263-1923 or rozgonyi@post-gazette.com) at the Post-Gazette.

Retrieval

• The ability to enter a search from a command line prompt or through a menu-driven interface. We want to be able to enter a search such as "turtles and byline (jones) and section (travel)" without having to go through a three-step process of adding each element of that search to the previous one. We do not expect all users would avail themselves of the command line search functionality, with many users preferring to use a step-by-step approach with pull-down lists to create a search statement.
• The ability to search using the Boolean operators AND, OR and NOT, alone or in combination (e.g., dogs AND cats OR mice; good BUTNOT evil; pirates and section (business) and keywords (chronology)).
• The ability to run proximity-based searches to find two words within so many words of one another (e.g., buttons /7 bows; gone /3 wind; wheel /p fortune).
• The ability to run proximity-based searches in a specified direction or in either direction. For example, the search statement "tiger pre/2 woods" would deliver documents in which the word "tiger" was within two words of the word "woods" but only where the word "tiger" occurred first (such as "Tiger Woods is incredible."). The search statement "tiger /2 woods" would deliver documents in which the word "tiger" was within two words of the word "woods," irrespective of their order (such as "He went into the woods. A tiger ate him.").
• The ability to run natural language queries (e.g., "what are the gravitational effects encountered at the event horizon of a black hole?"). The search engine would process this query in such a way as to give the requester those documents that most closely match the key concepts in the search statement.
• The ability to automatically search the plural of a term when the singular is entered and to search the singular when the plural is entered. This should not automatically include all variants of the root word, as this could widen the search parameters excessively and result in irrelevant hits being returned. For example, one may wish to search on the term "drugs" without getting hits on the words "drug" or "druggist."
• The ability to truncate a search term to find variant word endings (e.g., "enviro#" to find environment, environmental, etc.). For the reasons stated above, this truncation feature should be user-initiated and not a system default. One possibility is to have a checkbox on the query screen labeled "use stemming feature."
• The ability to report the number of hits retrieved by a search. This information should be available to the user as records are displayed (e.g., item 3 of 252, screen 4 of 31).
• The ability to immediately display search results upon execution of query
• A defined search box for entering queries. This is meant to give a clear indication to the user as to where they need to type their query.
• The ability to execute the query either by hitting the "Enter" key or by hitting an "execute query" button located right next to the "enter query" window.
• User-definable ability to limit the number of results returned from a query. This could be accomplished by having a pull-down window within which the user could input a maximum number of records displayed (e.g., 25, 50, 100). On the web interface, this could be a checkoff box near the query window.
• The ability to use "wild cards" in a search statement (e.g., "wild wom*n").
• The ability to transfer search results from one user to another, even if each is on a different platform. This could be used to allow a user located at a distance from another user to make available a group of hand-selected images (e.g., "here are the photos I would like to use with the election story tomorrow") or to make a set of stories available to a group of reporters or editors ("here is some background research for next week's Sunday cover story").
• The ability to modify a search statement without having to rekey the entire statement.
• The ability to interrupt a search or other function before it fully executes. (This would be the "oops" function, such as in "Oops, I didn't really mean to look for all occurrences of the word "the" like that.")
• The ability to clear a query and begin a new search by clicking on a single button.
• The ability to search all fields for search terms. For example, a search for the term "roadrunner" should deliver all documents with that term anywhere, as opposed to specifically having to direct the system to look in the keywords field for the term.
• The ability to access pull-down lists for specified fields. For example, if a user wishes to do a search for items within a certain section, we want a list of authorized section names readily available. Likewise for fields such as keywords, bylines, edition, graphic element, etc.). We would want to do this only with fields that have a defined set of authorized values associated.
• The ability to search on phrases (e.g., "to the hilt"; "national endowment for the arts"). The system should deliver only hits that match the phrase precisely, as opposed to trying to be helpful and delivering hits such as "national arts endowments were up 50% this past year."
• The ability to search in user-designated fields for search terms. For example, the search statement "newt gingrich and keywords (profile)" would deliver records that contained a reference to Newt Gingrich anywhere within the file and the word "profile " in the keywords field.
• The ability to combine full-text searching (i.e., searching on all fields) with fielded searching (e.g., "pirates and section (business) and date (1-1-96 to 6-2-98)").
• The ability to combine searches on different fields or within the same field (e.g., "keyword (drugs) and keyword (murder)"; "pittsburgh marathon and section (sports) and byline (collier)").
• The ability to designate which operation is performed first in a search. For example, in processing the search statement "(dogs or cats) and (mice or rats) and section (travel))," the system would process the "ors" first and then "and" the results together.
• The ability to display documents by relevancy of search terms, chronological order or reverse chronological order, per user preference. There will be times when a user wants to see the most recent documents first; the most relevant documents first, or the oldest documents first. If the system returns documents based on relevancy when the user needs to see the most current items, that is a problem.
• The ability to sort search results by specific fields (date, size, type, file type, etc.). This is related to but not the same as the above. For example, a user may have conducted a search on the system and then wish to sort the results in such a way as to see the Freehand documents that match the query.
• Automatic highlighting of search terms within the document, with the ability to go directly to that portion of the record that contains the search terms. For example, a user may enter a person's name as a search term and then find our annual murder roundup (a very long file) among the results. Rather than having to page down through the entire document to find the occurrence, there should be a away for the user to go straight to the page or screen on which the search terms appear. (This is often called a "keyword in context" display.)
• Automatic and/or customized routing of retrieved files to preassigned destination folders/directories. For example, the default could be to drop retrieved items into a folder on the user's desktop or another folder on their workstation. Other possible options (possibly available via a pull-down menu) would be to route items straight to a folder for a photo editor or technician to use or an output folder for transmission to another destination.
• Ability to retrieve an 18-MB high-res file from the archival medium and make it available for use within 30 seconds assuming there is no network traffic and we have a 10 BaseT client router on a 100 BaseT network.
• Ability to retrieve 15MB high-res file off of near-line storage (e.g., a RAID array) and make it available for use within 10 seconds.
• Ability to view image files within the database (whether on a RAID array or in a CD-ROM jukebox) using viewing software that is integral to the database or other products such as SCC PhotoGrid Pro. One concern here is that we have the ability to view photos after they have come in from the wire services, something we now do via the AP LeafDesk (which we are committed to replacing through this install). We want to use the archive viewer as an output interface to the pagination/production system. Since this viewing capability should be a function of the archiving application itself, it is likely that something like PhotoGrid would be used rarely. Nevertheless, we should be able to view a volume of files on the RAID array or a CD-ROM without having to be operating from within the archiving application.
• Ability to easily correct typing mistakes. This would apply primarily to librarians who add textual information to files to make them more accessible.
• User-definable ability to change display preferences (within preset limitations). For example, a user may wish to have text files display with the date as the first element and the headline as the second, whereas someone else may wish to see the date, page, section, edition, publication year and then the headline. This is the kind of thing a librarian is most likely to do, so this functionality would need to be integrated into the client they would most likely use (i.e., if the librarians are required to use the Mac or PC clients, this functionality would need to exist within those clients). Additionally, other users may wish to change the way images appear on their screen (i.e., the size of the thumbnail, the amount of textual information that displays with the thumbnail, the number of thumbnails that appear on the screen, etc.). The ability to change the amount of text displayed with a thumbnail should be available right from the display screen without the user having to change preferences. NOTE: This kind of functionality would be required for both editing and retrieval.
• Ability to tag and export files to an outside destination, such as a folder on the desktop, a folder on another device on the network, an attached or network printer or e-mail address. (A detailed description of the functionality needed for exporting images will be found later in this document.) There are numerous applications of this type of functionality, including:
  • A. The ability to export files to a web site from within the archive. This would include sending image files as cropped and watermarked JPEG (or other industry-standard format), graphic files as watermarked GIF or JPEG (or other industry-standard format), and text files as HTML-coded documents (or other industry-standard format).
  • B. The ability to export text files back into the Atex system. We want the user to be able to choose an "export to Atex" option.
  • C. The ability to make printouts of several previews (with text) or a page of thumbnails from within the archive. We want to be able to tag desired images and print directly to a network device via the web client.
  • D. The ability to send text, graphics and photo files to another publication for possible republication. We want to be able to accomplish this via FTP or email without leaving the archive (running the web client). This application should alert the user when a transmission exceeds a designated file size (determined by system administrator and based on the typical maximum for e-mail users incoming mail).
  • E. The ability to export a story or group of stories from the archive into a user's personal directory on the editorial/pagination system.
  • F. The ability to export PDF files.

• Ability to export files using user-definable compression ratios and file formats (e.g., JPEG, TIFF, GIF, EPS, etc.). As mentioned above, this would be an "export as" function that would allow the user to choose from a predetermined list of available formats. Site-definable list of stopwords. We would more than likely to opt for having little or no stop words, as we wish to be bale to search for phrases such as "to the hilt," "on the road" or "in brief" or names such as "george will" or "the who."
• Ability to retrieve all records updated in a certain period (e.g., last hour, last five minutes, since 10:00, etc.). The primary application of this functionality is to allow a user to call up the latest wire photos that have entered the system.
• Ability to display records in multiple ways, including: thumbnail display using icons for text images; headline display using headlines for text records and text from designated fields of graphic files; detailed display showing preview and IPTC info for images and full-text display for text items.
• Ability to permanently save search syntax on a user's machine to be called upon at the user's discretion. Some users may have searches that they run all the time, and this would allow them to keep going back to it without having to rekey every time.
• Ability to display a search history at user's discretion and to use that history to run searches. This would more than likely be used within a given search session to allow a user to go back to a previous query and rerun it. In the interest of decluttering the user interface, it might be best to have the search history as a displayable element, perhaps by clicking on a "show search history" button. Most users would use this functionality little if ever. Librarians, photo editors and more experienced searchers would be much more likely to take advantage of this.
• A separate interface for skilled versus novice users, with the novice users having a simplified search screen with lesser functionality as requested by publisher. The main intent would be to have a stripped-down client that might be suitable for reporters or outside users. Clearly, if we chose to put up our archives on a web server, we would want users to be able to find and display files but do little else.
• Ability to choose whether to search text only, images only, graphics only or any combination thereof.
• Ability to search the database by years. For example, our current text library allows a user to search for files in a specific year (such as the 1997 database), all years, or selected years. Our experience is that most users are looking for the most current information, while others want to find things no matter what year it is from.
• Ability to easily switch from editing to retrieval functions, and vice versa.
• Ability to incorporate a user-defined thesaurus.
• Ability to add terms to the thesaurus, with a function to prevent duplicate entries. In libraryese, this is referred to as maintaining authority control. We would probably want to have a "blessed" list and not allow anyone to add to it willy nilly.
• Ability to incorporate an acronym translation function. This would allow a user to enter the term "NASA" and have the system automatically search for "NASA" as well as "National Aeronautics and Space Administration," and vice versa.
• Remote access capability (for searching and editing). This would allow a user (with the right privileges) to dial in from a remote location and access the system.
• Option of doing a case-sensitive search. For example, a user may want to enter the search statement "AIDS and libraries" without getting hits on "library aids."
• Ability to weight search terms for increased search efficiency. For example, a user may wish to enter the search statement "what are the effects of the legalization of marijuana in the state of california?" and put greater weight on the term "marijuana" than on the other terms.
• Different display modes for different classes of users. For example, the displays a librarian sees may differ from the displays a photo editor or reporter sees.
• User-definable thumbnail displays (e.g., ability to change the number of thumbnails in the display and/or the amount of information displayed with each thumbnail).
• Different types of thumbnails. We want a 100K thumbnail with multiple components: 98x98 pixels, 128x128 pixels, 384x384 pixels, 768x768 pixels, in a 4.5 MB preview package.
• Ability to magnify image from thumbnail to mid-size preview without accessing the high-res file.
• Ability to magnify image from thumbnail to mid-size preview and then into the high-res seamlessly while high-res resides on live database before it is committed to the archival medium.
• The ability to tag thumbnails for grouping in projects, including images that have been: 1. edited, 2. captioned, 3. pre-pressed, 4. exported, 5. archived, etc., with a different color code for each tag.
• Ability to alter the status of an item by dragging its thumbnail from one window into another, with the windows designating such things as whether the files have been: 1. edited, 2. captioned, 3. pre-pressed, 4. exported, 5. archived, etc.
• Ability to display color thumbnails.
• Ability to hold files tagged for export in a spooler outside the user's editing terminal.
• Ability to search the full text of a PDF file and display a preview of it or the full PDF file.
• Ability to have multiple sessions open at one time. There are often situations when it is helpful to launch an application twice and switch back and forth between the two sessions. For example, right now I am working on this revised list of specs in Word, and I also have another session of Word launched in which I have open a list of specs crafted by some colleagues. Another example: when our library staff accepts calls from the public, it is often helpful to have a session open for searching the database and another session open for editing/enhancing files. This example indicates that this is a type of functionality that librarians would be likely to use.
• Ability to search records in near-line and archival storage with a single query.

Record acquisition

• Ability to translate field designations in use by front-end editorial system into field designations in use by the archive. We intend to feed text from our current text archive into the archive for the immediate future. We do not intend or expect that the archive vendor will need to handle Atex copy at all (though it is possible that we may reach a point where the archive will need to accept data from the Atex and the editorial/pagination system concurrently). We do expect the archive vendor to work with our editorial/pagination vendor to devise a reliable method whereby files from the editorial/pagination system are transferred to the archive system automatically on a regular basis. The archive vendor will need to map the data from the editorial/pagination system files into the appropriate fields in the archive (e.g., headlines go into the headline field, cutlines into the cutline field, etc.) while maintaining coherence between the various elements of a story. We would expect you to make this happen as long as the editorial/pagination system vendor can deliver the appropriate information and files to the archive.
• Ability to accept material from the front-end system into the archive edit system, whether at a predesignated time or continuously throughout the day. The intent is to have all typeset versions of the text files available for processing and entry into the database. Photos will also be entering the system continuously and should be available for editing virtually immediately.
• Creation of a new translation process should we change the front-end software we are using within two years of the purchase date of the system. This will be done at no additional charge.
• Automatic indexing of all IPTC header fields as images are acquired by the system. This may involve remapping data from one field to another (such as moving items from the headline field in the LeafDesk to the keywords field.
• Ability to move incoming wire feeds (including photos and graphics coming from AP, Reuters and GraphicsNet,) directly into designated folders/volumes as maximum-quality JPEGs. This will likely require a conversion process on some files.
• Ability to extract data from QuarkXPress pages and send it into the editing system with the appropriate field designations (headline, cutline, start of text, etc.) maintained. The issue here is that we have to get the content off of the paginated page and into the archive system. Our current text database has a utility with which we can translate elements off of a Quark page into text files that can be entered into the database, with database-appropriate coding already in place. This is not the solution we are advocating, but is rather the level of functionality we currently have. Another possible solution to the Quark problem would involve creating PDF files from the whole pages and exporting the individual elements (stories, photos, etc.) into the archive as unified pieces (i.e., the stories maintain their coherence and integrity throughout the process so that you don't end up with a database full of individual headlines, cutlines, stories and jump pages floating around aimlessly). We would not expect or want this to be a process that involved no human intervention. We would want the librarians to "bless" the stories before they go into the database and add any information they deem relevant (such as keywords).
• Automatic maximum-quality compression of uncompressed TIFF files as they enter the archive system.
• Automatic conversion of CMYK files to RGB files as they enter the archive system.
• Ability to hold multiple versions of the same file, corresponding to different typeset editions of the same story. Alluded to above, we need to be able to call up all "typeset" versions of a particular story. For example, on an election night, we might have six different versions of RESULT21 (the election story) move through the production system (three of which might get printed). The librarian will need to be able to call up every version of RESULT21 so that we can get the appropriate versions into the database. Ideally, we only want to see those versions of the story that actually got published. We would expect you to make this happen as long as the editorial/pagination system vendor can deliver the appropriate information and files to the archive.
• Ability to successfully import files from existing Fetch/AP Macarchive image database and existing graphics files, including AP LeafDesk, ANPA Macarchive, Reuters and AP GraphicsNet files. Test files will be supplied.
• Ability to successfully import text files from existing text archive database and map their content to fields within the archive database records. Additionally, the archive vendor must maintain a field for the text archive tag numbers. To keep our synchronization with our existing vendors (for corrections and deletions) we must keep the old numbers and, if possible, use a similar format for tagging new archive text records. This will involve discussions between the archive vendor and the various information vendors we deal with (such as Lexis-Nexis, UMI/DataTimes, Knight-Ridder Information).
• Ability to accept IPTC header info from SCC PhotoGrid Pro, SCC MediaGrid, AP LeafDesk, Reuters and AP GraphicsNet files and map that information to fields within the archive database records.
• Ability to execute a user's search automatically at preset intervals and notify the user when new data matching the search parameters has entered the system.
• Automatic attachment of an "acquire date" (including time of day) as files enter the database. Similarly, we would want to be able to attach a publication date to files as they are entered into the system (e.g., if we had a folder full of photos from that day's paper that we wished to enter into the database).
• An art assignment system that would consist of folders for individual assignments and include the ability to track these folders/assignments. Each folder would hold all the assignment and caption information available before the photographer/artist begins to add specifics on each record. Such information would be automatically added to the record as each file was scanned and/or funneled into the archive. This would allow us to attach all textual information in that folder to each file saved to that folder without having to save that information and attach it onto individual files manually. This system would include a textual display allowing the user to have access to and view status of existing folders/assignments in the system. This allows for tracking the status of works in progress from assignment generation to commitment to archival medium. Level of access to this directory would be restricted based on user class. Folders/assignments could be deleted automatically so many days after date of creation and also manually by authorized users. This assignment system would have a set up function that would incorporate dialog boxes and include the following fields and functions, with pull-down menus where indicated:

Photographer/Artist (with pull-down menu)

Dept./Section (with pull-down menu)

Editor (with pull-down menu)

Reporter (with pull-down menu)

Reporter at Shoot (with pull-down menu)

Requested by (with pull-down menu)

Request Date/Time (filled in automatically on export to request folder)

Color

B/W

Single picture or layout checkoff boxes (with a way to indicate the number of photos expected for the layout)

Mug shot check off box

Contact at site

Deadline Day/Time/Date

Authorizing Editor (with pull-down menu)

Story Slug

Working headline

Chatter (introductory text for graphics)

Publication Date

Location

Directions

Background Information

Approved by assignment editor

Category (e.g., football - high school)

Caption

Shoot date and time

Permissions (with pull-down menu)

Notes

Status (with pull-down menu allowing user to indicate whether the job is or has been unassigned, assigned, acknowledged, scanned, edited, captioned, prepressed, approved, archived, published or exported)

In use by

Ability to copy set-up info from one assignment folder into another.

Ability to cancel or save changes.

Ability to clear screen.

Ability to create a new assignment folder.

Ability to view a directory of existing folders/assignments, the display of which would include such elements as the assignment name, assignment number, number of files in folder, assign date, shoot date, and status.

Ability to call up assignments from that directory.

Ability to delete an assignment.

Ability to protect an assignment so that it does not automatically purge.

Ability to display and effect changes in the security status of an assignment (restricted function).

Ability to link two or more related assignments together and to display a listing or directory of the linked assignments.

Editing

• Dynamic keyword list with incremental matching of keywords and restricted ability to add keywords to the list. We would more than likely restrict the ability to add keywords to admin level users. We may want to incorporate a preexisting list of keywords
• Publishing history field capable of holding multiple publication dates, each of which would be searchable.
• Ability to add fields at our request. We anticipate that, with a few weeks or months of usage under our belts, we will finally begin to understand what we don't know, and will then be in a position to ask for things such as a field here or there. We would expect that such requests are considered routine and are not billed on a case-by-case basis.
• Ability to easily identify classes of files existing in the server. This would be used primarily by the librarians, who will need to call files up based on their status (archived, nonarchived, edited, ready for archive, etc.). We would plan to have all files go into the database and be held for a predetermined amount of time, after which they would be deleted unless they had been designated for archiving. Nevertheless, we would need to pull up nonarchived images as we go about archiving the photos from that day's paper or look to see what on the wires is worth adding to the archive.
• Spell-check function for data input by users. Again, this is a feature that would primarily be used by librarians as they edit/enhance records for the database. Should photographers ever do their captioning within the database application, this feature would be most useful.
• Ability to do side-by-side comparison of text files. This comes into play when trying to compare two versions of a story that seem identical but may have slight differences. The ability to have both of them up on the screen at the same time greatly facilitates the ability to differentiate between the two.
• Automatic tagging of files retrieved from archive to prevent needless rearchiving. An example of this comes from the Fetch/Macarchive product, in which files taken out of the archive and sent back to the LeafDesk are automatically tagged with the term "Macarchive" in the headline field, making it easy to find them and to tell from whence they came.
• A "rearchive" box that would be checked off to indicate that an image file from the archive is to be rearchived. A corollary to the above. There will be times when we pull an image out of the archive and choose to rearchive it for whatever reasons. This would result in two images being in the database.
• Integration of a suite of applications for production (detailed below), including the ability to select, color correct, set white point, set black point, tone, crop, rotate, tag and route without opening the high-res file. All such edits would be applied to the high-res upon call-out/export.
• Ability to batch edit images, either in folders or after tagging desired images while editing. This functionality should not be limit to one screen at a time (i.e., if there are 100 Super Bowl images, one should be able to apply an edit to all 100 at once. The types of functions to be performed on batches include:

ability to add textual information such as keywords

ability to set white point/ black point

ability to apply Look Up Tables (LUTs) for output

ability to do global tonal and color correction

ability to tag files with color coded mark

• Ability to rotate images as needed for display (both thumbnail and high-res).
• A "get info" button which would allow the user to display the image size for an individual image.
• Ability to cut-and-paste data from other sources into caption fields, headline field, etc.
• Pull-down lists (incorporating incremental matching) for designated fields, such as photographer, location, keywords, etc. Again, this would be intended for specific fields which had a list of valid designations attached to them. For example, we would more than likely choose to have a defined list of local locations rather than a list consisting of every location mentioned in the database. We would, of course, require the ability to edit these lists.
• Ability to dynamically update a retrieved set of images when new images are added to the system. This function would most likely be used by photo editors needing to be aware of the latest photos of the bombing in ________. In practice, a photo editor would often want to have his screen refresh as each new wire image enters the system.
• Text-editing functionality used during the preparation of stories for archiving. This would include but not be limited to the following features and functionality:

A. Ability to tag various files out of a directory listing and perform such functions on them as change status, delete, edit, etc.

B. Ability to abort or cancel changes that have been made to a text record and leave the record as if it had not been touched.

C. Ability to use "hot keys" to attach basic, frequently used information to files.

D. Ability to build a directory and then choose to edit the current file, mark and edit selected files or select and edit all files.

E. Ability to use customized pop-ups to call certain textual information into a file being edited. These pop-ups would be keyed to specific editing fields.

• Ability to correct and/or delete records as needed, with accompanying recovery of hard-disk system storage space. For purposes of updating the database and transmitting files to outside sources, the system will need to tag each file as a new record, a replacement record, or a deletion. This is currently done for our text files through the use of a tag that resides in the header of the record, as well as a unique identifier that the archive system creates for each record. Vendors can key off the header to automatically handle corrections and deletions this way. (For example, a file that comes through with a unique identifier that is identical to a file already in the database is either a correction or a deletion.) This will involve discussions between the archive vendor and the various information vendors we deal with (such as Lexis-Nexis, UMI/DataTimes, Knight-Ridder Information).
• Ability to display attached corrections prominently within the file. If a correction to a story is published, it is very important to draw the user's attention to it. We currently accomplish this by appending the text to the appropriate story/stories.
• Ability to associate a bar code with a database record. Such barcode would be keyed to the unique identifier of the database file and could be used to call up that file when scanned by a barcode reader. The archive system would need to be able to print the barcode(s) on the proof of an image or a page of thumbnails from the database. This would provide a very fast way to call up the record from the database. We would likely use this functionality for photos and graphics only.
• Ability to create a link between files in the database (e.g., to link the text of a story with its accompanying stories, photos and/or graphics and call up those items by clicking on a "show related items" button). For files coming off of the Atex system, this would most likely be accomplished by a librarian manually creating the link between the various individual elements. For files coming off of our new editorial/pagination system, we would want to archive related elements (text, photos, graphics, etc.) and still maintain the link between items. We would expect you to do this as long as the editorial/pagination system could deliver to the archive the information necessary to archive the files in a linked fashion automatically. Clearly this will involve coordination with the editorial/pagination system vendor.
• Ability to edit files prior to and subsequent to their commitment to archival storage. The main point here is that the attached information not reside only with the high-res image so that edits can be done on textual info, thumbnails, etc., without having to pull down the high-res file.
• Ability to move files into a "work-in-progress" area before committing them to the archival storage medium. This entails the idea of a "live" database for wire and local images as well as a way to move text files into a "completed" area or folder before they are committed to the archive. This is closely related to the need to do quality control work, checking enhanced or edited files before they are allowed entry to the permanent archive.
• The ability to set up macros or function keys that will perform specific functions. This should include system defaults as well as user-definable keys.
• The ability to access system functions via pull-down menus and/or clickable icons. All icons used in the system should have "tool tips" associated with them so the user can tell what they do.
• Ability to proof images from the archive via a pull-down or clickable icon. We would also want the user-definable ability to automatically proof images upon call out from the database. This would likely be restricted to certain user categories.
• Ability to compare two preview images side by side while simultaneously displaying results of a search in thumbnails. This would include the functionality to switch the "favored" image instantly with the frame being compared or with an image from the thumbnails being displayed.
• Site-definable ability to designate certain fields as "must fill in" (e.g., photographer's name, caption field, section, edition, etc.) before committing to archive. This would apply mainly to librarians as they edit/enhance files. Again, if photographers were to do their captioning from within the archive application, this could be a very useful feature.
• Ability to block the export of files that are tagged in a "rights" field to indicate that we do not have the right to republish/resell the image. This could be done in a negative way (by blocking all files that do not have "Pittsburgh Post-Gazette" in the ownership field) or by the use of a "red flag" button that would trigger a warning to the user and restrict the user from exporting the file. We would require the ability to override this feature as needed. This feature might actually prove most useful when doing the transmission of files to commercial vendors et al. We could set it up so that all files that did not have certain values in the ownership field would error out of the transmission process and show up in a report or else simply not go out.
• Ability to edit individual elements within fields without having to rekey the entire field.
• Ability to make global changes on database files (e.g., to change all occurrences of USAir in the keywords field to USAirways).
• Option to automatically prepress image files for B&W and color by exporting the files to hot folders running image editing software.
• Option to perform file conversions on files downloaded from such sources as Presslink, Wieck Photo, Scripps Howard, etc. These files would need to be converted into JPEGs before being automatically routed into the RAID array.
• A "date last modified" field.
• An image editing dialog box with "before" and "after" windows for an image being prepressed (such windows being viewable side by side concurrently), including the following features and functionality:
  • A. White and black points adjustment (auto and manual).
  • B. Ability to apply preferences.
  • C. Eyedropper function with readout in levels numbers 0-256 for each K and CMYK%.
  • D. Neutral point option.
  • E. Ability to "show points" for black, white and neutral.
  • F. Ability to set a numerical value for a field (such as white point).
  • G. Ability to clone.
  • H. Option for sharpening (with ability to control degree of sharpening).
  • I. Option for LUTs.
  • J. Option to modify saturation (in decimals with arrows for scrolling up and down).
  • K. Option to modify contrast (in decimals with arrows for scrolling up and down).
  • L. Ability to rotate image incrementally (with readout for degrees rotated).
  • M. Ability to crop.
  • N. Option to undo crop.
  • O. Ability to show or hide crop.
  • P. Option to preview cropped image (filling the window).
  • Q. Options to reset, set to previous, set to saved, set to original, OK and cancel.
  • R. Ability to preview the image with adjustments in effect (either individually or as a whole) for white, black and neutral, auto black, auto white and white clip.
  • (Note: With the exception of item "L", the above represents the industry-standard functionality available in the AP LeafDesk for at least the past five years.)

• An export images dialog box, including the following features and functionality:

A. Eyedropper function with readout numbers for R, G, B and K and CMYK.

B. Export options including: B&W, 24-bit color, 48-bit color, apply crop:

1. BW

2. 24-bit color

3. 48-bit color

4. apply crop

5. apply color adjustments

6. apply export LUT

7. apply default sharpening

8. apply custom sharpening

9. FPO mode

10. file type: (with pull down menu)

C. Display of image as adjusted via export options

D. Resolution options (such as pixels, inches, picas, points, etc.)

1. with option for predefined and numeric display of dpi

2. with option for "other dpi" with display for pixels per inch and pull downs for other measures (such as pixels, inches, picas, points, etc.)

E. Display of original image file size

F. Options for and display of predefined or custom file size for export

G. System-defined export file-size options with numeric display window adjacent to labels such as smaller, small, large, larger, custom, batch, etc.

• Ability to mark a thumbnail or preview image file "FPO" (For Position Only) over image area. Alternatively, an FPO tag can be applied which would be recognized by the editorial/pagination system in a way that would prevent a user from printing the page with the FPO element on it.

System architecture

• Perform system backups as a background process.
• Perform database updates and archiving to CD as background tasks. The archive system should automatically check the CD against the live database before the CD is approved for use and, only then, delete the affiliated files from the live system. The system should be able to produce a CD for the jukebox and another identical CD for off-site storage/backup.
• The system must be SQL-compatible and ODBC-compliant. We want to be able to issue a pure SQL query to the database and get results without having to go through the archive system.
• Ethernet capable.
• Support for Windows NT 4.0.
• Uninterruptible power supply compatible.
• Use of industry-standard storage devices and interfaces (RAID arrays, SCSI drives, CD-ROMS, DVD (if/when it arrives), CD-ROM burners, etc.). ISO 9660 compatibility is required.
• Ability to mix and match storage media (e.g., CD-ROM and RAID array or DVD).
• Multi-level password-protected system security (minimum of five user levels).
• PC and Mac clients with equal functionality for editing and retrieval. The Mac client should be a true client with the workload being shared between the Mac and the server, as opposed to running emulation software.
• A web client (compatible with current industry standards) with the ability to search and call items out of the database.
• A web client (compatible with current industry standards) equal in functionality with the PC and Mac clients. This gets back to the idea of different interfaces for different user categories. One web interface would support reporters and other nonexpert users. Another would support librarians and picture editors/technicians.
• Web-client software (compatible with current industry standards) for general public use that would offer highly limited access to all or a subset of all archived files. This would most likely be used in conjunction with a web site.
• Strategy for rebuilding the database in the event of damage to the archive.
• Scheme for replication of the database.
• Ability to watermark documents for uses such as creating a library of digital images on the World Wide Web.
• Minimum 4X integrated CD-ROM writer with option to automatically generate new CDs whenever the required amount of data is available.
• Ability to store graphics in native application file formats or to store them in user-designated formats (e.g., converting a QuarkXPress file to a PDF) and generate a cross-platform displayable preview file (either a JPEG or PDF preview). The archive vendor will archive and generate thumbnails for vector graphic files in the Encapsulated PostScript file (EPS) file format. It is our understanding that MacroMedia Freehand and Adobe Illustrator are both valid formats for archiving EPS files. Further, it is our understanding that, to see the same thumbnail/preview of an EPS in Quark for Windows, the EPS must be saved in a Windows application before being funneled into the archive (e.g., if you funnel a Mac EPS, only Mac Quark can display it).
• Ability to autoconvert native graphic files to a JPEG, TIFF or GIF upon export. This may fit into the item above. It would seem to be most useful for users on nongraphics workstations, including those using the web client.
• Ability to support up to 64 concurrent users.
• Compatibility with Adobe PhotoShop 4.0.
• Enough RAID 5 storage to hold 30 days of image files online. We estimate the need for a 40 GIG unit based on an anticipated influx of 500 wire images/day @ 1MB/image and 200 locally scanned images/day @ 1.5MB/image (plus 20% RAID overhead).
• Ability to store thumbnails, previews, IPTC header info in near-line storage and separately from the high-res images and graphic files. The intent here is that the attached info and thumbnails, previews should not be solely attached to the high-res file.

Miscellaneous

• Telephone support available 24 by 7 via an 800 number. Such support will include corrections of software errors, dial-in diagnostics and field service when necessary. Guaranteed response time of 3 hours from time of call.
• Ability to set a maximum number of displayable records for each user category. Basically, we don't want reporters sifting through 1,000 records to find the data they want, whereas the system administrator may have need to run queries that result in display sets numbering in the tens of thousands.
• Ability to retrieve all records updated in a certain period (e.g. today, yesterday, since last update, etc.) and transmit them to an outside destination via phone lines, FTP or some other means. The main intent for this functionality is to allow us to send files to outside sources we deal with, such as the commercial information vendors Dialog and Lexis-Nexis. There are several aspects of this feature, such as:

A. The system's ability to automatically keep track of the files that have been updated so an automatic transmission of the current update can be made.

B. The ability to automatically initiate a modem or FTP session to the intended destination and track the success of the transmission.

C. The ability to automatically make multiple attempts to complete the transmission if the first attempt is unsuccessful.

D. The ability to automatically generate reports on transmissions including such details as start and stop time, total byte count, file count, transmit file name, duration, and success.

E. The ability to automatically retransmit files that failed to go in the previous transmission.

F. The ability for the user (generally someone with admin level access) to initiate a retransmission of files that did not successfully make it to the intended destination. (This implies that the system stores the update files in a directory for a predetermined number of weeks. A minimum of four weeks would seem appropriate.) This retransmit ability should be accomplished via a menu-driven process. We do not want the user to have to go to a command line prompt and have to enter a complicated command to initiate a retransmit. It should be simplified, with the user being prompted for perhaps the date of the file and the intended destination.

• A utility for extracting data from the archive system for the creation of magnetic tapes, CD-ROMs, etc. This extraction should be executable by running a simple query on the database, the results of which are delivered in a standard format. For example, we currently use a utility on our text database to extract text records to FTP to an outside entity. We initiate a standard search query and the data is output in SIF format (Standard Intermediate Format), the same format we use for our daily transmissions to vendors. This is done via a very simple step-by-step utility and can be set up to run in advance so that, for example, two month's worth of extracts can be scheduled at one time.
• Written documentation for all aspects of the archive system, with enough copies for all of the appropriate user categories. Updates to the documentation will be provided as changes to the software warrant it, such as new releases or major upgrades. This is essential for our training efforts. Users will absolutely require quick reference cards to help them navigate the archive system. These will have to be available for different functions (editing, retrieval) and different classes of users (librarian, reporter, photo editor).
• Ability to easily add or delete users and perform other aspects of account administration.
• Ability to generate reports covering such aspects as data transmissions, database updates, database usage, record acquisition and database status.
• Automatic disconnect from system if user is idle for XX minutes (probably 10).
• Automatic messaging to system administrators whenever a serious error occurs (e.g., failed transmission, failed database update, failed read of a CD-ROM, etc.).
• Ability to convert files into other formats (such as HTML, GIF, etc.) for use on the World Wide Web or other systems.
• Ability to recover IPTC header/captioning information with high-res image even if the catalog/pointers become corrupted (i.e., redundant storage of IPTC info with thumbnail and high-res).
• Availability of a source code escrow agreement in the event of vendor leaving the business or deciding to no longer support the product.

Revised 9-23-97/tr

Archiving in News Libraries
SLA News Division Home