Chapter 7 - Archiving Document Images 4.0
- 1 ARCHIVING DOCUMENTS
- 1.1 How Images are Stored on Cartridges
- 1.2 Auto Archive
- 1.3 Archive Set up
- 1.4 General tab
- 1.5 Log Options tab
- 1.6 Auto Archive tab
- 1.7 Destination tab
- 1.8 Alerts tab
- 1.9 Running Archive Interactively
- 1.10 Running Archive as a Scheduled Task
- 1.11 Reviewing Archive Results
- 1.12 Other Archive Features
Archiving is the process of moving documents from a halFILE basket to permanent storage. Only indexed documents are archived and the archive process removes the document from the basket. Multiple baskets can be selected.
A Cartridge is a permanent storage device for images. Refer to Chapter 1 for information on setting up cartridges.
Archive is an automated process that is scheduled as a SQL Server Agent task or in Windows Task Scheduler. The options include:
Auto Archive with no cartridge rotation – this option runs the archive as a scheduled task, typically overnight. A pre-configured screen is used to select what baskets are archived to which cartridges.
Auto Archive with cartridge rotation – this runs archive as a scheduled task. It can also manage the process of creating new cartridges automatically and limiting the amount of the space used on a cartridge to facilitate backing up cartridges to CD or DVD.
The options are discussed in greater detail below.
How Images are Stored on Cartridges
When images are stored on cartridges, they are placed into specific directories. The highest directory level is the application id. The second directory level used depends on the folder option specified for the application (set using File-Application). The following illustrates the directories used for the various folder options.
No limits on number of images per folder
With this folder option, the user enters the folder id and there are no restrictions on how many images can be placed into the folder. The directory structure for this option is as follows:
Application Id _____ Folder Id
Therefore, if the application id is 'TST' and the folder is 'TEST' then images archived to cartridge are placed into the directory:
Limit number of images per folder and enter the folder id
With this option, the user enters the folder id and each folder is limited to a user-defined number of images. The directory structure on the archive cartridge for this option is as follows:
Application Id ______ Folder Id ______ Folder Counter
Therefore, if the application id is 'TST' and the folder is 'TEST' and the number of images per folder is 500, then the first 500 images are placed into the following directory.
Limit number of images per folder and auto assign the folder id
With this option, each folder is limited to a user-defined number of images and the system automatically defines the folder id. The directory structure on the archive cartridge is as follows:
Database Level Archiving
A option lets you include a database level folder in the archive folder structure. This option is defined on the General tab of the Auto Archive configuration screen (Tools-Configure Auto Archive menu in the halFILE Administrator). When this option is enabled, the folder structure is as follows:
Note: The system will never split the images for a document across two cartridges. If, based on the number of images per folder, there is not enough room in the folder counter directory, the document images will be copied to the next folder counter directory. Therefore, it is possible to have fewer images in a directory than is designated by the number of images per folder parameter.
halFILE's Archive utility includes an Auto Archive feature to archive a selected baskets to selected cartridges on a scheduled basis. The program, HFARCHIVE32.EXE, is run as a scheduled task on a Server or as a SQL Server Agent job. Multiple Archive sets can be set up to archive to different sets of cartridges.
Archive Set up
You can configure Archive by selecting the Tools-Archive Setup menu in the halFILE Administrator.. This brings up a screen with several tabs for defining the various Auto Archive options as shown below.
This tab is used to set up general auto archive options as follows:
- Enable Auto Archive Features - check this box to enable auto archive.
- Force recalculation of cartridges - as the screen suggests, this option is used when archiving to some NAS storage devices if copy requests periodically seem to fail.
- Enable database-level archiving - this option will create a database level folder beneath the application folder to further separate images on the cartridge.
- Low Disk Notify (in MB) – when the free space on the current archive drive reaches the disk space configured in this box, an alert e-mail and system message is sent as a notification that the drive is getting low on space (see Alerts section).
Log Options tab
This tab is used to configure how logging occurs during Auto Archive. It is strongly recommended that you use logging options and review the provided log files daily to ensure that archive is running properly.
- Disable Archive Logging - check this box to disable all logging.
- Append to archive logs - check this box to append to archive logs. Uncheck it to overwrite archive logs from previous jobs. It is recommended that you uncheck this box except when debugging problems.
- Turn on Debug log file - check this box to create an archive.dbg log file of detail activity. This is useful when debugging set up problems.
- Use Dated Log - check this box to create dated log files. Normally log files are named autoarchauto1.log (where 1 is the archive set number). With this option the log file is named HFARCHIVE32_YYYYMMDD_Auto1.log (where YYYYMMDD is year, month, and day, and 1 is the archive set number).
- Path to backup dated archive log files - if the Use Dated Log box is checked, enter the path to store these files. You must create the folder.
Auto Archive tab
The Auto Archive tab opens up the screen shown below. As you make changes to this form, it is important to remember that the Save Set button must be clicked to save those changes.
- Archive Set - You can have many archive sets. This drop down box lets you select the archive set to configure (<NEW> lets you configure the next available set).
It is recommended that documents for different databases be archived to different sets of cartridges. To do this you should set up an archive set for each database and name the cartridges in a way that the database is identified. One good method is to use the database id as part of the cartridge mask. So, if you have an application id of HAL and two databases with database ids of DD and TP then the first auto archive set could be set up to archive documents for application HAL, database DD to cartridges using the cartridge mask DD#####. The first cartridge is named DD00001. Then a second archive set would be set up to archive documents for application HAL, database TP to cartridges using the cartridge mask of TP#####. The first cartridge is then named TP00001.
- Set Description - enter a description of the archive set.
- Source tab - Select the Application, Database and Basket to archive and click Add to include it in the archive. Continue to select all the baskets for this archive set. The Add All button is used to add all the baskets for the currently selected database. If you delete a basket, you should remove it from the archive set using the Remove button.
- If the auto-rotate cartridges option is unchecked, archive automatically rotates to a new cartridge when a cartridge reaches a certain size. It is recommended that the auto-rotate cartridges box be checked.
- The Include ALL Baskets in the database check box can be used if you always want to archive all baskets in the database. In this case, it is not necessary to add the baskets into the list of items to be archived.
This tab, shown below, is used to configure the destination cartridges and drives for the auto archive. The Use Defaults button usually can configure this screen properly.
- Cart Name Mask - this field defines how new cartridges are named. Use the %appl% to substitute the Application ID and %doctype% to substitute the Document Type. Also, use # to designate the cartridge number, including as many digit placeholders as the maximum cartridge number will be. For example, if your Application ID was XXX and your Document Type ID was TP and the mask was %appl%%doctype%###, then the first cartridge created would be named XXXTP001. Cartridges are limited to 8 characters so your cartridge mask should not exceed 8 characters.
- Drive Sugffix - defines the suffix that is appended to the drive path when creating new cartridges. You do not need to include the cartridge name prefix, it will automatically be included.
- Cart Description - defines the cartridge description to use when creating a new cartridge. Again, you can use the %appl%, %doctype% and ### substitution values.
- Size limit (in MB)- defines the size limit in megabytes for new cartridges. When the files on the cartridge reach this size, then the cartridge is considered full and the system rotates to a new cartridge.
- Current Cart - this shows the current cartridge number being used by Auto Archive. When you first set up archive, you would create the first cartridge and enter a 1 in this box. For example, if you wanted cartridges named in the IMG00001 form, then you would use the Configure-Cartridges menu to create IMG00001. Then enter a 1 in this box. The auto-rotate feature will create IMG00002 and enter a 2 in this box when the IMG00001 cartridge becomes full.
- Cart type - defined the cartridge type to use for new cartridges. This should match the drive type used when the Archive Drive was created (Configure-Drives).
- Drive- select the drive to archive to from the list of drives provided. These drives are defined using the Configure-Drive menu.
NOTE: If you replace your image drives, you must update the Drive and Overflow Drive to properly record the target drive where images are archived.
- Overflow Drive - should the drive configured in the Drive box become full, then auto archive will begin using this drive to archive to, if it is defined.
- Save DB Information - checking the ON box will make the system create text files containing the data for all the images placed on the cartridge. This is a good disaster recovery technique and so we strongly recommend the use of this option.
- Log System Messages - checking the ON box will log messages to the System Message area which is displayed when users login to halFILE. Normally, this option is set to OFF since we recommend checking the log files or receiving the e-mail alert of archive activity.
This tab, shown below, is used to set up e-mail alerts of the Auto Archive activity to selected users. Enter the SMTP Server Address or Server Name in the SMTP Server box then enter the e-mail addresses to which the message should be sent. This feature requires HALSMTP.EXE, halFILE's SMTP e-mail program.
Running Archive Interactively
To run the archive routine, the command line is:
Where <n> is the number of the archive set. HFArchive32 Auto1 will run archive set \#1. For each basket being archived, the screen showing its progress will display. Once the basket is complete, it will disappear as the program prepares to archive the next basket in the archive set. If there are lots of documents in the basket, the screen may disappear for some time while the program calculates the size of files to be archived.
Running Archive as a Scheduled Task
HFARCHIVE32 can be set up as a scheduled NT task or as a job in SQL Server Agent using a batch file containing the same command line as above. You can set up multiple tasks to run different archive sets.
Be sure to configure the user that the task runs under as a network user who has the rights to run halFILE and has access the archive drives and basket folders.
Reviewing Archive Results
Auto Archive creates an autoarch.log file each time it runs. You can review this log to see if archive completed normally. You should also review baskets in halFILE using File \| Basket Status as well as search for archived documents to ensure that documents are being properly archived.
The Alerts section of the configuration screen is very useful for reviewing results. This sends an e-mail containing the log file from archive. The log file contains the number of documents archived as well as notifications when a cartridge becomes full.
Other Archive Features
Archive can also be set up to:
- Send an e-mail notification of auto archive activity.
- Save the database information for archived documents.
- Bypass the CD Stage Area requirement while maintaining CD-sized or DVD-sized cartridges.
- Automatically assign the next cartridge in sequence.