Evaluate TAR Reports

Eclipse provides several TAR-related reports to help you monitor review rounds and evaluate details regarding human review and TAR analysis of the project documents.

This project contains the following content:

TAR-Related Reports

The following table introduces these reports and explains when they are typically run and evaluated during a TAR project.

ClosedTAR-Related Reports

Report

Description

Run After [1]

T

V

C

TAR Project Summary

Provides an overview of the TAR project, with data ranging from the total number of documents in the TAR project (based on the category index selected for the project), to projected Responsive and Non Responsive documents for the project.

Note: If training/validation rounds have not been completed, all data will not be included.

X

X

X

TAR Project Validation Details

Presents detailed results for each validation round, including precision data for the review in terms of how accurately Responsive documents have been identified by the TAR process.

 

X

 

TAR Project Discrepancies

Identifies documents for which there is a disagreement between the reviewer and the TAR analysis regarding document responsiveness.

 

X

X

TAR Project Score Distribution

Presents distributed category scores (per validation round) for all rounds in the TAR project.

 

X

X

TAR Project Certification Details

Presents detailed results for the certification round, including precision data for the review in terms of how accurately Non Responsive documents have been identified by the TAR process.

 

 

X

[1] T= Training round; V = Validation rounds; C = Certification round

 

Basic TAR Report Procedure

To run a TAR report:

  1. Log in to Eclipse and open the needed case or batch.

  2. In the Case View pane, click Reports.

  3. Click the needed TAR report.

  4. Make additional selections as required by the report.

  5. After the report is generated, review results in the Reports tab. Use the following actions as needed:

    • Scroll vertically and/or horizontally to view all information.

      Click in the report and press the PAGE UP and PAGE DOWN keys for easy scrolling.

    • Click a column heading to sort by that column.

    • Enlarge or reduce a column’s width by dragging the right or left edge of the column’s heading.

  6. To the report, see the following procedure.

Export a TAR Report

To export a TAR report to a .CSV file:

  1. Run the report as explained in the previous procedure.

  2. At the bottom of the Reports tab, ensure that Export to CSV is selected in the actions list.

  3. Click Run.

  4. In the Save As dialog box, navigate an appropriate location for the file, enter a file name, and click Save. It is not necessary to include the file extension (.CSV).

  5. Open the file in a spreadsheet or other appropriate application.

TAR Report Details

Refer to the following tables for details about the data presented in TAR reports. Tables are listed in alphabetical order; data in each table is in the order it appears in each report.

ClosedTAR Project Certification Details Report

DAta

Description

Review Round

Certification Round (the type of review pass).

Batch Name

The name of the certification batch.

Review Document Total

Number of documents reviewed in the certification round.

Confirmed Non Responsive

Number of documents tagged as Non Responsive by the (human) reviewer.

% Combined Non Responsive

Percentage of documents in the certification round that were tagged as Non Responsive.

Certification Error %

The margin of error defined for the certification round, expressed as a plus/minus range (for example, +-3).

Estimated Missed Responsive Documents

The estimated number of responsive documents remaining in the TAR Project that have not been categorized as Responsive. This value is based on a formula that considers the following factors:

  • % Combined Non Responsive

  • Certification Error %

  • Number of documents in this TAR project categorized as Non Responsive.

 

ClosedTAR Project Discrepancies Report

Data

Description

Review Pass

Validation Round (the type of review pass).

Batch ID

Name of the validation batch.

DateTime

Date and time the reviewer tagged the document.

BegDoc

Image key of the document for which the human reviewer and the TAR analysis disagree.

Tag

Tag selected by reviewer (human’s assessment of document responsiveness).

Category

Category name (TAR assessment of document responsiveness).

Score

A measure of how closely the discrepant document matches the category.

SeedBegDoc

Image key of the document tagged by a reviewer as part of the training or validation review, which formed the basis for the TAR Category decision for the discrepant document.

 

ClosedTAR Project Score Distribution Report

DAta

Description

Validation Round

The number of the current validation round (each review pass increments the validation round by one).

Batch Name

Name of the validation batch.

Responsive 100-76

Number of documents in the Responsive category with a score of 100 to 76, as reported when the specified validation round is complete.

Responsive 75-51

Number of documents in the Responsive category with a score of 75 to 51, as reported when the specified validation round is complete.

Responsive 50-26

Number of documents in the Responsive category with a score of 50 to 26, as reported when the specified validation round is complete.

Responsive 25-1

Number of documents in the Responsive category with a score of 25 to 1, as reported when the specified validation round is complete.

Non Responsive 100-76

Number of documents in the Non Responsive category with a score of 100 to 76, as reported when the specified validation round is complete.

Non Responsive 75-51

Number of documents in the Non Responsive category with a score of 75 to 51, as reported when the specified validation round is complete.

Non Responsive 50-26

Number of documents in the Non Responsive category with a score of 50 to 26, as reported when the specified validation round is complete.

Non Responsive 25-1

Number of documents in the Non Responsive category with a score of 25 to 1, as reported when the specified validation round is complete.

 

ClosedTAR Project Summary Report

Data

Description

Documents in
Categorization Set

Total number of documents in the Analytics Index selected for the project.

Training Document Set

Number of training documents, as determined by Eclipse analysis after the training review pass is created (part of the TAR project creation).

Training Set - Responsive

Number of training documents tagged as Responsive by a (human) reviewer.

Training Set - Non Responsive

Number of training documents tagged as Non Responsive by a reviewer.

Training Set - Responsive Percentage

Percentage determined by dividing the Training Set Responsive value by the Training Document Set value (occurs after the training review pass is completed).

Projected - Responsive

Estimated number of Responsive documents in the entire TAR project, based on Eclipse analysis of the training set results (occurs after the training review pass is completed).

Projected -
Non Responsive

Estimated number of Non Responsive documents in the entire TAR project, based on Eclipse analysis of the training set results (occurs after the training review pass is completed).

Current Validation Round - Precision[1]

Precision from the last completed validation round for the selected TAR project.

Current Validation Round - Recall*

Recall from the last completed validation round for the selected TAR project.

Current Validation Round - FMeasure*

F Measure from the last completed validation round* for the selected TAR project.

Current Estimated Responsive Counts

Estimated number of Responsive documents in the entire TAR project, based on latest categorization of the project.

Current Estimated
Non Responsive Counts

Estimated number of Non Responsive documents in the entire TAR project, based on latest categorization of the project.

[1]Each batch within a validation review pass is considered to be a validation round. SeeTAR Project Validation Details Reportfor details on Precision, F Measure, and Recall.

 

ClosedTAR Project Validation Details Report

Data

Description

Review Round

Validation Round (the type of review pass).

Batch Name

The name of the batch.

Responsive - True

The number of documents for which the (human) reviewer and the Eclipse TAR analysis both tag the documents as Responsive.

Non Responsive - True

The number of documents for which the reviewer and the TAR analysis both tag the documents as Non Responsive.

Responsive - False

The number of documents for which the reviewer tags the documents as Responsive, but the TAR analysis considers the documents Non Responsive.

Non Responsive - False

The number of documents for which the reviewer tags the documents as Non Responsive, but the TAR analysis considers the documents Responsive.

Precision[1]

A measure of the accuracy of the identification of document responsiveness.

For validation rounds, this applies to documents tagged as Responsive. A higher precision value means a more accurate result, that is, that the documents that have been reviewed are indeed Responsive.

Recall*

A measure of completeness that is based on the number of documents tagged as Responsive as compared to the total number of truly responsive documents.

F Measure*

A method for factoring both precision and recall to indicate how well the TAR process has identified truly Responsive documents.

Sample Type

The type of sample used, Fixed or Statistical, plus values related to the sample type:

For Fixed, the number of documents selected for the sample will be listed.

For example, Fixed/40.

For Statistical, the selected confidence level and margin of error will be listed.

For example, Statistical/95, +-3

[1]Values are based on standard statistical calculations for Precision, Recall, and F Measure.