Skip to content

Running BIUTEE Tracer

Jump to bottom Edit New page
Roberto Zanoli edited this page May 20, 2015 · 1 revision

(See Running BIUTEE.)

Table of Contents

Visual Tracing Tool

BIUTEE's visual tracing tool assists in tracing various aspects and internal steps of the system.

Prerequisites

In order to run the visual tracing tool, you will need to install the prerequisite GraphViz:

  1. Download and install the program GraphViz from http://www.graphviz.org.
  2. Make sure that the executable dot (in {GraphVizInstallationFolder}\bin) is in your system's path:
    1. In Linux: export PATH=$PATH:{GraphVizInstallationFolder}\bin
    2. In Windows: set PATH=%PATH%;{GraphVizInstallationFolder}\bin

How to Run BIUTEE's Visual Tracing Tool

If you wish to use the tool, perform these steps:

  1. In Linux/Unix, run the script run_biutee.sh from $BIUTEE/workdir, with the mode visual. Configure the model and resources using the arguments, as explained here.
Alternatively, you can run the tool manually:
  1. Configure the model and resources in biutee.xml, under rte_pairs_preprocess and rte_pairs_train_and_test. More details here.
  2. Make sure that the EasyFirst parser server is running. More details here.
  3. Run:
    mvn -f $EOP/biutee/pom.xml exec:java -Dexec.mainClass=
    '''eu.excitementproject.eop.biutee.rteflow.systems.gui.VisualTracingTool'''
    -Dexec.args="biutee.xml"

Running a sample

This example will demonstrate how to use the tracer in order to annotate a T/H pair.

  1. Run the visual tracing tool as explained above.
  2. When prompted, click Initialize to initialize the system, and wait for this window to close:

  3. In the main window, choose from the menu File > Load and Select RTE-Pairs Dataset...:

  4. Load the dummy dataset file dummy_train.xml found in $BIUTEE/workdir.
  5. Select the 4th pair, using the numeric up-down control, and click on Select:

  6. Click Pre-Process in the main window. The status bar will display the current processing step of the text and the hypothesis. Once done, you will be able to view the trees of the text, as well as other linguistic annotations, in the tabs Text trees and Hypothesis tree respectively:

    If the text contains more than one sentence, you can browse the parse trees using the combobox at the bottom of the Text Trees tab.
  7. In order to annotate the pair of text and hypothesis, click the Automatic Mode tab. In this example, we will use the automatic proof generation. Click on the button Generate Proof to generate proof for T->H. After a short while (indicated by the progress bar), the proof will appear on the screen:

    After search is done, the progress bar indicates how likely it is that the text entails the hypothesis. 50% or more means entailment, while lower-than-50% means non-entailment.
  8. Look at the table at the lower part of the tab. It shows the sequence of transformations that have been applied so far.
  • The costs of each transformation are presented and highlighted by a color. Transformations with high cost, which cause the classifier to decide "no" (i.e., only if all of them had not been applied, the classifier would have been deciding "yes") are colored red. All other transformations are colored yellow or green, where green is for costs higher-than-median of all transformations-costs of current proof.
  • For each transformation, you can see the current feature-vector and the missing elements, by choosing in the menu: Options > Show Search Details.
  • The features that were changed by applying the current transformation are presented in bold.

Running scenarios and advanced options

This section will describe the different options for each running step. Note that in each tab you have a ? button which gives help about that tab.

Choosing the T/H pair

There are several options to choose the text and the hypothesis:

  1. Manually: type the text and the hypothesis on the text and hypothesis text boxes respectively.
  2. From recent pairs: choose pairs which you selected recently by choosing from the menu File > Load and Select recent pairs.... Navigate the recent pairs using the numeric up-down control. Once the wanted pair shows, click the button Select. The pair selection window will close, and in the main window, the text boxes of the text and hypothesis will be populated with the selected pair.

  3. From a RTE-pairs data set: choose pairs from a RTE1-5 dataset by choosing from the menu File > Load and Select RTE-Pairs Dataset.... Click on the button Load... to choose the required dataset. The first pair in the dataset will appear. Navigate the pairs using the numeric up-down control and click on Select as in choosing from recent pairs.
  4. From a RTE sum data set:
    1. Pre-process the RTE sum data set to obtain a serialized (.ser) dataset file. More details here.
    2. Load the pre-processed file: In the menu, choose File > Load RTE-Sum Dataset..., select the required .ser file and click Open.
    3. Choose the T/H pair: In the menu, choose File > Select Sentence From Dataset....

      Choose the selected topic, document and sentence and click Select. The selection will appear as the text.
    4. Type the hypothesis in the hypothesis text box in the main window.
    5. If you wish to return to RTE-pairs mode, choose File > Select Sentence From Dataset... and click on Discard.

Pre-processing the text and hypothesis

The pre-processing includes sentence splitting, parsing, NER and coreference resolution. After the text and hypothesis text boxes are populated, click on Pre-Process in the main window. Once the pre-processing is done, you will be able to view the following:

  1. The sentence split for the text appears in the menu window, next to the text.
  2. The parse tree(s) for the text appears in the Text Trees tab:

    If the text contains more than one sentence, you can browse the parse trees using the combobox at the bottom of the Text Trees tab.
  3. The parse tree for the hypothesis appears in the Hypothesis Tree tab:

  4. You can change the size of the trees by scrolling the mouse wheel while pressing Ctrl in the keyboard.
  5. You can simplify the visualization of the tree by choosing in the menu: Options > Show Concise Nodes:

  6. In the long nodes version, the nodes contain predicate truth annotation as +, - or ? before the lemma. You can see also clause truth, negation-uncertainty and signature by choosing in the menu Options > Show Semantic Annotations On Nodes.

Searching for the best proof

There are two ways to build a proof: manually and automatically.

  1. Manually:
    1. Click on the Manual Mode tab. In this tab, each row represents either the original parse tree of a sentence, or a generated parse tree. For each row, if the column Missing Elements is higher than zero, it means that it does not represent a complete proof, and more transformations should be applied.
    2. Apply all possible transformations on a parse tree by double-clicking the row:

      For convenience, we usually want that the table will not show any trees, but only trees that were generated as a result of the last double-click. For this, check the checkbox Display Only Last Generated Trees at the bottom.
    3. To view the last transformation's full description, put the mouse over the row. A tooltip will be displayed. To view the transformation visually, click on the row (not a double-click) and click on Display Selected Tree. This will switch to the View Tree (Manual) tab with the tree obtained from the last transformation:

  2. Automatically:
    1. Click on the Automatic Mode tab and click on the button Generate Proof at the bottom.
    2. After a short while (indicated by the progress bar), the proof will appear on the screen:

      After search is done, the progress bar indicates how likely it is that the text entails the hypothesis. 50% or more means entailment, while lower-than-50% means non-entailment.
    3. If you want to see the feature vector of the whole proof, in addition to the feature-vector of each transformation, choose from the menu: Options > Show Search Details. It will display under the transformations table.
In both modes (manual and automatic), you can view the trees of the different transformations in the proof.
  • Use the buttons <<, <, > and >> to see the original-text-tree, the tree-after-last-transformation, the transformation-before-current and the transformation-after-current (respectively).
  • You can change the sizes of the two parts of this tab by dragging the separator between them up and down.
  • The table at the lower part of the tab shows the sequence of transformations that have been applied so far.
  • The costs of each transformation are presented and highlighted by a color. Transformations with high cost, which cause the classifier to decide “no” (i.e., only if all of them had not been applied, the classifier would have been deciding “yes”) are colored red. All other transformations are colored yellow or green, where green is for costs higher-than-median of all transformations-costs of current proof.
  • For each transformation, you can see the current feature-vector and the missing elements, by choosing in the menu: Options > Show Search Details.
  • The features that were changed by applying the current transformation are presented in bold.
Add a custom footer

Documentation

Get Involved

Clone this wiki locally