| Subject: | possible bug in depth calculation of wup |
The following was reported by Hideki Shima of CMU.
-----------------------------------------------------
(2) WUP: Depth calculation is wrong
-----------------------------------------------------
Given ice#n#2 and person#n#1, the following comes out:
<trace>
HyperTree: *Root*#n#1 entity#n#1 physical_entity#n#1 object#n#1 ice#n#2
HyperTree: *Root*#n#1 entity#n#1 physical_entity#n#1 object#n#1 whole#n#2 living_thing#n#1 organism#n#1 person#n#1
HyperTree: *Root*#n#1 entity#n#1 physical_entity#n#1 causal_agent#n#1 person#n#1
Lowest Common Subsumers: object#n#1 (Depth=4)
Depth(ice#n#2) = 5
Depth(person#n#1) = 7
ice#n#2 person#n#1 0.666666666666667
</trace>
The correct value of Depth(person#n#1) would be 8, if you
count the nodes in the 2nd hypertree.
(Actually, none of the above hyper trees have a depth of 7.)
The correct wup value would 2 * 4 / (5 + 8) = 0.6154.
The same kind of issue was observed between ragweed_pollen#n#1
and whole#n#2 where the right value for Depth(ragweed_pollen#n#1)
seems to be 13, instead of 11.
This phenomenon has been seen in about 170 out of 10k
(=1.7%) randomly generated noun-noun pairs of synsets.
The current wup score is estimated to be higher than the expected
score by 0.02 on average over these 1.7% of pairs.