Here we show side-by-side comparisons of our Mechanical Turk-based
ground truth annotations and our saliency-based
heuristic annotations.
To compute
ground truth, we score each line based on the percentage of worker annotations that included that line. Lines with scores over 40% are included in the ground truth annotations.
To compute the
heuristic annotation, we do a weighted sum of saliency scores near each line. Lines with the top three scores are selected for the annotation, see paper for details.
Download annotation data, Mechanical Turk task code, and analysis scripts
here.
Images from
MIRFLICKR-25000 dataset.