Dataset Description for Similarity Learning

Dataset Name Link Suggested Metrics Some Papers that use the dataset Brief Description
WikiQA
  • MAP
  • MRR
  • SeqMatchSeq(MULT MAP=0.74, MRR=0.75)
  • BiMPM(MAP=0.71, MRR=0.73)
  • QA-Transfer(SQUAD* MAP=0.83, MRR=84.58, P@1=75.31)
 Question-Candidate_Answer1_to_N-Relevance1_to_N
SQUAD 2.0 Website
  • Exact Match
  • F1
 QA-Transfer(for pretraining) Question-Context-Answer_Range_in_context
Quora Duplicate Question Pairs Accuracy, F1
  • BiMPM(Acc=88.17%)
    		 Q1-Q2-DuplicateProbablity
    Sem Eval 2016 Task 3A genism-data(semeval-2016-2017-task3-subtaskA-unannotated)
    • MAP
    • AvgRecall
    • MRR
    • P
    • R
    • F1
    • Acc
    		 QA-Transfer(SQUAD* MAP=80.2, MRR=86.4, P@1=89.1) Question-Comment-SimilarityProbablity
    MovieQA Accuracy SeqMatchSeq(SUBMULT+NN test=72.9%, dev=72.1%) Plot-Question-Candidate_Answers
    InsuranceQA Website Accuracy SeqMatchSeq(SUBMULT+NN test1=75.6%, test2=72.3%, dev=77%) Question-Ground_Truth_Answer-Candidate_answer
    SNLI Accuracy
    • QA-Transfer(for pretraining)
    • SeqMatchSeq(SUBMULT+NN train=89.4%, test=86.8%)
    • BiMPM()Ensemble Acc=88.8%)
    		 Text-Hypothesis-Judgement
    TRECQA
    • MAP
    • MRR
    		 BiMPM(MAP:0.802, MRR:0.875) Question-Candidate_Answer1_to_N-relevance1_to_N
    SICK Website Accuracy QA-Transfer(Acc=88.2) sent1-sent2-entailment_label-relatedness_score

    More dataset info can be found at the SentEval repo.

    Links to Papers:

    Some useful examples

    SQUAD

    SQUAD

    SQUAD, WikiQA, SemEval, SICK

    alt

    MovieQA and InsuranceQA

    alt