{"id":4331,"date":"2017-08-04T19:06:27","date_gmt":"2017-08-04T10:06:27","guid":{"rendered":"http:\/\/blog.themusio.com\/?p=4331"},"modified":"2024-05-01T10:46:07","modified_gmt":"2024-05-01T01:46:07","slug":"coreference-resolution","status":"publish","type":"post","link":"https:\/\/blog.themusio.com\/?p=4331","title":{"rendered":"Coreference Resolution"},"content":{"rendered":"

Table of Contents<\/h1>\n
    \n
  1. Coreference Resolution<\/a>\n
      \n
    1. goal<\/a><\/li>\n
    2. motivation<\/a><\/li>\n
    3. ingredients<\/a><\/li>\n
    4. steps<\/a><\/li>\n
    5. outlook<\/a><\/li>\n
    6. resources<\/a><\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

      <\/a><\/p>\n

      Coreference Resolution<\/h1>\n

      <\/a><\/p>\n

      goal<\/h2>\n

      In this weeks blogpost we are going to shed light on coreference resolution in an end-to-end way as recently presented in the paper “End-to-end Neural Coreference Resolution”.<\/p>\n

      <\/a><\/p>\n

      motivation<\/h2>\n

      Solving tasks in an end-to-end fashion becomes more and more popular for several reason.
      \nTrying to optimize several components or modules of a system simultaneously can be very difficult due to their complex intertwining.
      \nBesides that multi-component system usually need a lot of hand-tuned parameters such as thresholds in order to decide which data to pass to the next stage.
      \nThis is also the case for previous work on coreference resolution which relies on parse trees and hand-engineered algorithms for aligning mentions on top of these.
      \nSystems designed in this way clearly do not easily generalize to other domains.<\/p>\n

      <\/a><\/p>\n

      ingredients<\/h2>\n

      scoring, bidirectional lstm, pruning, convolutional layer, word embeddings, character embeddings, metadata embedding<\/p>\n

      <\/a><\/p>\n

      steps<\/h2>\n

      The task of coreferencing can be summarized as detecting all expressions in some text which correspond to the same entity.
      \nCoreference resolution is a rudimentary task needed for tackling context understanding, question answering, information extraction and hence dialogue generation.
      \nIn more detail coreferencing refers to assigning an antecedent to every mention in an utterance, sentence or document.
      \nNow relying on spans of several words for detecting these mentions allows one to move a way from generating a parse tree.
      \nAnd an antecedent can be every span occurring previous to the actual mention.<\/p>\n

      The model architecture in the above mentioned paper consists of a unary and pairwise scoring of spans.
      \nThe unary scoring takes a span representation as input and tells us how likely it is that it is a mention.
      \nThe pairwise scoring consumes two span representations and additional metadata such as the speaker(s), the genre and the distance between those spans.
      \nSimple feed forward networks provide the parameters to learn a good scoring.
      \nAt this point we have to explain the structure of the span representations.
      \nFor properly detecting mentions not only the internal structure is of importance but also the context.
      \nHence the authors rely a bidirectional LSTM with a standard attention mechanism as of now being the standard for processing sequential data.
      \nPretrained word embeddings in form of 300-dimensional GloVe vectors as well as 50-dimensional vectors insensitive to word-ordering are fed to the LSTM.
      \nTo resolve the internal structure one-dimensional convolutional filters of size 3,4 and 5 are used, which provide further 8-dimensional embeddings.
      \nA span representation is then the concatenation of the result of the hidden states of the start and end position of the span, the attention state, as well as the size of the span.<\/p>\n

      Obviously, splitting utterances into all possible spans and scoring all of these pairwise is a highly computational expensive algorithm.
      \nTherefore a certain way of pruning is needed.
      \nFirst the possible number of words per span should be limited and it makes sense to only pairwise score those mentions with the highest mention scores.
      \nFurthermore one can neglect overlapping spans and only take a certain number of antecedents into account.
      \nThis seems to be a lot of hyperparameters to tune and might eventually lead to throwing a way of gold training examples.
      \nFortunately this is not the case.
      \nFor spans which do not appear in the training data or whose antecedents do not contain the gold standard the model requires the introduction of a dummy span, which gets a score of zero.
      \nFinally, all pairwise scored mentions should be clustered and represent now one entity.<\/p>\n

      The authors present gains in F1 scores on the OntoNotes benchmark and hence reach state-of-the-art performance.
      \nAdditionally the model allows to interpret the absence of coreference links and trace it back to either low mention scores or low mention ranking.
      \nThe attention mechanism also nicely shows the contribution of the individual words in the span to the coreference decision.<\/p>\n

      <\/a><\/p>\n

      outlook<\/h2>\n

      In the future additional information in the form of the output of a named entity recognizer or world knowledge could improve the models performance even more.
      \nAnother paper which we want to mention here is “Event Coreference Resolution by Iteratively Unfolding Inter-dependencies among Events”.
      \nIn addition to inter-document resolution of coreferences the authors tackle cross-document resolution and try to cluster mentions of real-world events.<\/p>\n

      <\/a><\/p>\n

      resources<\/h2>\n

      https:\/\/arxiv.org\/abs\/1707.07045<\/a>
      \n      https:\/\/arxiv.org\/abs\/1707.07344<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

      Table of Contents Coreference Resolution goal motivation ingredients steps outlook resources Coreference Resol […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[3642,3640],"tags":[],"class_list":["post-4331","post","type-post","status-publish","format-standard","hentry","category-ai-en","category-all-en"],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blog.themusio.com\/index.php?rest_route=\/wp\/v2\/posts\/4331","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.themusio.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.themusio.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.themusio.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.themusio.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4331"}],"version-history":[{"count":4,"href":"https:\/\/blog.themusio.com\/index.php?rest_route=\/wp\/v2\/posts\/4331\/revisions"}],"predecessor-version":[{"id":10845,"href":"https:\/\/blog.themusio.com\/index.php?rest_route=\/wp\/v2\/posts\/4331\/revisions\/10845"}],"wp:attachment":[{"href":"https:\/\/blog.themusio.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4331"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.themusio.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4331"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.themusio.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4331"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}