Participations in automatic summarization competitions

Every year since 2002, Guy Lapalme, with students from the RALI or with colleagues from other universities, have participated to all summarization tracks of the Document Understanding Conferences (DUC) and Text Analysis Conference (TAC) using different summarization approaches.

  • DUC 2002 : Atefeh Farzindar modified SumUM without changing the algorithm or manually developed patterns from a corpus of professional summarizers. Even though SumUM had not been developed on the newspaper type of text used at DUC2002, it ranked among the best systems of the competitions. [PDF]
  • DUC2003 : SumUM was modified to participate to the multiple document competition and obtained very good results. [PDF]
  • DUC2004 : Fouad Douzidia developed Lakhas (which means summarize in Arabic), a summarizer for news type texts based on a combination of methods used until then in English. Lakhas was used to produce short summaries in Arabic at the DUC2004 competition. These summaries were then machine translated into English. Even though this approach followed a different track from the one of the other competitors, the evaluation results were excellent and even the best one when using the same machine translation engine as the one used to translate the texts. The complete results of this experiments were described in His master thesis (in French). [PDF]
  • DUC2005 : CATS (Cats is an Automatic Text Summarizer) was developed by Atefeh Farzindar and Frédérik Rozon during the summer of 2005. The task was the summarization, in 250 words or less, of clusters of 20 newspaper articles dealing with the same subject. The summary had to deal only with a single aspect described by a topic described in a few lines. CATS performance was excellent compared to the other thirty participants to the competition. [PDF]
  • DUC2006 : Guy Lapalme joined a team form the Laboratoire Informatique d'Avignon (LIA) for developing a system combining many different approaches whose results were then combined. [PDF]
  • DUC2007 : Fabrizio Gotti, Guy Lapalme (RALI) et Luka Nérima, Éric Wehrli (LATL, Université de Genève) developed GOFAISum (Good Old Fashion AI Summarizer) using a completely symbolic approach. Texts are parsed with FIPS producing an XML syntactic analysis which is then processed with a series of XSLT stylesheets. [PDF]
  • TAC2008 : Pierre-Étienne Genest followed the GOFAISum approach to deal with update summaries. The system, called NESS, ranked well in the competition results. [PDF].
  • TAC2009: RALI presented to contributions to this competition:
    • HexTAC: a submission created using an interface allowing human summarizer to create sentence level extraction summaries. [PDF] [Project page]
    • Ness2: a two step approach to the update task. [PDF]
  • TAC2010: Pierre-Étienne Genest tested a text generation approach to generate selected sentences to insert in the summary.  Minipar was used to parse the original texts and Simple-NLG was used to create the sentences. [PDF
  • TAC2011: Improvement in the TAC2010 approach with a new theoretical framework.[PDF]
  • No summarization track at TAC in 2012 and 2013
  • TAC2014Biomedical Summarization Track : summarize the impact of a paper by studying its citations from other papers. Bruno Malenfant submitted results of a system he developed at the beginning of his PhD dealing with this subject. [PDF]