{"id":489929,"date":"2018-06-11T09:08:49","date_gmt":"2018-06-11T16:08:49","guid":{"rendered":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/?post_type=msr-event&#038;p=489929"},"modified":"2025-08-06T11:57:08","modified_gmt":"2025-08-06T18:57:08","slug":"microsoft-sigir-2018","status":"publish","type":"msr-event","link":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/event\/microsoft-sigir-2018\/","title":{"rendered":"Microsoft @ SIGIR 2018"},"content":{"rendered":"\n\n<p><strong>Venue:<\/strong> <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/uunions.umich.edu\/league\" target=\"_blank\" rel=\"noopener\">Michigan League<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/maps.studentlife.umich.edu\/building\/michigan-league\" target=\"_blank\" rel=\"noopener\">Location on Campus Map<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>)<\/p>\n<p><strong>Website:<\/strong> <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/sigir.org\/sigir2018\/\" target=\"_blank\" rel=\"noopener\">SIGIR 2018<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span id=\"label-external-link\" class=\"sr-only\" aria-hidden=\"true\">Opens in a new tab<\/span><\/p>\n<p>SIGIR is a major international forum for presentation of the latest state-of-the-art research and demonstration of new systems and methods for connecting people with information: from Web search engines, recommender systems, and social network technology to compelling applications in health, legal, educational, and other domains, research at SIGIR spans both academia and industry.<\/p>\n<h2>Program Committee members<\/h2>\n<p><a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/pauben\/\">Paul Bennett<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Short Paper Chair<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/jfgao\/\">Jianfeng Gao<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, AI Track Co-chair<\/p>\n<h2>Invited Speakers<\/h2>\n<p><strong>Distributional Representation of Complex Semantics<\/strong> (Keynote at <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/kg4ir.github.io\/\" target=\"_blank\" rel=\"noopener\">KG4IR workshop<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>)<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/kuansanw\/\" target=\"_blank\" rel=\"noopener\">Kuansan Wang<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<\/p>\n<p><strong>Lessons from Building a Large-scale Commercial IR-based Chatbot for an Emerging Market<\/strong><br \/>\nPuneet Agrawal and Manoj Kumar Chinnakotla, Microsoft<\/p>\n<p><strong>Causal Inference over Longitudinal Data to Support Expectation Exploration<\/strong><br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/emrek\/\">Emre Kiciman<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/jiyinhe.github.io\/ProfS2018\/\" target=\"_blank\" rel=\"noopener\"><strong>Search and Recommendation in the Enterprise<\/strong><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/pauben\/\" rel=\"noopener\">Paul Bennett<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<\/p>\n<h2>Workshops<\/h2>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" target=\"_blank\" href=\"http:\/\/lnd4ir.github.io\/\">Learning from Limit\/Noisy data for IR<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br \/>\nHamed Zamani (UMass Amherst), Mostafa Dehghani (Univ. of Amsterdam), Fernando Diaz (Microsoft Research \u2013 Montreal), Hang Li (Toutiao AI Lab), Nick Craswell (Microsoft)<\/p>\n<h2>Microsoft attendees<\/h2>\n<p>Amjad Abu-Jbara, Microsoft<br \/>\nOmar Alonso, Microsoft<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/hassanam\/\">Ahmed Awadallah<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/pauben\/\">Paul Bennett<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<br \/>\nEdward Cui, Microsoft<br \/>\nWeiwei Deng, Microsoft<br \/>\nFernando Diaz, Microsoft Research \u2013 Montreal<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/sdumais\/\">Susan Dumais<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/adamfo\/\">Adam Fourney<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research AI<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/emrek\/\">Emre Kiciman<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<br \/>\nXiaoliang Ling, Microsoft<br \/>\nPawel Pietrusinski, Microsoft<br \/>\nMona Soliman Habib, Microsoft<br \/>\nHui Su, Microsoft<\/p>\n<h2>Career Opportunities<\/h2>\n<h4><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/careers.microsoft.com\/us\/en\/job\/463001\/ML-Engineer\" target=\"_blank\" rel=\"noopener\">ML Engineer<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/h4>\n<p style=\"padding-left: 30px\">AI & Research (AI&R) at Hyderabad, India comprises of highly motivated researchers, engineers, product managers and data-scientists building end-to-end web-scale and enterprise-scale AI systems. We seek talented, energetic, creative and passionate ML engineers with ability to enhance and apply research to ship and build high-quality products and services.<\/p>\n<p><span id=\"label-external-link\" class=\"sr-only\" aria-hidden=\"true\">Opens in a new tab<\/span><\/p>\n<h2>Full Papers<\/h2>\n<p><strong>Calendar-Aware Proactive Email Recommendation<\/strong><br \/>\nQian Zhao (University of Minnesota); Paul Bennett (Microsoft); Adam Fourney (Microsoft); Anne Thompson (Microsoft); Shane Williams (Microsoft); Adam D. Troy (Microsoft); Susan Dumais (Microsoft)<\/p>\n<p><strong>Characterizing and Supporting Question Answering in Human-to-Human Communication<\/strong><br \/>\nXiao Yang (The Pennsylvania State University); Ahmed Hassan Awadallah (Microsoft); Madian Khabsa (Apple); Wei Wang (Microsoft); Miaosen Wang (Microsoft)<\/p>\n<p><strong>Deep Domain Adaptation Hashing with Adversarial Learning<\/strong><br \/>\nFuchen Long (University of Science and Technology of China); Ting Yao (Microsoft); Qi Dai (Microsoft); Xinmei Tian (University of Science and Technology of China); Jiebo Luo (University of Rochester); Tao Mei (Microsoft)<\/p>\n<p><strong>Measuring the Utility of Search Engine Result Pages<\/strong><br \/>\nLeif Azzopardi (University of Strathclyde); Paul Thomas (Microsoft); Nick Craswell (Microsoft)<\/p>\n<p><strong>Natural Language Interfaces with Fine-Grained User Interaction: A Case Study on Web APIs<\/strong><br \/>\nYu Su (University of California Santa Barbara); Ahmed Hassan Awadallah (Microsoft); Miaosen Wang (Microsoft); Ryen White (Microsoft)<\/p>\n<p><strong>Towards Better Text Understanding and Retrieval through Kernel Entity Salience Modeling<\/strong><br \/>\nChenyan Xiong (Carnegie Mellon University); Zhengzhong Liu (Carnegie Mellon University); Jamie Callan (Carnegie Mellon University); Tie-Yan Liu (Microsoft)<\/p>\n<h2>Short Papers<\/h2>\n<p><strong>Ad Click Prediction in sequence with Long Short-Term Memory Networks: An externality-aware model<\/strong><br \/>\nWeiwei Deng (Microsoft); Xiaoliang Ling (Microsoft); Yang Qi (Microsoft); Tunzi Tan (School of Mathematical Sciences @ University of Chinese Academy of Sciences); Eren Manavoglu (Microsoft); Qi Zhang (Microsoft)<\/p>\n<p><strong>Assessing the Readability of Web Search Results for Searchers with Dyslexia<\/strong><br \/>\nAdam Fourney (Microsoft); Meredith Ringel Morris (Microsoft); Abdullah Ali (University of Washington); Laura Vonessen (University of Washington)<\/p>\n<p><strong>Attention-driven Factor Model for Explainable Personalized Recommendation<\/strong><br \/>\nJingwu Chen (Institute of Computing Technology, Chinese Academy of Sciences); Fuzhen Zhuang (Institute of Computing Technology, Chinese Academy of Sciences); Xin Hong (Institute of Computing Technology, Chinese Academy of Sciences); Xiang Ao (Institute of Computing Technology, Chinese Academy of Sciences); Xing Xie (Microsoft); Qing He (Institute of Computing Technology, Chinese Academy of Sciences)<\/p>\n<p><strong>Cross Domain Regularization for Neural Ranking Models using Adversarial Learning<\/strong><br \/>\nDaniel Cohen (University of Massachusetts Amherst); Bhaskar Mitra (Microsoft); Katja Hofmann (Microsoft); Bruce Croft (University of Massachusetts Amherst)<\/p>\n<p><strong>Multi-level Abstraction Convolutional Model with Weak Supervision for Information Retrieval<\/strong><br \/>\nYifan Nie (University of Montreal); Alessandro Sordoni (Maluuba \u2013 Microsoft); Jian-Yun Nie (University of Montreal)<\/p>\n<p><strong>Optimizing Query Evaluations using Reinforcement Learning for Web Search<\/strong><br \/>\nCorby Rosset (Microsoft); Damien Jose (Microsoft); Gargi Ghosh (Microsoft); Bhaskar Mitra (Microsoft); Saurabh Tiwary (Microsoft)<\/p>\n<p><strong>Quantitative Information Extraction From Social Data<\/strong><br \/>\nOmar Alonso (Microsoft); Thibault Sellam (Columbia University)<\/p>\n<p><strong>Testing the Cluster Hypothesis with Focused and Graded Relevance Judgments<\/strong><br \/>\nEilon Sheetrit (Technion \u2013 Israel Institute of Technology); Anna Shtok (Technion \u2013 Israel Institute of Technology); Oren Kurland (Technion, Israel Institute of Technology); Igal Shprincis (Microsoft, Herzliya, Israel)<\/p>\n<p><strong>Transparent Tree Ensembles<\/strong><br \/>\nAlexander Moore (Microsoft); Vanessa Murdock (Microsoft); Yaxiong Cai (Microsoft); Kristine Jones (Microsoft)<\/p>\n<h2>SIRIP Industry Papers<\/h2>\n<p><strong>Puneet Agrawal and Manoj Kumar Chinnakotla. Lessons from Building a Large-scale Commercial IR-based Chatbot for an Emerging Market<\/strong><br \/>\nPuneet Agrawal (Microsoft); Manoj Kumar Chinnakotla (Microsoft)<span id=\"label-external-link\" class=\"sr-only\" aria-hidden=\"true\">Opens in a new tab<\/span><\/p>\n<p>The <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-academic-graph\/\">Microsoft Academic Graph<\/a> makes it possible to gain analytic insights about any of the entities within it: publications, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/authors\/0\/\" target=\"_blank\" rel=\"noopener\">authors<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/institutions\/0\/\" target=\"_blank\" rel=\"noopener\">institutions<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/topics\/0\/\" target=\"_blank\" rel=\"noopener\">topics<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/journals\/0\/\" target=\"_blank\" rel=\"noopener\">journals<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, and <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/conferences\/0\/\" target=\"_blank\" rel=\"noopener\">conferences<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>. Below, we present historical trend analysis about the SIGIR\u2013 Special Interest Group on Information Retrieval\u2013Conference.<\/p>\n<p>You can generate your own insights by accessing the Microsoft Academic Graph through the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/labs.cognitive.microsoft.com\/en-us\/project-academic-knowledge\" target=\"_blank\" rel=\"noopener\">Academic Knowledge API<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> or through <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/azure.microsoft.com\/en-us\/services\/storage\/data-lake-storage\/\" target=\"_blank\" rel=\"noopener\">Azure Data Lake Store<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (please <a href=\"mailto:academicapi@microsoft.com\">contact us<\/a> for the latter option). If you would like to learn how we generated the insights below, please see the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/Azure-Samples\/academic-knowledge-analytics-visualization\" target=\"_blank\" rel=\"noopener\">repository with source code<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>.<\/p>\n<p><em>Click on each image for current trends and data hosted by <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-academic-graph\/\" target=\"_blank\" rel=\"noopener\">Microsoft Academic Graph<\/a>.<\/em><\/p>\n<h2>SIGIR paper output<\/h2>\n<p>The chart below shows the evolution of the number of conference papers for each conference year.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSection\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492926 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/1-5b341f6a20917-1024x594.jpg\" alt=\"SIGIR Analytics\" width=\"1024\" height=\"594\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/1-5b341f6a20917-1024x594.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/1-5b341f6a20917-300x174.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/1-5b341f6a20917-768x445.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/1-5b341f6a20917.jpg 1801w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<p>In the following chart, the black bars represent average numbers of references per conference paper for each year. The data show that recent publications tend to cite more references. The green bars show the average number of citations of conference papers written in a given year. Note that the citations are raw counts and not normalized by the age of publications. This is because the \u201ccorrect\u201d way to normalize the citation counts turns out to be a nontrivial problem and may well be application dependent. Please treat the raw data presented as an invitation to conduct research on this topic!<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSection64300c408074167a6155\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492932 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/2-5b341fecb9887-1024x583.jpg\" alt=\"\" width=\"1024\" height=\"583\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/2-5b341fecb9887-1024x583.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/2-5b341fecb9887-300x171.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/2-5b341fecb9887-768x437.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/2-5b341fecb9887.jpg 1808w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<p>That being said, a visible trend is that older publications tend to receive more citations because they have more time for researchers to recognize the contributions of the paper. There are, however, notable exceptions, the first in 1994, due to several highly cited papers:<\/p>\n<ul>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2085989833\" target=\"_blank\" rel=\"noopener\">David D. Lewis, William A. Gale \u201cA sequential algorithm for training text classifiers.\u201d<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2014415866\" target=\"_blank\" rel=\"noopener\">Stephen E. Robertson, Steve Walker \u201cSome simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval.\u201d<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2105106523\" target=\"_blank\" rel=\"noopener\">Ellen M. Voorhees \u201cQuery expansion using lexical-semantic relations\u201d<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<\/ul>\n<p>The second result, in 1998 and 1999, sees the technique of language model for information retrieval being introduced, leading quite a few papers to be highly cited in the ensuing years. However, in 2000, when the concept of discounted cumulative gain (DCG) is first proposed, most citations of the work go to the journal version (TOIS) of the work published two years later. That might explain why there is a deep decline in the citation counts of SIGIR 2000 relative to adjacent years.<\/p>\n<h2>Memory of references<\/h2>\n<p>How old are the papers cited by SIGIR papers? Follow a given year\u2019s column to see the age of papers cited in conference papers published that year. For example, in 2017, SIGIR papers collectively cited 683 papers published in 2016, 657 papers published in 2015, and so on.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSection5aa6788f30dda3c4e571\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-491648 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-1024x526.jpg\" alt=\"\" width=\"1024\" height=\"526\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-1024x526.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-300x154.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-768x394.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3.jpg 1461w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<p><em>*If some years appear to cite publications from the future, it is most likely because they cited books. When a new edition of the book appeared, it replaced the previous one in the Microsoft Academic Graph and the citation appears to be from the future. In this representation, to generate a cleaner view, we removed all instances of papers citing papers more than two years in the future.<\/em><\/p>\n<h2>Outgoing references<\/h2>\n<p>What venues do SIGIR papers cite?<\/p>\n<p>The pie chart shows the top 10 venues cited by\u00a0SIGIR papers over time.\u00a0SIGIR, CIKM, and WWW emerge as the top three.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSection877cd069ea401bcc0b78\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492938 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-5b34216b5161e-1024x882.jpg\" alt=\"SIGIR Analytics - Top Referenced Venues\" width=\"1024\" height=\"882\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-5b34216b5161e-1024x882.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-5b34216b5161e-300x258.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-5b34216b5161e-768x662.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-5b34216b5161e.jpg 1098w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<p>The 100 percent stacked bar chart below shows the percent of references given by SIGIR\u00a0papers to each of the top 20 venues, year by year.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSection9e43685ae60ee089bd8d\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492941 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/4-5b3421f9a3e73-1024x585.jpg\" alt=\"SIGIR Analytics - Top Venue Reference Over Time\" width=\"1024\" height=\"585\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/4-5b3421f9a3e73-1024x585.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/4-5b3421f9a3e73-300x171.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/4-5b3421f9a3e73-768x439.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/4-5b3421f9a3e73.jpg 1808w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<h2>Incoming citations<\/h2>\n<p>What venues cite SIGIR papers?<\/p>\n<p>The pie chart below shows the top 10 venues of all time that cite SIGIR papers.\u00a0 SIGIR is the top one, followed by CIKM, and Information Processing and Management. See the table for year-by-year details of citations coming from each of the top 10 venues.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSection28f59f85bd62a0050642\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492944 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/5-5b34224942898-1024x813.jpg\" alt=\"SIGIR Analytics - Top Citing Venues\" width=\"1024\" height=\"813\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/5-5b34224942898-1024x813.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/5-5b34224942898-300x238.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/5-5b34224942898-768x610.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/5-5b34224942898.jpg 1184w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<p>The 100 percent stacked bar chart below shows the citation distribution from the top 20 citing venues, year by year.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSection94444bc9501c90220d25\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492947 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/6-5b3422dd1f418-1024x583.jpg\" alt=\"SIGIR Analytics - Top venues citations over time\" width=\"1024\" height=\"583\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/6-5b3422dd1f418-1024x583.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/6-5b3422dd1f418-300x171.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/6-5b3422dd1f418-768x437.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/6-5b3422dd1f418.jpg 1813w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<h2>Most-cited authors<\/h2>\n<p>Who are the most-cited authors of all time in SIGIR papers? The interactive chart below ranks the most-cited authors by using number of publications cited by the conference and number of citations received from the conference. Authors do not have to have published in SIGIR to appear on this chart.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSection040b54e3b60130152e09\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492950 size-full\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/7-5b342339bf8d7.jpg\" alt=\"SIGIR Analytics - Most-cited authors\" width=\"653\" height=\"482\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/7-5b342339bf8d7.jpg 653w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/7-5b342339bf8d7-300x221.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/7-5b342339bf8d7-80x60.jpg 80w\" sizes=\"auto, (max-width: 653px) 100vw, 653px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<p>Who are the rising stars among the top cited authors in SIGIR? The 100 percent stacked bar chart below shows the citation distribution by the top 20 authors, year by year.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSection1bca5b290bd7905a3072\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-491666 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-1024x581.jpg\" alt=\"\" width=\"1024\" height=\"581\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-1024x581.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-300x170.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-768x435.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9.jpg 1806w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<h2>Top institutions<\/h2>\n<p>The bubble chart visualizes the top institutions at SIGIR by citation count. The size of the bubble is proportional to the total number of publications from that institution at SIGIR.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSection4fd5e755853297631ae0\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492953 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/8-5b34237f1d30f-1024x881.jpg\" alt=\"SIGIR Analytics - Top Institutions\" width=\"1024\" height=\"881\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/8-5b34237f1d30f-1024x881.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/8-5b34237f1d30f-300x258.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/8-5b34237f1d30f-768x661.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/8-5b34237f1d30f.jpg 1060w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<p>Get the most current data and also explore the top institutions at the conference in more detail by clicking the chart. Once on the underlying Microsoft PowerBI dashboard, click on a column to rank the top institutions by publication or citation count.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSectioncd55d0220d0d9b53d015\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492956 size-full\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-5b3423bfaf4b7.jpg\" alt=\"SIGIR Analytics - Top institutions\" width=\"673\" height=\"543\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-5b3423bfaf4b7.jpg 673w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-5b3423bfaf4b7-300x242.jpg 300w\" sizes=\"auto, (max-width: 673px) 100vw, 673px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<h2>Top authors<\/h2>\n<p>The next three charts show author rankings according to different criteria.<\/p>\n<p>The bubble chart displays SIGIR authors ranked by citation count, with bubble size being relative to publication count.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSection93ab9db27cc380484980\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-491675 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/12-1024x831.jpg\" alt=\"SIGIR Conference Analytics - 12\" width=\"1024\" height=\"831\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/12-1024x831.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/12-300x244.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/12-768x623.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/12.jpg 1270w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<p>Get the most current data and also explore the top authors at the conference in more detail by clicking the chart. Once on the underlying Microsoft PowerBI dashboard, you can also explore the top conference authors in more detail. Click on a column to rank the top authors by Microsoft Academic rank, publication, or citation count.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSectionc28b3e42437b6370a536\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-491678 size-full\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/13.jpg\" alt=\"SIGIR Conference Analytics - 13\" width=\"736\" height=\"680\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/13.jpg 736w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/13-300x277.jpg 300w\" sizes=\"auto, (max-width: 736px) 100vw, 736px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<p>The bubble chart below visualizes author rank, which is calculated by Microsoft Academic by using a formula that is less susceptible to citation counts than similar measures. The X axis shows author rank. The higher an author\u2019s rank, the closer they are to the right side. The Y axis normalizes the rank by publication count and enables us to identify impactful authors who might not have had a very large number of publications. The closer an author is to the top, the higher their normalized rank. Of course, the area of the chart that represents the highest rank is the top right corner.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSectione8aeea536041799471c1\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-491681 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/14-1024x821.jpg\" alt=\"SIGIR Microsoft Analytics - 14\" width=\"1024\" height=\"821\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/14-1024x821.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/14-300x241.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/14-768x616.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/14.jpg 1282w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<p>Stephen Roberson is an interesting case. Although he is one of the most influential authors in the information retrieval field, he\u2019s only ranked at the 19<sup>th<\/sup> place for SIGIR conference. It turns out the Stephen\u2019s best work is not published at SIGIR. BM25F is published at CIKM in 2004 [1], then in a booklet in 2009 [2]. He got his fame mostly from Okapi, published first at 1994 TREC [3] through 1999 [4], again, at TREC. His most well-cited work at SIGIR is an approximation to 2-Poisson model [5], and a CAL paper with the Bing team using pseudo-relevance feedback [6] that is no longer in the production. He co-authored a paper questioning the use of language modeling techniques for IR [7] which, unfortunately, prevailed until today against his predictions.<\/p>\n<ol>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2085030399\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., et al. \u201cSimple BM25 Extension to Multiple Weighted Fields.\u201d Proceedings of the Thirteenth ACM International Conference on Information and Knowledge Management, 2004, pp. 42\u201349.<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2155482025\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., and Hugo Zaragoza. \u201cThe Probabilistic Relevance Framework: BM25 and Beyond.\u201d Foundations and Trends in Information Retrieval, vol. 3, no. 4, 2009, pp. 333\u2013389.<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/1482214997\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., et al. \u201cOkapi at TREC.\u201d Overview of the Third Text REtrieval Conference, no. 500207, 1994, pp. 109\u2013123.<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/1587004086\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., and Steve Walker. \u201cOkapi\/Keenbow at TREC-8.\u201d TREC, 1999.<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2014415866\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., and Steve Walker. \u201cSome Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval.\u201d Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1994, pp. 232\u2013241.<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2104049510\" target=\"_blank\" rel=\"noopener\">Cao, Guihong, et al. \u201cSelecting Good Expansion Terms for Pseudo-Relevance Feedback.\u201d Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2008, pp. 243\u2013250.<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/1997211290\" target=\"_blank\" rel=\"noopener\">Allan, James, et al. \u201cChallenges in Information Retrieval and Language Modeling: Report of a Workshop Held at the Center for Intelligent Information Retrieval, University of Massachusetts Amherst, September 2002.\u201d International ACM SIGIR Conference on Research and Development in Information Retrieval, vol. 37, no. 1, 2003, pp. 31\u201347.<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<\/ol>\n<p>We hope you have enjoyed the analytic insights into this conference made possible by the <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-academic-graph\/\" target=\"_blank\" rel=\"noopener\">Microsoft Academic Graph<\/a>! Please visit our <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-academic-graph\/\" target=\"_blank\" rel=\"noopener\">Microsoft Academic Graph<\/a> page to learn how you can use our knowledge graph to generate your own custom analytics about an institution, a topic, an author, a publication venue, or any combination of these.<span id=\"label-external-link\" class=\"sr-only\" aria-hidden=\"true\">Opens in a new tab<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Venue: Michigan League (opens in new tab) (Location on Campus Map (opens in new tab)) Website: SIGIR 2018 (opens in new tab)Opens in a new tab SIGIR is a major international forum for presentation of the latest state-of-the-art research and demonstration of new systems and methods for connecting people with information: from Web search engines, [&hellip;]<\/p>\n","protected":false},"featured_media":490106,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr_startdate":"2018-07-08","msr_enddate":"2018-07-12","msr_location":"Ann Arbor, Michigan, USA","msr_expirationdate":"","msr_event_recording_link":"","msr_event_link":"http:\/\/sigir.org\/sigir2018\/attend\/","msr_event_link_redirect":false,"msr_event_time":"","msr_hide_region":true,"msr_private_event":false,"msr_hide_image_in_river":0,"footnotes":""},"research-area":[13556,13555],"msr-region":[197900],"msr-event-type":[197941],"msr-video-type":[],"msr-locale":[268875],"msr-program-audience":[],"msr-post-option":[],"msr-impact-theme":[],"class_list":["post-489929","msr-event","type-msr-event","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-research-area-search-information-retrieval","msr-region-north-america","msr-event-type-conferences","msr-locale-en_us"],"msr_about":"<!-- wp:msr\/event-details {\"title\":\"Microsoft @ SIGIR 2018\",\"backgroundColor\":\"grey\",\"image\":{\"id\":490106,\"url\":\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/SIGIR_Conf_Header_06_2018_1920x720.jpg\",\"alt\":\"\"}} \/-->\n\n<!-- wp:msr\/content-tabs --><!-- wp:msr\/content-tab {\"title\":\"About\"} --><!-- wp:freeform --><p><strong>Venue:<\/strong> <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/uunions.umich.edu\/league\" target=\"_blank\" rel=\"noopener\">Michigan League<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/maps.studentlife.umich.edu\/building\/michigan-league\" target=\"_blank\" rel=\"noopener\">Location on Campus Map<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>)<\/p>\n<p><strong>Website:<\/strong> <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/sigir.org\/sigir2018\/\" target=\"_blank\" rel=\"noopener\">SIGIR 2018<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span id=\"label-external-link\" class=\"sr-only\" aria-hidden=\"true\">Opens in a new tab<\/span><\/p>\n<p>SIGIR is a major international forum for presentation of the latest state-of-the-art research and demonstration of new systems and methods for connecting people with information: from Web search engines, recommender systems, and social network technology to compelling applications in health, legal, educational, and other domains, research at SIGIR spans both academia and industry.<\/p>\n<h2>Program Committee members<\/h2>\n<p><a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/pauben\/\">Paul Bennett<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Short Paper Chair<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/jfgao\/\">Jianfeng Gao<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, AI Track Co-chair<\/p>\n<h2>Invited Speakers<\/h2>\n<p><strong>Distributional Representation of Complex Semantics<\/strong> (Keynote at <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/kg4ir.github.io\/\" target=\"_blank\" rel=\"noopener\">KG4IR workshop<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>)<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/kuansanw\/\" target=\"_blank\" rel=\"noopener\">Kuansan Wang<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<\/p>\n<p><strong>Lessons from Building a Large-scale Commercial IR-based Chatbot for an Emerging Market<\/strong><br \/>\nPuneet Agrawal and Manoj Kumar Chinnakotla, Microsoft<\/p>\n<p><strong>Causal Inference over Longitudinal Data to Support Expectation Exploration<\/strong><br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/emrek\/\">Emre Kiciman<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/jiyinhe.github.io\/ProfS2018\/\" target=\"_blank\" rel=\"noopener\"><strong>Search and Recommendation in the Enterprise<\/strong><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/pauben\/\" rel=\"noopener\">Paul Bennett<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<\/p>\n<h2>Workshops<\/h2>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" target=\"_blank\" href=\"http:\/\/lnd4ir.github.io\/\">Learning from Limit\/Noisy data for IR<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br \/>\nHamed Zamani (UMass Amherst), Mostafa Dehghani (Univ. of Amsterdam), Fernando Diaz (Microsoft Research \u2013 Montreal), Hang Li (Toutiao AI Lab), Nick Craswell (Microsoft)<\/p>\n<h2>Microsoft attendees<\/h2>\n<p>Amjad Abu-Jbara, Microsoft<br \/>\nOmar Alonso, Microsoft<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/hassanam\/\">Ahmed Awadallah<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/pauben\/\">Paul Bennett<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<br \/>\nEdward Cui, Microsoft<br \/>\nWeiwei Deng, Microsoft<br \/>\nFernando Diaz, Microsoft Research \u2013 Montreal<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/sdumais\/\">Susan Dumais<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/adamfo\/\">Adam Fourney<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research AI<br \/>\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/emrek\/\">Emre Kiciman<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Microsoft Research<br \/>\nXiaoliang Ling, Microsoft<br \/>\nPawel Pietrusinski, Microsoft<br \/>\nMona Soliman Habib, Microsoft<br \/>\nHui Su, Microsoft<\/p>\n<h2>Career Opportunities<\/h2>\n<h4><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/careers.microsoft.com\/us\/en\/job\/463001\/ML-Engineer\" target=\"_blank\" rel=\"noopener\">ML Engineer<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/h4>\n<p style=\"padding-left: 30px\">AI &amp; Research (AI&amp;R) at Hyderabad, India comprises of highly motivated researchers, engineers, product managers and data-scientists building end-to-end web-scale and enterprise-scale AI systems. We seek talented, energetic, creative and passionate ML engineers with ability to enhance and apply research to ship and build high-quality products and services.<\/p>\n<p><span id=\"label-external-link\" class=\"sr-only\" aria-hidden=\"true\">Opens in a new tab<\/span><\/p>\n<!-- \/wp:freeform --><!-- \/wp:msr\/content-tab --><!-- wp:msr\/content-tab {\"title\":\"Accepted Papers\"} --><!-- wp:freeform --><h2>Full Papers<\/h2>\n<p><strong>Calendar-Aware Proactive Email Recommendation<\/strong><br \/>\nQian Zhao (University of Minnesota); Paul Bennett (Microsoft); Adam Fourney (Microsoft); Anne Thompson (Microsoft); Shane Williams (Microsoft); Adam D. Troy (Microsoft); Susan Dumais (Microsoft)<\/p>\n<p><strong>Characterizing and Supporting Question Answering in Human-to-Human Communication<\/strong><br \/>\nXiao Yang (The Pennsylvania State University); Ahmed Hassan Awadallah (Microsoft); Madian Khabsa (Apple); Wei Wang (Microsoft); Miaosen Wang (Microsoft)<\/p>\n<p><strong>Deep Domain Adaptation Hashing with Adversarial Learning<\/strong><br \/>\nFuchen Long (University of Science and Technology of China); Ting Yao (Microsoft); Qi Dai (Microsoft); Xinmei Tian (University of Science and Technology of China); Jiebo Luo (University of Rochester); Tao Mei (Microsoft)<\/p>\n<p><strong>Measuring the Utility of Search Engine Result Pages<\/strong><br \/>\nLeif Azzopardi (University of Strathclyde); Paul Thomas (Microsoft); Nick Craswell (Microsoft)<\/p>\n<p><strong>Natural Language Interfaces with Fine-Grained User Interaction: A Case Study on Web APIs<\/strong><br \/>\nYu Su (University of California Santa Barbara); Ahmed Hassan Awadallah (Microsoft); Miaosen Wang (Microsoft); Ryen White (Microsoft)<\/p>\n<p><strong>Towards Better Text Understanding and Retrieval through Kernel Entity Salience Modeling<\/strong><br \/>\nChenyan Xiong (Carnegie Mellon University); Zhengzhong Liu (Carnegie Mellon University); Jamie Callan (Carnegie Mellon University); Tie-Yan Liu (Microsoft)<\/p>\n<h2>Short Papers<\/h2>\n<p><strong>Ad Click Prediction in sequence with Long Short-Term Memory Networks: An externality-aware model<\/strong><br \/>\nWeiwei Deng (Microsoft); Xiaoliang Ling (Microsoft); Yang Qi (Microsoft); Tunzi Tan (School of Mathematical Sciences @ University of Chinese Academy of Sciences); Eren Manavoglu (Microsoft); Qi Zhang (Microsoft)<\/p>\n<p><strong>Assessing the Readability of Web Search Results for Searchers with Dyslexia<\/strong><br \/>\nAdam Fourney (Microsoft); Meredith Ringel Morris (Microsoft); Abdullah Ali (University of Washington); Laura Vonessen (University of Washington)<\/p>\n<p><strong>Attention-driven Factor Model for Explainable Personalized Recommendation<\/strong><br \/>\nJingwu Chen (Institute of Computing Technology, Chinese Academy of Sciences); Fuzhen Zhuang (Institute of Computing Technology, Chinese Academy of Sciences); Xin Hong (Institute of Computing Technology, Chinese Academy of Sciences); Xiang Ao (Institute of Computing Technology, Chinese Academy of Sciences); Xing Xie (Microsoft); Qing He (Institute of Computing Technology, Chinese Academy of Sciences)<\/p>\n<p><strong>Cross Domain Regularization for Neural Ranking Models using Adversarial Learning<\/strong><br \/>\nDaniel Cohen (University of Massachusetts Amherst); Bhaskar Mitra (Microsoft); Katja Hofmann (Microsoft); Bruce Croft (University of Massachusetts Amherst)<\/p>\n<p><strong>Multi-level Abstraction Convolutional Model with Weak Supervision for Information Retrieval<\/strong><br \/>\nYifan Nie (University of Montreal); Alessandro Sordoni (Maluuba \u2013 Microsoft); Jian-Yun Nie (University of Montreal)<\/p>\n<p><strong>Optimizing Query Evaluations using Reinforcement Learning for Web Search<\/strong><br \/>\nCorby Rosset (Microsoft); Damien Jose (Microsoft); Gargi Ghosh (Microsoft); Bhaskar Mitra (Microsoft); Saurabh Tiwary (Microsoft)<\/p>\n<p><strong>Quantitative Information Extraction From Social Data<\/strong><br \/>\nOmar Alonso (Microsoft); Thibault Sellam (Columbia University)<\/p>\n<p><strong>Testing the Cluster Hypothesis with Focused and Graded Relevance Judgments<\/strong><br \/>\nEilon Sheetrit (Technion \u2013 Israel Institute of Technology); Anna Shtok (Technion \u2013 Israel Institute of Technology); Oren Kurland (Technion, Israel Institute of Technology); Igal Shprincis (Microsoft, Herzliya, Israel)<\/p>\n<p><strong>Transparent Tree Ensembles<\/strong><br \/>\nAlexander Moore (Microsoft); Vanessa Murdock (Microsoft); Yaxiong Cai (Microsoft); Kristine Jones (Microsoft)<\/p>\n<h2>SIRIP Industry Papers<\/h2>\n<p><strong>Puneet Agrawal and Manoj Kumar Chinnakotla. Lessons from Building a Large-scale Commercial IR-based Chatbot for an Emerging Market<\/strong><br \/>\nPuneet Agrawal (Microsoft); Manoj Kumar Chinnakotla (Microsoft)<span id=\"label-external-link\" class=\"sr-only\" aria-hidden=\"true\">Opens in a new tab<\/span><\/p>\n<!-- \/wp:freeform --><!-- \/wp:msr\/content-tab --><!-- wp:msr\/content-tab {\"title\":\"Conference Analytics\"} --><!-- wp:freeform --><p>The <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-academic-graph\/\">Microsoft Academic Graph<\/a> makes it possible to gain analytic insights about any of the entities within it: publications, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/authors\/0\/\" target=\"_blank\" rel=\"noopener\">authors<\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/institutions\/0\/\" target=\"_blank\" rel=\"noopener\">institutions<\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/topics\/0\/\" target=\"_blank\" rel=\"noopener\">topics<\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/journals\/0\/\" target=\"_blank\" rel=\"noopener\">journals<\/a>, and <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/conferences\/0\/\" target=\"_blank\" rel=\"noopener\">conferences<\/a>. Below, we present historical trend analysis about the SIGIR\u2013 Special Interest Group on Information Retrieval\u2013Conference.<\/p>\n<p>You can generate your own insights by accessing the Microsoft Academic Graph through the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/labs.cognitive.microsoft.com\/en-us\/project-academic-knowledge\" target=\"_blank\" rel=\"noopener\">Academic Knowledge API<\/a> or through <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/azure.microsoft.com\/en-us\/services\/storage\/data-lake-storage\/\" target=\"_blank\" rel=\"noopener\">Azure Data Lake Store<\/a> (please <a href=\"mailto:academicapi@microsoft.com\">contact us<\/a> for the latter option). If you would like to learn how we generated the insights below, please see the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/Azure-Samples\/academic-knowledge-analytics-visualization\" target=\"_blank\" rel=\"noopener\">repository with source code<\/a>.<\/p>\n<p><em>Click on each image for current trends and data hosted by <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-academic-graph\/\" target=\"_blank\" rel=\"noopener\">Microsoft Academic Graph<\/a>.<\/em><\/p>\n<h2>SIGIR paper output<\/h2>\n<p>The chart below shows the evolution of the number of conference papers for each conference year.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492926 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/1-5b341f6a20917-1024x594.jpg\" alt=\"SIGIR Analytics\" width=\"1024\" height=\"594\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/1-5b341f6a20917-1024x594.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/1-5b341f6a20917-300x174.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/1-5b341f6a20917-768x445.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/1-5b341f6a20917.jpg 1801w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/p>\n<p>In the following chart, the black bars represent average numbers of references per conference paper for each year. The data show that recent publications tend to cite more references. The green bars show the average number of citations of conference papers written in a given year. Note that the citations are raw counts and not normalized by the age of publications. This is because the \u201ccorrect\u201d way to normalize the citation counts turns out to be a nontrivial problem and may well be application dependent. Please treat the raw data presented as an invitation to conduct research on this topic!<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection64300c408074167a6155\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492932 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/2-5b341fecb9887-1024x583.jpg\" alt=\"\" width=\"1024\" height=\"583\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/2-5b341fecb9887-1024x583.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/2-5b341fecb9887-300x171.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/2-5b341fecb9887-768x437.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/2-5b341fecb9887.jpg 1808w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/p>\n<p>That being said, a visible trend is that older publications tend to receive more citations because they have more time for researchers to recognize the contributions of the paper. There are, however, notable exceptions, the first in 1994, due to several highly cited papers:<\/p>\n<ul>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2085989833\" target=\"_blank\" rel=\"noopener\">David D. Lewis, William A. Gale \u201cA sequential algorithm for training text classifiers.\u201d<\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2014415866\" target=\"_blank\" rel=\"noopener\">Stephen E. Robertson, Steve Walker \u201cSome simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval.\u201d<\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2105106523\" target=\"_blank\" rel=\"noopener\">Ellen M. Voorhees \u201cQuery expansion using lexical-semantic relations\u201d<\/a><\/li>\n<\/ul>\n<p>The second result, in 1998 and 1999, sees the technique of language model for information retrieval being introduced, leading quite a few papers to be highly cited in the ensuing years. However, in 2000, when the concept of discounted cumulative gain (DCG) is first proposed, most citations of the work go to the journal version (TOIS) of the work published two years later. That might explain why there is a deep decline in the citation counts of SIGIR 2000 relative to adjacent years.<\/p>\n<h2>Memory of references<\/h2>\n<p>How old are the papers cited by SIGIR papers? Follow a given year\u2019s column to see the age of papers cited in conference papers published that year. For example, in 2017, SIGIR papers collectively cited 683 papers published in 2016, 657 papers published in 2015, and so on.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection5aa6788f30dda3c4e571\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-491648 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-1024x526.jpg\" alt=\"\" width=\"1024\" height=\"526\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-1024x526.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-300x154.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-768x394.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3.jpg 1461w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/p>\n<p><em>*If some years appear to cite publications from the future, it is most likely because they cited books. When a new edition of the book appeared, it replaced the previous one in the Microsoft Academic Graph and the citation appears to be from the future. In this representation, to generate a cleaner view, we removed all instances of papers citing papers more than two years in the future.<\/em><\/p>\n<h2>Outgoing references<\/h2>\n<p>What venues do SIGIR papers cite?<\/p>\n<p>The pie chart shows the top 10 venues cited by\u00a0SIGIR papers over time.\u00a0SIGIR, CIKM, and WWW emerge as the top three.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection877cd069ea401bcc0b78\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492938 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-5b34216b5161e-1024x882.jpg\" alt=\"SIGIR Analytics - Top Referenced Venues\" width=\"1024\" height=\"882\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-5b34216b5161e-1024x882.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-5b34216b5161e-300x258.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-5b34216b5161e-768x662.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-5b34216b5161e.jpg 1098w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/p>\n<p>The 100 percent stacked bar chart below shows the percent of references given by SIGIR\u00a0papers to each of the top 20 venues, year by year.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection9e43685ae60ee089bd8d\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492941 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/4-5b3421f9a3e73-1024x585.jpg\" alt=\"SIGIR Analytics - Top Venue Reference Over Time\" width=\"1024\" height=\"585\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/4-5b3421f9a3e73-1024x585.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/4-5b3421f9a3e73-300x171.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/4-5b3421f9a3e73-768x439.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/4-5b3421f9a3e73.jpg 1808w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/p>\n<h2>Incoming citations<\/h2>\n<p>What venues cite SIGIR papers?<\/p>\n<p>The pie chart below shows the top 10 venues of all time that cite SIGIR papers.\u00a0 SIGIR is the top one, followed by CIKM, and Information Processing and Management. See the table for year-by-year details of citations coming from each of the top 10 venues.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection28f59f85bd62a0050642\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492944 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/5-5b34224942898-1024x813.jpg\" alt=\"SIGIR Analytics - Top Citing Venues\" width=\"1024\" height=\"813\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/5-5b34224942898-1024x813.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/5-5b34224942898-300x238.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/5-5b34224942898-768x610.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/5-5b34224942898.jpg 1184w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/p>\n<p>The 100 percent stacked bar chart below shows the citation distribution from the top 20 citing venues, year by year.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection94444bc9501c90220d25\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492947 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/6-5b3422dd1f418-1024x583.jpg\" alt=\"SIGIR Analytics - Top venues citations over time\" width=\"1024\" height=\"583\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/6-5b3422dd1f418-1024x583.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/6-5b3422dd1f418-300x171.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/6-5b3422dd1f418-768x437.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/6-5b3422dd1f418.jpg 1813w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/p>\n<h2>Most-cited authors<\/h2>\n<p>Who are the most-cited authors of all time in SIGIR papers? The interactive chart below ranks the most-cited authors by using number of publications cited by the conference and number of citations received from the conference. Authors do not have to have published in SIGIR to appear on this chart.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection040b54e3b60130152e09\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492950 size-full\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/7-5b342339bf8d7.jpg\" alt=\"SIGIR Analytics - Most-cited authors\" width=\"653\" height=\"482\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/7-5b342339bf8d7.jpg 653w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/7-5b342339bf8d7-300x221.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/7-5b342339bf8d7-80x60.jpg 80w\" sizes=\"auto, (max-width: 653px) 100vw, 653px\" \/><\/a><\/p>\n<p>Who are the rising stars among the top cited authors in SIGIR? The 100 percent stacked bar chart below shows the citation distribution by the top 20 authors, year by year.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection1bca5b290bd7905a3072\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-491666 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-1024x581.jpg\" alt=\"\" width=\"1024\" height=\"581\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-1024x581.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-300x170.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-768x435.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9.jpg 1806w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/p>\n<h2>Top institutions<\/h2>\n<p>The bubble chart visualizes the top institutions at SIGIR by citation count. The size of the bubble is proportional to the total number of publications from that institution at SIGIR.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection4fd5e755853297631ae0\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492953 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/8-5b34237f1d30f-1024x881.jpg\" alt=\"SIGIR Analytics - Top Institutions\" width=\"1024\" height=\"881\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/8-5b34237f1d30f-1024x881.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/8-5b34237f1d30f-300x258.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/8-5b34237f1d30f-768x661.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/8-5b34237f1d30f.jpg 1060w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/p>\n<p>Get the most current data and also explore the top institutions at the conference in more detail by clicking the chart. Once on the underlying Microsoft PowerBI dashboard, click on a column to rank the top institutions by publication or citation count.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSectioncd55d0220d0d9b53d015\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-492956 size-full\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-5b3423bfaf4b7.jpg\" alt=\"SIGIR Analytics - Top institutions\" width=\"673\" height=\"543\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-5b3423bfaf4b7.jpg 673w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-5b3423bfaf4b7-300x242.jpg 300w\" sizes=\"auto, (max-width: 673px) 100vw, 673px\" \/><\/a><\/p>\n<h2>Top authors<\/h2>\n<p>The next three charts show author rankings according to different criteria.<\/p>\n<p>The bubble chart displays SIGIR authors ranked by citation count, with bubble size being relative to publication count.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection93ab9db27cc380484980\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-491675 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/12-1024x831.jpg\" alt=\"SIGIR Conference Analytics - 12\" width=\"1024\" height=\"831\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/12-1024x831.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/12-300x244.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/12-768x623.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/12.jpg 1270w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/p>\n<p>Get the most current data and also explore the top authors at the conference in more detail by clicking the chart. Once on the underlying Microsoft PowerBI dashboard, you can also explore the top conference authors in more detail. Click on a column to rank the top authors by Microsoft Academic rank, publication, or citation count.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSectionc28b3e42437b6370a536\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-491678 size-full\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/13.jpg\" alt=\"SIGIR Conference Analytics - 13\" width=\"736\" height=\"680\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/13.jpg 736w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/13-300x277.jpg 300w\" sizes=\"auto, (max-width: 736px) 100vw, 736px\" \/><\/a><\/p>\n<p>The bubble chart below visualizes author rank, which is calculated by Microsoft Academic by using a formula that is less susceptible to citation counts than similar measures. The X axis shows author rank. The higher an author\u2019s rank, the closer they are to the right side. The Y axis normalizes the rank by publication count and enables us to identify impactful authors who might not have had a very large number of publications. The closer an author is to the top, the higher their normalized rank. Of course, the area of the chart that represents the highest rank is the top right corner.<\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSectione8aeea536041799471c1\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-491681 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/14-1024x821.jpg\" alt=\"SIGIR Microsoft Analytics - 14\" width=\"1024\" height=\"821\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/14-1024x821.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/14-300x241.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/14-768x616.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/14.jpg 1282w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/p>\n<p>Stephen Roberson is an interesting case. Although he is one of the most influential authors in the information retrieval field, he\u2019s only ranked at the 19<sup>th<\/sup> place for SIGIR conference. It turns out the Stephen\u2019s best work is not published at SIGIR. BM25F is published at CIKM in 2004 [1], then in a booklet in 2009 [2]. He got his fame mostly from Okapi, published first at 1994 TREC [3] through 1999 [4], again, at TREC. His most well-cited work at SIGIR is an approximation to 2-Poisson model [5], and a CAL paper with the Bing team using pseudo-relevance feedback [6] that is no longer in the production. He co-authored a paper questioning the use of language modeling techniques for IR [7] which, unfortunately, prevailed until today against his predictions.<\/p>\n<ol>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2085030399\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., et al. \u201cSimple BM25 Extension to Multiple Weighted Fields.\u201d Proceedings of the Thirteenth ACM International Conference on Information and Knowledge Management, 2004, pp. 42\u201349.<\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2155482025\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., and Hugo Zaragoza. \u201cThe Probabilistic Relevance Framework: BM25 and Beyond.\u201d Foundations and Trends in Information Retrieval, vol. 3, no. 4, 2009, pp. 333\u2013389.<\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/1482214997\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., et al. \u201cOkapi at TREC.\u201d Overview of the Third Text REtrieval Conference, no. 500207, 1994, pp. 109\u2013123.<\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/1587004086\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., and Steve Walker. \u201cOkapi\/Keenbow at TREC-8.\u201d TREC, 1999.<\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2014415866\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., and Steve Walker. \u201cSome Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval.\u201d Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1994, pp. 232\u2013241.<\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/2104049510\" target=\"_blank\" rel=\"noopener\">Cao, Guihong, et al. \u201cSelecting Good Expansion Terms for Pseudo-Relevance Feedback.\u201d Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2008, pp. 243\u2013250.<\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/academic.microsoft.com\/#\/detail\/1997211290\" target=\"_blank\" rel=\"noopener\">Allan, James, et al. \u201cChallenges in Information Retrieval and Language Modeling: Report of a Workshop Held at the Center for Intelligent Information Retrieval, University of Massachusetts Amherst, September 2002.\u201d International ACM SIGIR Conference on Research and Development in Information Retrieval, vol. 37, no. 1, 2003, pp. 31\u201347.<\/a><\/li>\n<\/ol>\n<p>We hope you have enjoyed the analytic insights into this conference made possible by the <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-academic-graph\/\" target=\"_blank\" rel=\"noopener\">Microsoft Academic Graph<\/a>! Please visit our <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-academic-graph\/\" target=\"_blank\" rel=\"noopener\">Microsoft Academic Graph<\/a> page to learn how you can use our knowledge graph to generate your own custom analytics about an institution, a topic, an author, a publication venue, or any combination of these.<span id=\"label-external-link\" class=\"sr-only\" aria-hidden=\"true\">Opens in a new tab<\/span><\/p>\n<!-- \/wp:freeform --><!-- \/wp:msr\/content-tab --><!-- \/wp:msr\/content-tabs -->","tab-content":[{"id":0,"name":"About","content":"SIGIR is a major international forum for presentation of the latest state-of-the-art research and demonstration of new systems and methods for connecting people with information: from Web search engines, recommender systems, and social network technology to compelling applications in health, legal, educational, and other domains, research at SIGIR spans both academia and industry.\r\n<h2>Program Committee members<\/h2>\r\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/pauben\/\">Paul Bennett<\/a>, Short Paper Chair\r\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/jfgao\/\">Jianfeng Gao<\/a>, AI Track Co-chair\r\n<h2>Invited Speakers<\/h2>\r\n<strong>Distributional Representation of Complex Semantics<\/strong> (Keynote at <a href=\"https:\/\/kg4ir.github.io\/\" target=\"_blank\" rel=\"noopener\">KG4IR workshop<\/a>)\r\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/kuansanw\/\" target=\"_blank\" rel=\"noopener\">Kuansan Wang<\/a>, Microsoft Research\r\n\r\n<strong>Lessons from Building a Large-scale Commercial IR-based Chatbot for an Emerging Market<\/strong>\r\nPuneet Agrawal and Manoj Kumar Chinnakotla, Microsoft\r\n\r\n<strong>Causal Inference over Longitudinal Data to Support Expectation Exploration<\/strong>\r\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/emrek\/\">Emre Kiciman<\/a>, Microsoft Research\r\n\r\n<a href=\"https:\/\/jiyinhe.github.io\/ProfS2018\/\" target=\"_blank\" rel=\"noopener\"><strong>Search and Recommendation in the Enterprise<\/strong><\/a>\r\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/pauben\/\" rel=\"noopener\">Paul Bennett<\/a>, Microsoft Research\r\n<h2>Workshops<\/h2>\r\n<a href=\"http:\/\/lnd4ir.github.io\/\">Learning from Limit\/Noisy data for IR<\/a>\r\nHamed Zamani (UMass Amherst), Mostafa Dehghani (Univ. of Amsterdam), Fernando Diaz (Microsoft Research \u2013 Montreal), Hang Li (Toutiao AI Lab), Nick Craswell (Microsoft)\r\n<h2>Microsoft attendees<\/h2>\r\nAmjad Abu-Jbara, Microsoft\r\nOmar Alonso, Microsoft\r\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/hassanam\/\">Ahmed Awadallah<\/a>, Microsoft Research\r\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/pauben\/\">Paul Bennett<\/a>, Microsoft Research\r\nEdward Cui, Microsoft\r\nWeiwei Deng, Microsoft\r\nFernando Diaz, Microsoft Research \u2013 Montreal\r\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/sdumais\/\">Susan Dumais<\/a>, Microsoft Research\r\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/adamfo\/\">Adam Fourney<\/a>, Microsoft Research AI\r\n<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/people\/emrek\/\">Emre Kiciman<\/a>, Microsoft Research\r\nXiaoliang Ling, Microsoft\r\nPawel Pietrusinski, Microsoft\r\nMona Soliman Habib, Microsoft\r\nHui Su, Microsoft\r\n<h2>Career Opportunities<\/h2>\r\n<h4><a href=\"https:\/\/careers.microsoft.com\/us\/en\/job\/463001\/ML-Engineer\" target=\"_blank\" rel=\"noopener\">ML Engineer<\/a><\/h4>\r\n<p style=\"padding-left: 30px\">AI &amp; Research (AI&amp;R) at Hyderabad, India comprises of highly motivated researchers, engineers, product managers and data-scientists building end-to-end web-scale and enterprise-scale AI systems. We seek talented, energetic, creative and passionate ML engineers with ability to enhance and apply research to ship and build high-quality products and services.<\/p>"},{"id":1,"name":"Accepted Papers","content":"<h2>Full Papers<\/h2>\r\n<strong>Calendar-Aware Proactive Email Recommendation<\/strong>\r\nQian Zhao (University of Minnesota); Paul Bennett (Microsoft); Adam Fourney (Microsoft); Anne Thompson (Microsoft); Shane Williams (Microsoft); Adam D. Troy (Microsoft); Susan Dumais (Microsoft)\r\n\r\n<strong>Characterizing and Supporting Question Answering in Human-to-Human Communication<\/strong>\r\nXiao Yang (The Pennsylvania State University); Ahmed Hassan Awadallah (Microsoft); Madian Khabsa (Apple); Wei Wang (Microsoft); Miaosen Wang (Microsoft)\r\n\r\n<strong>Deep Domain Adaptation Hashing with Adversarial Learning<\/strong>\r\nFuchen Long (University of Science and Technology of China); Ting Yao (Microsoft); Qi Dai (Microsoft); Xinmei Tian (University of Science and Technology of China); Jiebo Luo (University of Rochester); Tao Mei (Microsoft)\r\n\r\n<strong>Measuring the Utility of Search Engine Result Pages<\/strong>\r\nLeif Azzopardi (University of Strathclyde); Paul Thomas (Microsoft); Nick Craswell (Microsoft)\r\n\r\n<strong>Natural Language Interfaces with Fine-Grained User Interaction: A Case Study on Web APIs<\/strong>\r\nYu Su (University of California Santa Barbara); Ahmed Hassan Awadallah (Microsoft); Miaosen Wang (Microsoft); Ryen White (Microsoft)\r\n\r\n<strong>Towards Better Text Understanding and Retrieval through Kernel Entity Salience Modeling<\/strong>\r\nChenyan Xiong (Carnegie Mellon University); Zhengzhong Liu (Carnegie Mellon University); Jamie Callan (Carnegie Mellon University); Tie-Yan Liu (Microsoft)\r\n<h2>Short Papers<\/h2>\r\n<strong>Ad Click Prediction in sequence with Long Short-Term Memory Networks: An externality-aware model<\/strong>\r\nWeiwei Deng (Microsoft); Xiaoliang Ling (Microsoft); Yang Qi (Microsoft); Tunzi Tan (School of Mathematical Sciences @ University of Chinese Academy of Sciences); Eren Manavoglu (Microsoft); Qi Zhang (Microsoft)\r\n\r\n<strong>Assessing the Readability of Web Search Results for Searchers with Dyslexia<\/strong>\r\nAdam Fourney (Microsoft); Meredith Ringel Morris (Microsoft); Abdullah Ali (University of Washington); Laura Vonessen (University of Washington)\r\n\r\n<strong>Attention-driven Factor Model for Explainable Personalized Recommendation<\/strong>\r\nJingwu Chen (Institute of Computing Technology, Chinese Academy of Sciences); Fuzhen Zhuang (Institute of Computing Technology, Chinese Academy of Sciences); Xin Hong (Institute of Computing Technology, Chinese Academy of Sciences); Xiang Ao (Institute of Computing Technology, Chinese Academy of Sciences); Xing Xie (Microsoft); Qing He (Institute of Computing Technology, Chinese Academy of Sciences)\r\n\r\n<strong>Cross Domain Regularization for Neural Ranking Models using Adversarial Learning<\/strong>\r\nDaniel Cohen (University of Massachusetts Amherst); Bhaskar Mitra (Microsoft); Katja Hofmann (Microsoft); Bruce Croft (University of Massachusetts Amherst)\r\n\r\n<strong>Multi-level Abstraction Convolutional Model with Weak Supervision for Information Retrieval<\/strong>\r\nYifan Nie (University of Montreal); Alessandro Sordoni (Maluuba \u2013 Microsoft); Jian-Yun Nie (University of Montreal)\r\n\r\n<strong>Optimizing Query Evaluations using Reinforcement Learning for Web Search<\/strong>\r\nCorby Rosset (Microsoft); Damien Jose (Microsoft); Gargi Ghosh (Microsoft); Bhaskar Mitra (Microsoft); Saurabh Tiwary (Microsoft)\r\n\r\n<strong>Quantitative Information Extraction From Social Data<\/strong>\r\nOmar Alonso (Microsoft); Thibault Sellam (Columbia University)\r\n\r\n<strong>Testing the Cluster Hypothesis with Focused and Graded Relevance Judgments<\/strong>\r\nEilon Sheetrit (Technion \u2013 Israel Institute of Technology); Anna Shtok (Technion \u2013 Israel Institute of Technology); Oren Kurland (Technion, Israel Institute of Technology); Igal Shprincis (Microsoft, Herzliya, Israel)\r\n\r\n<strong>Transparent Tree Ensembles<\/strong>\r\nAlexander Moore (Microsoft); Vanessa Murdock (Microsoft); Yaxiong Cai (Microsoft); Kristine Jones (Microsoft)\r\n<h2>SIRIP Industry Papers<\/h2>\r\n<strong>Puneet Agrawal and Manoj Kumar Chinnakotla. Lessons from Building a Large-scale Commercial IR-based Chatbot for an Emerging Market<\/strong>\r\nPuneet Agrawal (Microsoft); Manoj Kumar Chinnakotla (Microsoft)"},{"id":2,"name":"Conference Analytics","content":"The <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-academic-graph\/\">Microsoft Academic Graph<\/a> makes it possible to gain analytic insights about any of the entities within it: publications, <a href=\"https:\/\/academic.microsoft.com\/#\/authors\/0\/\" target=\"_blank\" rel=\"noopener\">authors<\/a>, <a href=\"https:\/\/academic.microsoft.com\/#\/institutions\/0\/\" target=\"_blank\" rel=\"noopener\">institutions<\/a>, <a href=\"https:\/\/academic.microsoft.com\/#\/topics\/0\/\" target=\"_blank\" rel=\"noopener\">topics<\/a>, <a href=\"https:\/\/academic.microsoft.com\/#\/journals\/0\/\" target=\"_blank\" rel=\"noopener\">journals<\/a>, and <a href=\"https:\/\/academic.microsoft.com\/#\/conferences\/0\/\" target=\"_blank\" rel=\"noopener\">conferences<\/a>. Below, we present historical trend analysis about the SIGIR\u2013 Special Interest Group on Information Retrieval\u2013Conference.\r\n\r\nYou can generate your own insights by accessing the Microsoft Academic Graph through the <a href=\"https:\/\/labs.cognitive.microsoft.com\/en-us\/project-academic-knowledge\" target=\"_blank\" rel=\"noopener\">Academic Knowledge API<\/a> or through <a href=\"https:\/\/azure.microsoft.com\/en-us\/services\/storage\/data-lake-storage\/\" target=\"_blank\" rel=\"noopener\">Azure Data Lake Store<\/a> (please <a href=\"mailto:academicapi@microsoft.com\">contact us<\/a> for the latter option). If you would like to learn how we generated the insights below, please see the <a href=\"https:\/\/github.com\/Azure-Samples\/academic-knowledge-analytics-visualization\" target=\"_blank\" rel=\"noopener\">repository with source code<\/a>.\r\n\r\n<em>Click on each image for current trends and data hosted by <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-academic-graph\/\" target=\"_blank\" rel=\"noopener\">Microsoft Academic Graph<\/a>.<\/em>\r\n<h2>SIGIR paper output<\/h2>\r\nThe chart below shows the evolution of the number of conference papers for each conference year.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-492926 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/1-5b341f6a20917-1024x594.jpg\" alt=\"SIGIR Analytics\" width=\"1024\" height=\"594\" \/><\/a>\r\n\r\nIn the following chart, the black bars represent average numbers of references per conference paper for each year. The data show that recent publications tend to cite more references. The green bars show the average number of citations of conference papers written in a given year. Note that the citations are raw counts and not normalized by the age of publications. This is because the \u201ccorrect\u201d way to normalize the citation counts turns out to be a nontrivial problem and may well be application dependent. Please treat the raw data presented as an invitation to conduct research on this topic!\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection64300c408074167a6155\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-492932 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/2-5b341fecb9887-1024x583.jpg\" alt=\"\" width=\"1024\" height=\"583\" \/><\/a>\r\n\r\nThat being said, a visible trend is that older publications tend to receive more citations because they have more time for researchers to recognize the contributions of the paper. There are, however, notable exceptions, the first in 1994, due to several highly cited papers:\r\n<ul>\r\n \t<li><a href=\"https:\/\/academic.microsoft.com\/#\/detail\/2085989833\" target=\"_blank\" rel=\"noopener\">David D. Lewis, William A. Gale \u201cA sequential algorithm for training text classifiers.\u201d<\/a><\/li>\r\n \t<li><a href=\"https:\/\/academic.microsoft.com\/#\/detail\/2014415866\" target=\"_blank\" rel=\"noopener\">Stephen E. Robertson, Steve Walker \u201cSome simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval.\u201d<\/a><\/li>\r\n \t<li><a href=\"https:\/\/academic.microsoft.com\/#\/detail\/2105106523\" target=\"_blank\" rel=\"noopener\">Ellen M. Voorhees \u201cQuery expansion using lexical-semantic relations\u201d<\/a><\/li>\r\n<\/ul>\r\nThe second result, in 1998 and 1999, sees the technique of language model for information retrieval being introduced, leading quite a few papers to be highly cited in the ensuing years. However, in 2000, when the concept of discounted cumulative gain (DCG) is first proposed, most citations of the work go to the journal version (TOIS) of the work published two years later. That might explain why there is a deep decline in the citation counts of SIGIR 2000 relative to adjacent years.\r\n<h2>Memory of references<\/h2>\r\nHow old are the papers cited by SIGIR papers? Follow a given year\u2019s column to see the age of papers cited in conference papers published that year. For example, in 2017, SIGIR papers collectively cited 683 papers published in 2016, 657 papers published in 2015, and so on.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection5aa6788f30dda3c4e571\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-491648 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-1024x526.jpg\" alt=\"\" width=\"1024\" height=\"526\" \/><\/a>\r\n\r\n<em>*If some years appear to cite publications from the future, it is most likely because they cited books. When a new edition of the book appeared, it replaced the previous one in the Microsoft Academic Graph and the citation appears to be from the future. In this representation, to generate a cleaner view, we removed all instances of papers citing papers more than two years in the future.<\/em>\r\n<h2>Outgoing references<\/h2>\r\nWhat venues do SIGIR papers cite?\r\n\r\nThe pie chart shows the top 10 venues cited by\u00a0SIGIR papers over time.\u00a0SIGIR, CIKM, and WWW emerge as the top three.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection877cd069ea401bcc0b78\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-492938 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/3-5b34216b5161e-1024x882.jpg\" alt=\"SIGIR Analytics - Top Referenced Venues\" width=\"1024\" height=\"882\" \/><\/a>\r\n\r\nThe 100 percent stacked bar chart below shows the percent of references given by SIGIR\u00a0papers to each of the top 20 venues, year by year.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection9e43685ae60ee089bd8d\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-492941 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/4-5b3421f9a3e73-1024x585.jpg\" alt=\"SIGIR Analytics - Top Venue Reference Over Time\" width=\"1024\" height=\"585\" \/><\/a>\r\n<h2>Incoming citations<\/h2>\r\nWhat venues cite SIGIR papers?\r\n\r\nThe pie chart below shows the top 10 venues of all time that cite SIGIR papers.\u00a0 SIGIR is the top one, followed by CIKM, and Information Processing and Management. See the table for year-by-year details of citations coming from each of the top 10 venues.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection28f59f85bd62a0050642\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-492944 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/5-5b34224942898-1024x813.jpg\" alt=\"SIGIR Analytics - Top Citing Venues\" width=\"1024\" height=\"813\" \/><\/a>\r\n\r\nThe 100 percent stacked bar chart below shows the citation distribution from the top 20 citing venues, year by year.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection94444bc9501c90220d25\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-492947 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/6-5b3422dd1f418-1024x583.jpg\" alt=\"SIGIR Analytics - Top venues citations over time\" width=\"1024\" height=\"583\" \/><\/a>\r\n<h2>Most-cited authors<\/h2>\r\nWho are the most-cited authors of all time in SIGIR papers? The interactive chart below ranks the most-cited authors by using number of publications cited by the conference and number of citations received from the conference. Authors do not have to have published in SIGIR to appear on this chart.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection040b54e3b60130152e09\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-492950 size-full\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/7-5b342339bf8d7.jpg\" alt=\"SIGIR Analytics - Most-cited authors\" width=\"653\" height=\"482\" \/><\/a>\r\n\r\nWho are the rising stars among the top cited authors in SIGIR? The 100 percent stacked bar chart below shows the citation distribution by the top 20 authors, year by year.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection1bca5b290bd7905a3072\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-491666 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-1024x581.jpg\" alt=\"\" width=\"1024\" height=\"581\" \/><\/a>\r\n<h2>Top institutions<\/h2>\r\nThe bubble chart visualizes the top institutions at SIGIR by citation count. The size of the bubble is proportional to the total number of publications from that institution at SIGIR.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection4fd5e755853297631ae0\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-492953 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/8-5b34237f1d30f-1024x881.jpg\" alt=\"SIGIR Analytics - Top Institutions\" width=\"1024\" height=\"881\" \/><\/a>\r\n\r\nGet the most current data and also explore the top institutions at the conference in more detail by clicking the chart. Once on the underlying Microsoft PowerBI dashboard, click on a column to rank the top institutions by publication or citation count.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSectioncd55d0220d0d9b53d015\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-492956 size-full\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/9-5b3423bfaf4b7.jpg\" alt=\"SIGIR Analytics - Top institutions\" width=\"673\" height=\"543\" \/><\/a>\r\n<h2>Top authors<\/h2>\r\nThe next three charts show author rankings according to different criteria.\r\n\r\nThe bubble chart displays SIGIR authors ranked by citation count, with bubble size being relative to publication count.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSection93ab9db27cc380484980\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-491675 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/12-1024x831.jpg\" alt=\"SIGIR Conference Analytics - 12\" width=\"1024\" height=\"831\" \/><\/a>\r\n\r\nGet the most current data and also explore the top authors at the conference in more detail by clicking the chart. Once on the underlying Microsoft PowerBI dashboard, you can also explore the top conference authors in more detail. Click on a column to rank the top authors by Microsoft Academic rank, publication, or citation count.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSectionc28b3e42437b6370a536\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-491678 size-full\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/13.jpg\" alt=\"SIGIR Conference Analytics - 13\" width=\"736\" height=\"680\" \/><\/a>\r\n\r\nThe bubble chart below visualizes author rank, which is calculated by Microsoft Academic by using a formula that is less susceptible to citation counts than similar measures. The X axis shows author rank. The higher an author\u2019s rank, the closer they are to the right side. The Y axis normalizes the rank by publication count and enables us to identify impactful authors who might not have had a very large number of publications. The closer an author is to the top, the higher their normalized rank. Of course, the area of the chart that represents the highest rank is the top right corner.\r\n\r\n<a href=\"https:\/\/msit.powerbi.com\/view?r=eyJrIjoiOTc3ZGY0NDEtM2QwMy00MGVmLWExODgtZTk1YWVmYzNmYzNkIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&amp;pageName=ReportSectione8aeea536041799471c1\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-491681 size-large\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/14-1024x821.jpg\" alt=\"SIGIR Microsoft Analytics - 14\" width=\"1024\" height=\"821\" \/><\/a>\r\n\r\nStephen Roberson is an interesting case. Although he is one of the most influential authors in the information retrieval field, he\u2019s only ranked at the 19<sup>th<\/sup> place for SIGIR conference. It turns out the Stephen\u2019s best work is not published at SIGIR. BM25F is published at CIKM in 2004 [1], then in a booklet in 2009 [2]. He got his fame mostly from Okapi, published first at 1994 TREC [3] through 1999 [4], again, at TREC. His most well-cited work at SIGIR is an approximation to 2-Poisson model [5], and a CAL paper with the Bing team using pseudo-relevance feedback [6] that is no longer in the production. He co-authored a paper questioning the use of language modeling techniques for IR [7] which, unfortunately, prevailed until today against his predictions.\r\n<ol>\r\n \t<li><a href=\"https:\/\/academic.microsoft.com\/#\/detail\/2085030399\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., et al. \u201cSimple BM25 Extension to Multiple Weighted Fields.\u201d Proceedings of the Thirteenth ACM International Conference on Information and Knowledge Management, 2004, pp. 42\u201349.<\/a><\/li>\r\n \t<li><a href=\"https:\/\/academic.microsoft.com\/#\/detail\/2155482025\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., and Hugo Zaragoza. \u201cThe Probabilistic Relevance Framework: BM25 and Beyond.\u201d Foundations and Trends in Information Retrieval, vol. 3, no. 4, 2009, pp. 333\u2013389.<\/a><\/li>\r\n \t<li><a href=\"https:\/\/academic.microsoft.com\/#\/detail\/1482214997\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., et al. \u201cOkapi at TREC.\u201d Overview of the Third Text REtrieval Conference, no. 500207, 1994, pp. 109\u2013123.<\/a><\/li>\r\n \t<li><a href=\"https:\/\/academic.microsoft.com\/#\/detail\/1587004086\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., and Steve Walker. \u201cOkapi\/Keenbow at TREC-8.\u201d TREC, 1999.<\/a><\/li>\r\n \t<li><a href=\"https:\/\/academic.microsoft.com\/#\/detail\/2014415866\" target=\"_blank\" rel=\"noopener\">Robertson, Stephen E., and Steve Walker. \u201cSome Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval.\u201d Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1994, pp. 232\u2013241.<\/a><\/li>\r\n \t<li><a href=\"https:\/\/academic.microsoft.com\/#\/detail\/2104049510\" target=\"_blank\" rel=\"noopener\">Cao, Guihong, et al. \u201cSelecting Good Expansion Terms for Pseudo-Relevance Feedback.\u201d Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2008, pp. 243\u2013250.<\/a><\/li>\r\n \t<li><a href=\"https:\/\/academic.microsoft.com\/#\/detail\/1997211290\" target=\"_blank\" rel=\"noopener\">Allan, James, et al. \u201cChallenges in Information Retrieval and Language Modeling: Report of a Workshop Held at the Center for Intelligent Information Retrieval, University of Massachusetts Amherst, September 2002.\u201d International ACM SIGIR Conference on Research and Development in Information Retrieval, vol. 37, no. 1, 2003, pp. 31\u201347.<\/a><\/li>\r\n<\/ol>\r\nWe hope you have enjoyed the analytic insights into this conference made possible by the <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-academic-graph\/\" target=\"_blank\" rel=\"noopener\">Microsoft Academic Graph<\/a>! Please visit our <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-academic-graph\/\" target=\"_blank\" rel=\"noopener\">Microsoft Academic Graph<\/a> page to learn how you can use our knowledge graph to generate your own custom analytics about an institution, a topic, an author, a publication venue, or any combination of these."}],"msr_startdate":"2018-07-08","msr_enddate":"2018-07-12","msr_event_time":"","msr_location":"Ann Arbor, Michigan, USA","msr_event_link":"http:\/\/sigir.org\/sigir2018\/attend\/","msr_event_recording_link":"","msr_startdate_formatted":"July 8, 2018","msr_register_text":"Watch now","msr_cta_link":"http:\/\/sigir.org\/sigir2018\/attend\/","msr_cta_text":"Watch now","msr_cta_bi_name":"Event Register","featured_image_thumbnail":"<img width=\"960\" height=\"360\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/SIGIR_Conf_Header_06_2018_1920x720.jpg\" class=\"img-object-cover\" alt=\"Microsoft at SIGIR 2018\" decoding=\"async\" loading=\"lazy\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/SIGIR_Conf_Header_06_2018_1920x720.jpg 1920w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/SIGIR_Conf_Header_06_2018_1920x720-300x113.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/SIGIR_Conf_Header_06_2018_1920x720-768x288.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/SIGIR_Conf_Header_06_2018_1920x720-1024x384.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2018\/06\/SIGIR_Conf_Header_06_2018_1920x720-1600x600.jpg 1600w\" sizes=\"auto, (max-width: 960px) 100vw, 960px\" \/>","event_excerpt":"SIGIR is a major international forum for presentation of the latest state-of-the-art research and demonstration of new systems and methods for connecting people with information: from Web search engines, recommender systems, and social network technology to compelling applications in health, legal, educational, and other domains, research at SIGIR spans both academia and industry. Program Committee members Paul Bennett, Short Paper Chair Jianfeng Gao, AI Track Co-chair Invited Speakers Distributional Representation of Complex Semantics (Keynote at&hellip;","msr_research_lab":[199561,199565],"related-researchers":[],"msr_impact_theme":[],"related-academic-programs":[],"related-groups":[],"related-projects":[],"related-opportunities":[],"related-publications":[],"related-videos":[],"related-posts":[492719,493160,493454,493484],"_links":{"self":[{"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-event\/489929","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-event"}],"about":[{"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-event"}],"version-history":[{"count":6,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-event\/489929\/revisions"}],"predecessor-version":[{"id":1147104,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-event\/489929\/revisions\/1147104"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/media\/490106"}],"wp:attachment":[{"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/media?parent=489929"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=489929"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=489929"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=489929"},{"taxonomy":"msr-video-type","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-video-type?post=489929"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=489929"},{"taxonomy":"msr-program-audience","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-program-audience?post=489929"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=489929"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=489929"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}