{"id":731851,"date":"2021-03-09T01:40:04","date_gmt":"2021-03-09T09:40:04","guid":{"rendered":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/?post_type=msr-project&#038;p=731851"},"modified":"2025-01-29T08:43:01","modified_gmt":"2025-01-29T16:43:01","slug":"project-litmus","status":"publish","type":"msr-project","link":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/project-litmus\/","title":{"rendered":"Project LITMUS"},"content":{"rendered":"<section class=\"mb-3 moray-highlight\">\n\t<div class=\"card-img-overlay mx-lg-0\">\n\t\t<div class=\"card-background  has-background-racing-green card-background--full-bleed\">\n\t\t\t\t\t<\/div>\n\t\t<!-- Foreground -->\n\t\t<div class=\"card-foreground d-flex mt-md-n5 my-lg-5 px-g px-lg-0\">\n\t\t\t<!-- Container -->\n\t\t\t<div class=\"container d-flex mt-md-n5 my-lg-5 align-self-center\">\n\t\t\t\t<!-- Card wrapper -->\n\t\t\t\t<div class=\"w-100 w-lg-col-5\">\n\t\t\t\t\t<!-- Card -->\n\t\t\t\t\t<div class=\"card material-md-card py-5 px-md-5\">\n\t\t\t\t\t\t<div class=\"card-body \">\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n<h1 class=\"wp-block-heading h2\" id=\"project-litmus\">Project LITMUS<\/h1>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n<h2 class=\"wp-block-heading\" id=\"litmus-linguistically-inspired-training-and-testing-of-multilingual-systems\">LITMUS: <strong>L<\/strong>inguistically <strong>I<\/strong>nspired <strong>T<\/strong>raining and testing of <strong>MU<\/strong>ltilingual <strong>S<\/strong>ystems<\/h2>\n\n\n\n<p>Transformer-based Language Models have revolutionized the field of NLP and have shown great improvements in various benchmarks and are being used to power various NLP applications today. Multilingual versions of these models can also potentially serve many low-resource languages for which labeled data is not available by making use of the zer0-shot paradigm. However, there remain challenges associated with deploying such models, including the question of evaluating the models across a wide variety of languages which may not be present in standard evaluation benchmarks.<\/p>\n\n\n\n<div class=\"wp-block-media-text has-video  has-vertical-margin-small  has-vertical-padding-none  is-stacked-on-mobile\"><figure class=\"wp-block-media-text__media video-wrapper\"><div class=\"yt-consent-placeholder\" role=\"region\" aria-label=\"Video playback requires cookie consent\" data-video-id=\"20dR8oKc9j0\" data-poster=\"https:\/\/img.youtube.com\/vi\/20dR8oKc9j0\/maxresdefault.jpg\"><iframe class=\"media-text__video\" data-src=\"https:\/\/www.youtube-nocookie.com\/embed\/20dR8oKc9j0?enablejsapi=1&rel=0\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen aria-hidden=\"true\" tabindex=\"-1\"><\/iframe><div class=\"yt-consent-placeholder__overlay\"><button class=\"yt-consent-placeholder__play\"><svg width=\"42\" height=\"42\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><g fill=\"none\" fill-rule=\"evenodd\"><circle fill=\"#000\" opacity=\".556\" cx=\"21\" cy=\"21\" r=\"21\"\/><path stroke=\"#FFF\" d=\"M27.5 22l-12 8.5v-17z\"\/><\/g><\/svg><span class=\"yt-consent-placeholder__label\">Video playback requires cookie consent<\/span><\/button><\/div><\/div><\/figure><div class=\"wp-block-media-text__content\">\n<p>The goal of Project LITMUS is to discover strategies to evaluate massive multilingual models and also suggest data collection and training strategies to improve the performance of these models.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-content-justification-left is-horizontal is-content-justification-left is-layout-flex wp-container-core-buttons-is-layout-7e5fce0a wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/github.com\/microsoft\/Litmus\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub<\/a><\/div>\n<\/div>\n<\/div><\/div>\n\n\n\n<p>You may also be interested in the other projects our group works on: Project <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/ellora\/\">ELLORA<\/a>&nbsp;(Enabling Low Resource Languages) and <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/melange\/\">Project M\u00e9lange<\/a> (Understanding Mixed Language).<\/p>\n\n\n","protected":false},"excerpt":{"rendered":"<p>The goal of Project LITMUS is to discover strategies to evaluate massive multilingual models and also suggest data collection and training strategies to improve the performance of these models.<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13545],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-731851","msr-project","type-msr-project","status-publish","hentry","msr-research-area-human-language-technologies","msr-locale-en_us","msr-archive-status-complete"],"msr_project_start":"2020-07-01","related-publications":[732127,812008,816697],"related-downloads":[],"related-videos":[1122060],"related-groups":[144940],"related-events":[],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[],"slides":[],"related-researchers":[{"type":"user_nicename","display_name":"Kalika Bali","user_id":32477,"people_section":"Section name 0","alias":"kalikab"},{"type":"user_nicename","display_name":"Tanuja Ganu","user_id":38883,"people_section":"Section name 0","alias":"taganu"},{"type":"user_nicename","display_name":"Sunayana Sitaram","user_id":37287,"people_section":"Section name 0","alias":"susitara"}],"msr_research_lab":[199562],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/731851","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":12,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/731851\/revisions"}],"predecessor-version":[{"id":1124850,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/731851\/revisions\/1124850"}],"wp:attachment":[{"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/media?parent=731851"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=731851"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=731851"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=731851"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=731851"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}