{"id":689079,"date":"2020-09-03T09:05:38","date_gmt":"2020-09-03T16:05:38","guid":{"rendered":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/?p=689079"},"modified":"2022-11-01T08:58:19","modified_gmt":"2022-11-01T15:58:19","slug":"expressive-pixels-a-new-visual-communication-platform-to-support-creativity-accessibility-and-innovation","status":"publish","type":"post","link":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/blog\/expressive-pixels-a-new-visual-communication-platform-to-support-creativity-accessibility-and-innovation\/","title":{"rendered":"Expressive Pixels: A new visual communication platform to support creativity, accessibility, and innovation"},"content":{"rendered":"<aside id=accordion-99af219b-8566-4640-aaf4-3043d0d9bb70 class=\"msr-table-of-contents-block accordion mb-5 pb-0\" data-bi-aN=\"table-of-contents\">\n\t<button class=\"btn btn-collapse bg-gray-100 mb-0 display-flex justify-content-between\" type=\"button\" data-mount=\"collapse\" data-target=\"#accordion-collapse-99af219b-8566-4640-aaf4-3043d0d9bb70\" aria-expanded=\"true\" aria-controls=\"accordion-collapse-99af219b-8566-4640-aaf4-3043d0d9bb70\">\n\t\t<span class=\"msr-table-of-contents-block__label subtitle\">In this article<\/span>\n\t\t<span class=\"msr-table-of-contents-block__current mr-4 text-gray-600 font-weight-normal\" aria-hidden=\"true\"><\/span>\n\t<\/button>\n\t<div id=\"accordion-collapse-99af219b-8566-4640-aaf4-3043d0d9bb70\" class=\"msr-table-of-contents-block__collapse-wrapper collapse show\" data-parent=\"#accordion-99af219b-8566-4640-aaf4-3043d0d9bb70\">\n\t\t<div class=\"accordion-body bg-gray-100 border-top pt-4\">\n\t\t\t<ol class=\"msr-table-of-contents-block__list\">\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"#origins-the-power-of-an-idea-to-enrich-communication-and-peoples-lives\" class=\"msr-table-of-contents-block__list-item-link\">Origins: The power of an idea to enrich communication and people\u2019s lives<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"#evolution-of-the-sparklet-and-platform-design-custom-built-hardware-and-firmware-for-broad-access-to-expressive-pixels\" class=\"msr-table-of-contents-block__list-item-link\">Evolution of the Sparklet and platform design: Custom-built hardware and firmware for broad access to Expressive Pixels<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"#expressive-pixels-and-microsoft-makecode-a-perfect-pairing\" class=\"msr-table-of-contents-block__list-item-link\">Expressive Pixels and Microsoft MakeCode: A perfect pairing<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"#building-an-interactive-community-for-both-creators-and-developers\" class=\"msr-table-of-contents-block__list-item-link\">Building an interactive community for both creators and developers<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"#creativity-is-collective-growing-the-reach-of-expressive-pixels\" class=\"msr-table-of-contents-block__list-item-link\">Creativity is collective: Growing the reach of Expressive Pixels<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t<\/ul>\n\t\t<\/div>\n\t<\/div>\n\t<span class=\"msr-table-of-contents-block__progress-bar\"><\/span>\n<\/aside>\n\n\n\n<p>The need to express oneself is innate for every person in the world, and its roots run through art, technology, communication, and the acts of learning and building things from the ground up. It\u2019s no coincidence, then, that a new platform being released by Microsoft Research, called <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/product\/expressive-pixels\/\">Expressive Pixels<\/a>, stems from this belief. Expressive Pixels introduces an authoring app combined with open-source firmware, peripherals, documentation, and APIs that allow users and makers to create animations and then display them on a range of colorful LED display devices. With applications in areas spanning from accessibility to productivity, creativity, and education, the technology\u2019s potential for growth is limited only by the imaginations of its users and community\u2014and <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/p\/expressive-pixels\/9mtc56w1rxqh?cid=msft_web_appsforwindows_spotlight&activetab=pivot:overviewtab\">the app can now be downloaded for free from the Microsoft store<\/a>.<\/p>\n\n\n\n<p>The concept began with a simple idea\u2014empowering individuals who require alternative tools for communication with others in their lives. The collaboration began in the Microsoft Research <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/group\/enable\/#:~:text=The%20Enable%20group%20in%20Microsoft%20Research%20creates%20assistive,the%20areas%20of%20eye-controlled%20or%20sound-based%20user%20interfaces.\">Enable Group<\/a> in 2015 and has since grown to include members of the Microsoft Research Redmond Lab and the Small, Medium, & Corporate Business team. Its unique path to realization has led the Expressive Pixels team to embrace the creation of hardware display devices that integrate with other maker devices, an educational opportunity with support for Microsoft MakeCode, and a full open-source release of the firmware for developers. To learn more about the Expressive Pixels journey, read on or explore one of its many facets by clicking on a topic above.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"yt-consent-placeholder\" role=\"region\" aria-label=\"Video playback requires cookie consent\" data-video-id=\"qTP_vLQSP9E\" data-poster=\"https:\/\/img.youtube.com\/vi\/qTP_vLQSP9E\/maxresdefault.jpg\"><iframe aria-hidden=\"true\" tabindex=\"-1\" title=\"Welcome to Microsoft Expressive Pixels (1 of 9)\" width=\"500\" height=\"281\" data-src=\"https:\/\/www.youtube-nocookie.com\/embed\/qTP_vLQSP9E?feature=oembed&rel=0&enablejsapi=1\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><div class=\"yt-consent-placeholder__overlay\"><button class=\"yt-consent-placeholder__play\"><svg width=\"42\" height=\"42\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><g fill=\"none\" fill-rule=\"evenodd\"><circle fill=\"#000\" opacity=\".556\" cx=\"21\" cy=\"21\" r=\"21\"\/><path stroke=\"#FFF\" d=\"M27.5 22l-12 8.5v-17z\"\/><\/g><\/svg><span class=\"yt-consent-placeholder__label\">Video playback requires cookie consent<\/span><\/button><\/div><\/div>\n<\/div><\/figure>\n\n\n\n<h2 id=\"origins-the-power-of-an-idea-to-enrich-communication-and-peoples-lives\">Origins: The power of an idea to enrich communication and people\u2019s lives<\/h2>\n\n\n\n<p>Expressive Pixels did not begin with a hardware or software concept. Instead, it emerged from a participatory design collaboration between the Enable Group and members of the ALS Community. Enable researchers and technologists partnered closely with people living with ALS (PALS), caregivers, families, clinicians, non-profit partners, and assistive technology companies to identify, design, and test new experiences and technologies designed to improve the lives of people and communities affected by speech and mobility disability.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"745\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EXP_Graphic-ALS-Collaborators-1024x745.jpg\" alt=\"A group of people are smiling and laughing while conversing in a room. \" class=\"wp-image-689121\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EXP_Graphic-ALS-Collaborators-1024x745.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EXP_Graphic-ALS-Collaborators-300x218.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EXP_Graphic-ALS-Collaborators-768x559.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EXP_Graphic-ALS-Collaborators.jpg 1100w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>Enable team members and collaborators from the ALS community spend quality time together at a meetup on the Microsoft campus in late 2017. Front row, from left to right: Tom Lowell, Ann Paradiso, Jeremy Best, and Hallie Phillips. Back row: Jon Campbell, Patra West, Dwayne Lamb, and Marc West.<\/figcaption><\/figure>\n\n\n\n<p>From the outset, the Enable team established an ethos of inclusive, user-centered design, deep empathy, and a scrappy, can-do approach to innovating and problem solving within an extreme set of constraints.<\/p>\n\n\n\n<p>The extended team has always embraced the spirit of designing not just <strong>for <\/strong>the user, but <strong>with <\/strong>the user, resulting in a fruitful, five-year collaboration with the PALS community on a range of <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/augmentative-and-alternative-communication\/#!publications\">research studies<\/a>, hackathon projects, exploratory prototypes, and released software.<\/p>\n\n\n\n\t<div class=\"border-bottom border-top border-gray-300 mt-5 mb-5 msr-promo text-center text-md-left alignwide\" data-bi-aN=\"promo\" data-bi-id=\"1141385\">\n\t\t\n\n\t\n\t<div class=\"row pt-3 pb-4 align-items-center\">\n\t\t\t\t\t\t<div class=\"msr-promo__media col-12 col-md-5\">\n\t\t\t\t<a class=\"bg-gray-300 display-block\" href=\"https:\/\/ai.azure.com\/labs\" aria-label=\"Azure AI Foundry Labs\" data-bi-cN=\"Azure AI Foundry Labs\" target=\"_blank\">\n\t\t\t\t\t<img decoding=\"async\" class=\"w-100 display-block\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2025\/06\/Azure-AI-Foundry_1600x900.jpg\" \/>\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"msr-promo__content p-3 px-5 col-12 col-md\">\n\n\t\t\t\t\t\t\t\t\t<h2 class=\"h4\">Azure AI Foundry Labs<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t<p id=\"azure-ai-foundry-labs\" class=\"large\">Get a glimpse of potential future directions for AI, with these experimental technologies from Microsoft Research.<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t<div class=\"wp-block-buttons justify-content-center justify-content-md-start\">\n\t\t\t\t\t<div class=\"wp-block-button\">\n\t\t\t\t\t\t<a href=\"https:\/\/ai.azure.com\/labs\" aria-describedby=\"azure-ai-foundry-labs\" class=\"btn btn-brand glyph-append glyph-append-chevron-right\" data-bi-cN=\"Azure AI Foundry Labs\" target=\"_blank\">\n\t\t\t\t\t\t\tAzure AI Foundry\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div><!--\/.msr-promo__content-->\n\t<\/div><!--\/.msr-promo__inner-wrap-->\n\t<\/div><!--\/.msr-promo-->\n\t\n\n\n<p>One central theme the team tackled was enhancing the experience of augmentative and alternative communication (AAC) systems, which allow people with limited mobility and speech to control computers and speech devices using their eyes, a head mouse, or other alternate forms of access based on individual needs. Many PALS rely on eye tracking or head mouse input for computer access and synthetic speech in the late stages of disease progression.<\/p>\n\n\n\n<p>It made sense for the team to invest heavily in eye tracking as a primary input because the eye muscles are typically more resilient to ALS than the muscles needed for movement, speech, swallowing, and breathing. Eye-tracking technologies and AAC systems have improved over the years, but the barriers to access, entry, and adoption remain insurmountable for many users, leading to device abandonment and social isolation when users are no longer able to speak or gesture.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/video\/galactic-bell-star-music-demo-expressive-pixels\/\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"591\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPX-Figure-2-Image-1024x591.jpg\" alt=\"A woman sits in front of a computer screen and controls a set of tubular bells using a control interface on a table. \" class=\"wp-image-689133\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPX-Figure-2-Image-1024x591.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPX-Figure-2-Image-300x173.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPX-Figure-2-Image-768x443.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPX-Figure-2-Image.jpg 1188w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption>The Hands-Free Music project is an inclusive music creation and performance suite, comprising multiple instruments, interfaces, and input modalities and output options. <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/video\/galactic-bell-star-music-demo-expressive-pixels\/\">The Galactic Bell Star<\/a>, pictured above is an inclusive musical instrument that can be played with multiple inputs, including eye tracking, adaptive switches, touch, and physical mallets. It uses integrated Sparklets to visualize the music.<\/figcaption><\/figure>\n\n\n\n<p>Internally, the Enable team worked closely with collaborators across the company, including researchers in the Microsoft Research <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/group\/ability\/\">Ability team<\/a> and engineers in the Microsoft Research Redmond Central Engineering Test Team. They knew that typing words and phrases letter-by-letter using just the eyes was time-consuming and error prone, which created several opportunities for innovation unique to Microsoft Research.<\/p>\n\n\n\n<p>The team suspected that secondary, LED-based displays could serve as an important adjunct to AAC-based communication by serving as a visual proxy for body language, social cues, and to convey emotion\u2014all critical aspects of nonverbal communication that can be compromised by ALS. They conducted research around expressivity and creative endeavors central to the human experience, such as playing or composing music.<\/p>\n\n\n\n<p>They developed <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/nam06.safelinks.protection.outlook.com\/?url=https%3A%2F%2Fnewed.any0.dpdns.org%2Fen-us%2Fresearch%2Fproject%2Fmicrosoft-hands-free-music%2F&data=02%7C01%7Cv-alhage%40microsoft.com%7C5371aa0e41fe4ce4db7208d84fd31afb%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637347115404663474&sdata=LRzn%2FAKUx9Vha%2FT%2Bm0uclGfPjaPbs9G8kGN8TknoVK4%3D&reserved=0\">hands-free, multi-modal interfaces and musical instruments<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, built an innovative speech keyboard with an integrated awareness display, and investigated various approaches to improve speed and accuracy for eye-tracking input. The speech keyboard was a key reference design for Windows Eye Control, and the awareness display seeded the idea for Expressive Pixels.<\/p>\n\n\n\n<h2 id=\"evolution-of-the-sparklet-and-platform-design-custom-built-hardware-and-firmware-for-broad-access-to-expressive-pixels\">Evolution of the Sparklet and platform design: Custom-built hardware and firmware for broad access to Expressive Pixels<\/h2>\n\n\n\n<p>After brainstorming with the Enable team on how to develop a prototyping environment to author animations for their awareness display, Gavin Jancke\u2014General Manager Engineering, Microsoft Research\u2014was hooked on this project right away, so much so that he devoted a couple years of his hobby time to make Expressive Pixels a reality. Jancke began by working on the animation authoring app for the project, but his role quickly evolved to hardware design when the team realized there was no existing displays of sufficient resolution or standalone capability in the market. As he got more involved, he came up with a new set of hardware designs and started prototyping.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"768\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPX-Blog-image-2-1024x768.jpg\" alt=\"A man peers through a microscope while soldering a small red motherboard. The view from the microscope is displayed on a laptop on the desk next to the microchip.\" class=\"wp-image-689127\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPX-Blog-image-2-1024x768.jpg 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPX-Blog-image-2-300x225.jpg 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPX-Blog-image-2-768x576.jpg 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPX-Blog-image-2-80x60.jpg 80w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPX-Blog-image-2-240x180.jpg 240w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPX-Blog-image-2.jpg 1163w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>Microsoft Research General Manager Engineering Gavin Jancke, pictured here reworking a component on the Sparklet motherboard, had to learn a wide range of new skills to help him design and build an LED device that would work well with Expressive Pixels. In the end, growing his skills in electrical engineering and developing firmware has allowed him to get involved in other projects at Microsoft in ways he wouldn\u2019t have before. <\/figcaption><\/figure>\n\n\n\n<p>The end goal, Jancke soon realized, required gaining knowledge of electrical engineering, which would lead to an improved hardware design that could reach more people. He set to work creating a new display device that could display much more resolution. The outcome is the Sparklet: a Bluetooth-capable high-resolution display, that was battery powered, in a single package. \u201cThe thing with the Sparklet is, no one has created an end-to-end device that is packaged up and ready to go,\u201d says Jancke of the hardware. This lowers the bar of entry for people without electronics knowledge to jump right in and experiment with the basic Expressive Pixels capabilities.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>\u201cThis is from schematics, PCB [printed circuit board] design, firmware which is C-level code running on the microprocessor, communications protocol which is Bluetooth and USB, all the way through to a full-blown fluid user interface experience in C#. This is what I call full stack!\u201d<\/p><cite>Gavin Jancke on the process of creating the Expressive Pixels platform<\/cite><\/blockquote>\n\n\n\n<p>The story of the Expressive Pixels doesn\u2019t end there. The team is working on new versions that can accomplish different tasks\u2014such as triggering animations synchronized on musical notes from a MIDI interface or from external accessibility-capable buttons and adaptive switches, for example. Plus, Expressive Pixels has many opportunities for makers and developers to work with and program LED devices beyond the Sparklet. Even for Jancke himself, this hardware work opened doors in his own career, enabling him to undertake other, more advanced hardware engineering projects in his day job. Designing the hardware and firmware for Sparklet gave him the confidence to face new challenges in <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/prism\/\">Passthrough AR<\/a>, &nbsp;<a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/project-emma\/\">Project Emma<\/a>, and <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/project-eclipse\/\">Project Eclipse<\/a>\u2014three projects he has worked on since where skills developed while working on Expressive Pixels have been integral.<\/p>\n\n\n\n\n\n<p>Going under the hood of Expressive Pixels reveals more about how it is designed to foster inclusive end-experiences for users with a variety of needs and goals\u2014many of its design considerations are made to get more people involved in more ways. The firmware incorporates a mix-and-match philosophy so that makers and developers can use parts based on their individual needs. The application logic implements the features of Expressive Pixels (such as animation rendering, file Storage, and behaviors). It is also designed with a hardware abstraction layer so that it can be used across different hardware implementations. This provides system-level capabilities that isolate the application logic from the underlying type of microprocessor and development toolchain (such as Arduino, Nordic SDK, or other platforms that makers use).<\/p>\n\n\n\n<p>More specifically, the firmware includes support for Bluetooth and USB connectivity, enlisting a COBS (Consistent Overhead Byte Stuffing) streaming protocol to enable robust communications between two devices. The Expressive Pixels authoring app produces animations using a compression video codec that allows for optimizing the transmission and storage of image frames for constrained microcontrollers, and there\u2019s support for storing multiple animations on devices. See Figure 1 below for a visual layout of the design, and explore the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/microsoft\/ExpressivePixels\/wiki\">Expressive Pixels documentation<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> for more details.<\/p>\n\n\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"577\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-1024x577.png\" alt=\"A box on the left shows external activations: Bluetooth, USB, a red button, and MIDI. Bluetooth and USB connect to the display device through a COBS streaming interface in the firmware. The app also integrates with the streaming protocol via Bluetooth or USB. Buttons and MIDI connect to the display device through Expressive Pixels Core and Expressive Pixels Platform in the firmware. EPX animation format in the firmware integrates with the display device. Finally, the app also intergrates with MakeCode devices via Buetooth and USB using the EPX animation format. \" class=\"wp-image-689352\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-1024x577.png 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-300x169.png 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-768x433.png 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-1066x600.png 1066w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-655x368.png 655w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-343x193.png 343w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-640x360.png 640w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-960x540.png 960w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-1280x720.png 1280w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo.png 1400w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>Figure 1: Expressive Pixels platform diagram, consisting of external activation devices and connections, the app, firmware, LED display device, and MakeCode devices. The firmware is made up of a COBS Streaming protocol and two-part firmware: ExpressivePixelsCore and ExpressivePixelsPlatform.<\/figcaption><\/figure>\n\n\n\n<h2 id=\"expressive-pixels-and-microsoft-makecode-a-perfect-pairing\">Expressive Pixels and Microsoft MakeCode: A perfect pairing<\/h2>\n\n\n\n<p>After developing the authoring app and firmware, the team saw the potential fun factor that it opened up to makers. However, there was something more that began to take root throughout the process\u2014the power of learning and experimentation inherent in the design of the platform.<\/p>\n\n\n\n<p>After creating the user interface for Expressive Pixels and refining the compelling visual nature of animated LED displays, the team realized there was an untapped educational resource in <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/makecode\">Microsoft MakeCode<\/a>, which could be integrated with Expressive Pixels. The goal here was twofold: create a way for the maker community to easily adapt the platform for their own purposes and encourage people to develop their own scenarios around expression. The authoring environment of MakeCode, with block-style programming, gives those who are interested in learning the basics of code an accessible entry point.<\/p>\n\n\n\n<p>MakeCode lowers the barrier for entry for programming small devices, such as the BBC micro:bit, which are usually relegated to those who learn C programming. The barrier is lowered to such a point that people who are completely new to programming can create new inventions with relative ease. Integrating displays with expressive capability opens up a new range of creative and inventive scenarios, and it allows makers a new range of potential for experimentation. For example, a small robot could display a small stop sign when the robot came into a pre-determined proximity with an object.<\/p>\n\n\n\n<p>Developments in MakeCode and connecting to other hardware could also lead to new opportunities in accessibility and beyond. With the power of MakeCode and Expressive Pixels together, people with little knowledge of code can start to use the technology in tangential areas like costume design and fashion.<\/p>\n\n\n\n<p>So what does it take for Expressive Pixels to link up with MakeCode and the micro:bit? About 4 MakeCode jigsaw programming blocks. Ultimately, connecting Expressive Pixels to MakeCode creates the opportunity for people to learn in a fun way and for makers to expand the possibilities of the platform. This vision is one of collaboration at large scale, a way for people to learn and grow the technology at the same time.<\/p>\n\n\n\n<h2 id=\"building-an-interactive-community-for-both-creators-and-developers\">Building an interactive community for both creators and developers<\/h2>\n\n\n\n<p>The beauty of Expressive Pixels is that there are many ways for various people to engage with the technology\u2014from makers to learners to artists and more. With Expressive Pixels, the platform encourages people to become creators. From the bottom up, the Expressive Pixels app and its hardware considerations were designed to give as many people access to the underlying technology as possible. It\u2019s easy to get started, but there is a depth built into the platform that encourages even the most serious developers to get involved and lend their talents.<\/p>\n\n\n\n<p>The <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/microsoft\/ExpressivePixels\">firmware source code is now open source and available on GitHub<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, so if you are a developer interested in modifying or experimenting with the code in ways beyond the limits of the app, the resources are available for creating the next evolutions of Expressive Pixels now.<\/p>\n\n\n\n<h2 id=\"creativity-is-collective-growing-the-reach-of-expressive-pixels\">Creativity is collective: Growing the reach of Expressive Pixels<\/h2>\n\n\n\n<p>There are many different groups and areas that Expressive Pixels has encountered on its path to being released, but one thing that the team has certainly learned along the way: the more people involved, the merrier. In collaboration with people who have a wide range of backgrounds, abilities, and perspectives, the team is looking out for new ways of making this technology work to meet a diverse set of needs. There are future irons in the fire.<\/p>\n\n\n\n<p>Members of the Expressive Pixels team are hustling to test 3D-printed mounts and various other designs for makers and the community to use for modification, and they have already created prototypes for alternate switch access and eye tracking triggering. Designs for these are and will continue to be released for makers and creatives to use and modify along with the software. Down the road, the dream for Expressive Pixels includes integration with Windows Eye Control and other universal productivity scenarios\u2014imagine being able to sync your Sparklet with your Microsoft Teams and Outlook status to show people outside your physical office or room whether you\u2019re available or busy. The team has already started testing these out in their home offices.<\/p>\n\n\n\n<p>Coming full circle, the Enable Group has recently been mentoring student teams at University of Washington and University College London to develop a collection of eye tracking-based applications built on top of eye-tracking APIs and samples the team released for developers. Projects included an eye-tracking piano interface, a drawing app that allows users to create mandalas and other shapes with their eyes, and two eye-tracking games. They plan to introduce Expressive Pixels to the next group of students and are excited to see where their imaginations will take the platform in the future.<\/p>\n\n\n\n<p><strong>Acknowledgments<\/strong><\/p>\n\n\n\n<p><em>Expressive Pixels and its winding journey could not have been possible without so many people. We are forever inspired by the collaborators, partners, advisors, and friends who have dedicated their time and energy in service to our mission. Learn more about our contributors at the <a href=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/project\/microsoft-expressive-pixels\/people\/\" target=\"_blank\" rel=\"noreferrer noopener\">Expressive Pix<\/a>els team page.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The need to express oneself is innate for every person in the world, and its roots run through art, technology, communication, and the acts of learning and building things from the ground up. It\u2019s no coincidence, then, that a new platform being released by Microsoft Research, called Expressive Pixels, stems from this belief. Expressive Pixels [&hellip;]<\/p>\n","protected":false},"author":38838,"featured_media":689352,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[],"msr_hide_image_in_river":0,"footnotes":""},"categories":[1],"tags":[],"research-area":[13551,13554,13560],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-689079","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-graphics-and-multimedia","msr-research-area-human-computer-interaction","msr-research-area-programming-languages-software-engineering","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[144928],"related-projects":[439290],"related-events":[],"related-researchers":[],"msr_type":"Post","featured_image_thumbnail":"<img width=\"960\" height=\"540\" src=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-960x540.png\" class=\"img-object-cover\" alt=\"\" decoding=\"async\" loading=\"lazy\" srcset=\"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-960x540.png 960w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-300x169.png 300w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-1024x577.png 1024w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-768x433.png 768w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-1066x600.png 1066w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-655x368.png 655w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-343x193.png 343w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-640x360.png 640w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo-1280x720.png 1280w, https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-content\/uploads\/2020\/09\/1400x788_Expressive_Pixels_Still_NoLogo.png 1400w\" sizes=\"auto, (max-width: 960px) 100vw, 960px\" \/>","byline":"","formattedDate":"September 3, 2020","formattedExcerpt":"The need to express oneself is innate for every person in the world, and its roots run through art, technology, communication, and the acts of learning and building things from the ground up. It\u2019s no coincidence, then, that a new platform being released by Microsoft&hellip;","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/posts\/689079","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/users\/38838"}],"replies":[{"embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/comments?post=689079"}],"version-history":[{"count":44,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/posts\/689079\/revisions"}],"predecessor-version":[{"id":894471,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/posts\/689079\/revisions\/894471"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/media\/689352"}],"wp:attachment":[{"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/media?parent=689079"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/categories?post=689079"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/tags?post=689079"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=689079"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=689079"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=689079"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=689079"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=689079"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=689079"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=689079"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=689079"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}