{"id":694401,"date":"2020-10-01T09:39:47","date_gmt":"2020-10-01T16:39:47","guid":{"rendered":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/?p=694401"},"modified":"2020-10-06T16:28:44","modified_gmt":"2020-10-06T23:28:44","slug":"archai-can-design-your-neural-network-with-state-of-the-art-neural-architecture-search-nas","status":"publish","type":"post","link":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/blog\/archai-can-design-your-neural-network-with-state-of-the-art-neural-architecture-search-nas\/","title":{"rendered":"Archai can design your neural network with state-of-the-art neural architecture search (NAS)"},"content":{"rendered":"\n
\"\"\/<\/figure>\n\n\n\n

The goal of neural architecture search (NAS) (opens in new tab)<\/span><\/a> is to have computers automatically search for the best-performing neural networks. Recent advances in NAS methods have made it possible to build problem-specific networks that are faster, more compact, and less power hungry than their handcrafted counterparts. Unfortunately, many NAS methods rely on an array of tricks that aren\u2019t always documented in a way that\u2019s easy to discover. While these tricks result in neural networks with greater accuracy, they often cloud the performance of the search algorithm themselves. Since different NAS methods use different enhancements and some none at all, NAS techniques have become difficult for researchers to compare. The use of a variety of enhancements has also made NAS methods difficult to reproduce (opens in new tab)<\/span><\/a>. Once-promising methods may disappoint when an attempt is made to transfer them to other datasets. Additionally, engineers trying to use NAS often find it challenging to understand the implications of advertised advances because of a deluge of research claims, an inability to fairly compare methods side by side, fragmented code bases in research repos, hyperparameters that aren\u2019t carefully managed, and a lack of plug-and-play for individual techniques.<\/p>\n\n\n\n

\n\t
\n\t\t
\n\t\t\t\t\t\tSOURCE CODE<\/span>\n\t\t\tGitHub: Archai<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n

We\u2019ve sought to address many of these concerns with a goal of making state-of-the-art NAS research more widely usable. We\u2019ve asked, can we find the right abstractions to unify many of these methods? A unified NAS framework would help enable the adoption of NAS algorithms in industry and support reproducibility, as well as fair evaluation, in research. Such a framework would also accelerate algorithmic innovation by allowing the research community to pursue even higher ambitions in its application of NAS, as well as to conduct searches in novel spaces that might yield architectures we haven\u2019t yet imagined. With this goal in mind, we\u2019ve developed Archai<\/a>, an-open source project now available on GitHub. Archai<\/em>, short for Architecture A<\/em>I, means \u201cfirst principles,\u201d which captures the spirit of the work we\u2019re doing.<\/p>\n\n\n\n

Archai enables execution of standard NAS algorithms with a single command line. Currently, Differentiable Architecture Search (DARTS) (opens in new tab)<\/span><\/a>, Petridish<\/a>, Differentiable ArchiTecture Approximation (DATA) (opens in new tab)<\/span><\/a>, and eXperts Neural Architecture Search (XNAS) (opens in new tab)<\/span><\/a> are implemented. Archai makes it easy to add new algorithms, experiment with many well-known datasets, and add new datasets through unified interfaces. Additionally, Archai enables the isolation of hyperparameters via a configuration system that makes assumptions and settings explicit. The behaviors of architecture search systems are sensitive to these hyperparameters. With unified hyperparameter configuration controls, different algorithms can be tested on the same playing field.<\/p>\n\n\n\n\t

\n\t\t\n\n\t\t

\n\t\tSpotlight: Event Series<\/span>\n\t<\/p>\n\t\n\t

\n\t\t\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t\"Research\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t
\n\n\t\t\t\t\t\t\t\t\t

Microsoft Research Forum<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

Join us for a continuous exchange of ideas about research in the era of general AI. Watch the first four episodes on demand.<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t\t\n\t\t\t\t\t\t\tWatch on-demand\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t<\/div>\n\t<\/div>\n\t\n\n\n

At the core of Archai are several interfaces that provide abstractions for common components of NAS algorithms. This reduces the code duplication, making new algorithm development faster and easier. Archai also uses a common model description language based on YAML that is extensible and \u201ccompilable\u201d to a PyTorch model. Because all the algorithms share exactly the same components, including the ones for training and evaluation, they can be written more compactly. Having common components also sets the stage for fairer comparison and easier reproducibility.<\/p>\n\n\n\n

Key features of Archai<\/h3>\n\n\n\n