Intel’s neuromorphic computing initiative

Dubbed Pohoiki Springs, the new system based on Intel Labs’ “Loihi” neuromorphic processor is expected to be available to the research community soon. It will contain up to 768 Loihi chips, totalling 100 million neurons. Introduced in November 2017, Loihi is Intel’s fifth and most complex chip in a family of different neuromorphic devices, and its architecture is optimized for Spiking Neural Networks (SNNs). Compared to ‘regular’ Artificial Neural Networks, SNNs can be considered more ‘similar’ to biological neural networks in that they incorporate time as an explicit dependency in computations, and their neurons will fire (produce a spike) only when certain parameters reach a specific threshold. In the outgoing ‘spike train’, information is represented by the frequency of spikes or the timing between them. SNNs promise great benefits over ‘regular’ ANNs in terms of performance and power consumption. Among the reasons currently preventing a wider adoption of SNNs in practical AI applications is that conventional processing architectures – such as CPUs and GPUs – are not ideally suited to implement these networks. Hence, the need for SNN-specialized architectures such as the Loihi chip.

The Loihi chip. Image credit: Intel

The Loihi chip

Loihi is a 60 mm2 chip fabricated in Intel’s 14nm FinFET process, containing 2.07 billion transistors. It offers 128 neuromorphic cores supporting up to 128,000 neurons and 128 million synapses. The architecture (described in a paper published on IEEE Micro, January 2018) also includes three embedded x86 processor cores, a Network-on-Chip that transports all communication between cores in the form of packetized messages, and memories that store the connectivity, configuration, and dynamic state of all neurons mapped to the cores. The chip integrates a wide range of novel features for the field, such as hierarchical connectivity, dendritic compartments, synaptic delays, and programmable synaptic learning rules. A fully digital and asynchronous architecture, Loihi approximates the continuous time dynamics of SNNs through a fixed-size discrete timestep model. However, instead of using a clock it uses a ‘barrier synchronization mechanism’ where the updates of all synaptic state are delayed until the end of a periodic learning epoch time. This also eliminates the power waste associated with a continuously running clock. The Intel Labs paper also describes the solutions adopted to overcome a number of architectural hurdles, such as the internal connectivity challenge. Starting in fourth quarter 2017, Intel has developed four different Loihi-based systems: Wolf Mountain, with four processors, for remote access; Nahuku, an Arria10 expansion board with up to 32 processors, for cloud and local use; Kapoho Bay, with one or two processors and a USB host interface; and the new, already-mentioned Pohoiki Springs system. Intel is also providing a specific software development kit for Loihi.

Wolf Mountain system. Image credit: Intel Corporation

Kapoho Bay system. Image credit: Intel Corporation

The advantages of spike-based computation

The above-mentioned Intel Labs’ paper also includes some significant test results. Running a spiking convolutional form of the Locally Competitive Algorithm, Loihi can solve LASSO (least absolute shrinkage and selection operator) optimization problems with over three orders of magnitude superior energy-delay-product compared to conventional solvers running on a technologically comparable CPU (same process, voltage supply and die area). According to Intel, this test provides an unambiguous example of spike-based computation outperforming all known conventional solutions. Since the publication of that paper, new researches on Loihi have been carried out throughout the world. Some of them were briefly described by Mike Davies, Director of the Neuromorphic Computing Lab at Intel Labs, in a speech at the Neuro-Inspired Computational Elements Workshop (NICE) that was held last March in Albany, New York. According to Davies, sparse communication in time optimizes energy efficiency; also, spikes efficiently compute many rate-based models; provide efficient and natural processing of temporal data; support event-based algorithms that have nothing to do with rates; and efficiently implement phasor networks. In all examples studied by Intel so far, benefits of spiking networks compared to conventional architectures increase with increasing problem scale. Researches quoted by Davies include a keyword spotting application where Loihi proved to be the most energy-efficient architecture for real-time inference, providing 5-10x lower energy than closest conventional DNN architecture; an application on MNIST image processing where the Intel neuromorphic chip achieved the same accuracy level of a LSTM (Long short-term memory) network; and a robot arm application with results outperforming standard PD & PID control algorithms. Davies also mentioned some examples of future research directions, such as unsupervised dictionary learning, generalization to data manifold learning, hierarchical LCA for adversarial-robust inference. A specific research direction concerns the quest for a ‘neuromorphic version’ of error backpropagation – the algorithm currently used to train ‘regular’ neural networks, which, though very effective, has no direct equivalent in biological brains. The Loihi research community is also addressing this issue, with works about eligibility propagation, surrogate gradient learning, and dendritic cortical microcircuits that approximate the backpropagation algorithm.

The Intel Neuromorphic Research Community

On March 1^st, 2018, the Santa Clara microprocessor giant announced the formation of the ‘Intel Neuromorphic Research Community’ (INRC), aimed at creating a network of collaborators spanning academic, government and industry research groups. Research vectors pursued by INRC include neuromorphic theory (abstracting neuroscience understanding and relating it to practical computational models); spiking neural network algorithms (development of neural network dynamics, features and learning rules); neuromorphic applications (systems and software that use Loihi and future Intel neuromorphic architectures to solve real-world problems); programming models (new paradigms for conceptualizing and specifying the structure and emergent behavior of neuromorphic systems); and event-driven sensing and control technologies (novel and efficient approaches for interfacing spike-based computing systems with real-world data and systems). Qualified proponents may receive grants for their work as well as access to a software development kit and a Loihi test system. Due to high demand for Loihi development systems, Intel is working to enable cloud access to a scalable, multiuser Loihi-based system, dubbed ‘Neuromorphic Research as a Service’ (NRaaS). As of March 2019, the INRC community involved more than fifty organizations in industry and academia, working on more than forty projects. The common goal: advancing neuromorphic computing from a promise to a competitive technology.

This entry was posted on Friday, July 5th, 2019 at 7:11 am. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.