Part of the  

Solid State Technology

  and   

The Confab

  Network

About  |  Contact

Posts Tagged ‘AI’

Deep Learning Could Boost Yields, Increase Revenues

Thursday, March 23rd, 2017

thumbnail

By Dave Lammers, Contributing Editor

While it is still early days for deep-learning techniques, the semiconductor industry may benefit from the advances in neural networks, according to analysts and industry executives.

First, the design and manufacturing of advanced ICs can become more efficient by deploying neural networks trained to analyze data, though labelling and classifying that data remains a major challenge. Also, demand will be spurred by the inference engines used in smartphones, autos, drones, robots and other systems, while the processors needed to train neural networks will re-energize demand for high-performance systems.

Abel Brown, senior systems architect at Nvidia, said until the 2010-2012 time frame, neural networks “didn’t have enough data.” Then, a “big bang” occurred when computing power multiplied and very large labelled data sets grew at Amazon, Google, and elsewhere. The trifecta was complete with advances in neural network techniques for image, video, and real-time voice recognition, among others.

During the training process, Brown noted, neural networks “figure out the important parts of the data” and then “converge to a set of significant features and parameters.”

Chris Rowen, who recently started Cognite Ventures to advise deep-learning startups, said he is “becoming aware of a lot more interest from the EDA industry” in deep learning techniques, adding that “problems in manufacturing also are very suitable” to the approach.

Chris Rowen, Cognite Ventures

For the semiconductor industry, Rowen said, deep-learning techniques are akin to “a shiny new hammer” that companies are still trying to figure out how to put to good use. But since yield questions are so important, and the causes of defects are often so hard to pinpoint, deep learning is an attractive approach to semiconductor companies.

“When you have masses of data, and you know what the outcome is but have no clear idea of what the causality is, (deep learning) can bring a complex model of causality that is very hard to do with manual methods,” said Rowen, an IEEE fellow who earlier was the CEO of Tensilica Inc.

The magic of deep learning, Rowen said, is that the learning process is highly automated and “doesn’t require a fab expert to look at the particular defect patterns.”

“It really is a rather brute force, naïve method. You don’t really know what the constituent patterns are that lead to these particular failures. But if you have enough examples that relate inputs to outputs, to defects or to failures, then you can use deep learning.”

Juan Rey, senior director of engineering at Mentor Graphics, said Mentor engineers have started investigating deep-learning techniques which could improve models of the lithography process steps, a complex issue that Rey said “is an area where deep neural networks and machine learning seem to be able to help.”

Juan Rey, Mentor Graphics

In the lithography process “we need to create an approximate model of what needs to be analyzed. For example, for photolithography specifically, there is the transition between dark and clear areas, where the slope of intensity for that transition zone plays a very clear role in the physics of the problem being solved. The problem tends to be that the design, the exact formulation, cannot be used in every space, and we are limited by the computational resources. We need to rely on a few discrete measurements, perhaps a few tens of thousands, maybe more, but it still is a discrete data set, and we don’t know if that is enough to cover all the cases when we model the full chip,” he said.

“Where we see an opportunity for deep learning is to try to do an interpretation for that problem, given that an exhaustive analysis is impossible. Using these new types of algorithms, we may be able to move from a problem that is continuous to a problem with a discrete data set.”

Mentor seeks to cooperate with academia and with research consortia such as IMEC. “We want to find the right research projects to sponsor between our research teams and academic teams. We hope that we can get better results with these new types of algorithms, and in the longer term with the new hardware that is being developed,” Rey said.

Many companies are developing specialized processors to run machine-learning algorithms, including non-Von Neumann, asynchronous architectures, which could offer several orders of magnitude less power consumption. “We are paying a lot of attention to the research, and would like to use some of these chips to solve some of the problems that the industry has, problems that are not very well served right now,” Rey said.

While power savings can still be gained with synchronous architectures, Rey said brain-inspired projects such as Qualcomm’s Zeroth processor, or the use of memristors being developed at H-P Labs, may be able to deliver significant power savings. “These are all worth paying attention to. It is my feeling that different architectures may be needed to deal with unstructured data. Otherwise, total power consumption is going through the roof. For unstructured data, these types of problem can be dealt with much better with neuromorphic computers.”

The use of deep learning techniques is moving beyond the biggest players, such as Google, Amazon, and the like. Just as various system integrators package the open source modules of the Hadoop data base technology into a more-secure offering, several system integrators are offering workstations packaged with the appropriate deep-learning tools.

Deep learning has evolved to play a role in speech recognition used in Amazon’s Echo. Source: Amazon

Robert Stober, director of systems engineering at Bright Computing, bundles AI software and tools with hardware based on Nvidia or Intel processors. “Our mission statement is to deploy deep learning packages, infrastructure, and clusters, so there is no more digging around for weeks and weeks by your expensive data scientists,” Stober said.

Deep learning is driving new the need for new types of processors as well as high-speed interconnects. Tim Miller, senior vice president at One Stop Systems, said that training the neural networks used in deep learning is an ideal task for GPUs because they can perform parallel calculations, sharply reducing the training time. However, GPUs often are large and require cooling, which most systems are not equipped to handle.

David Kanter, principal consultant at Real World Technologies, said “as I look at what’s driving the industry, it’s about convolutional neural networks, and using general-purpose hardware to do this is not the most efficient thing.”

However, research efforts focused on using new materials or futuristic architectures may over-complicate the situation for data scientists outside of the research arena. At the International Electron Devices Meeting (IEDM 2017), several research managers discussed using spin torque magnetic (STT-MRAM) technology, or resistive RAMs (ReRAM), to create dense, power-efficient networks of artificial neurons.

While those efforts are worthwhile from a research standpoint, Kanter said “when proving a new technology, you want to minimize the situation, and if you change the software architecture of neural networks, that is asking a lot of programmers, to adopt a different programming method.”

While Nvidia, Intel, and others battle it out at the high end for the processors used in training the neural network, the inference engines which use the results of that training must be less expensive and consume far less power.

Kanter said “today, most inference processing is done on general-purpose CPUs. It does not require a GPU. Most people I know at Google do not use a GPU. Since the (inference processing) workload load looks like the processing of DSP algorithms, it can be done with special-purpose cores from Tensilica (now part of Cadence) or ARC (now part of Synopsys). That is way better than any GPU,” Kanter said.

Rowen was asked if the end-node inference engine will blossom into large volumes. “I would emphatically say, yes, powerful inference engines will be widely deployed” in markets such as imaging, voice processing, language recognition, and modeling.

“There will be some opportunity for stand-alone inference engines, but most IEs will be part of a larger system. Inference doesn’t necessarily need hundreds of square millimeters of silicon. But it will be a major sub-system, widely deployed in a range of SoC platforms,” Rowen said.

Kanter noted that Nvidia has a powerful inference engine processor that has gained traction in the early self-driving cars, and Google has developed an ASIC to process its Tensor deep learning software language.

In many other markets, what is needed are very low power consumption IEs that can be used in security cameras, voice processors, drones, and many other markets. Nvidia CEO Jen Hsung Huang, in a blog post early this year, said that deep learning will spur demand for billions of devices deployed in drones, portable instruments, intelligent cameras, and autonomous vehicles.

“Someday, billions of intelligent devices will take advantage of deep learning to perform seemingly intelligent tasks,” Huang wrote. He envisions a future in which drones will autonomously find an item in a warehouse, for example, while portable medical instruments will use artificial intelligence to diagnose blood samples on-site.

In the long run, that “billions” vision may be correct, Kanter said, adding that the Nvidia CEO, an adept promoter as well as an astute company leader, may be wearing his salesman hat a bit.

“Ten years from now, inference processing will be widespread, and many SoCs will have an inference accelerator on board,” Kanter said.

Memristor Variants and Models from Knowm

Friday, January 22nd, 2016

thumbnail

By Ed Korczynski, Sr. Technical Editor

Knowm Inc. (www.knowm.org), a start-up pioneering next-generation advanced computing architectures and technology, recently announced the availability of two new variations of memristors targeting different neuromorphic applications. The company also announced raw device data available for purchase to help researchers develop and improve memristor models. These new Knowm offerings enable the next step in the R&D of radically new chips for pattern-recognition, machine-learning, and artificial intelligence (AI) in general.

There is general consensus between industry and academia and government that future improvements in computing are now severely limited by the amount of energy it takes to use Von Neumann architectures. Consequently, the US Whitehouse has issued a grand challenge with the Energy-Efficient Computing: from Devices to Architectures (E2CDA) program (http://www.nsf.gov/pubs/2016/nsf16526/nsf16526.htm) actively soliciting proposals through March 28, 2016.

The Figure shows a schematic cross-section of Knowm’s memristor devices—with Tin (Sn) and Chromium (Cr) metal layers as the new options to tungsten (W)—along with the device I/V curves for each. “They differ in their activation threshold,” explained Knowm CEO and co-founder Alex Nugent in an exclusive interview with Solid State Technology. “As the activation thresholds become smaller you get reduced data retention, but higher cycle endurance. As that threshold increases you have to dissipate more energy per event, and the more energy you dissipate the faster it will burn-out.” Knowm’s two new memristors, as well as the company’s previously announced device, are now available as unpackaged raw dice with masks designed for research probe stations.

Figure: Schematic cross-section of Knowm’s memristor devices using Tin (Sn) or Chromium (Cr) or tungsten (W) metal layers, along with the device I/V curves for each. (Source: Knowm)

Knowm is working on the simultaneous co-optimization of the entire “stack” from memristors to circuit architectures to application-specific algorithms. “The potential of memristors is so huge that we are seeing exponential growth in the literature, a sort of gold rush as engineers race to design new circuits and re-envision old circuits,” commented Knowm CEO and co-founder Alex Nugent. “The problem is that in the race to publish, circuit designers are adopting models that do not adequately describe real devices.” Knowm’s raw data includes AC, DC, pulse response, and retention for different memristors.

Additional memristors are being developed by Knowm’s R&D lab partner Dr. Kris Campbell of  Boise State University (http://coen.boisestate.edu/kriscampbell/), using different metal layers to achieve different activation thresholds beyond the three shown to date. “She has discovered an algorithm for creating memristors along this dimension,” said Nugent. “From a physics perspective it makes sense that there would be devices with high cycle endurance but reduced data retention.”

“In the future what I image is a single chip with multiple memristors on it. Some will be volatile and very fast, while others will be slow,” continued Nugent. “Just like analog design today uses different capacitors, future neuromophic chips would likely use memristors optimized for different changes in adaptation threshhold. If you think about memristors as fundamental elements—as per Leon Chua (https://en.wikipedia.org/wiki/Leon_O._Chua)—then it makes sense that we’ll need different memristors.”

The applications spaces for these devices have intrinsically different requirements for speed and retention. For example, to exploit these devices for pattern recognition and/or anomaly detection (keeping track of confidence in making temporal predictions) it seems best to choose relatively high activation thresholds because the number of operations is unlikely to burn-out devices. Conversely, for circuits that constantly solve optimization problems the best memristors would require low burn-out and thus low activation thresholds. However, analog applications are generally problematic because the existing memristors leak current, such that stored values degrade over time.

Knowm is shipping devices today, mostly to university researchers, and has tested thousands of devices itself. The Knowm memristors can be fabricated at <500°C using industry-standard unit-process steps, allowing for eventual integration with silicon CMOS “back-end” metallization layers. While still in early R&D, this technology could provide much of the foundation for post-Moore’s-Law silicon ICs.

—E.K.

Knowm First to Deliver Configurable Artificial Neural Networks using Bi-Directional Learning Memristors

Wednesday, September 2nd, 2015

thumbnail

By Ed Korczynski, Sr. Technical Editor

Knowm Inc., a start-up pioneering next-generation advanced computing architectures and technology, today announced the availability of artificial neural-network (ANN) chips built using memristors with bi-directional incremental learning capability. “We have been dreaming about this device and developing the theory for how to apply them to best maximize their potential for more than a decade,” said Alex Nugent, CEO and co-founder of Knowm. “The problem I set out to solve in 2001 was the massive discrepancy between how computers model brains and how neurons function. This result is truly a monumental technical milestone.”

Memristors with the bi-directional incremental resistance change property are the foundation for developing ANN such as Knowm’s recently announced Thermodynamic RAM (kT-RAM). Intended for high computing power jobs like machine learning (ML), autonomous robotics, and intelligent internet assistants, kT-RAM radically increases the efficiency of synaptic integration and adaptation operations such as pattern recognition and inference. The company has released an API for organizations and individual developers.

The Figure shows how Knowm’s architecture is adaptive, and based on the principle of Anti-Hebbian and Hebbian (AHaH) learning in neurons—following Hebb’s famous observation that “neurons that fire together wire together.” Hebbian learning reduces the synaptic resistance, while anti-Hebbian learning increases the resistance. The adaptive architecture means that thousands of memristors could be connected in parallel to do large-scale pattern recognition, or individual memristors could be mapped into a decision-tree to produce combinatorial optimization.

New “thermodynamic RAM” (kT-RAM) artificial neural network (ANN) architecture from Knowm is inherent adaptive, and built with memristors capable of bi-directional incremental resistance changes for efficient learning. (Source: Knowm)

The most famous ANN chip had been True North, but IBM could not develop memristors technology so that chip uses a fixed architecture with digital SRAM transistor arrays. The use of SRAM arrays means that True North chips require 21 pJ energy per synaptic integration, while Knowm’s memristor arrays can perform the same function with less than a thousandth of the energy (1-10 fJ).

Since the principle of Hebbian-learning is well known, many R&D teams around the world have tried and failed to find a material stack that allows for controlled incremental increase and decrease in resistance. An ideal memristor with the following properties would allow a single 2-terminal device to provide both Anti-Hebbian and Hebbian learning in artificial synapses:

  • BEOL CMOS compatible fabrication,
  • Voltage dependent  and low voltage thresholds of adaptation,
  • Non-Volatile with high cycle endurance,
  • Resistance ranges from ~100kΩ to ~100MΩ, and
  • Bi-directional incremental changes in resistance.

Fortunately, Dr. Kris Campbell of Boise State University had been researching the electronic properties of chalcogenide compounds, and her group was able to find the right material. Working with Knowm on this patent-pending technology, Campbell can create memristors that adjust resistance in incremental steps in both directions, instead of being limited to incremental change in only one direction alternating with a non-incremental “re-set” step. Using voltage pulses nominally 100 ns allows for learning (though shorter pulse lengths also work) using 0.6V to decrease resistance or -0.3V to increase resistance, and then a 20 mV pulse can easily read the learning level. The memristor stack of materials—including a silver layer as source of ions to diffuse through and alter the resistance of the calchogenide layer (still secret)—is only a few tens of nanometers thick, and can be formed in a single physical-vapor deposition (PVD) chamber in a time-scale of minutes.

Knowm has cycled these memristors billions of times, so reliability has been shown and the company is confident that it now has the building blocks in place for the creation of powerful and efficient ANN chips. “This is a low-level resource for adaptive learning,” explained Nugent to Solid State Technology. “It’s important to say that we’re not trying to do what others are doing with digital non-volatile memory. What we started out to do is to use memristors as synaptic connectors.” Earlier this year, Knowm announced the commercial availability of the first kT-RAM products:  discrete memristor chips, a Back End of Line (BEOL) CMOS+memristor service at Boise State, and the first “Knowm Anomaly” application.

Non-Volatile Memory (NVM) using memristors in cross-point arrays for digital Resistance RAM (ReRAM) has been pursued by many companies for many years. While there are inherent specification differences between digital NVM and analog ANN, in general it is more difficult to meet the device requirements for ANN. “In terms of the NVM, I feel pretty good about it already,” said Campbell to Solid State Technology. “Work I had been doing with Air Force Research Lab starting in about 2008 had been studying NVM and cross-point arrays.”