Part of the  

Solid State Technology

  and   

The Confab

  Network

About  |  Contact

Posts Tagged ‘Coseer’

Enabling the A.I. Era

Monday, October 23rd, 2017

thumbnail

By Pete Singer, Editor-in-Chief

There’s a strongly held belief now that the way in which semiconductors will be designed and manufactured in the future will be largely determined by a variety of rapidly growing applications, including artificial intelligence/deep learning, virtual and augmented reality, 5G, automotive, the IoT and many other uses, such as bioelectronics and drones.

The key question for most semiconductor manufacturers is how can the benefit from these trends? One of the goals of a recent panel assembled by Applied Materials for an investor day in New York was to answer that question.

Jay Kerley, Praful Krishna, Mukash Khare, Matt Johnson and Christos Georgiopoulos (left to right)

The panel, focused on “enabling the A.I. era,” was moderated by Sundeep Bajikar (former Sellside Analyst, ASIC Design Engineer). The panelists were: Christos Georgiopoulos (former Intel VP, professor), Matt Johnson (SVP in Automotive at NXP), Jay Kerley (CIO of Applied Materials), Mukesh Khare (VP of IBM Research) and Praful Krishna (CEO of Coseer). The panel discussion included three debates: the first one was “Data: Use or Discard”; the second was “Cloud versus Edge”; and the third was “Logic versus Memory.”

“There’s a consensus view that there will be an explosion of data generation across multiple new categories of devices,” said Bajikar, noting that the most important one is the self-driving car.  NXP’s Johnson responded that “when it comes to data generation, automotive is seeing amazing growth.” He noted the megatrends in this space: the autonomy, connectivity, the driver experience, and electrification of the vehicle. “These are changing automotive in huge ways. But if you look underneath that, AI is tied to all of these,” he said.

He said that estimates of data generation by the hour are somewhere from 25 gigabytes per hour on the low end, up to 250 gigabytes or more per hour on the high end. or even more in some estimates. “It’s going to be, by the second, the largest data generator that we’ve seen ever, and it’s really going to have a huge impact on all of us.”

Georgiopoulos agrees that there’s an enormous amount of infrastructure that’s getting built right now. “That infrastructure is consisting of both the ability to generate the data, but also the ability to process the data both on the edge as well as on the cloud,” he said. The good news is that sorting that data may be getting a little easier. “One of the more important things over the last four or five years has been the quality of the data that’s getting generated, which diminishes the need for extreme algorithmic development,” he said. “The better data we get, the more reasonable the AI neural networks can be and the simpler the AI networks can be for us to extract information that we need and turn the data information into dollars.”

Edge computing describes a computing topology in which information processing, and content collection and delivery, are placed closer to the sources of this information. Connectivity and latency challenges, bandwidth constraints and greater functionality embedded at the edge favors distributed models. Jay Kerley (CIO of Applied Materials) addressed the debate of cloud vs edge computing, noting it was a factor of data, then actual value and finally intelligence. “There’s no doubt that with the pervasiveness of the edge and billions of devices, data is going to be generated exponentially. But the true power comes in harnessing that data in the core. Taking it and turning it into actual intelligence. I believe that it’s going to happen in both places, and as a result of that, the edge is not going to only generate data, it’s going to have to consume data, and it’s going to have to make decisions. When you’re talking about problems around latency, maybe problems around security, problems around privacy, that can’t be overcome, the edge is going to have to be able to make decisions,” he said.

Kerley said there used to be a massive push to build data centers, but that’s changed. “You want to shorten the latency to the edge, so that data centers are being deployed in a very pervasive way,” he said. What’s also changing is that cloud providers have a huge opportunity to invest in the edge, to make the edge possible. “If they don’t, they are going to get cut out,” he added. “They’ve got to continue to invest to make access into the cloud as easy, and as frictionless as possible. At the end of the day, with all that data coming into these cloud data centers, the processing of that information, turning it into actual intelligence, turning it into value, is absolutely critical.”

Mukesh Khare (VP of IBM Research) also addressed the value of data. “We all believe that data is our next natural resource. We’re not going to discard it. You’re going to go and figure out how to generate value out of it,” he said.

Khare said that today, most artificial intelligence is too complex. It requires, training, building models and then doing inferencing using those models. “The reason there is good in artificial intelligence is because of the exponential increase in data, and cheap compute. But, keep in mind that, the compute that we are using right now is the old compute. That compute was built to do spreadsheet, databases, the traditional compute.

“Since that compute is cheap and available, we are making use of it. Even with the cheap and available compute in cloud, it takes months to generate those models. So right now, most of the training is still being done in cloud. Whereas, inferencing, making use from that model is done at the edge. However, going forward, it is not possible because the devices at the edge are continuously generating so much data that you cannot send all the data back to the cloud, generate models, and come back on the edge.”

“Eventually, a lot of training needs to move to the edge as well,” Khare said. This will require some innovation so that the compute, which is being done right now in cloud, can be transferred over to edge with low-power devices, cheap devices. Applied Materials’ Kerley added that innovation has to happen not only at the edge, but in the data center and at the network layer, as well as in the software frameworks. “Not only the AI frameworks, but what’s driving compression, de-duplication at the storage layer is absolutely critical as well,” he said.

NXP’s Johnson also weighed in on the edge vs cloud debate with the opinion that both will be required for automotive. “For automotive to do what it needs to, both need to evolve,” he said. “In the classic sense of automotive, the vehicle would be the edge, which needs access to the cloud frequently, or non-stop. I think it’s important to remember that the edge values efficiency. So, efficiency, power, performance and cost are all very important to make this happen,” he added.

Automotive security adds another degree of complexity. “If you think of something that’s always connected, and has the ability to make decisions and control itself, the security risk is very high. And it’s not just to the consumer of the vehicle, but also to the company itself that’s providing these vehicles. It’s actually foundational that the level of safety, security, reliability, that we put into these things is as good as it can be,” Johnson said.

Georgiopoulos said a new compute model is required for A.I. “It’s important to understand that the traditional workloads that we all knew and loved for the last forty years, don’t apply with A.I. They are completely new workloads that require very different type of capabilities from the machines that you build,” he said.  “With these new kind of workloads, you’re going to require not only new architectures, you’re going to require new system level design. And you’re going to require new capabilities like frameworks. He said TensorFlow, which is an open-source software library for machine intelligence originally developed by researchers and engineers working on the Google Brain Team, seems to be the biggest framework right now. “Google made it public for only one very good reason. The TPU that they have created runs TensorFlow better than any other hardware around. Well, guess what? If you write something on TensorFlow, you want to go to the Google backend to run it, because you know you’re going to get great results. These kind of architectures are getting created right now that we’re going to see a lot more of,” he said.

Georgiopoulos said this “architecture war” is by no means over. “There are no standardized ways by which you’re going to do things. There is no one language that everybody’s going to use for these things. It’s going to develop, and it’s going to develop over the next five years. Then we’ll figure out which architecture may be prevalent or not. But right now, it’s an open space,” he said.

IBM’s Khare, weighed in on how transistors and memory will need to evolve to meet the demands of new AI computer architectures, “For artificial intelligence in our world, we have to think very differently. This is an inflection, but this is the kind of inflection that world has not seen for last 60 years.” He said the world has gone from tabulating system era (1900 to 1940) to the programmable system era in 1950s, which we are still using. “We are entering the era of what we call cognitive computing, which we believe started in 2011, when IBM first demonstrated artificial intelligence through our Watson System, which played Jeopardy,” he said.

Khare said “we are still using the technology of programmable systems, such as logic, memory, the traditional way of thinking, and applying it to AI, because that’s the best we’ve got.”

AI needs more innovation at all levels, Khare said. “You have to think about systems level optimization, chip design level optimization, device level optimization, and eventually materials level optimization,” he said.  “The artificial workloads that are coming out are very different. They do not require the traditional way of thinking — they require the way the brain thinks. These are the brain inspired systems that will start to evolve.”

Khare believes analog compute might hold the answer. “Analog compute is where compute started many, many years ago. It was never adopted because the precision was not high enough, so there were a lot of errors. But the brain doesn’t think in 32 bits, our brain thinks analog, right? So we have to bring those technologies to the forefront,” he said. “In research at IBM we can see that there could be several orders of magnitude reduction in power, or improvement in efficiency that’s possible by introducing some of those concepts, which are more brain inspired.”