Intel has announced a significant advance for its neuromorphic research processor, named Codehi. The company has now extended its implementation of Loihi to 64 processors, allowing it to create a system with over 8 million neurons (8.3 million). This new configuration (codenamed Pohoiki Beach) delivers 1000 times better performance than conventional processors in applications such as sparse coding, graph search, and constraint satisfaction problems. Intel says the new Pohoiki range offers 10,000 times more energy efficiency than conventional processor architectures in this type of test.
Neuromorphic Computing is a subset of the computer that attempts to imitate the architecture of the brain with the help of modern technological analogues. Instead of implementing a typical CPU clock, for example. Loihi is based on a peaked neural network architecture. The basic Loihi processor contains 128 neuromorphic cores, three Lakefield processor cores (Intel Quark) and an off-chip communications network. Theoretically, Loihi can handle up to 4,096 on-chip cores and 16,384 chips, although Intel has announced that it does not intend to market a design of this size.
"Thanks to the Loihi chip, we were able to demonstrate a power consumption 109 times lower than that of a real-time graphics processor, and a power consumption 5 times lower than that of an inference material. IoT specialized, "said Chris Eliasmith. CEO of Applied Brain Research and Professor at the University of Waterloo. "Even better, while the network is multiplied by 50, Loihi retains performance results in real time and uses 30% more power, while the IoT hardware consumes 500% more power and is no longer in time. real."
The implementation of Pohoiki Beach is not the largest deployment planned for the neuromorphic chip. Intel announces its intention to roll out an even larger concept, dubbed Pohoiki Springs, which will offer "unprecedented levels of performance and efficiency for enhanced neuromorphic workloads."
We covered the advances and research in neuromorphic computing for several years at ET. The work done on these processors is closely related to the work done in AI and the machine intelligence in general, but it's not just about how to perform AI / ML workloads. on existing chips. The ultimate goal is to build processors closer to the human brain.
One of the quirks of computing is that the analogies between the functioning of the human brain and the operation of computers are widespread. Human brains and conventional computers do not overlap very little on their functioning. Transistors are not equivalent to neurons and the pulsed neural network that Loihi uses to transmit information about his own processor cores is intended to be closer to biological processes that humans and other animals use than traditional silicon.
Projects like this one have several long-term research goals, but one of the most fundamental is to better understand how the brain works to replicate some of their energy efficiency. The human brain operates at around 20W. The Exascale supercomputer, considered the minimum for advanced neural simulation of anything more complex than an earthworm, should consume megawatts of power per supercomputer. The difference between these numbers explains why we are primarily interested in the long-term energy efficiency and computing potential of the brain. Architectures such as Loihi are not just an effort to write programs that mimic what humans can do. the goal is to copy aspects of our neurology as well. This makes their progress a little more interesting.
Background image: Tim Herman / Intel Corporation
Google's DeepMind division has made considerable efforts to apply artificial intelligence to issues such as computer vision and climate change, but there is still room for gaming. DeepMind first dominated the game of Go, then switched to StarCraft II, beat some of the best players in the world early this year. Now, you could have a chance of play against IA AlphaStarbut you will probably be destroyed.
Before challenging the professional players, Deepmind Simulated more than 200 years of StarCraft II gameplay to form the bot. It's a convolutive neural network that began by absorbing reruns of StarCraft II pro matches. By using competing models, DeepMind has trained several "agents" capable of building and fighting, as well as human players – better, in fact. AlphaStar has won 10 of 11 matches against professional players.
Previous matches were an impressive demonstration of AI prowess. AlphaStar had a better understanding of resource allocation, coverage and micromanagement of units than most human actors. Naturally, his ability to transparently control multiple units has also helped. Although, he did not take as much action as human players to win.
The experiment will run on Blizzard's European servers, where a small number of humans will be paired with AlphaGo in part 1v1. They will not know it, but players have to sign up for a chance to compete with the AI. Unfortunately, there is no way to make sure you can play against AlphaStar.
Blizzard will let DeepMind manage several different agents on Battle.net, and their operation will be different from that of the last demo. The new AlphaStar will be able to play against or one of the three StarCraft II races (it was only before Protoss). It also relies on a normal view of the game by the camera, while the former used a bird's eye view of the entire map. DeepMind has also limited alphaStar shares per minute (APM).
DeepMind is primarily interested in testing AlphaStar in matches where players change their strategies, and keeping the secrecy of the match ensures a controlled test. After being presumably murdered by artificial intelligence, players will see their rankings affected as if they had played a human opponent. DeepMind will use the results of this test to inform future research on AI, and the results of the matches will be included in a future scientific article.
There is hardly a day without a story about false news. It reminds me of a quote from the favorite radio journalist of my youth: "If you do not like the news, go out and do it yourself." The revolutionary linguistic model of OpenAI, the 1.5 billion parameter of GPT-2, close enough for the group to decide that it is too dangerous to publish publicly, at least for the moment. However, OpenAI has now released two smaller versions of the template, as well as tools for adjusting them with your own text. So, without much effort, and using considerably less GPU-2 that will be able to generate text from scratch, you can create an optimized version of GPT-2. answer questions similar to those with which you train.
GPT-2 (Generative Pre-Trained Transformer Version 2) is based on a version of the very powerful Transformer Neural Network Warning. What excited OpenAI students so much was discovering that she could handle a number of language tasks without being directly trained in these tasks. Once preformed with his massive corpus of Reddit data and with the appropriate prompts, he did a fair job in answering questions and translating languages. As far as semantic knowledge is concerned, it's certainly not at all Watson, but this type of unsupervised learning is particularly exciting because it saves a lot of time and money for data labeling for a supervised learning.
For such a powerful tool, working with GPT-2 is fortunately quite simple, as long as you know at least a little Tensorflow. Most of the tutorials I've found are also based on Python. It is therefore very useful to have at least a basic knowledge of programming in Python or in a similar language. Currently, OpenAI has released two pre-formed versions of GPT-2. One (117 million) has 117 million parameters, while the other (345 million) has 345 million. As might be expected, the larger version requires more GPU memory and takes longer to exercise. You can train on your CPU, but it's going to be really slow.
The first step is to download one or both models. Fortunately, most tutorials, including the ones we're going to browse, use Python code to do it for you. Once downloaded, you can run the pre-formed template to generate text automatically or in response to a prompt you provided. But there is also a code that allows you to build on the pre-formed model by tweaking it on a data source of your choice. Once you have adapted your model to your satisfaction, you just have to run it and provide appropriate prompts.
There are a number of tutorials on this subject, but my favorite is Max Woolf. In fact, until the publication of OpenAI, I was working with his RNN text generator, which I borrowed for his work on GPT-2. He provided a complete package on GitHub to download, adjust and run the GPT-2 based model. You can even take it directly in a PyPl package. The readme file guides you through the process, with some suggestions on how to change different settings. If you have a large number of GPUs at hand, this is an excellent approach, but since the 345M model requires most 16GB GPUs for training or tuning, you may need to turn to a cloud GPU.
Fortunately, there is a way to make free use of a powerful graphics processor in the cloud: Google's Colab. It's not as flexible as the Google Compute Engine account today, and you have to reload everything, but did I say it was free? During my tests, I got a Tesla T4 or K80 GPU when I booted a laptop, one of the two being fast enough to train these models at a reasonable pace. The best part is that Woolf has already written a Colab notebook echoing the local python code version of gpt2-simple. Just like the desktop version, you can simply follow or modify parameters to experiment. It's more complicated to integrate the data in Colab, but the notebook will also guide you.
Now that powerful language templates have been published on the Web and there are plenty of tutorials to use them, perhaps the most difficult part of your project is creating the dataset you want to use for tuning. If you want to replicate the experiences of others by having him generate Shakespeare or write Star Trek dialogue, you can simply hook one that is online. In my case, I wanted to see how the models would be used to generate articles like those found on ExtremeTech. I've had access to a catalog of over 12,000 items from the last 10 years. I was able to group them into a text file and use them as a basis for fine tuning.
Eleven I had my body of 12,000 ExtremeTech articles, I started trying to train the GPT-2 simplified on the GPU Nvidia 1080 of my desktop. Unfortunately, the 8GB RAM of the GPU was not enough. So I switched to 117M model training on my 4-core i7. It was not terribly terrible, but it would have taken a week to make a real fall, even with the smaller of the two models. So I moved to Colab and the 345M model. The training was much, much, faster, but it was annoying to handle the session resets and the unpredictability of the GPUs I was getting for each session.
After that, I bit the ball, opened a Google Compute Engine account, and decided to take advantage of Google's $ 300 credit to new customers. If you do not master the configuration of a virtual machine in the cloud, the task can be a bit difficult, but there are many guides online. This is simple if you start with one of the preconfigured virtual machines on which Tensorflow is already installed. I chose a Linux version with 4 vCPU. Even though my desktop system is Windows, the same Python code worked perfectly on both. You must then add to GPU, which in my case has taken a request for permission from Google's technical support. I guess this is because GPU-equipped machines are more expensive and less flexible than CPU-only machines, so they have a certain validation process. It only took a few hours and I was able to launch VM with a Tesla T4. When I logged in for the first time (using the built-in SSH), he reminded me that I had to install the Nvidia drivers for the T4 and gave me the command which I'd I needed it.
Then you need to configure a file transfer client such as WinSCP and start using your template. Once you have loaded your code and data, created a virtual Python environment (optional) and loaded the necessary packages, you can proceed in the same way as on your desktop. I trained my model in increments of 15,000 steps and I downloaded the model checkpoints each time. I would have them as a reference. This can be particularly important if you have a small set of training data, as too much training can lead to over-adaptation and worsening of your model. It is therefore useful to have control points.
Speaking of control points, like models, they are great. So, you will probably want to add a disk to your VM. By arranging the disc separately, you can still use it for other projects. The automatic editing process is a bit awkward (it seems like it can be a checkbox, but it is not). Fortunately, just do it at eleven. After installing my virtual machine with the code, the template, and the training data I needed, I dropped them. The T4 was able to run about one step every 1.5 seconds. The virtual machine I had set up was about $ 25 a day (do not forget that the virtual machines do not shut down themselves, you have to stop them if you do not want to be billed, and the persistent disk continues to be charged even at that time).
To save some money, I transferred the model checkpoints (.zip file) to my desktop. I could then shut down the virtual machine (save a dollar or two per hour) and interact with the model locally. You get the same result in both cases because the model and the control point are the same. The traditional method to evaluate the success of your training is to retain some of it in the form of a validation game. If the loss continues to decrease, what is the probability that you started calculating the loss when you run your model on the data you provided for validation? simply "remember" your entry and return it to you. This reduces its ability to process new information.
After experimenting with different types of prompts, I focused on feeding the model (which I nicknamed The Oracle) in the first sentences of current ExtremeTech articles and on the results. After 48 hours (106,000 steps in this case) of training on T4, here is an example:
The more information the model has on a subject, the more it starts to generate plausible text. We write a lot about Windows Update, so I thought I'd let the model give it to try:
With something as subjective as text generation, it's hard to know how to go with the formation of a template. This is all the more true as every time a prompt is submitted, you get a different answer. If you want plausible or fun answers, it's best to generate multiple samples for each prompt and browse them yourself. In the case of the Windows Update prompt, we powered the model using the same prompt after a few hours of training, and it seemed like the extra work might have been helpful:
I was impressed, but not blown away, by the crude predictive performance of GPT-2 (at least the public version) compared to simpler solutions such as textgenrnn. What I did not realize was that later, it's versatility. GPT-2 is general enough to handle a wide variety of use cases. For example, if you give him a pair of French and English sentences in the form of a prompt, followed by a sentence in French only, the translation of the money is plausible. Or if you give these question-answer pairs, followed by a question, it's a decent job to find a plausible answer. If you're generating interesting text or articles, consider sharing them because it's definitely a learning experience for all of us.
As artificial intelligence continues to progress rapidly, we still have a long way to go to develop the sensors needed to translate the physical world into computer-based data. While vision and sound were a long way ahead, our other senses have little practical application in the digital world, but this is not the case with robots.
MIT recently created a new robot using GelSight Sensors which allows him to see the objects he touches and to create a 3D map of the texture to better understand it. The video below shows how GelSight technology, generally used for aerospace applications, can "see" what it touches.
GelSight certainly offers an impressive and detailed way of translating the real world into digital information. But that does not make a smart robot, but just very informative fingers that require intelligence to be controlled. Aware of this potential, MIT has created a robot with an artificial intelligence model that exercises on objects that it touches with the help of detailed three-dimensional maps generated by its GelSight sensors. While the robot does not really see what it touches in a traditional optical sense, it receives so much data through its sensors that it can translate that data into visual information and learn from it just like anything else. Ordinary image-oriented convoluted neuron network (CNN).
The MIT robot was trained in 12,000 video recordings of touch data from GelSight sensors, broken down into still image frames, of 200 household objects. Combined with tactile data, this allows the robot to understand the materials that its sensors touch. In one conversation with Engadget, CSAIL Ph.D. student and lead author of this project, explained what their system can now achieve:
"Looking at the scene, our model can imagine the sensation of touching a flat surface or a sharp edge, and by blindly touching, our model can predict interaction with the environment only from tactile feelings. more power to the robot and reduce the data we might need for tasks involving manipulation and seizure of objects. "
The MIT system, which is still in its infancy, is working and it is thanks to their approach. Many researchers and developers in artificial intelligence tend to create models based on the functioning of the human brain, but this often makes no sense. In some cases, we make I want it to work like a human being because his goal is either to approach us or to help us learn more about ourselves by simulating human processes. In most other cases, however, addressing the development of artificial intelligence by imposing a human framework negates the many non-human benefits that software and hardware have to offer.
MIT has chosen to use a much more accurate and capable sensor that any human can only approximate and make the most of the computing power available to AI. By making choices that exploit the benefits of computers rather than forcing humans, they have created a robot that can surpass humans in blind identification tasks. In specific cases, he succeeds already.
Although this does not seem to be the most important problem to solve, touch actually plays an important role in robotics. Niche applications might benefit from a robotic ability to feel the difference between cotton and nylon, but wider applications have much more to offer. For a robot without touching, all objects have the same feeling. He may be able to understand some things visually, but it's rarely helpful.
Think about how you would look after your day if everything you touched felt the same, or more exactly, looked like nothing at all. You would not know what force to use when you plug a cable. You would not be able to understand the practical differences between a printed image of sandpaper and the sandpaper itself.
By providing the robot with a sense of touch and the ability to learn from it, this robot can better judge the materials it touches. He can learn faster and more accurately than he can do simply with standard visuals. He can then use this information to adjust his actions based on the materials that he handles – or at least, it's the ideal goal for the future. If robots can understand touch, they are less likely to cause unintentional damage. At the present time, if you asked most intelligent robots to carry a water balloon, they would not know how to hold it without destroying it. The sense of touch gives robots the ability, through a well trained AI model, to know how to handle different types of objects and to act accordingly.
Although MIT has only created a smarter and smarter robot component, it's still a step in the right direction. A robot designed to understand and incorporate the data it acquires in a tactile way has far better implications for general safety than others. That's how creating backups against potential accidents.
From breast cancer For brain aneurysms, artificial intelligence continues to impose itself as a valuable diagnostic tool. Researchers at Stanford University have has created predictive AI to detect the likelihood of aneurysms during brain scans with great precision.
Although relatively rare, brain aneurysms present suddenly and have a very short time for treatment before becoming fatal. According to the Foundation of Brain AneurysmsAbout 30,000 people in the United States suffer from this disease each year, with 15% going to the hospital and 40% of all cases resulting in death. Of the survivors, 66% suffer from a decrease in neurological function. This constitutes an accurate and early diagnosis of the most important factor in the prevention of cerebral aneurysm, but this has proven to be a very difficult task for health professionals.
Research at Stanford University has recognized the difficulty of this problem and created a tool to solve it. Kristen Yeom, associate professor of radiology and co-lead author of the paper, explains why AI is an essential part of the diagnostic process:
Searching for an aneurysm is one of the most critical and intensive tasks. Given the significant challenges posed by complex neurovascular anatomy and the possible fatal outcome of a missed aneurysm, this has prompted me to apply advances in computer science and vision to neuroimaging.
Brain aneurysms occur when the wall of a brain artery swells, but the real problems do not begin until these bulges begin to flow or burst. The larger they become, the more difficult the treatment becomes if they break. While other diagnostic efforts seek to detect the signs of a disease before it can noticeably manifest themselves, brain aneurysms have various origins, ranging from addiction to cancer, going through various issues related to blood. The main problem remains the detection of existing aneurysms. Before breaking, cerebral aneurysms usually do not require any symptoms and their removal requires corrective surgery. This leads to a careful and thorough diagnostic process, with clinicians not wishing to undergo brain surgery without the assurance of its necessity.
The results of the creation of their artificial intelligence tool HeadXNet. The importance of avoiding misdiagnosis meant that HeadXNet could not exercise undue influence over the physician's decisions. In addition, cerebral sweeps are complete three-dimensional models, much more complex than flat images that convolutional neural networks are generally trained to understand. To solve these problems, the Yeom team manually tagged each voxel in the learning data to specify whether it contained an aneurysm. After the training, HeadXNet provided its answer only in the form of an overlay to locate the locations of the brain with the highest probability of aneurysm without actually exerting an additional influence likely to occur. ############################################################################ 39, influence a misdiagnosis.
The Yeom team tested HeadXNet with eight clinicians and 115 brain exams. This little test gave positive results. Using this tool, clinicians have correctly identified more aneurysms and reduced the number of diagnostic discrepancies between them. Even with these promising results, HeadXNet will not be part of the diagnostic process in the near future. Beyond the need to continue developments and testing to ensure its safe use in larger populations, brain content readers are not designed to integrate with such machine learning technologies as than HeadXNet. The widespread use of this technology requires more data, testing and development before the general population can benefit.
HeadXNet nonetheless represents a significant step forward in the resolution process of a complicated and fatal problem of underfunded research. It also demonstrates the greatest benefit of using artificial intelligence as a collaborative partner and not a replacement for the human. With the right motivations and the right implementations, the AI has a lot to offer to humanity. Although we must also consider the worstwe continue to see the results of hoping for the best.
Top picture credit: Storyblocks