Bias amp vs bias amp 2
In the past year, I’ve been getting the same mix of “there’s nothing new here” and “this seems like the wrong direction.” Over the past year as I’ve been speaking to people about the data-centric AI movement, I’ve been getting flashbacks to when I was speaking to people about deep learning and scalability 10 or 15 years ago. As a programming paradigm, this seems like too much work.” I did manage to convince him the other person I did not convince. NeurIPS workshop paper advocating using CUDA, a platform for processing on GPUs, for deep learning-a different senior person in AI sat me down and said, “CUDA is really complicated to program. I remember when my students and I published the first Having 50 thoughtfully engineered examples can be sufficient to explain to the neural network what you want it to learn.” “In many industries where giant data sets simply don’t exist, I think the focus has to shift from big data to good data. I think he felt that the action couldn’t just be in scaling up, and that I should instead focus on architecture innovation. One very senior person pulled me aside and warned me that starting Google Brain would be bad for my career. Ng: Over a decade ago, when I proposed starting the Google Brain project to use Google’s compute infrastructure to build very large neural networks, it was a controversial step. It’s funny to hear you say that, because your early work was at a consumer-facing company with millions of users. While that paradigm of machine learning has driven a lot of economic value in consumer software, I find that that recipe of scale doesn’t work for other industries. Having said that, a lot of what’s happened over the past decade is that deep learning has happened in consumer-facing companies that have large user bases, sometimes billions of users, and therefore very large data sets. But I’m confident that if a semiconductor maker gave us 10 times more processor power, we could easily find 10 times more video to build such models for vision. Many researchers are working on this, and I think we’re seeing early signs of such models being developed in computer vision. The compute power needed to process the large volume of images for video is significant, and I think that’s why foundation models have arisen first in NLP. Ng: I think there is a scalability problem.
What needs to happen for someone to build a foundation model for video?
BIAS AMP VS BIAS AMP 2 FREE
Foundation models offer a lot of promise as a new paradigm in developing machine learning applications, but also challenges in terms of making sure that they’re reasonably fair and free from bias, especially if many of us will be building on top of them. For example, GPT-3 is an example of a foundation model. Ng: This is a term coined by Percy Liang and some of my friends at Stanford to refer to very large models, trained on very large data sets, that can be tuned for specific applications. When you say you want a foundation model for computer vision, what do you mean by that? Having said that, it only applies to certain problems, and there’s a set of other problems that need small data solutions. So I think that this engine of scaling up deep learning algorithms, which has been running for something like 15 years now, still has steam in it. I think there’s lots of signal to still be exploited in video: We have not been able to build foundation models yet for video because of compute bandwidth and the cost of processing video, as opposed to tokenized text.
I’m excited about NLP models getting even bigger, and also about the potential of building foundation models in computer vision. Do you agree that it can’t go on that way?Īndrew Ng: This is a big question. Some people argue that that’s an unsustainable trajectory. The great advances in deep learning over the past decade or so have been powered by ever-bigger models crunching ever-bigger amounts of data.
Ng’s current efforts are focused on his company And that’s what he told IEEE Spectrum in an exclusive Q&A. So when he says he has identified the next big shift in artificial intelligence, people listen. He pioneered the use of graphics processing units (GPUs) to train deep learning models in the late 2000s with his students at Stanford University, cofounded Google Brain in 2011, and then served for three years as chief scientist for Baidu, where he helped build the Chinese tech giant’s AI group. Andrew Ng has serious street cred in artificial intelligence.