Danny Hillis, Connection Machine Legacy 5: Cloud, GPUs, People, August 2016


You had mentioned I think when I was visiting you last, that the Connection Machine you saw as sort of a dead end. Were you talking about hardware? Because what you have just told me is that all the concepts that – I mean I know that you were, you wanted … Well I don’t think it was a dead end, I think it was ahead of its time, and the technology wasn’t quite there to do what we wanted. But it definitely showed that something like that was possible, and it got people thinking in a way that as the technology came along, that you could build, you know, you could do that economically on a large enough scale that people sort of knew how to program it, knew how to use it, and so on. I remember when I wrote the first Scientific American article on the Connection Machine, I had an hypothesis, I said look, once we had this, there’s no need to have a computer … the power of the computer in your home so much. The real power will be out, you know, in some centralized location. I didn’t call it the Cloud but that’s where you need computations, because computations want to be near each other. They don’t need to be near you once you have the bandwidth for that. And I said so you can imagine your have big computers that you use something across the nation, and then the Scientific American people made me – they were like, “that’s just too implausible. We’ll let you say, like, ‘in the same city.'” So I was like, “Okay, well fine.” So you know, one city, someplace in the center of the city, but they actually – that was the negotiation of them pushing it back because it just sounded too crazy. Wow. Well can we say that the CM-1 as a SIMD machine, that the hardware experience there flowed – hardware and software experience there – flowed into the GPU processors. And the CM-5 … Yeah. A GPU chip or even a microprocessor that has a GPU unit or a vector unit on it, you know, they have SIMD instructions on them, so you know, those are fundamentally little Connection Machines and I think if you read … its clear … I think probably people like Nvidia even acknowledge that, you know, there’re little Connection Machines on the chip, and they can put several – they actually don’t have all the things the Connection Machine had, because they don’t have the … they don’t have the connection part of it so much, but they have the map part more than they have the reduce part. Or they have the connection part, the communicate part. They’re able to do reduction. … So that’s one sense in which it exists, and then, you know, the Cloud is much more like the CM-5 it’s, you know, racks and racks of micro processors that are connected by a fast network. And so, you know, that was very much the architecture that we evolved toward, and they’re programmed very much like the CM-5 was programmed. Although they actually don’t have special hardware for reduce network anymore they just do that with the general network. And they don’t have special hardware for the map, like the graphics processors do. So it turns out for the neural networks actually the more efficient way to do it is on a bunch of the graphics processors because they do have that special, they do have that special hardware. So both things definitely take elements from the way that we did things on the Connection Machine. It’s interesting though, that in many ways the software is more primitive than … we really had it so that you can take your Fortran program and compile it to run on 10,000 processors and that pretty much doesn’t exist now. Why not??? The technology kind of lost in the whole meltdown of the super computer industry and … you know, I think it’s getting … there are DARPA programs to deal with it … again. But it’s funny seeing a technology get lost in your lifetime. Yeah, yeah. It will get rebuilt eventually. But people were able to do enough with the simpler programming paradigms … you know, the simple kind of things like MapReduce, which was definitely one of the things we used but wasn’t the only thing we used. But that’s turned out to be powerful enough to do an awful lot. So it sounds like even though Thinking Machines went belly up after, you know, little over 10 years that … in some sense it was worthwhile, both the hardware and the software and the ideas have gone on and … Well also the people have gone on, I mean it’s amazing the people that … some of them have gone on to win Nobel Prizes, or start giant research institutes, or found companies or … So what’s interesting is if you would – actually what would have been the best investment portfolio would’ve just been to invest in everybody in that building – whatever they did! And so, you know, fantastic people if you remember, like, you know, Eric Lander was just starting to play with biology using the … and is now the head of one of the biggest biology Institutes. Sydney Brenner you know was … … only insiders knew who Sydney Brenner was, he certainly hadn’t won the Nobel Prize at that point, you know. So it was things like Brewster’s search engine and his archiving the Internet just seemed like crazy ideas, nobody understood what they were. So I think a lot of … I think, you know, people went on to do kind of amazing things and … so maybe that’s how it has it’s biggest impact was a set of people got – it’s like the Manhattan Project in that sense, a set of people got together, and sort of inspired each other to go off and do great things. And yeah we did do a bunch of stuff and we did, I mean, the project that Dick Feynman was working, on which was quantum computing, and it was so absurd that only Dick Feynman was working on it, you know, that’s become a whole field! And I’m quite sure that Thinking Machines was the first company that ever worked on that! Just because they had this nutty guy called Richard Feynman who had this nutty idea that he was working on in his spare time. Yeah, yeah, yeah. So well I think it was … I think we were very lucky to have been at that time in that place, and it was certainly an extraordinary thing. If I had known even a quarter as much about this business and how the world works as I do now, it would have been able to save it for long enough to, you know, for the web come along and parallel computing to come along, but you know, I was a bad business person, made a lot of stupid mistakes and you know, I can now go back and see all kinds of things that I did wrong that I would never do today, but it wasn’t – it wasn’t what I knew about, it wasn’t what I was paying attention to then. Yeah and it did go belly up right when the web started out didn’t it, that was that was a bit of an … And remember, we were working on a web server and nobody – we couldn’t raise any money on it because nobody knew what the web was, right? Right. Sigh. But in fact, you know, a lot of those people went to Sun, and turned it – helped turn it from a workstation company into a web company. So ….

Comments 1

  • Great conversation
    Thanks for uploading this, Tamiko.
    It was a long time since I read The Pattern on the Stone .
    Time to revisit some chapters.

Leave a Reply

Your email address will not be published. Required fields are marked *