February 23, 2009
Bee in the Bonnet, Bug on the Loose
Please note that contributed articles, blog entries, and comments posted on EDACafe.com are the views and opinion of the author and do not necessarily represent the views and opinions of the management and staff of Internet Business Systems and its subsidiary web-sites.
Dave Patterson: That was a clever thing for the instructor to say, but it’s not true. In any laptop, or desktop, or server, there’s always an OS running plus other applications, so you can easily keep a couple of cores busy. If your system is multicore, for instance, you can actually set up a performance measurement, so you can see at any time how busy each cores is.
Q: So alternatively, why not just ship systems with one big, honking single-core processor, and let clever software do the gatekeeping and load balancing between applications running on that single core?
Dave Patterson: That’s actually the attractiveness of parallel processors – you don’t have to design huge new hardware. You can just use multicores.
What actually happened until just a few years ago is, they built bigger and bigger microprocessors so programmers didn’t have to work out the problems of parallel programming. But then, the processor used more and more power, and at 100 watts per chip, that was the most it could handle without exotic packaging.
By 2004 or 2005, we ran out of what we could do with your ‘honking’ single-core microprocessor and the only path forward was to replace it with 2, more power efficient cores. But, if we could still do it with a single processor, it’s true – we wouldn’t have to [bother with the problems] or multicore. If you could buy a 15 GHz processor, [we’d be set]. If we could have 500 Watt chips, we could have 15 GHz, but the only possible way to get there is through multicore.
Now the situation is more than just choosing between the lesser of two evils, single core or multicore. There’s actually only one fork left in the road. [Meanwhile], people have worked for 15 years to build hardware that preserves the single processor illusion for programmers. It’s something that many smart people have worked on in the past, and it’s where we have failed. But the future of the industry [rests in our ability to get] more performance in this pretty well trodden area. It’s remarkable that it’s taken so long.
Q: So, what is the secret sauce to parallelizing complex algorithms for coding?
Dave Patterson: You mean, how to just cut the Gordian Knot? Well, we’re not there yet. Or another way of describing it – do you know football? What we need is the Hail Mary pass.
But even then, it’s a lot easier to throw the Hail Mary pass than it is to catch it. A bunch of us – not just at Berkeley, but in lots of places – are trying to figure out a way we can catch it. We’re trying to demonstrate ways to help more people build parallel programs, but it’s really hard to do. We’re all researchers involved in work that will make these ideas more operable.
Q: I heard Kurt Keutzer’s talk at DAC last year about creating buckets for different types of programs, an attempt to classify programs into buckets that lend themselves more or less easily to parallel programming – if I understood the gist of his presentation.
Dave Patterson: Kurt’s office is right across the hall from mine here in Soda Hal! Yes, Kurt’s right. This is the El Dorado, this idea of identifying design patterns in software architecture. If we think more carefully about the architecture of the software while we’re building the software, many of those applications would lend themselves to parallel execution.
[Unfortunately], often people have ignored software engineering, so we’re betting that by carefully architecting your software into one of these well-known patterns, it will be easier to parallel program.
Q: Are you telling me that we’re only now starting to teach tidy programming practices?
Dave Patterson: No, what I’m saying is the difficulty in teaching is that you’ve got just about one semester to teach a student to program. During that semester, the students write maybe 1000 lines of code, maybe 10,000, with which to learn the programming principles of software engineering. It’s very difficult to teach these things, to teach concepts that are important industrial concerns when they don’t necessarily work in the classroom.
[Also], it’s hard to quantize [the value] of good programming principles. People have been talking about software architecture for a long time, but sill it’s hard to tell how seriously people take it.
Kurt Keutzer always knew, particularly when he was CTO at Synopsys, that software architecture is overwhelmingly important. It’s his insight that has led us to examine the hypotheses that we should be sensitive to design patterns – and now we’re demonstrating it. People are starting to get impressive results, and we won’t need to have the Single Golden Truth about parallel programming to solve the problem.
That Hail Mary pass? It’s in the air, and researchers everywhere have a sense of urgency that comes from wanting to beat their competition in catching the pass. It’s really an unusual circumstance – lots of people in industry needing the answer and waiting for us for us to find it.
Q: If it’s so important to industry, why doesn’t industry solve it themselves?
Dave Patterson: They are! They’re funding work at Illinois, at Stanford, at Berkeley, and other places. Not all of the best minds are at one company or at one university.
Q: And whoever gets the answer will own the patent?
Dave Patterson: I’m actually a patent skeptic – I’ve never filed a patent. It’s too easy to get around patents. The more important thing is to demonstrate an idea, and then to convince people that you’ve got the answer.
I like the Berkeley Software Distribution Model, the BSDM. We’re happy to have our software in open source. In areas like pharmacology, where it takes 10 years to do research, it’s a different story. But, that’s not the case in IT. Our philosophy is to work openly with industry.
Q: So, who’s in this race with you to catch the Hail Mary pass?
Dave Patterson: University of Illinois. Also, Microsoft and Intel, Stanford, AMD, Nvidia, and others. Here at Berkeley we’ve assembled a really great team and we’re making really exciting progress right now. Of course, everyone I mentioned has got a big team, as well, and good people working to solve this for the country and the world. It’s also really important that we’re all taking different approaches to finding the solution.
Q: Why not have a NASA/NACA-type program funded by the government? The American aviation industry was built on basic research, wind tunnels, etc, funded by the U.S. Government in the first half of the 20th century. If the parallel programming problem is so important, why not do it the NASA/NACA way?
Dave Patterson: Actually, when I was President of the ACM, I wrote several editorials on this topic. The Administration was cutting back on funding for academics. I wrote that some years down the road we’d need solutions, but since DARPA wasn’t funding the work, we wouldn’t have them when we needed them.
Why wasn’t DARPA funding brilliant students at great universities? They were funding industry instead – shorter-term stuff – giving hundreds of millions of dollars to companies like Cray and Sun and IBM. Actually, the person in charge of DARPA [responsible for those decisions]? His term just ended yesterday!
I’m pretty much on the record saying that what he did was terrible and American IT research has suffered as a result. If he had not changed the model 8 years ago, made decisions to the detriment of our country, it would be a very different world today.
[Meanwhile], other countries like China and India have stepped up their investment in IT. Parallel computing is something they’ll try to do research on. Industry everywhere wants somebody to catch [that Hail Mary pass]. If somebody catches the pass in China or Korea or India – the first people to make the catch will have a head start.
Q: How will you know when you see the answer? Maybe you’ll see the solution published one day in the Communications of the ACM?
You can find the full EDACafe event calendar here.
To read more news, click here.
-- Peggy Aycinena, EDACafe.com Contributing Editor.
Be the first to review this article