Linus Torvalds, Patterson and different views (or different worlds?)

I am working in the “Open Source Learning Lab” project which will be the new Eduforge.org

In this context, looking for content for the Topics of Eduforge, I came through a nice presentation about the History of Linux by Linus Torvalds . It is an 85 minutes  story, told in a very friendly way with  questions from the audience, about a week or two after September 11, 2001, sponsored by the Computer History Museum. It is (after the overview of the Museum) the most viewed video of the 60 excellent ones that are stored there.  It is enjoyable listening to a “young” Torvalds, speaking about the 10 years of Linux! I recommend it.

But which is the link of the video with this blog?

One of the last questions at the end of the video, was about SMP (Symmetric Multiprocessing, the use of multiple CPUs) . Torvalds answers something like “I don’t see this coming in at least five years, but definitely it is a very interesting place to be, and it will involve NUMA (Non-Uniform Memory Access, a computer memory design used in multiprocessors)”.

That was enough to catch my attention, and started to search if the interest of Linus continued after those five years…What I found was a sort of soap opera, pretty much a conversation in different languages…

Someone posted this in Real World Technologies

“…Quoted from the white paper “The Landscape of Parallel Computing Research: A View from Berkeley”. This paper just gave Berkeley $10M over 5 years from MS and Intel to research the future of parallel computing. Happy reading…”

The first answer with the title “disappointing” said

Name: Linus Torvalds (torvalds@osdl.org) 2/14/08

Ugh. They seem to make essentially all of their arguments based on their “dwarfs” (shouldn’t that be “vertically challenged algorithm”?).

And their dwarfs in turn seem entirely selected to then support the end result they wanted. Can anybody say “circular argument” ten times fast?

Apart from the obvious graphics thing, none of their loads seem at all relevant to “general purpose computing”, they are all essentially about scientific computing.

And we already pretty much know the solution to scientific computing: throw lots of cheap hardware on it (where “cheap” is then defined by what is mass-produced for other reasons).

Designing future hardware around the needs of scientific computing seems ass-backwards. It’s putting the cart in front of the horse.

Linus

Which after a couple of threads -and good comments from others, received this answer (abridged)

Name: David Patterson (pattrsn@cs.berkeley.edu) 2/15/08

Since we spent almost 2 years of our lives working on the this report, I’d add my perspective to this discussion.
* The goal is to raise the level of abstraction to allow people the space that we’ll need to be able to make the manycore bet work, rather than to be hamstrung by 15-year old legacy code written in 30-year old progamming languages.

…

* Based on our 2 year investigation, we make the provactive claim that your programming language, compiler, libraries, computer architecture … better be able to handle these design patterns well, because they will be important in the upcoming decade in many apps. There are likely more design patterns that these 13, but they include, for example
– Finite State Machines
– Branch and Bound
– Graph Algorithms
which aren’t in most people’s lists of scientific computing problems.

Our bet is that the best applications, the best programming languages, the best libraries,… have not yet been written.

The challenge for this next generation of software to be correct, efficient, and scale with the increasing number of processors, without overburdening programmers. If we as field can succeed at this amazingly difficult challenge, the future looks good. If not, then performance increases we have relied upon for decades will come to an abrupt halt, likely diminishing the future of the IT industry.

Dave Patteron, UC Berkeley

Which received this answer from Torvalds (abridged)

David Patterson (pattrsn@cs.berkeley.edu) on 2/15/08 wrote:
>
>* The goal is to raise the level of abstraction to allow people the space that we’ll need to be able to make the manycore bet work, rather than to be hamstrung by 15-year old legacy code written in 30-year old progamming languages.

Well, you basically start off by just assuming it can work, and that nothing else can.

That’s a big assumption. It’s by no means something you should take for granted. It’s a wish that hasn’t come to fruition so far, and quite frankly, I don’t think people are really any closer to a solution today than they were two decades ago.

The fact that you can find application domains where it does work isn’t new.

We’ve had our CM-5’s, we’ve had our Occam programs, and there’s no question they worked. The question is whether they work for general-purpose computing, and that one is still unanswered, I think.

…

The problem isn’t CPU time, it’s memory and IO footprint.

And I don’t think that is unheard of elsewhere. The core algorithm could well be parallelizable, but the problem isn’t the linear CPU speed, it’s the things outside the CPU.

>Our bet is that the best applications, the best programming languages, the best libraries,… have not yet been written.

If we are looking at a 100+ core future, I certainly agree.

>If we as field can succeed at this amazingly difficult challenge, the future looks good. If not, then performance increases we have relied upon for decades will come to an abrupt halt, likely dimishing the future of the IT industry.

Here’s my personal prediction, and hey, it’s just that: a guess:
(a) we’ll continue to be largely dominated by linear  issues in a majority of loads.
(b) this may well mean that the future of GP computing ends up being about small, and low power, and being absolutely everywhere (== really dirty cheap).

IOW, the expectation of exponential performance scaling may simply not be the thing that we even want. Yeah, we’ll get it for those nice parallel loads, but rather than expect everything to get there, maybe we should just look forward to improving IT in other directions than pure performance.

If the choice becomes one of “parallel but fast machines” and “really small and really cheap and really low power and ‘fast enough’ ones with just a coule of cores”, maybe people will really pick the latter.

Especially if it proves that the parallel problem really isn’t practically solvable for a lot of things that people want to do.

Pessimistic? It depends on what you look forward to.

Linus

My comment?

If the funding went to a university somewhere else and not Berkeley, probably Torvalds  wouldn’t comment.

If someone else was making the observation, Patterson probably wouldn’t even notice it and answer.

Is this a personalities game, or there is some potential here? Funding goes to the most unusual places with the most unrealistic expectations. As Torvalds said in his speech in 2001, you never start a project in programming knowing exactly how will be the final result…

And I’m not saying that he is wrong in his observations, but it sounds that both are more focused on the “right/wrong” position and defending their pride.

Patterson also said in 2006 that “…maybe we should be putting a big RAMP box out there on the Internet for the open source community, to let them play with a highly scalable processor and see what ideas they can come up with. I guess that’s the right question: What can we do to engage the open source community to get innovative people… The parallel solutions may not come from academia or from research labs as they did in the past”.

Are these two worlds coming closer?

Nicolas Erdody

Wellington