[Cplex] Keywords, compliance, and a parallel UNCOL.

Nelson, Clark clark.nelson at intel.com
Sat Jun 29 01:01:41 CEST 2013

> I realize that the group is focused on C, but if C++ buy-in is
> wanted, from discussions I've seen, I'm pretty sure that keywords
> are not going work out. Would there be any interest in moving
> towards something similar to the C++ attribute syntax? For
> example:
> [[spawn]] for(int i = 0; i < 32; i++) {...

Concerning keywords: There are a few people in the C++ committee who are
strongly opposed to requiring support for parallelism in the compiler, but
the clear majority (at least in the parallelism study group) are at least
weakly in favor of it. Personally, I have no idea what's eventually going to
happen in WG21.

But I know one thing for sure: attributes are not a practical solution to
the problem of standardization. The consensus in the C++ committee seems to
be that if a feature has any semantics, it's not appropriate to express it
as an attribute. There are at least three features of C++11 that were
originally proposed to be attributes, in order to avoid having to invent a
new keyword. But they were deemed to have semantics, so an alternate syntax
was eventually found.

Basically, nothing that's interesting can be expressed as an attribute. And
parallelism is interesting.

> My compliance question goes along with the syntax. Assuming these
> features go into the standard, should they be marked optional for
> compliance? Is there any feeling that vendors would choose not to
> adopt a standard if it requires these constructs? If they aren't
> required, I'm thinking keywords again might not be the best
> implementation. An implementation could be free to ignore
> attributes they don't support.

A penetrating question.

Of course they will first go into a Technical Specification, which no one
will be required to implement. But let's go ahead and jump to the eventual

In my opinion, what we should be trying to achieve is support for
parallelism, not concurrency -- in other words, using multiple threads as a
way of accelerating a computation, not as a way of writing programs that
require multiple things to be going on at once, such as producers
interacting with consumers. Concurrency is just intrinsically a harder
problem than parallelism, and it's not clear to me that we would be doing
the world a favor if we were to make it easier to attempt without putting a
lot of thought into it.

There are certainly circumstances in which adding another thread is going to
slow down the computation, not speed it up. And that may be true even when
there's only one thread already in use. If some component of the system is
aware of that fact, or is even able to make a shrewd guess that it's likely,
I'd like it to be understood that that component is free to decline to add
another thread.

So I think it's important that all parallelism should be considered optional
-- whether invoked by a syntax that involves keywords, pragmas, or
attributes. The syntax needs to be recognized, of course, but it could wind
up being ineffective.

On the other hand, if the invoking syntax involves a call to a function
named, say, thrd_create, I think the implementation had better assume that
the goal might be concurrency, not parallelism, and do its best to comply.

> Finally, the UNCOL comment:  I don't disagree with the idea that a
> common scheduling substrate/runtime is needed, but I'm worried
> about falling into the trap of trying to create the impossible.
> Given the different requirements we've already seen voiced between
> "regular" and "HPC" users desires for control, I'm concerned that
> there would be the need for too many knobs to try to support all
> these different visions of parallelism.

I absolutely share your concerns.

> So, is the focus here just
> on a common substrate that works with the Cilk and OpenMP models,
> or is it define a substrate upon which someone could build their
> own vision of parallelism which would then compose with C, OpenMP,
> Cilk, etc?

The proposal seems to be the latter; a wide variety of languages and systems
were mentioned.

> As a follow on to the above, is the focus here going to be simply
> shared memory parallelism or will spawn be allowed to off-load
> work to an accelerator? Data movement becomes critical if that is
> allowed, and would probably add to the complexity of this base
> runtime.

Another very good question. My impression is that there is no mature
language support for distributed-memory systems; in other words, we're short
of existing practice that's ready for standardization. (MPI is mature, of
course, but it's library, not language.)

But I sure wouldn't mind learning otherwise.


More information about the Cplex mailing list