I know that we'll be doing some of the assignments in these parallel programming languages, but I'm curious to see how much abstractions/mechanisms can actually help to parallelize and speed-up programs. For instance, it seems like the slow-downs in previous slides can occur because of the programmer, such as unevenly distributing workload or causing high communication latencies between two dependent processes. How much does CUDA help to parallelize code require a talented developer or does it only provide the tools, leaving a significant portion of parallelization up to the programmer?
I know that we'll be doing some of the assignments in these parallel programming languages, but I'm curious to see how much abstractions/mechanisms can actually help to parallelize and speed-up programs. For instance, it seems like the slow-downs in previous slides can occur because of the programmer, such as unevenly distributing workload or causing high communication latencies between two dependent processes. How much does CUDA help to parallelize code require a talented developer or does it only provide the tools, leaving a significant portion of parallelization up to the programmer?