Parallel Programming in C with MPI and OpenMP

Michael Jay Quinn

Mentioned 2

This volume gives a high-level overview of parallel architectures, including processor arrays, centralized multi-processors, distributed multi-processors, commercial multi-computers and commodity clusters. A six-chapter tutorial introduces 25 MPI functions by developing parallel programs to solve a series of increasingly difficult problems. Each program is taken from problem description through design and analysis to implementation and benchmarking on an actual commodity cluster, providing the reader with a wealth of examples.

More on

Mentioned in questions and answers.

I am working to become a scientific programmer. I have enough background in Math and Stat but rather lacking on programming background. I found it very hard to learn how to use a language for scientific programming because most of the reference for SP are close to trivial.

My work involves statistical/financial modelling and none with physics model. Currently, I use Python extensively with numpy and scipy. Done R/Mathematica. I know enough C/C++ to read code. No experience in Fortran.

I dont know if this is a good list of language for a scientific programmer. If this is, what is a good reading list for learning the syntax and design pattern of these languages in scientific settings.

this might be useful: the nature of mathematical modeling

Writing Scientific Software: A Guide to Good Style is a good book with overall advice for modern scientific programming.

I'm a scientific programmer who just entered the field in the past 2 years. I'm into more biology and physics modeling, but I bet what you're looking for is pretty similar. While I was applying to jobs and internships there were two things that I didn't think would be that important to know, but caused me to end up missing out on opportunities. One was MATLAB, which has already been mentioned. The other was database design -- no matter what area of SP you're in, there's probably going to be a lot of data that has to be managed somehow.

The book Database Design for Mere Mortals by Michael Hernandez was recommended to me as being a good start and helped me out a lot in my preparation. I would also make sure you at least understand some basic SQL if you don't already.

One issue scientific programmers face is maintaining a repository of code (and data) that others can use to reproduce your experiments. In my experience this is a skill not required in commercial development.

Here are some readings on this:

These are in the context of computational biology but I assume it applies to most scientific programming.

Also, look at Python Scripting for Computational Science.

I am getting ready to do some research in parallel computing next semester, and I'd love to have the MPI down pat by the start of the school year. However, the top-ranked hits on Amazon are very old, the newest being from 1999. Are there any more modern texts? Thanks very much.

This one is the one that I have, and I am quite fond of it:

It is pretty much the same age as the ones you found (2003), but MPI did not change much over that time period.