The Wayback Machine - https://web.archive.org/web/20100106152936/http://datamining.anu.edu.au:80/~ole/pypar/

Pypar has moved to Google Code

You can now browse the Subversion repository for pypar: here

To checkout pypar anonymously (read only)
svn checkout http://pypar.googlecode.com/svn pypar

To checkout pypar as a Google Code user
svn checkout https://pypar.googlecode.com/svn pypar --username

If you wish to participate in the development or just become part of the community, email Ole.Moller.Nielsen@gmail.com or subscribe to the mailing list at http://groups.google.com/group/pypar-discuss?pli=1

You can still browse the old repository at Sourceforge

You can also download the last (ANU) version of pypar here: pypar_1.9.3.tgz

Back to Ole Nielsen's Home Page

Pypar (last update: 24 April 2007)

Parallel Programming in the spirit of Python!

Pypar is an efficient but easy-to-use module that allows programs/scripts written in the Python programming language to run in parallel on multiple processors and communicate using message passing. Pypar provides bindings to an important subset of the message passing interface standard MPI. Other Python MPI bindings available from other developers include: PyMPI, Scientific Python and pythonMPI.

Pypar is characterised by the following features

Pypar is flexible:
General Python objects of any type can be communicated - not just arrays. Optional functionality is available through keyword arguments.
Pypar is easy to use:
The user need only specify what to send and to which processor. Pypar takes care of details about data types, size, dimension and MPI specifics such as tags, communicators and buffers. A typical send and receive pair looks like

pypar.send(x, destination=1) # Send object x to processor 1

y=pypar.receive(source=1) # Receive object y from processor 1
Pypar is efficient:
Full bandwidth of C-MPI programs is achieved for consecutive Numerical arrays. Latency is less than twice that of pure C-MPI programs. Test programs to verify this are included (pytiming, ctiming.c).
Pypar is easy to install:
Installs using standard distutils.
Pypar is light weight:
Pypar consists of only two files: pypar.py and the c-extension mpiext.c. It depends only Python, an MPI library (C), Numeric Python and a C compiler.
Pypar does not require the Python interpreter to be modified or recompiled:
Parallel python programs use the standard Python and need merely import the pypar module. This means for example that you can upgrade Python independently of your parallel codes.
Pypar is easy to extend:
With the framework in place it is not hard to include additional MPI functionality should the need arise. In fact, many contributors have done exactly that in the past (see README for a list of contributors) Alternatively, drop me a note if there is some functionality you would like to see included and I'll add it to my 'todo' list.

The Pypar module has been released as open source under the GNU General Public License.

Open Source - Of Course

You can take a look at the source code without having to download and unpack the tarball:

pypar.py: The Python interface
mpiext.c: The C-extension

Documentation

There is some documentation in the README and DOC files that come with the Pypar distribution. Here are the online copies

You can also try out the

PyPar Tutorial (pdf)

written for the our Summerschool in computational mathematics. The text mentions files to be copied from a particular directory - those files are now the pypar distribution. Two other related tutorials from the summerschool are available here

Links to people who have contributed and projects that use pypar

Here is a few links to papers and projects relating to pypar projects.html

Download

You can download the source, tests documentation and examples for Pypar at http://datamining.anu.edu.au/software/pypar/pypar_1.9.3.tgz

Example

Here is an example of a simple program using Pypar ring_example.py. It basically communicates some text in a ring structure. To run it on four MPI processes, say, simply issue the command:

mpirun -np 4 python ring_example.py

and thanks to Constantinos Makassikis for helping me render this code nicely in html with code2blog. Thanks heaps!

import pypar                         # The Python-MPI interface 

numproc = pypar.size()               # Number of processes as specified by mpirun
myid =    pypar.rank()               # Id of of this process (myid in [0, numproc-1]) 
node =    pypar.get_processor_name() # Host name on which current process is running

print 'I am proc %d of %d on node %s' %(myid, numproc, node)

if myid == 0: 
    # Actions for process 0 
    
    msg = 'P0'  
    pypar.send(msg, destination=1)        # Send message to proces 1 (right hand neighbour)
    msg = pypar.receive(source=numproc-1) # Receive message from last process

    print 'Processor 0 received message "%s" from processor %d' %(msg, numproc-1)

else:         
    # Actions for all other processes

    source = myid-1                                  
    destination = (myid+1)%numproc                   

    msg = pypar.receive(source)                      
    msg = msg + 'P' + str(myid)                      # Update message     

    pypar.send(msg, destination)                     

pypar.finalize()

| Home Page | Data Mining |

Ole Nielsen - <ole.moller.nielsen at gmail.com>

Last modified: December 23 2009 (/home/ole/public_html/software/pyparweb/index.php)

Dec	JAN	Mar
	06
2008	2010	2011

	pypar.send(x, destination=1)	# Send object x to processor 1
	y=pypar.receive(source=1)	# Receive object y from processor 1