Discussion of the Myrinet interconnect and GM

tomanj2 | 18 Sep 2006 21:47

Isend max.message size?

Hello,

I'm here for first time and I'm interested about myrinet technology.
Now, I'm also student of Czech Technical University in Praque, chair of Computer
Science.
I am implementing Segmented version of paralel quicksort alghoritm which I
tested on university's myrinet cluster called star(star.felk.cvut.cz).

The problem is:
When I used MPI_Isend function for exchanging data between processors (sending
for ex. one huge message containing thousands of MPI_DOUBLE members!) and test
the end of the Isend by MPI_Test function...the repeat loop will never end
(means that data won't be send).

When the count (number of members are less) then everything is O.K. and MPI_Test
 will finish both loops.

I try to write simple code for better undertanding:

.....
MPI_Isend(databuf,count,MPI_DOUBLE,dest,tag,MPI_COMM_WORLD,&request1);
MPI_Isend(indexbuf,count, MPI_UNSIGNED,dest,tag,MPI_COMM_WORLD,&request2);
do {
      MPI_Test(&request1, &flg, &status);
    } while (!flg);
do {
      MPI_Test(&request2, &flg, &status);
    } while (!flg);

/* dealocation of buffers */

(Continue reading)

headers

Patrick Geoffray | 18 Sep 2006 22:18

Re: Isend max.message size?

Hi Jiri,

tomanj2 <at> fel.cvut.cz wrote:
> The problem is:
> When I used MPI_Isend function for exchanging data between processors (sending
> for ex. one huge message containing thousands of MPI_DOUBLE members!) and test
> the end of the Isend by MPI_Test function...the repeat loop will never end
> (means that data won't be send).
> 
> When the count (number of members are less) then everything is O.K. and MPI_Test
>  will finish both loops.

> Is it a problem with count parameter (number of items) resp.  message is too
> huge? What is the maximal message size that I can send?

The maximum message size is 4GB (length coded on 32 bits), 2GB if signed 
types are used somewhere. However, I suspect that you are far from these 
limits. In most MPI implementations, there is a different behavior if 
the message size is below or above a specific threshold (~32 KB). If the 
message is small enough, it will be sent wherever the matching receive 
as been posted on the other side. In this case, there is the possibility 
that the message will be unexpected (receive not yet posted), so it 
would need to be saved in temporary buffer. If the message is too big, 
it is not reasonable to risk it to be unexpected, so the data is sent 
only if the receive has been posted, there is a rendez-vous step 
(synchronization).

I would suspect that you do not post a valid matching MPI receive and, 
depending on the size of the message, the send completes eagerly or not.

(Continue reading)