[IPython-User] Question about schedulers

Fernando Perez fperez.net@gmail....
Wed Jun 6 13:26:32 CDT 2012


On Wed, Jun 6, 2012 at 2:01 AM, Darren Govoni <darren@ontrenet.com> wrote:
>  Let's say I have 10 engines and a list of 20 objects I want to map
> using a load balanced view. How will the scheduler distribute the 20
> messages? Assuming all engines are equal, will the first 10 objects be
> distributed to 1 engine each and the second 10 objects will wait for an
> engine to be free then go there? Or will all 20 messages be spread to
> the engines at the same time?

The load balancing is dynamic, so tasks will go to the engine as they
become available.  If you have an estimate of your task durations it's
still a good idea to schedule the longest first so they start
earliest, rather than having one long task starting last.  But if you
have no prior knowledge of this kind, at least the dynamic scheduling
gives you the best possible results.

Cheers,

f


More information about the IPython-User mailing list