[IPython-user] Cannot start ipcluster

Brian Granger ellisonbg.net@gmail....
Sat Oct 17 23:50:56 CDT 2009


> Without ipcluster local -n 4, when I just launch an ipython shell and
> execute these two lines, I get an error:
>
>
What if you do it *with* first doing ipcluster local -n 4 in another
terminal window (even if there are all those warnings)?

Cheeers,

Brian


>
> I[1]: from IPython.kernel.client import MultiEngineClient
> /usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
> DeprecationWarning: the sha module is deprecated; use the hashlib module
> instead
>   import sha
> /usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
> DeprecationWarning: the sets module is deprecated
>
> I[2]: mec = MultiEngineClient()
>
> ---------------------------------------------------------------------------
> ConnectionRefusedError                    Traceback (most recent call last)
>
> /home/gsever/Desktop/<ipython console> in <module>()
>
> /home/gsever/Desktop/python-repo/ipython/IPython/kernel/client.pyc in
> get_multiengine_client(furl_or_file)
>      67     """
>      68     client =
> blockingCallFromThread(_client_tub.get_multiengine_client,
> ---> 69         furl_or_file)
>      70     return client.adapt_to_blocking_client()
>      71
>
> /home/gsever/Desktop/python-repo/ipython/IPython/kernel/twistedutil.pyc in
> blockingCallFromThread(f, *a, **kw)
>      70         @raise: any error raised during the callback chain.
>      71         """
> ---> 72         return
> twisted.internet.threads.blockingCallFromThread(reactor, f, *a, **kw)
>      73
>      74 else:
>
> /usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/internet/threads.pyc
> in blockingCallFromThread(reactor, f, *a, **kw)
>     112     result = queue.get()
>     113     if isinstance(result, failure.Failure):
> --> 114         result.raiseException()
>     115     return result
>     116
>
> /usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/failure.pyc
> in raiseException(self)
>     324         information if available.
>     325         """
> --> 326         raise self.type, self.value, self.tb
>     327
>     328
>
> ConnectionRefusedError: Connection was refused by other side: 111:
> Connection refused.
>
>
> I had installed Twisted and other parallel computing requirements following
> the instructions that were listed on the documentation pages.
>
>
>
>> mec.get_ids()
>>
>> It may all be working...all you see are deprecation warnings related to
>> Python 2.6.
>>
>> One more word.  The parallel computing stuff is not working in trunk right
>> now (I am fixing it),
>> so please stick with 0.10.
>>
>
> I was bitten by the trunk a few times. For over a month I am using the
> downgraded (0.10) version :)
>
>
>>
>> Cheers,
>>
>> Brian
>>
>> On Sat, Oct 17, 2009 at 5:41 PM, Gökhan Sever <gokhansever@gmail.com>wrote:
>>
>>> Hello,
>>>
>>> I want to experiment IPython's parallel computing functionality. This far
>>> I couldn't progress much because ipcluster instantiation stalls giving the
>>> following messages without dropping me into the main IPython shell.
>>>
>>> My intention is parallelise a small Python script that calls an external
>>> set of scripts that process the dataset I have in-hand. It is not a huge
>>> computing power demanding task but in my Intel 2.5Ghz Dual Core 2 it takes
>>> about 1.5 hours to process the whole dataset. Looking at the system monitor
>>> I see that the workload is not equally distributed in between CPUs (one of
>>> them usually much lazier than the other.) I am sure parallezing the code run
>>> would boost the processing speed. In my dataset I have 17 folders and each
>>> folder is independent from each other. My script visits each folder and
>>> calls the main external script via subprocess module's call function.
>>> Processing starts with the first folder, and doesn't work on the next folder
>>> unless the processing finishes with the previous folder. Basically, what I
>>> really want is to put externally called scripts into separate threads, so
>>> that I don't need to wait the previous job to be done during the processing
>>> process.
>>>
>>> From the IPython parallel computing documentation, it seems like what I
>>> want is doable in IPython. However I need some advice whether my
>>> understanding is correct in this aspect. Also for the solution of the below
>>> warning messages.
>>>
>>> Thanks.
>>>
>>>
>>> [gsever@ccn Desktop]$  ipcluster local -n 4
>>> /usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>>> instead
>>>   import sha
>>> /usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>>> DeprecationWarning: the sets module is deprecated
>>> 2009-10-17 18:59:37-0500 [-] Log opened.
>>> 2009-10-17 18:59:37-0500 [-] Process ['ipcontroller',
>>> '--logfile=/home/gsever/.ipython/log/ipcontroller'] has started with
>>> pid=11066
>>> 2009-10-17 18:59:37-0500 [-] Waiting for controller to finish starting...
>>> 2009-10-17 18:59:38-0500 [-]
>>> '/usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>>> instead\n  import sha\n'
>>> 2009-10-17 18:59:38-0500 [-]
>>> '/usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>>> DeprecationWarning: the sets module is deprecated\n'
>>> 2009-10-17 18:59:39-0500 [-] Controller started
>>> 2009-10-17 18:59:39-0500 [-] Process ['ipengine',
>>> '--logfile=/home/gsever/.ipython/log/ipengine11066-'] has started with
>>> pid=11067
>>> 2009-10-17 18:59:39-0500 [-] Process ['ipengine',
>>> '--logfile=/home/gsever/.ipython/log/ipengine11066-'] has started with
>>> pid=11068
>>> 2009-10-17 18:59:39-0500 [-] Process ['ipengine',
>>> '--logfile=/home/gsever/.ipython/log/ipengine11066-'] has started with
>>> pid=11069
>>> 2009-10-17 18:59:39-0500 [-] Process ['ipengine',
>>> '--logfile=/home/gsever/.ipython/log/ipengine11066-'] has started with
>>> pid=11070
>>> 2009-10-17 18:59:39-0500 [-] Engines started with pids: [11067, 11068,
>>> 11069, 11070]
>>> 2009-10-17 18:59:39-0500 [-]
>>> '/usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>>> instead\n  import sha\n'
>>> 2009-10-17 18:59:39-0500 [-]
>>> '/usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>>> instead\n  import sha\n'
>>> 2009-10-17 18:59:39-0500 [-]
>>> '/usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>>> DeprecationWarning: the sets module is deprecated\n'
>>> 2009-10-17 18:59:40-0500 [-]
>>> '/usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>>> DeprecationWarning: the sets module is deprecated\n'
>>> 2009-10-17 18:59:40-0500 [-]
>>> '/usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>>> instead\n  import sha\n'
>>> 2009-10-17 18:59:40-0500 [-]
>>> '/usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>>> instead\n  import sha\n'
>>> 2009-10-17 18:59:40-0500 [-]
>>> '/usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>>> DeprecationWarning: the sets module is deprecated\n'
>>> 2009-10-17 18:59:40-0500 [-]
>>> '/usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>>> DeprecationWarning: the sets module is deprecated\n'
>>>
>>>
>>> Here is my system info:
>>>
>>> ================================================================================
>>> Platform     :
>>> Linux-2.6.29.6-217.2.3.fc11.i686.PAE-i686-with-fedora-11-Leonidas
>>> Python       : ('CPython', 'tags/r26', '66714')
>>> IPython      : 0.10
>>> NumPy      : 1.4.0.dev
>>>
>>> ================================================================================
>>>
>>> --
>>> Gökhan
>>>
>>> _______________________________________________
>>> IPython-user mailing list
>>> IPython-user@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-user
>>>
>>>
>>
>
>
> --
> Gökhan
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20091017/2770f55c/attachment.html 


More information about the IPython-user mailing list