[IPython-user] Cannot start ipcluster

Gökhan Sever gokhansever@gmail....
Sun Oct 18 00:06:41 CDT 2009


On Sat, Oct 17, 2009 at 11:50 PM, Brian Granger <ellisonbg.net@gmail.com>wrote:

>
> Without ipcluster local -n 4, when I just launch an ipython shell and
>> execute these two lines, I get an error:
>>
>>
> What if you do it *with* first doing ipcluster local -n 4 in another
> terminal window (even if there are all those warnings)?
>
>
I can't drop into ipython in this case :(

Doing "ps -aux | grep ip" yields

gsever   15467  0.7  0.3  20700 14264 pts/2    S+   00:02   0:00
/usr/bin/python /usr/bin/ipcluster local -n 4
gsever   15468  1.1  0.5  40424 23544 pts/2    S+   00:02   0:00
/usr/bin/python /usr/bin/ipcontroller
--logfile=/home/gsever/.ipython/log/ipcontroller
gsever   15469  1.1  0.4  36980 20332 pts/2    S+   00:02   0:00
/usr/bin/python /usr/bin/ipengine
--logfile=/home/gsever/.ipython/log/ipengine15468-
gsever   15470  1.1  0.5  37396 20668 pts/2    S+   00:02   0:00
/usr/bin/python /usr/bin/ipengine
--logfile=/home/gsever/.ipython/log/ipengine15468-
gsever   15471  0.9  0.4  37064 20392 pts/2    S+   00:02   0:00
/usr/bin/python /usr/bin/ipengine
--logfile=/home/gsever/.ipython/log/ipengine15468-
gsever   15472  1.2  0.4  36976 20328 pts/2    S+   00:02   0:00
/usr/bin/python /usr/bin/ipengine
--logfile=/home/gsever/.ipython/log/ipengine15468-

and system monitor shows this tasks as sleeping. See the screenshot :
http://img148.imageshack.us/img148/6240/screenshotsystemmonitor.png

I hope this is not another Fedora 11 related bug.


> Cheeers,
>
> Brian
>
>
>>
>> I[1]: from IPython.kernel.client import MultiEngineClient
>> /usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>> instead
>>   import sha
>> /usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>> DeprecationWarning: the sets module is deprecated
>>
>> I[2]: mec = MultiEngineClient()
>>
>>
>> ---------------------------------------------------------------------------
>> ConnectionRefusedError                    Traceback (most recent call
>> last)
>>
>> /home/gsever/Desktop/<ipython console> in <module>()
>>
>> /home/gsever/Desktop/python-repo/ipython/IPython/kernel/client.pyc in
>> get_multiengine_client(furl_or_file)
>>      67     """
>>      68     client =
>> blockingCallFromThread(_client_tub.get_multiengine_client,
>> ---> 69         furl_or_file)
>>      70     return client.adapt_to_blocking_client()
>>      71
>>
>> /home/gsever/Desktop/python-repo/ipython/IPython/kernel/twistedutil.pyc in
>> blockingCallFromThread(f, *a, **kw)
>>      70         @raise: any error raised during the callback chain.
>>      71         """
>> ---> 72         return
>> twisted.internet.threads.blockingCallFromThread(reactor, f, *a, **kw)
>>      73
>>      74 else:
>>
>> /usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/internet/threads.pyc
>> in blockingCallFromThread(reactor, f, *a, **kw)
>>     112     result = queue.get()
>>     113     if isinstance(result, failure.Failure):
>> --> 114         result.raiseException()
>>     115     return result
>>     116
>>
>> /usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/failure.pyc
>> in raiseException(self)
>>     324         information if available.
>>     325         """
>> --> 326         raise self.type, self.value, self.tb
>>     327
>>     328
>>
>> ConnectionRefusedError: Connection was refused by other side: 111:
>> Connection refused.
>>
>>
>> I had installed Twisted and other parallel computing requirements
>> following the instructions that were listed on the documentation pages.
>>
>>
>>
>>> mec.get_ids()
>>>
>>> It may all be working...all you see are deprecation warnings related to
>>> Python 2.6.
>>>
>>> One more word.  The parallel computing stuff is not working in trunk
>>> right now (I am fixing it),
>>> so please stick with 0.10.
>>>
>>
>> I was bitten by the trunk a few times. For over a month I am using the
>> downgraded (0.10) version :)
>>
>>
>>>
>>> Cheers,
>>>
>>> Brian
>>>
>>> On Sat, Oct 17, 2009 at 5:41 PM, Gökhan Sever <gokhansever@gmail.com>wrote:
>>>
>>>> Hello,
>>>>
>>>> I want to experiment IPython's parallel computing functionality. This
>>>> far I couldn't progress much because ipcluster instantiation stalls giving
>>>> the following messages without dropping me into the main IPython shell.
>>>>
>>>> My intention is parallelise a small Python script that calls an external
>>>> set of scripts that process the dataset I have in-hand. It is not a huge
>>>> computing power demanding task but in my Intel 2.5Ghz Dual Core 2 it takes
>>>> about 1.5 hours to process the whole dataset. Looking at the system monitor
>>>> I see that the workload is not equally distributed in between CPUs (one of
>>>> them usually much lazier than the other.) I am sure parallezing the code run
>>>> would boost the processing speed. In my dataset I have 17 folders and each
>>>> folder is independent from each other. My script visits each folder and
>>>> calls the main external script via subprocess module's call function.
>>>> Processing starts with the first folder, and doesn't work on the next folder
>>>> unless the processing finishes with the previous folder. Basically, what I
>>>> really want is to put externally called scripts into separate threads, so
>>>> that I don't need to wait the previous job to be done during the processing
>>>> process.
>>>>
>>>> From the IPython parallel computing documentation, it seems like what I
>>>> want is doable in IPython. However I need some advice whether my
>>>> understanding is correct in this aspect. Also for the solution of the below
>>>> warning messages.
>>>>
>>>> Thanks.
>>>>
>>>>
>>>> [gsever@ccn Desktop]$  ipcluster local -n 4
>>>> /usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>>>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>>>> instead
>>>>   import sha
>>>> /usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>>>> DeprecationWarning: the sets module is deprecated
>>>> 2009-10-17 18:59:37-0500 [-] Log opened.
>>>> 2009-10-17 18:59:37-0500 [-] Process ['ipcontroller',
>>>> '--logfile=/home/gsever/.ipython/log/ipcontroller'] has started with
>>>> pid=11066
>>>> 2009-10-17 18:59:37-0500 [-] Waiting for controller to finish
>>>> starting...
>>>> 2009-10-17 18:59:38-0500 [-]
>>>> '/usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>>>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>>>> instead\n  import sha\n'
>>>> 2009-10-17 18:59:38-0500 [-]
>>>> '/usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>>>> DeprecationWarning: the sets module is deprecated\n'
>>>> 2009-10-17 18:59:39-0500 [-] Controller started
>>>> 2009-10-17 18:59:39-0500 [-] Process ['ipengine',
>>>> '--logfile=/home/gsever/.ipython/log/ipengine11066-'] has started with
>>>> pid=11067
>>>> 2009-10-17 18:59:39-0500 [-] Process ['ipengine',
>>>> '--logfile=/home/gsever/.ipython/log/ipengine11066-'] has started with
>>>> pid=11068
>>>> 2009-10-17 18:59:39-0500 [-] Process ['ipengine',
>>>> '--logfile=/home/gsever/.ipython/log/ipengine11066-'] has started with
>>>> pid=11069
>>>> 2009-10-17 18:59:39-0500 [-] Process ['ipengine',
>>>> '--logfile=/home/gsever/.ipython/log/ipengine11066-'] has started with
>>>> pid=11070
>>>> 2009-10-17 18:59:39-0500 [-] Engines started with pids: [11067, 11068,
>>>> 11069, 11070]
>>>> 2009-10-17 18:59:39-0500 [-]
>>>> '/usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>>>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>>>> instead\n  import sha\n'
>>>> 2009-10-17 18:59:39-0500 [-]
>>>> '/usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>>>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>>>> instead\n  import sha\n'
>>>> 2009-10-17 18:59:39-0500 [-]
>>>> '/usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>>>> DeprecationWarning: the sets module is deprecated\n'
>>>> 2009-10-17 18:59:40-0500 [-]
>>>> '/usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>>>> DeprecationWarning: the sets module is deprecated\n'
>>>> 2009-10-17 18:59:40-0500 [-]
>>>> '/usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>>>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>>>> instead\n  import sha\n'
>>>> 2009-10-17 18:59:40-0500 [-]
>>>> '/usr/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-i686.egg/twisted/python/filepath.py:12:
>>>> DeprecationWarning: the sha module is deprecated; use the hashlib module
>>>> instead\n  import sha\n'
>>>> 2009-10-17 18:59:40-0500 [-]
>>>> '/usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>>>> DeprecationWarning: the sets module is deprecated\n'
>>>> 2009-10-17 18:59:40-0500 [-]
>>>> '/usr/lib/python2.6/site-packages/foolscap-0.4.2-py2.6.egg/foolscap/banana.py:2:
>>>> DeprecationWarning: the sets module is deprecated\n'
>>>>
>>>>
>>>> Here is my system info:
>>>>
>>>> ================================================================================
>>>> Platform     :
>>>> Linux-2.6.29.6-217.2.3.fc11.i686.PAE-i686-with-fedora-11-Leonidas
>>>> Python       : ('CPython', 'tags/r26', '66714')
>>>> IPython      : 0.10
>>>> NumPy      : 1.4.0.dev
>>>>
>>>> ================================================================================
>>>>
>>>> --
>>>> Gökhan
>>>>
>>>> _______________________________________________
>>>> IPython-user mailing list
>>>> IPython-user@scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-user
>>>>
>>>>
>>>
>>
>>
>> --
>> Gökhan
>>
>
>


-- 
Gökhan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20091018/278a1429/attachment.html 


More information about the IPython-user mailing list