[IPython-user] [IPython-dev] Twisted 8...
Barry Wark
barrywark@gmail....
Wed Apr 2 01:23:08 CDT 2008
On Tue, Apr 1, 2008 at 9:58 PM, Brian Granger <ellisonbg.net@gmail.com> wrote:
> I figured it would mostly "just work". I wouldn't even worry about
> the failure and error. The error is a particularly nasty one that
> probably only appears sometimes. I can look into it at some point.
> The failure is something in the new daemon that Min is working on - it
> is possible that the test simply doesn't pass right now.
OK. Good to know.
>
> The bigger thing that we need to decide is what to do with
> twisted.web2. Currently we ship a hacked version of it in
> ipython1.externals. We use it for the notebook and other http
> protocols related things. I don't like shipping it like this, but
> given its state, I am not sure we have any other options. Minimally,
> we will have to update the version of web2 that we provide to the
> current trunk, so things work with Twisted 8.0
This is well out of my knowlege of the codebase, so I'll leave
anything related to twisted.web2 for others.
>
> But, let's make a branch for Twisted 8.0 related stuff. We would also
> need to update documentation things and the setupext stuff.
Done. I modified setupext to allow Twisted >=2.5 and modified
ipython1.kernel.twistedutils to define blockingCallFromThread as a
wrapper to twisted.internet.threads.blockingCallFromThread for Twisted
>=8 and the existing version for Twisted <8.
>
> Brian
>
>
>
> On Tue, Apr 1, 2008 at 10:03 PM, Barry Wark <barrywark@gmail.com> wrote:
> > Just FYI, it looks like it won't be that bad getting things to work
> > with Twisted 8. I modified setupext to not barf if twisted's
> > version.major > 2 and installed ipython1. I'll go ahead and branch an
> > ipython1-twisted8 and start fixing the last couple of test... Here's
> > the output of 'trial ipython1' on my machine (twisted 8.0.1):
> >
> > ipython1.core.tests.test_shell
> > BasicShellTest
> > testCommand ... [OK]
> > testExecute ... [OK]
> > testPutGet ... [OK]
> > testUpdate ... [OK]
> > ipython1.daemon.tests.test_daemonservice
> > BasicDaemonServiceTest
> > testIDaemonServiceDeferreds ... [OK]
> > testIDaemonServiceInterface ... [OK]
> > testIDaemonServiceInterfaceMethods ... [OK]
> > testRestart_0 ... [OK]
> > testRestart_1 ... [OK]
> > testShutdown ... [FAIL]
> > testSignal_0 ... [OK]
> > testSignal_1 ... [OK]
> > testSignal_10 ... [OK]
> > testSignal_11 ... [OK]
> > testSignal_12 ... [OK]
> > testSignal_13 ... [OK]
> > testSignal_14 ... [OK]
> > testSignal_15 ... [OK]
> > testSignal_16 ... [OK]
> > testSignal_17 ... [OK]
> > testSignal_18 ... [OK]
> > testSignal_19 ... [OK]
> > testSignal_2 ... [OK]
> > testSignal_20 ... [OK]
> > testSignal_21 ... [OK]
> > testSignal_22 ... [OK]
> > testSignal_23 ... [OK]
> > testSignal_24 ... [OK]
> > testSignal_25 ... [OK]
> > testSignal_26 ... [OK]
> > testSignal_27 ... [OK]
> > testSignal_28 ... [OK]
> > testSignal_29 ... [OK]
> > testSignal_3 ... [OK]
> > testSignal_30 ... [OK]
> > testSignal_31 ... [OK]
> > testSignal_32 ... [OK]
> > testSignal_33 ... [OK]
> > testSignal_4 ... [OK]
> > testSignal_5 ... [OK]
> > testSignal_6 ... [OK]
> > testSignal_7 ... [OK]
> > testSignal_8 ... [OK]
> > testSignal_9 ... [OK]
> > testStartBadProcess_0 ... [OK]
> > testStartBadProcess_1 ... [OK]
> > testStartBadProcess_2 ... [OK]
> > testStartProcess_0 ... [OK]
> > testStartProcess_1 ... [OK]
> > testStartProcess_2 ... [OK]
> > ExternalProcessTest
> > testPoll ... [OK]
> > ipython1.kernel.tests.test_controllerservice
> > BasicControllerServiceTest
> > testBadUnregister ... [OK]
> > testIControllerCoreInterface ... [OK]
> > testIControllerCoreInterfaceMethods ... [OK]
> > testRegisterCallables ... [OK]
> > testRegisterUnregisterEngine ... [OK]
> > testRegisterUnregisterMultipleEngines ... [OK]
> > ipython1.kernel.tests.test_enginepb
> > EnginePBTest
> > testClearQueue ... [OK]
> > testExecuteFailures_0 ... [OK]
> > testExecuteFailures_1 ... [OK]
> > testExecuteFailures_2 ... [OK]
> > testExecuteFailures_3 ... [OK]
> > testExecuteFailures_4 ... [OK]
> > testExecuteFailures_5 ... [OK]
> > testExecuteFailures_6 ... [OK]
> > testExecute_0 ... [OK]
> > testExecute_1 ... [OK]
> > testExecute_10 ... [OK]
> > testExecute_11 ... [OK]
> > testExecute_2 ... [OK]
> > testExecute_3 ... [OK]
> > testExecute_4 ... [OK]
> > testExecute_5 ... [OK]
> > testExecute_6 ... [OK]
> > testExecute_7 ... [OK]
> > testExecute_8 ... [OK]
> > testExecute_9 ... [OK]
> > testGetResultDefault ... [OK]
> > testGetResultFailure ... [OK]
> > testGetResult_0 ... [OK]
> > testGetResult_1 ... [OK]
> > testGetResult_10 ... [OK]
> > testGetResult_11 ... [OK]
> > testGetResult_2 ... [OK]
> > testGetResult_3 ... [OK]
> > [ERROR]
> > testGetResult_4 ... [OK]
> > testGetResult_5 ... [OK]
> > testGetResult_6 ... [OK]
> > testGetResult_7 ... [OK]
> > testGetResult_8 ... [OK]
> > testGetResult_9 ... [OK]
> > testIEngineCoreDeferreds ... [OK]
> > testIEngineCoreInterface ... [OK]
> > testIEngineCoreInterfaceMethods ... [OK]
> > testIEngineQueuedDeferreds ... [OK]
> > testIEngineQueuedInterface ... [OK]
> > testIEngineQueuedInterfaceMethods ... [OK]
> > testIEngineSerializedDeferreds ... [OK]
> > testIEngineSerializedInterface ... [OK]
> > testIEngineSerializedInterfaceMethods ... [OK]
> > testKeys ... [OK]
> > testMessageSizeLimit ... [OK]
> > testNull ... [OK]
> > testPullFunction ... [OK]
> > testPullNameError ... [OK]
> > testPullSerializedFailures ... [OK]
> > testPushFunction ... [OK]
> > testPushFunctionGlobal ... [OK]
> > testPushPullArray ... [OK]
> > testPushPullFailures ... [OK]
> > testPushPullSerialized ... [OK]
> > testPushPull_0 ... [OK]
> > testPushPull_1 ... [OK]
> > testPushPull_2 ... [OK]
> > testPushPull_3 ... [OK]
> > testPushPull_4 ... [OK]
> > testQueueStatus ... [OK]
> > ipython1.kernel.tests.test_engineservice
> > BasicEngineServiceTest
> > testClearProperties ... [OK]
> > testDelHasProperties ... [OK]
> > testExecuteFailures_0 ... [OK]
> > testExecuteFailures_1 ... [OK]
> > testExecuteFailures_2 ... [OK]
> > testExecuteFailures_3 ... [OK]
> > testExecuteFailures_4 ... [OK]
> > testExecuteFailures_5 ... [OK]
> > testExecuteFailures_6 ... [OK]
> > testExecute_0 ... [OK]
> > testExecute_1 ... [OK]
> > testExecute_10 ... [OK]
> > testExecute_11 ... [OK]
> > testExecute_2 ... [OK]
> > testExecute_3 ... [OK]
> > testExecute_4 ... [OK]
> > testExecute_5 ... [OK]
> > testExecute_6 ... [OK]
> > testExecute_7 ... [OK]
> > testExecute_8 ... [OK]
> > testExecute_9 ... [OK]
> > testGetResultDefault ... [OK]
> > testGetResultFailure ... [OK]
> > testGetResult_0 ... [OK]
> > testGetResult_1 ... [OK]
> > testGetResult_10 ... [OK]
> > testGetResult_11 ... [OK]
> > testGetResult_2 ... [OK]
> > testGetResult_3 ... [OK]
> > testGetResult_4 ... [OK]
> > testGetResult_5 ... [OK]
> > testGetResult_6 ... [OK]
> > testGetResult_7 ... [OK]
> > testGetResult_8 ... [OK]
> > testGetResult_9 ... [OK]
> > testGetSetProperties ... [OK]
> > testIEngineCoreDeferreds ... [OK]
> > testIEngineCoreInterface ... [OK]
> > testIEngineCoreInterfaceMethods ... [OK]
> > testIEnginePropertiesInterface ... [OK]
> > testIEnginePropertiesInterfaceMethods ... [OK]
> > testIEngineSerializedDeferreds ... [OK]
> > testIEngineSerializedInterface ... [OK]
> > testIEngineSerializedInterfaceMethods ... [OK]
> > testKeys ... [OK]
> > testPullFunction ... [OK]
> > testPullNameError ... [OK]
> > testPullSerializedFailures ... [OK]
> > testPushFunction ... [OK]
> > testPushFunctionGlobal ... [OK]
> > testPushPullArray ... [OK]
> > testPushPullFailures ... [OK]
> > testPushPullSerialized ... [OK]
> > testPushPull_0 ... [OK]
> > testPushPull_1 ... [OK]
> > testPushPull_2 ... [OK]
> > testPushPull_3 ... [OK]
> > testPushPull_4 ... [OK]
> > testStrictDict ... [OK]
> > QueuedEngineServiceTest
> > testClearProperties ... [OK]
> > testClearQueue ... [OK]
> > testDelHasProperties ... [OK]
> > testExecuteFailures_0 ... [OK]
> > testExecuteFailures_1 ... [OK]
> > testExecuteFailures_2 ... [OK]
> > testExecuteFailures_3 ... [OK]
> > testExecuteFailures_4 ... [OK]
> > testExecuteFailures_5 ... [OK]
> > testExecuteFailures_6 ... [OK]
> > testExecute_0 ... [OK]
> > testExecute_1 ... [OK]
> > testExecute_10 ... [OK]
> > testExecute_11 ... [OK]
> > testExecute_2 ... [OK]
> > testExecute_3 ... [OK]
> > testExecute_4 ... [OK]
> > testExecute_5 ... [OK]
> > testExecute_6 ... [OK]
> > testExecute_7 ... [OK]
> > testExecute_8 ... [OK]
> > testExecute_9 ... [OK]
> > testGetResultDefault ... [OK]
> > testGetResultFailure ... [OK]
> > testGetResult_0 ... [OK]
> > testGetResult_1 ... [OK]
> > testGetResult_10 ... [OK]
> > testGetResult_11 ... [OK]
> > testGetResult_2 ... [OK]
> > testGetResult_3 ... [OK]
> > testGetResult_4 ... [OK]
> > testGetResult_5 ... [OK]
> > testGetResult_6 ... [OK]
> > testGetResult_7 ... [OK]
> > testGetResult_8 ... [OK]
> > testGetResult_9 ... [OK]
> > testGetSetProperties ... [OK]
> > testIEngineCoreDeferreds ... [OK]
> > testIEngineCoreInterface ... [OK]
> > testIEngineCoreInterfaceMethods ... [OK]
> > testIEnginePropertiesInterface ... [OK]
> > testIEnginePropertiesInterfaceMethods ... [OK]
> > testIEngineQueuedDeferreds ... [OK]
> > testIEngineQueuedInterface ... [OK]
> > testIEngineQueuedInterfaceMethods ... [OK]
> > testIEngineSerializedDeferreds ... [OK]
> > testIEngineSerializedInterface ... [OK]
> > testIEngineSerializedInterfaceMethods ... [OK]
> > testKeys ... [OK]
> > testPullFunction ... [OK]
> > testPullNameError ... [OK]
> > testPullSerializedFailures ... [OK]
> > testPushFunction ... [OK]
> > testPushFunctionGlobal ... [OK]
> > testPushPullArray ... [OK]
> > testPushPullFailures ... [OK]
> > testPushPullSerialized ... [OK]
> > testPushPull_0 ... [OK]
> > testPushPull_1 ... [OK]
> > testPushPull_2 ... [OK]
> > testPushPull_3 ... [OK]
> > testPushPull_4 ... [OK]
> > testQueueStatus ... [OK]
> > testStrictDict ... [OK]
> > ipython1.kernel.tests.test_multiengine
> > BasicMultiEngineTestCase
> > testClearProperties ... [OK]
> > testClearQueue ... [OK]
> > testDelHasProperties ... [OK]
> > testExecuteAll ... [OK]
> > testExecuteFailures_0 ... [OK]
> > testExecuteFailures_1 ... [OK]
> > testExecuteFailures_2 ... [OK]
> > testExecuteFailures_3 ... [OK]
> > testExecuteFailures_4 ... [OK]
> > testExecuteFailures_5 ... [OK]
> > testExecuteFailures_6 ... [OK]
> > testGetIDs ... [OK]
> > testGetResultAll ... [OK]
> > testGetResultDefault ... [OK]
> > testGetResultFailure ... [OK]
> > testGetSetProperties ... [OK]
> > testIEngineMultiplexerDeferreds ... [OK]
> > testIEngineMultiplexerInterfaceMethods ... [OK]
> > testIMultiEngineInterface ... [OK]
> > testInvalidEngineID ... [OK]
> > testNoEnginesRegistered ... [OK]
> > testPullFunction ... [OK]
> > testPullFunctionAll ... [OK]
> > testPushFunction ... [OK]
> > testPushFunctionAll ... [OK]
> > testPushPull ... [OK]
> > testPushPullAll ... [OK]
> > testPushPullSerialized ... [OK]
> > testQueueStatus ... [OK]
> > SynchronousMultiEngineTestCase
> > testClearProperties ... [OK]
> > testClearQueue ... [OK]
> > testDelHasProperties ... [OK]
> > testExecute ... [OK]
> > testGetIDs ... [OK]
> > testGetResult ... [OK]
> > testGetSetProperties ... [OK]
> > testISynchronousMultiEngineInterface ... [OK]
> > testPushPull ... [OK]
> > testPushPullFunction ... [OK]
> > testPushPullSerialized ... [OK]
> > testQueueStatus ... [OK]
> > testResetAndKeys ... [OK]
> > test_clear_pending_deferreds ... [OK]
> > ipython1.kernel.tests.test_multienginexmlrpc
> > FullSynchronousMultiEngineTestCase
> > testClearProperties ... [OK]
> > testClearQueue ... [OK]
> > testDelHasProperties ... [OK]
> > testExecute ... [OK]
> > testGetIDs ... [OK]
> > testGetResult ... [OK]
> > testGetSetProperties ... [OK]
> > testISynchronousMultiEngineInterface ... [OK]
> > testMap ... [OK]
> > testMapNonblocking ... [OK]
> > testPushPull ... [OK]
> > testPushPullFunction ... [OK]
> > testPushPullSerialized ... [OK]
> > testQueueStatus ... [OK]
> > testResetAndKeys ... [OK]
> > testRun ... [OK]
> > testRunNonblocking ... [OK]
> > testScatterGather ... [OK]
> > testScatterGatherNonblocking ... [OK]
> > testScatterGatherNumpy ... [OK]
> > testScatterGatherNumpyNonblocking ... [OK]
> > testZipPull ... [OK]
> > testZipPullNonblocking ... [OK]
> > test_clear_pending_deferreds ... [OK]
> > ipython1.kernel.tests.test_newserialized
> > SerializedTestCase
> > testNDArraySerialized ... [OK]
> > testPickleSerialized ... [OK]
> > testSerializedInterfaces ... [OK]
> > ipython1.kernel.tests.test_pendingdeferred
> > PendingDeferredManagerTest
> > testBasic ... [OK]
> > test_double_get ... [OK]
> > test_get_after_callback ... [OK]
> > test_get_after_errback ... [OK]
> > test_get_before_callback ... [OK]
> > test_get_before_errback ... [OK]
> > test_nested_deferreds ... [OK]
> > test_noresult_noblock ... [OK]
> > test_save_get_delete ... [OK]
> > test_save_then_delete ... [OK]
> > test_with_callbacks ... [OK]
> > test_with_errbacks ... [OK]
> > ipython1.kernel.tests.test_task
> > BasicTaskControllerTestCase
> > testAbort ... [OK]
> > testAbortType ... [OK]
> > testClears ... [OK]
> > testRecoveryTasks ... [OK]
> > testSetupNS ... [OK]
> > testSimpleRetries ... [OK]
> > testTaskIDs ... [OK]
> > testTaskResults ... [OK]
> > ipython1.kernel.tests.test_taskxmlrpc
> > TaskTest
> > testAbort ... [OK]
> > testAbortType ... [OK]
> > testClears ... [OK]
> > testRecoveryTasks ... [OK]
> > testSetupNS ... [OK]
> > testSimpleRetries ... [OK]
> > testTaskIDs ... [OK]
> > testTaskResults ... [OK]
> > ipython1.notebook.tests.test_notebook
> > MissingSQLAlchemy
> > testSkipNBTests ... [OK]
> >
> > ===============================================================================
> > [FAIL]: ipython1.daemon.tests.test_daemonservice.BasicDaemonServiceTest.testShutdown
> >
> > Traceback (most recent call last):
> > File "/Users/barry/Development/vendors/ipython1-dev/ipython1/daemon/tests/daemonservicetest.py",
> > line 172, in <lambda>
> > d.addCallback(lambda r: self.assertEquals(len(r), len(validCommands)))
> > twisted.trial.unittest.FailTest: not equal:
> > a = 1
> > b = 3
> >
> > ===============================================================================
> > [ERROR]: ipython1.kernel.tests.test_enginepb.EnginePBTest.testGetResult_3
> >
> > Traceback (most recent call last):
> > Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean.
> > Selectables:
> > <Broker #0 on 10201>
> > -------------------------------------------------------------------------------
> > Ran 339 tests in 37.882s
> >
> > FAILED (failures=1, errors=1, successes=338)
> >
> >
> >
> >
> > On Tue, Apr 1, 2008 at 7:55 PM, Barry Wark <barrywark@gmail.com> wrote:
> > > Same on OS X. Installation is easy (via setuptools or distutils). If I
> > > find time in the next day or two, I'll make a twisted8 branch of
> > > ipython1-dev and will try to get things working. Brian, shoot me a
> > > note if you're going to do the same; I'm sure your efforts will be
> > > more efficient than mine.
> > >
> > > barry
> > >
> > >
> > >
> > > On Tue, Apr 1, 2008 at 7:11 PM, Glenn H Tarbox, PhD <glenn@tarbox.org> wrote:
> > > > I don't fully understand all the issues with setuptools and the variants
> > > > other than there is a substantial debate. I've seen threads and IRC
> > > > traffic on this on the twisted channels and elsewhere so I know there
> > > > are issues.
> > > >
> > > > However, I've been running trunk Twisted for a while... 8.0 did bring a
> > > > number of fixes but the rather dramatic jump in numbering is almost
> > > > entirely about changing the version numbering scheme itself (the 8 is
> > > > for 2008), not a rearchitecture.
> > > >
> > > > By the time code makes it to the trunk on Twisted, its pretty solid.
> > > > I've had no trouble and given the number of tests and buildbots (I host
> > > > 4), I'd be surprised if we found significant issues.
> > > >
> > > > On linux the installation is trivial and solid... can't speak to the
> > > > other platforms.
> > > >
> > > > -glenn
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, 2008-04-02 at 01:26 +0200, Gael Varoquaux wrote:
> > > > > On Tue, Apr 01, 2008 at 04:11:52PM -0700, Barry Wark wrote:
> > > > > > Just to play devil's advocate, it's not our fault that linux distros
> > > > > > lag behind in their packaged version of Twisted ;)
> > > > >
> > > > > How old is the twisted 8.0 release? I have the feeling it is only a
> > > > > couple of weeks old. In this case you can't call not packaging in a
> > > > > stable release lagging behind, but simply normal procedure of validating
> > > > > before shipping. Do you run mission-critical servers? Do you know what it
> > > > > means to have a lab stopped from working if their is a flaw on the
> > > > > packages you supported on one of your servers. I can tell you, it feels
> > > > > very bad. So supporting leading edge is great when you are dealing with
> > > > > your own box, and you are a devlopper happy to play around, but elsewhere
> > > > > more conservative decisions can also make sense.
> > > > >
> > > > > > On the more practical side, why not use Python's pacakge system (i.e.
> > > > > > setuptools) to install Twisted (I'd guess that all linux distros that
> > > > > > would include ipython would also have setuptools packages)?
> > > > >
> > > > > Because it s not Python's package system. It is a hack to work around the
> > > > > fact that Python has no package system. It induces a crazy amount of side
> > > > > effect and misbehaviors. Don't get me wrong, I use setuptools a lot, and
> > > > > appreciate the fonctionnality they add, but I can hardly see them as a
> > > > > robust solution for packaging. It is funny you should mention them, AFAIK
> > > > > until this (leading edge) release, twisted itself could not be installed
> > > > > using setuptools.
> > > > >
> > > > > I am not sure if you have followed, but currently setuptools are a
> > > > > burning subject. IMHO Python packaging is a burning subject, and I am
> > > > > tired of loosing a ridiculous amount of time to deal with different
> > > > > version of libraries. I am afraid there is no good solution. Provide
> > > > > several almost good solutions is a good idea, and setuptools do figure
> > > > > amongst these, but I must admit I wouldn't force people to rely on
> > > > > setuptools. I bites back.
> > > > >
> > > > > > using a setuptools-compatible Twisted might allow the ipython1
> > > > > > installer to install (maybe even precompiled) Twisted eggs if Twisted
> > > > > > isn't present. Just a thought.
> > > > >
> > > > > This kind of reasonning has brought setuptools to what it is currently:
> > > > > something very problematic on secure and high-reliability servers. Indeed
> > > > > you seem to be suggesting that the installer will be doing some magic to
> > > > > download the requirements if they are not their. Think about it, this is
> > > > > both a unacceptable security hole, and something that forbids any kind of
> > > > > alternate packaging (for instance for automated deployement on a
> > > > > cluster).
> > > > >
> > > > > I am just trying to warn people that setuptools is a neat toy, but not a
> > > > > solution for everybody. I use it a lot on my development boxes, but never
> > > > > on my servers.
> > > > >
> > > > > My 2 cents,
> > > > >
> > > > > Gaël
> > > >
> > > > > _______________________________________________
> > > > > IPython-dev mailing list
> > > > > IPython-dev@scipy.org
> > > > > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
> > > > --
> > > >
> > > > Glenn H. Tarbox, PhD | Don't worry about people stealing your ideas. If your ideas
> > > > 206-494-0819 | are any good, you'll have to ram them down people's throats
> > > > glenn@tarbox.org (gtalk) + ghtdak on aim/freenode | ^ Howard Aiken, IBM engineer
> > > >
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > IPython-dev mailing list
> > > > IPython-dev@scipy.org
> > > > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
> > > >
> > >
> > _______________________________________________
> > IPython-user mailing list
> > IPython-user@scipy.org
> > http://lists.ipython.scipy.org/mailman/listinfo/ipython-user
> >
>
More information about the IPython-user
mailing list