[SciPy-dev] Re: scipy_distutils under cygwin using DF

[Pearu Peterson]

On Wed, 31 Mar 2004, [ISO-8859-15] Berthold Höllmann wrote:

> I just checke out the latest CVS, and now get:
> 
> ---snip---
> hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core
> $ pwd
> /home/hoel/unix/work/CVS/scipy/scipy_core
> 
> hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core
> $ ls -l /home/hoel/unix
> lrwxrwxrwx    1 hoel     Kein           94 Oct 27 14:52 /home/hoel/unix -> /cygdrive/h
> 
> hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core
> $ python
> Enthought Edition build 1028
> Python 2.3 (#46, Aug 11 2003, 09:34:05) [MSC v.1200 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import os.path
> >>> os.path.realpath(r'C:\Programme')
> 'C:\\Programme'
> >>> os.path.realpath(r"C:\Programme\Microsoft Visual Studio")
> 'C:\\Programme\\Microsoft Visual Studio'
> >>> os.path.realpath(r"C:\Programme\Microsoft Visual Studio\\")
> 'C:\\Programme\\Microsoft Visual Studio'
> >>> os.path.realpath(r'"C:\Programme\Microsoft Visual Studio\\"')
> 'h:\\work\\CVS\\scipy\\scipy_core\\"C:\\Programme\\Microsoft Visual Studio\\"'
> >>> ^Z
> $
> ---snip---
> 
> >
> > when python was fired up from the H: disk?
> >
> 
> So the problem seems to be, the executable name got quoted to early (I
> think this was my fault, I moved "quote_arg" to an too early position,
> but on I think now it is called again at a wrong position. What I get
> now under WinXP is
> 
> ---snip---
> customize MSVCCompiler
> customize MSVCCompiler using build_clib
> 0
> customize CompaqVisualFCompiler
> customize CompaqVisualFCompiler using build_clib
> splitcmdline('DF /what')
> running build_ext
> No module named msvccompiler
> customize MSVCCompiler
> customize MSVCCompiler using build_ext
> customize CompaqVisualFCompiler
> customize CompaqVisualFCompiler using build_ext
> splitcmdline('DF /what')
> building 'SXFPyBase' extension
> compling C sources
> C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include-Ic:\Python23\PC /Tcbuild\src\fortranobject.c /Fobuild\temp.win32-2.3\Release\build\src\fortranobject.obj
> Could not locate executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe"
> Executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" does not exist
> 
> error: Command ""C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\fortranobject.c /Fobuild\temp.win32-2.3\Release\build\src\fortranobject.obj" failed with exit status 1
> make: *** [build] Error 1
> ---snip---
> 
> The line after "compling C sources" looks wrong (should'nt the word
> "compling" in command/build_clib.py be replaced by "compiling" at 4
> instances?) but it comes from CCompiler_spawn and the cl.exe part goes
> properly quoted to exec_command. Maybe the "_nt_quote_args" and the
> part leading to "log.info" should be exchanged for clarity in
> CCompiler_spawn.
> 
> ---snip---
> # Using customized CCompiler.spawn.
> def CCompiler_spawn(self, cmd, display=None):
>     if type(cmd) is type([]) and os.name == 'nt':
>         cmd = _nt_quote_args(cmd)
>     if display is None:
>         display = cmd
>         if type(display) is type([]): display = ' '.join(display)
>     log.info(display)
>     s,o = exec_command(cmd)
>     if s:
>         if type(cmd) is type([]):
>             cmd = ' '.join(cmd)
>         print o
>         raise DistutilsExecError,\
>               'Command "%s" failed with exit status %d' % (cmd, s)
> CCompiler.spawn = new.instancemethod(CCompiler_spawn,None,CCompiler)
> ---snip---


I fixed this issue by removing quotes in find_executable function.
It is in CVS now and seems to work, I was able to build scipy_core
with MSVC 6.0 compiler!


> Back to my problem: It came out to the definition of the include
> directory '-I"f:\DATA\FBE\ESC\Devel\include"'. The execution via
> "cmd.exe" does not like the quotes (") around the include path, so I
> removed them, just to stumble over the next problem:
> 
> "-If:\DATA\FBE\ESC\Devel\include"
> 
> as an option to DF is interpreted as
> "/iface=\DATA\FBE\ESC\Devel\include" and leads to an syntax error in
> DF. Luckily for me, "f:" is a network directory so I can write
> "\\Glf\Vol1\...", but I guess a general solution should be found.
> 
> > Hmm, Python docs say that  os.path.realpath is available only for Unix. 
> > This could provide a fix.
> >
> >> ---snip---
> >>     s,o=exec_command('echo path=%path%')
> >>     log.debug((s,o))
> >>     assert s==0 and o!='',(s,o)
> >> 
> >>     s,o=exec_command('%s -c "import sys;sys.stderr.write(sys.platform)"' \
> >>                      % pythonexe)
> >>     log.debug((s,o))
> >>     assert s==0 and o=='win32',(s,o)
> >> ---snip---
> >> 
> >> With this I get:
> >> 
> >> ---snip---
> >> $ python scipy_distutils/exec_command.py
> >> ...
> >> AssertionError: (0, 'COMMAND \'c:\\\\Python23\\\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"\' FAILED: win32')
> >> ---snip---
> >> 
> >> The problem is, the test writes to stderr, but a result on stderr
> >> leads "_exec_command" to generate the "... FAILED ..."
> >
> > I think I have fixed that yesterday.
> >
> >
> >> AssertionError: (0, '$AAA')
> >> ---snip---
> >> 
> >> which I also don't understand
> >
> > Ok, I could reproduce this on Sun and this is now fixed in CVS.
> 
> python scipy_distutils/exec_command.py
> 
> OK on Linux and Solaris (except for the failing svn test, but I have
> no svn installed, so this is expected ;-).
> 
> But on WinXP I still get 
> 
> ---snip---
> hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core
> $ python scipy_distutils/exec_command.py
> splitcmdline('a   b  cc')
> splitcmdline('a')
> splitcmdline('a "  b  cc"')
> splitcmdline('"a bcc"  -h')
> splitcmdline('"\\"a \\" bcc" -h')
> splitcmdline(" 'a bcc'  -h")
> splitcmdline("'\\'a \\' bcc' -h")
> Using cygwin echo in win32 environment is not supported
> splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'AAA\',\'\')"')
> splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'AAA\')"')
> splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"')
> splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"')
> splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"')
> splitcmdline('echo path=%path%')
> splitcmdline('c:\\Python23\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"')
> Traceback (most recent call last):
>   File "scipy_distutils/exec_command.py", line 601, in ?
>     test(use_tee=0)
>   File "scipy_distutils/exec_command.py", line 502, in test_nt
>     assert s==0 and o=='win32',(s,o)
> AssertionError: (0, 'COMMAND \'c:\\\\Python23\\\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"\' FAILED: win32')
> ---snip---
>                                    
> maybe line 502 in exec_command.py could be changed to 
> 
> ---snip---
>     assert s==0 and o[-5:]=='win32',(s,o)
> ---snip---

No, that would be cheating;-) Recent CVS seems to fix this issue as well.

Regards,
Pearu