[AstroPy] Problem reading VOTable produced by HLA

Michael Droettboom mdroe@stsci....
Tue Feb 26 09:20:37 CST 2013


On 02/26/2013 09:51 AM, Andy Ptak wrote:
> Hello,
> I am trying out reading VOTables using and am having trouble. 
>  Specifically reading tables returned by the Hubble Legacy Archive. 
>  The query was:
> http://hla.stsci.edu/cgi-bin/hlaSIAP.cgi?inst=ACS&pos=11.888333,-25.288056&size=0.001389
> which I attached.
>
> I am using astropy installed by pip on the enthought distribution on a 
> mac, so its version 0.2 and numpy got upgraded to 1.7.0 (if that matters).
> First, is there a way to have a votable imported directly from query 
> without saving it to a physical file first?  Maybe its best practice 
> to save VO queries to a file to trace errors, in case the votable is 
> very large, etc. but when you know you are getting something back that 
> is small and (as in this case) you mainly want a list of fields for 
> further work, saving the full votable seams like an unnecessary extra 
> step.

You can pass an http url to astropy.io.votable.parse, e.g.:

In [2]: from astropy.io import votable

In [3]: 
votable.parse("http://hla.stsci.edu/cgi-bin/hlaSIAP.cgi?inst=ACS&pos=11.888333,-25.288056&size=0.001389")

>
> The errors I am getting are below.  I got similar errors when using a 
> different votable call I found on the HLA website (basically the same 
> except using acsSIAP.cgi?strict=1
> Is this a problem with the Votable being returned by stsci, a bug or 
> am I doing something wrong?

Interestingly, I'm not able to reproduce the exception here.  (This is 
on Linux, with Python 2.7, astropy 0.2 and numpy 1.7.0).

Would you mind tinkering around in the debugger for me?  Turn on the 
IPython debugger with "%pdb", reproduce the crash, and then go up two 
levels and print out "array_chunk"?  I'm curious if there's some messed 
up values getting in there.

Thanks,
Mike

>
> Thanks,
> Andy Ptak
>
>
>
> In [4]: from astropy.io.votable import parse
> In [6]: votable = parse("test1.xml")
> WARNING: W42: test1.xml:3:0: W42: No XML namespace specified 
> [astropy.io.votable.exceptions]
> WARNING: W03: test1.xml:7:4: W03: Implictly generating an ID from a 
> name 'INPUT:POS' -> 'INPUT_POS' [astropy.io.votable.exceptions]
> WARNING: W03: test1.xml:8:4: W03: Implictly generating an ID from a 
> name 'INPUT:SIZE' -> 'INPUT_SIZE' [astropy.io.votable.exceptions]
> WARNING: W03: test1.xml:9:4: W03: Implictly generating an ID from a 
> name 'INPUT:FORMAT' -> 'INPUT_FORMAT' [astropy.io.votable.exceptions]
> WARNING: W03: test1.xml:10:4: W03: Implictly generating an ID from a 
> name 'INPUT:imagetype' -> 'INPUT_imagetype' 
> [astropy.io.votable.exceptions]
> WARNING: W03: test1.xml:11:4: W03: Implictly generating an ID from a 
> name 'INPUT:inst' -> 'INPUT_inst' [astropy.io.votable.exceptions]
> WARNING: W03: test1.xml:12:4: W03: Implictly generating an ID from a 
> name 'INPUT:hrcmatch' -> 'INPUT_hrcmatch' [astropy.io.votable.exceptions]
> WARNING: W03: test1.xml:13:4: W03: Implictly generating an ID from a 
> name 'INPUT:zoom' -> 'INPUT_zoom' [astropy.io.votable.exceptions]
> WARNING: W03: test1.xml:14:4: W03: Implictly generating an ID from a 
> name 'INPUT:autoscale' -> 'INPUT_autoscale' 
> [astropy.io.votable.exceptions]
> WARNING: W03: test1.xml:15:4: W03: Implictly generating an ID from a 
> name 'INPUT:asinh' -> 'INPUT_asinh' [astropy.io.votable.exceptions]
> WARNING: W06: test1.xml:16:4: W06: Invalid UCD 
> 'VOX:STC_CoordRefFrame': UCD has invalid character ':' in 
> 'VOX:STC_CoordRefFrame' [astropy.io.votable.exceptions]
> WARNING: W06: test1.xml:17:4: W06: Invalid UCD 
> 'VOX:WCS_CoordProjection': UCD has invalid character ':' in 
> 'VOX:WCS_CoordProjection' [astropy.io.votable.exceptions]
> WARNING: W06: test1.xml:19:6: W06: Invalid UCD 
> 'VOX:Image_AccessReference': UCD has invalid character ':' in 
> 'VOX:Image_AccessReference' [astropy.io.votable.exceptions]
> WARNING: W06: test1.xml:82:6: W06: Invalid UCD 'VOX:Image_Format': UCD 
> has invalid character ':' in 'VOX:Image_Format' 
> [astropy.io.votable.exceptions]
> WARNING: W06: test1.xml:91:6: W06: Invalid UCD 'VOX:Image_Title': UCD 
> has invalid character ':' in 'VOX:Image_Title' 
> [astropy.io.votable.exceptions]
> WARNING: W06: test1.xml:94:6: W06: Invalid UCD 'VOX:Image_Naxis': UCD 
> has invalid character ':' in 'VOX:Image_Naxis' 
> [astropy.io.votable.exceptions]
> WARNING: W06: test1.xml:97:6: W06: Invalid UCD 'VOX:Image_Scale': UCD 
> has invalid character ':' in 'VOX:Image_Scale' 
> [astropy.io.votable.exceptions]
> WARNING: W06: test1.xml:100:6: W06: Invalid UCD 'VOX:Image_Naxes': UCD 
> has invalid character ':' in 'VOX:Image_Naxes' 
> [astropy.io.votable.exceptions]
> WARNING: W06: test1.xml:103:6: W06: Invalid UCD 
> 'VOX:WCS_CoordRefPixel': UCD has invalid character ':' in 
> 'VOX:WCS_CoordRefPixel' [astropy.io.votable.exceptions]
> WARNING: W06: test1.xml:106:6: W06: Invalid UCD 
> 'VOX:WCS_CoordRefValue': UCD has invalid character ':' in 
> 'VOX:WCS_CoordRefValue' (suppressing further warnings of this type...) 
> [astropy.io.votable.exceptions]
> ERROR: OverflowError: Python int too large to convert to C long 
> [numpy.ma.core]
> ---------------------------------------------------------------------------
> OverflowError                             Traceback (most recent call 
> last)
> /Volumes/Apps_and_Docs/aptak/Dropbox/HLA/<ipython-input-6-61a1dc44cf49> in 
> <module>()
> ----> 1 votable = parse("test1.xml")
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/astropy/io/votable/table.pyc 
> in parse(source, columns, invalid, pedantic, chunk_size, table_number, 
> table_id, filename, _debug_python_based_parser)
>     114 _debug_python_based_parser=_debug_python_based_parser) as 
> iterator:
>     115         return tree.VOTableFile(
> --> 116             config=config, pos=(1, 1)).parse(iterator, config)
>     117
>     118
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/astropy/io/votable/tree.pyc 
> in parse(self, iterator, config)
>    3084             if start:
>    3085                 tag_mapping.get(tag, self._add_unknown_tag)(
> -> 3086                     iterator, tag, data, config, pos)
>    3087             elif tag == 'DESCRIPTION':
>    3088                 if self.description is not None:
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/astropy/io/votable/tree.pyc 
> in _add_resource(self, iterator, tag, data, config, pos)
>    3015         resource = Resource(config=config, pos=pos, **data)
>    3016         self.resources.append(resource)
> -> 3017         resource.parse(self, iterator, config)
>    3018
>    3019     def _add_coosys(self, iterator, tag, data, config, pos):
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/astropy/io/votable/tree.pyc 
> in parse(self, votable, iterator, config)
>    2871             if start:
>    2872                 tag_mapping.get(tag, self._add_unknown_tag)(
> -> 2873                     iterator, tag, data, config, pos)
>    2874             elif tag == 'DESCRIPTION':
>    2875                 if self.description is not None:
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/astropy/io/votable/tree.pyc 
> in _add_table(self, iterator, tag, data, config, pos)
>    2828         table = Table(self._votable, config=config, pos=pos, 
> **data)
>    2829         self.tables.append(table)
> -> 2830         table.parse(iterator, config)
>    2831
>    2832     def _add_info(self, iterator, tag, data, config, pos):
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/astropy/io/votable/tree.pyc 
> in parse(self, iterator, config)
>    2212                             'TABLEDATA', data.iterkeys(), 
> config, pos)
>    2213                         self.array = self._parse_tabledata(
> -> 2214                             iterator, colnumbers, fields, config)
>    2215                         break
>    2216                     elif tag == 'BINARY':
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/astropy/io/votable/tree.pyc 
> in _parse_tabledata(self, iterator, colnumbers, fields, config)
>    2357
>    2358         array = _resize(array, alloc_rows)
> -> 2359         array[numrows:] = array_chunk
>    2360         if alloc_rows != 0:
>    2361             array.mask[numrows:] = mask_chunk
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc 
> in __setslice__(self, i, j, value)
>    3067
>    3068         """
> -> 3069         self.__setitem__(slice(i, j), value)
>    3070
>    3071
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc 
> in __setitem__(self, indx, value)
>    3031                 _mask = ndarray.__getattribute__(self, '_mask')
>    3032             # Set the data, then the mask
>
> -> 3033             ndarray.__setitem__(_data, indx, dval)
>    3034             ndarray.__setitem__(_mask, indx, mval)
>    3035         elif hasattr(indx, 'dtype') and (indx.dtype == MaskType):
>
> OverflowError: Python int too large to convert to C long

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/astropy/attachments/20130226/e3901e8a/attachment.html 


More information about the AstroPy mailing list