[SciPy-User] scipy central comments

Skipper Seabold jsseabold@gmail....
Tue Sep 6 12:33:09 CDT 2011


On Tue, Sep 6, 2011 at 1:25 PM, Kevin Dunn <kgdunn@gmail.com> wrote:
> On Mon, Sep 5, 2011 at 21:01, Skipper Seabold <jsseabold@gmail.com> wrote:
>> On Mon, Sep 5, 2011 at 8:40 PM, Kevin Dunn <kgdunn@gmail.com> wrote:
<snip>
>>> By the way, does SciPy/NumPy have a way to load data from a URL like R?
>>>
>>> I've looked for this but can't seem to find anything on it. In R it is
>>> so nice to be able to say to someone:
>>> data = read.table('http://datasets.connectmv.com/file/ammonia.csv')
>>>
>>> rather that doing a two step: download and load.
>>>
>>
>>
>> There is the DataSource class, though there are other ways this could
>> be accomplished. I'm not sure that there's a function to do it yet.
>>
>> ds = np.lib.DataSource()
>> fp = ds.open('http://datasets.connectmv.com/file/ammonia.csv')
>> from StringIO import StringIO
>> arr = np.genfromtxt(StringIO(fp.read()), names=True)
>>
>> Or you could use urllib
>>
>> import urllib
>> fp2 = urllib.urlopen('http://datasets.connectmv.com/file/ammonia.csv')
>> arr2 = np.genfromtxt(StringIO(fp2.read()), names=True)
>>
>> If there's nothing else
>>
>> def loadurl(url, *args, **kwargs):
>>    from urllib import urlopen
>>    from cStringIO import StringIO
>>    fp = urlopen(url)
>>    return np.genfromtxt(StringIO(fp.read()), *args, **kwargs)
>>
>> arr3 = loadurl('http://datasets.connectmv.com/file/ammonia.csv')
>
> Thanks Skipper - I wasn't aware of the np.lib.DataSource class. It
> seems that it can handle compressed data sources as well.
>
> All 3 methods work great. Would you mind adding those code snippets to
> SciPy Central for others to see?
>

http://scipy-central.org/item/25/1/load-a-url-into-an-array

Someone feel free to make a pull request with loadurl, if it's useful.
I'll try to remember to do it later.

Skipper


More information about the SciPy-User mailing list