[IPython-User] slow startup time possibly because of sqlite

Thomas Kluyver takowl@gmail....
Tue Jan 29 12:05:53 CST 2013


- Dumping to file on exit: sadly not as simple as just dumping the raw data
in memory - as far as I know, you have to connect to a file, query the
in-memory database, and insert the results into the file database. We don't
want to be doing all that on shutdown, especially if the file is on NFS
where it could have problems.
- Loading other history files - you can create a separate HistoryAccessor
already. Wrapping that in magic commands or whatever isn't a priority, at
least for me.

Both could probably be implemented as extensions if you want to play around
with them.

Thomas


On 29 January 2013 07:21, klo uo <klonuo@gmail.com> wrote:

> Do you think it would be useful, to allow option to write current session
> to memory as in Matthias example, and:
>
>  - allow dumping history from memory to file by appending, on exit
>  - allow IPython function to load stored history file, on demand
>
>
> On Mon, Jan 28, 2013 at 11:32 PM, Thomas Kluyver <takowl@gmail.com> wrote:
>
>> On 28 January 2013 19:33, Matthias BUSSONNIER <
>> bussonniermatthias@gmail.com> wrote:
>>
>>> IIRC we had some issues with history on NFS, but maybe it was only on
>>> windows.
>>
>>
>> It's not, unfortunately - SQLite over NFS is a general problem. And as
>> far as we know, there's no way to easily detect whether a directory is on
>> NFS. The workaround is to set HistoryManager.hist_file to a location that
>> will be stored locally, or to :memory: (in which case, history will be lost
>> between sessions).
>>
>> Thanks for the detailed analysis of the problem. From what you say,
>> connecting from different threads may be the issue. We have a
>> HistorySavingThread which opens its own connection to the database to write
>> command history. Python's sqlite3 module has an undocumented option to
>> share a single database connection between threads. We've been reluctant to
>> play with it, but you might want to experiment with it.
>>
>> Best wishes,
>> Thomas
>>
>> _______________________________________________
>> IPython-User mailing list
>> IPython-User@scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-user
>>
>>
>
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20130129/2a94987d/attachment.html 


More information about the IPython-User mailing list