[SciPy-User] TimeSeries and Milliseconds for aircraft data

Ariel Rokem arokem@berkeley....
Wed Feb 24 00:32:34 CST 2010

Hi everyone,

Gokhan is referring to this:


For an exposition, see our Scipy conference proceedings paper (a pdf of
which can be found here:

We are still working on it. The intention of the library is to support
analysis of data from neuroscience experiments, because we are
neuroscientists, but so far, I don't think that we have made any design
decisions that would preclude other scientists from using our time-series
objects. In fact, the time-series objects we have designed support temporal
resolutions as fast as picoseconds (the representation of time is done in
int64, in order to avoid float-precision issues). It is still under
development and we have yet to make a release of this, but the code (in
development) is already available on github and the tests therein can direct
you on the possible usage:




On Tue, Feb 23, 2010 at 7:23 AM, Gökhan Sever <gokhansever@gmail.com> wrote:

> On Tue, Feb 23, 2010 at 3:35 AM, Chris Jesse <
> chris.jesse@flightdataservices.com> wrote:
>> Hi All,
>> I've got lots of timeseries data from aircraft sensors (such as altitude,
>> temperatures, landing gear up/down, airspeed..) which I was hoping to load
>> into a timeseries array. However, I've just read that it only supports time
>> resolutions down to a second. Is there a solution for my data which contains
>> samples from hundreds of different parameters at different frequencies from
>> 8Hz to 1/64Hz over an 8 hour period?
>> I need to be able to interpolate between values, mask values when samples
>> are corrupt / outside of operational boundaries and store values as
>> accurately to the nearest 1/16th of a second if possible.
>> Your help is much appreciated! Thanks,
>> Chris
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User@scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
> Hi Chris,
> Here at the University of North Dakota we work with aircraft measurements
> as well. Similar to your case depends on the campaign [1] sometimes the
> number of measured parameters reach over hundred. We use a QNX 4.25 based
> data acquisition system called M300 where all the probe data are connected
> in. M300 saves everything in a binary file which we later process using our
> ADPAA package [2] It is mostly IDL coded and mainly developed by Dr. David
> Delene [3] He is also the main responsible person for the aircraft
> measurements/deployments in our department.
> ADPAA produces files using a pre-determined ASCII data structure shown here
> [4] The system supports frequencies up to 100 Hz but typically the highest
> we sample is at 25 Hz. There might be exceptional cases where we might need
> higher sample rates. However 1-4 Hz is sufficiently enough when sampling
> clouds with cloud-aerosol microphysical instruments. (PCASP, FSSP, CCN) (I
> am disregarding instruments like holographic particle detectors for the
> simplicity of my response :) which we don't often get to fly with.)
> Anyway after some advertisement back to your question: in our system all
> the data are time-stamped with a main "Time [seconds]; UT seconds from
> midnight on day aircraft flight started" value. It simplifies data analysis
> a lot in my view where the Python comes into action actually. Once the ASCII
> files are created (we also output to NetCDF containers but I find easy to
> work on simple text files.) I read the data using NumPy and I use the
> MaskedArray module to deal with the missing values in the data. I publish my
> code along with data on the ccnworks [5] page where I mostly commit for my
> thesis using ground and airborne measurements and scientific Python
> ecosystem. For instance to read the ASCII data files (also known as NASA
> formatted) I use the generic script [6] As a part of my thesis I am working
> on this script [7] recently. I use the lab data [8] so far. In logn-fit.py I
> have a section where I do masking on the data like:
> mask02 = (pcasp.data['Time'] > 13800) & (pcasp.data['Time'] <= 14400)
> This is one of the most important part of the code --to extract the right
> portion of the required data. In this case I am interested in time periods
> based on the supersaturation values from the DMT-CCNC instrument. The next
> step will be to analyse the cloud base data. As far as I know there is no
> easy solution for this. What do PCASP probe shows when the aircraft is
> levelling below a cloud deck :) The only way I know is to mask manually and
> most logically using the common time-stamp in all data files.
> I remember Berkeley guys work on a project called NiPy but it is for
> neuro-scientists mostly and not much design consideration in it for airborne
> measurement scientists in mind :)
> I hope extra information doesn't bother you and let me know your opinions.
> [1] http://atmoswiki.aero.und.edu/atmos/citation/home
> [2] http://sourceforge.net/projects/adpaa/
> [3] http://aerosol.atmos.und.edu/
> [4] http://aerosol.atmos.und.edu/ADPAA/fileheader.html
> [5] http://code.google.com/p/ccnworks/
> [6] http://code.google.com/p/ccnworks/source/browse/trunk/nasafile.py
> [7]
> http://code.google.com/p/ccnworks/source/browse/trunk/thesis/part1/logn-fit.py
> [8]
> http://code.google.com/p/ccnworks/source/browse/trunk/thesis/part1/20090225_034214/09_02_25_03_42_14.lab.clean
> --
> Gökhan
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user

Ariel Rokem
Helen Wills Neuroscience Institute
University of California, Berkeley
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20100223/c44b59bf/attachment-0001.html 

More information about the SciPy-User mailing list