[SciPy-User] TimeSeries and Milliseconds for aircraft data
Tue Feb 23 09:23:03 CST 2010
On Tue, Feb 23, 2010 at 3:35 AM, Chris Jesse <
> Hi All,
> I've got lots of timeseries data from aircraft sensors (such as altitude,
> temperatures, landing gear up/down, airspeed..) which I was hoping to load
> into a timeseries array. However, I've just read that it only supports time
> resolutions down to a second. Is there a solution for my data which contains
> samples from hundreds of different parameters at different frequencies from
> 8Hz to 1/64Hz over an 8 hour period?
> I need to be able to interpolate between values, mask values when samples
> are corrupt / outside of operational boundaries and store values as
> accurately to the nearest 1/16th of a second if possible.
> Your help is much appreciated! Thanks,
> SciPy-User mailing list
Here at the University of North Dakota we work with aircraft measurements as
well. Similar to your case depends on the campaign  sometimes the number
of measured parameters reach over hundred. We use a QNX 4.25 based data
acquisition system called M300 where all the probe data are connected in.
M300 saves everything in a binary file which we later process using our
ADPAA package  It is mostly IDL coded and mainly developed by Dr. David
Delene  He is also the main responsible person for the aircraft
measurements/deployments in our department.
ADPAA produces files using a pre-determined ASCII data structure shown here
 The system supports frequencies up to 100 Hz but typically the highest
we sample is at 25 Hz. There might be exceptional cases where we might need
higher sample rates. However 1-4 Hz is sufficiently enough when sampling
clouds with cloud-aerosol microphysical instruments. (PCASP, FSSP, CCN) (I
am disregarding instruments like holographic particle detectors for the
simplicity of my response :) which we don't often get to fly with.)
Anyway after some advertisement back to your question: in our system all the
data are time-stamped with a main "Time [seconds]; UT seconds from midnight
on day aircraft flight started" value. It simplifies data analysis a lot in
my view where the Python comes into action actually. Once the ASCII files
are created (we also output to NetCDF containers but I find easy to work on
simple text files.) I read the data using NumPy and I use the MaskedArray
module to deal with the missing values in the data. I publish my code along
with data on the ccnworks  page where I mostly commit for my thesis using
ground and airborne measurements and scientific Python ecosystem. For
instance to read the ASCII data files (also known as NASA formatted) I use
the generic script  As a part of my thesis I am working on this script
 recently. I use the lab data  so far. In logn-fit.py I have a section
where I do masking on the data like:
mask02 = (pcasp.data['Time'] > 13800) & (pcasp.data['Time'] <= 14400)
This is one of the most important part of the code --to extract the right
portion of the required data. In this case I am interested in time periods
based on the supersaturation values from the DMT-CCNC instrument. The next
step will be to analyse the cloud base data. As far as I know there is no
easy solution for this. What do PCASP probe shows when the aircraft is
levelling below a cloud deck :) The only way I know is to mask manually and
most logically using the common time-stamp in all data files.
I remember Berkeley guys work on a project called NiPy but it is for
neuro-scientists mostly and not much design consideration in it for airborne
measurement scientists in mind :)
I hope extra information doesn't bother you and let me know your opinions.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the SciPy-User