I have data sets of Heart Rate Variability data and i am trying to determine if the data collected each day is similar or related for data collected the next day. Please how can i go about this. I am looking to see if Dynamic time warping or cross correlation can do this but i am confused as to how to go about it. I am open to suggestions. I am hoping to write my code using either Matlab or Python
I have tried using Dynamic time warping and Cross correlation to compare the signals
Related
Data Science
How to handle dimensionality differences over time or between subjects
Asked today
Modified today
Viewed 9 times
0
Note: This question has in mind tabular data, rather than imaging/NLP.
In the situation of collecting data over long periods of time, instruments may change and collect more precise data. This leads to the dimensionality of the data changing over time. In its simplest form this might be a feature called FeatureA was collected, but over time the instrument allows us to collect more detailed features that aim to replace FeatureA with new features FeatureA1, FeatureA2 and FeatureA3.
We probably dont want to throw the data away with the less precise FeatureA, so how do we incorporate these new features? If it was a straight replacement of one feature with another one feature I might go for a time-varying multilevel model, but I can't see a way of applying this where the dimensionality increases, at least with most libraries.
Similarly, if sub-cohorts are using slightly different instruments to detect the same thing, but the dimensionality is different, how could we input them into the same model.
Encoder-Decoder RNNs can handle the problem of differing input sizes quite elegantly so perhaps there is some inspiration there - maybe code up tabular data inputs as tensors like you would with word vectors.
Perhaps dimensionality reduction techniques like PCA/Autoencoders might work?
Does anyone have any suggestions?
I am a geophysics student and I am trying to predict shear wave velocity which is numerical data. I feel since it is numerical data it'd be regression analysis but the problem I have now is that I don't have a shear wave log I can use as a target which then makes the project unsupervised, How do I go about it, please?
I want to if it's possible to predict numerical data because I have tried picking out random logs I feel will predict it but how do I check the accuracy
The solution inhere for you is to make data out of the signal data. I was also working on similar kind of problem where I was to predict the intensity of fall and data that I got was signal data having x,y,z axis. I managed to solve the problem by initially creating the data using clustering methodology according to my use case.Now since I have supervised data I proceded with futher analysis and predictions.
Sorry if my question makes no sense to you.
I'm troubled because as of now I have a program that can recognize sign language. The whole process it takes to recognize a gesture done by a user goes like this:
User performs a gesture.
The gesture done by the user is recorded by Kinect and is saved as a temp data.
That temp data is then compared to all words in the library using Dynamic Time Warping.
After comparing the temp data to all the words in the library, the one which the temp data is most similar to is printed on a screen.
Now what troubles me is:
Is Dynamic Time Warping a learning algorithm?
If not, how do I perform cross validation using Dynamic Time Warping?
No, DTW is a method used to look at the similarity between two time sequences. It is an algorithm but not a learning algorithm. In your case, signal generated from Kinect is compared with the signals already saved for different words using DTW algorithm.
As I am fairly new to RapidMiner, I have a Historical Financial Data Set (with attributes Date, Open, Close, High, Low, Volume Traded) from Yahoo Finance and I am trying to find a way to segment it such as in the image below:
I am also planning on performing this segmentation on more than one of such Data Sets and then comparing between each segmentation (i.e. Segment 1 for Data Set A against Segment 1 for Data Set B), so I would preferably require an equal number of segments each.
I am aware that certain extensions are available within the RapidMiner Marketplace, however I do not believe that any of them have what I am looking for. Your assistance is much appreciated.
Edit: I am currently trying to replicate the Voting-Based Outlier Mining for Multiple Time Series (V-BOMM) with multiple data sets. So far, I am able to perform the operation by recording and comparing common dates against each other.
However, I would like to enhance the process to compare Segments rather than simply dates. I have gone through the existing functionalities of RapidMiner, and thus far I don't believe any fit my requirements.
I have also considered Dynamic Time Warping, but I can't seem to find an available functionality in RapidMiner.
Ultimate question: Can someone guide me to functionalities that can help replicate the segmentation in the attached image such that the segments can be compared between Historic Data Sets in RapidMiner? Also, can someone guide me on how to implement Dynamic Time Warping using RapidMiner?
I would use the new version of the Time Series extension, using the windowing features to segment the time series into whatever parts you want. There is a nice explanation of the new tools in the blog section of the community.
I have a number of sensors measuring a Temperature (or some other physical attribute) data. Does anyone know of any clustering method that can tell which sensors are showing similar patterns and behaviors? My series are showing some trends with cycles.
I am very new to Time series analysis.
Thank you,
Basic K-means clustering works fine for most kinds of sensor data. You will need to take time slices to avoid auto-regressive issues. Check out the proc in R