I had a problem with the irregular point by time as shown below. As some of the data wasn't recorded at the same point of time is there anyway to show all my data at a estimated point.
http://postimg.org/image/6cr7ktqln/
I doesn't have enough reputation to post images so I use the above link
The highcharts library has no "estimation" algorithm, so you need calculate these points on your own in the preprocessing.
Related
I am trying to figure out how to create this overlaid plot of time-series data, where one of the series should "look" like a histogram.
The problem is I could not figure out how to combine/overlay a histogram with time series data and line/scatter plot and get the histogram xbins to work with the date time data, etc.
So I was also trying to use a bar chart, and create a "pseudo histogram" by removing the gaps between bars, adding outlines, and so forth but that seems fruitless as I don't see a way to control all the borders/lines to that level of control.
The result I am looking for is roughly like so;
Which to me looks like the best match for a plot type should be a histogram, but again I could not figure out how to make that work overlaid with the same x axis as the line/scatter time-series data.
Can anyone offer ideas or point me to an example that might help me understand how to do this ?
I guess I also need to figure out how to align the y-axis scales of the two series also, but that I expect is a different topic...
I am specifically using plotly.js / Javascript
The line chart view below has too many data points and I'd like to reduce the frequency or smooth the line so that it's more readable, I can't find anything in the Charts documentation on this issue, is it possible?
I'm experimenting using reservoir computing techniques to classify images, but I'm not sure how to convert an arbitrary image to a time series.
I found this approach but it doesn't seem to be general enough.
Thanks!
As defined in that article, a time series is just a single-value function of one variable. However, an image is, in general, a multi-value function of two variables. So, in order to convert from an image to a 'time series', you're projecting down from a higher dimensional space to a lower dimensional one (for example, the radial scanning technique described collapses the image as a whole into an outline, which reduces the dimension to one). A key point is that these projections all 'lose data'. Since they're all lossy, there isn't going to be a 'general' solution that works for all uses of all images.. choosing what data you can afford to lose based on your intended application is a key aspect of using this technique. So, I guess my answer is that there is no single general way to convert an image to a 'time series' that works well for all applications.
I would think along those lines.
An image is a static two dimensional array of pixels recorded of a period in time.
A time series is non-static just like a video is a series of images going from one frame to the next in time.
Not sure I answered the question but I hope this helps.
I'm actually trying to detect characteristics of the time series for a very big region composed of many smaller subregions (in my case pixels). I don't know much about this, so the only way I can come up with is an averaged time series for the entire region, although I know this would definitely conceal many features by averaging.
I'm just wondering if there are any widely used techniques that can detect the common features of a suite of time series? like pattern recognition or time series classification?
Any ideas/suggestions are much appreciated!
Thanks!
Some extra explanations: I'm dealing with remote sensing images of several years with a time step of 7 days. So for each pixel, there is a time series associated, with values extracted from this pixel on different dates.So if I define a region consisting of many pixels, is there a way to detect or extract some common features charactering all or most of the time series of pixels within this region? Such as the shape of the time series, or a date around which there's an obvious increase in the values?
You could compute the correlation matrix for the pixels. This would simply be:
corr = np.zeros((npix,npix))
for i in range(npix):
for j in range(npix):
corr(i,j) = sum(data(i,:)*data(j,:))/sqrt(sum(data(i,:)**2)*sum(data(j,:)**2))
If you want more information, you can compute this as a function of time, i.e. divide your time series into blocks (say minutes) and compute the correlation for each of them. Then you can see how the correlation changes over time.
If the correlation changes a lot, you may be more interested in the cross-power spectrum of the pixels. This is defined as
cpow(i,j,:) = (fft(data(i,:))*conj(fft(data(j,:)))
This will tell you how much pixel i and j tend to change together on various time-scales. For example, they could be moving in unison in time-scales of a second (1 Hz), but also have changes on a time-scale of, say, 10 seconds which are not correlated with each other.
It all depends on what you need, really.
So I've recorded some data from an Android GPS, and I'm trying to find the peaks of these graphs, but I haven't been able to find anything specific, perhaps because I'm not too sure what I'm looking for. I have found some MatLab functions, but I can't find the actual algorithms that do it. I need to do this in Java, but I should be able to translate code from other languages.
As you can see, there are lots of 'mini-peaks', but I just want the main ones.
Your solution depends on what you want to do with the data. If you want to do very serious things then you should most likely use (Fast) Fourier Transforms, and extract both the phase and frequency output from it. But that's very computationally intensive and takes a long while to program. If you just want to do something simple that doesn't require a lot of computational resources, then here's a suggestion:
For that exact problem i implemented the below algorithm a few hours ago. I invented the algorithm myself so i do not know if it has a name already, but it is working great on very noisy data.
You need to determine the average peak-to-peak distance and call that PtP. Do that measurement any what you like. Judging from the graph in your case it appears to be about 35. In my code i have another algorithm i invented to do that automatically.
Then choose a random starting index on the graph. Poll every new datapoint from then on and wait until the graph has either risen or fallen from the starting index level by about 70% of PtP. If it was a fall then that's a tock. If it was a rise then that's a tick. Store that level as the last tick or tock height. Produce a 'tick' or 'tock' event at this index.
Continue forward in the data. After ticks, if the data continues to rise after that point then store that level as the new 'height-of-tick' but do not produce a new tick event. After tocks, if the data continues to fall after that point then store that level as the new 'depth-of-tock' but do not produce a new tock event.
If last event was a tock then wait for a tick, if last event was a tick then wait for a tock.
Each time you detect a tick, then that should be a peak! Good luck.
I think what you want to do is run this through some sort of low-pass filter. Depending on exactly what you want to get out of this dataset, a simple "box car" filter might be
sufficient: at each point, take the average of the N samples centered on that point,
and take the average as the filtered value. The larger N is, the more aggressively smoothed the filtered data will be.
I guess you have lots of points... Calculate mean value of them, subtract it from all point's values and get highest point value (negative or positive) from each range where points have same sign till they change it. I hope I am clear...
With particulary nasty and noisy data I usually use smoothing. Easiest example of smoothing is moving average. Then you can find peacks on that moving average. And then you simply go back to your original data and take the closest peak to one you found on moving average.
I've done some looking into peak detection and I can tell you that if your data doesn't behave, it could mess up your algorithm. Off the top of my head, you could try: Pick a threshold, i.e threshold = 250. If data is above threshold, find the max at that period. This is assuming that the data you have has a mean about 230. Not sure how fancy you want to get. Hope that helps.