Standard Deviation only using daily values to show on every time chart in Pine Script - standard-deviation

I am trying to make a Standard Deviation overlay using only the daily inputs, and have it overlay that info on any time frame chart. So, even if I look at an hourly chart, I will still see the daily deviations overlayed on the chart. I made one that changes with whatever time frame I am looking at. I started with another public one to make this:
study(title="Standard Deviation",shorttitle="SD",overlay=true)
length = input(20, minval=1)
src = input(open, title="Source")
sd = stdev(src, length)
piv=open
plotOpen = plot(piv,title="Open",color=black,trackprice=true,linewidth=2)
plotR05 = plot(piv+(0.5*sd),title="+0.5", color=red,trackprice=true,linewidth=2)
plotS05 = plot(piv-(0.5*sd),title="-0.5", color=red,trackprice=true,linewidth=2)
plotR10 = plot(piv+sd,title="1", color=blue,trackprice=true,linewidth=2)
plotS10 = plot(piv-sd,title="-1", color=blue,trackprice=true,linewidth=2)
plotR15 = plot(piv+(1.5*sd),title="+1.5", color=green,trackprice=true,linewidth=2)
plotS15 = plot(piv-(1.5*sd),title="-1.5", color=green,trackprice=true,linewidth=2)
plotR20 = plot(piv+(2*sd),title="+2", color=orange,trackprice=true,linewidth=2)
plotS20 = plot(piv-(2*sd),title="-2", color=orange,trackprice=true,linewidth=2)

I am trying to make a Standard Deviation overlay using only the daily inputs, and have it overlay that info on any time frame chart.
You can use TradingView's security() function for that. That function can load price data from any time frame and/or instrument, including the daily data from the current instrument.
With security() your code can thus calculate the daily standard deviation regardless of which time frame the script currently runs on.
For example:
study(title="Standard Deviation",shorttitle="SD",overlay=true)
length = input(20, minval=1)
src = input(open, title="Source")
// Load daily stddev
dailyStd = security(tickerid, "D", stddev(src, length))
piv=open
plotOpen = plot(piv,title="Open",color=black,trackprice=true,linewidth=2)
plotR05 = plot(piv+(0.5*dailyStd),title="+0.5", color=red,trackprice=true,linewidth=2)
plotS05 = plot(piv-(0.5*dailyStd),title="-0.5", color=red,trackprice=true,linewidth=2)
plotR10 = plot(piv+dailyStd,title="1", color=blue,trackprice=true,linewidth=2)
plotS10 = plot(piv-dailyStd,title="-1", color=blue,trackprice=true,linewidth=2)
plotR15 = plot(piv+(1.5*dailyStd),title="+1.5", color=green,trackprice=true,linewidth=2)
plotS15 = plot(piv-(1.5*dailyStd),title="-1.5", color=green,trackprice=true,linewidth=2)
plotR20 = plot(piv+(2*dailyStd),title="+2", color=orange,trackprice=true,linewidth=2)
plotS20 = plot(piv-(2*dailyStd),title="-2", color=orange,trackprice=true,linewidth=2)
Give it a try to see if this approach better serves your goal.

for version 5
//#version=5
indicator("Standart Deviation", shorttitle="SD", overlay=true)
length = input.int(30, minval=1)
src = input.source(open,"Source")
//load daily stdev
dailyStd = request.security(syminfo.tickerid,"D",ta.stdev(src,length))
piv=open
plotOpen = plot(piv,title="Open", color=color.white, trackprice = true, linewidth=2)
plotStdResistance = plot(piv+dailyStd,title="RESISTANCE",color=color.red,trackprice=true,linewidth=1)
plotStdSupport = plot(piv-dailyStd,title="SUPPORT", color=color.green, trackprice = true, linewidth=1)

Related

Unable to label my significant correlation on the corrplot

So I am trying to plot a corrplot. The code I am using is
corrplot::corrplot(cor.Phenotyping.All.scaled$r, type = "full", diag = TRUE,
method = "color", tl.cex = 0.4, tl.col = 'black', outline = F,
order = "original", p.mat = NULL,
pch.col = "white", sig.level = c(.001), pch.cex = .5 , insig = "label_sig",
col = colorRampPalette(c("darkred","white","midnightblue"))(100),
cl.pos = "b")
The code does make a plot but the significant correlations are not labelled as asterisk in the plot. My significant value is p<0.001 and significant correlations are present in my data. I have tried multiple ways i have tried extracting significant p-values and putting them to make a graph using but still I cannot get asterisk. Desperate for help

How to synchronize sensor data with different sampling rates in octave?

I want to implement a kalman sensor fusion based upon accelerometer, rotation vector and WLAN. I load sensor data from CSV file for each android sensor(Accelerometer, Rotation vector and WLAN) by using dlmread. Each sensor has different sampling rates from 2ms to 200ms. My problem is that I do not know how to access the data for implementing a sensor fusion because the values are out of sync.
accelData = dlmread("1.csv", ",", 1, 0);
accelX = accelData(:, 15);
accelY = accelData(:, 16);
accelZ = accelData(:, 17);
rotVecData = dlmread("11.csv", ",", 1, 0);
rot1 = rotVecData(:, 15);
rot2 = rotVecData(:, 16);
rot3 = rotVecData(:, 17);
rot4 = rotVecData(:, 18);
rot5 = rotVecData(:, 19);
dtA = 0.002;#2ms
samplesA = length(accelX);
tAccel = 0:dtA:(samplesA*dtA - dtA);
dtR = dtA*5;#10ms
samplesR = length(rot1);
tRot = 0:dtR:(samplesR*dtR - dtR);
for i = 1 : samplesR
r = [rot1(i), rot2(i), rot3(i), rot4(i), rot5(i)];
[azimuth] = Orientation(r, 90);
a(end+1) = rad2deg(azimuth);
end
figure
plot(tAccel, accelX, tRot, rot1)
In the example I could plot 2 values of different sensors in correct time interval. The output is the following:
The problem here is that I can not iterate over the values for further calculations. Because the accel(i) value takes place with a rate of 2ms while the i-th element in rot1 has a rate of 20ms. So each sensor array from CSV has different sizes but all of them takes place in the same time interval.
In the shown figure it is correct but I do not know how to iterate over the data in one loop corresponding to the different sampling rates?
Note: The values depends on each other(Pedestrian dead reckoning based on rotation vector and accelerometer based step detection).

Google Earth Engine: mask clouds and map a function over an image collection of different sensors

I want to combine all the Landsat sensors from 1985 up today in Google Earth Engine, remove the clouds and calculate the time-series of the NBR index. As a new GEE user I have the following:
// find all data and filter them by date
var lst5 = ee.ImageCollection('LANDSAT/LT5_SR').filterDate('1984-10-01', '2011-10-01');
var lst7 = ee.ImageCollection('LANDSAT/LE7_SR').filterDate('2011-10-01', '2013-04-07');
var lst8 = ee.ImageCollection('LANDSAT/LC8_SR').filterDate('2013-04-07', '2018-05-01');
var lst7_08 = ee.ImageCollection('LANDSAT/LE7_SR').filterDate('2007-12-01', '2008-02-01');
var lst7_92 = ee.ImageCollection('LANDSAT/LT4_SR').filterDate('1992-01-02', '1992-04-01');
// Combine all landsat data, 1985 through 2015
var everything = ee.ImageCollection(lst5.merge(lst7));
everything = everything.merge(lst8);
everything = everything.merge(lst7_08);
everything = everything.merge(lst7_92);
var alltogether = ee.ImageCollection(everything.filterDate('1984-01-01', '2018-05-01'));
From this point, I do not know how to remove the clouds and calculate the NBR index (NBR index here) for every image in my final collection.
Can anyone help me?
Thank you.
EDIT:
I think that I need to map a normalizedDifference function over my collection in order to get the NBR index but I am not sure how to do this for my collection with the different sensors.
You've got quite a lot going on here, but here's what I think you want. You should check this very carefully to ensure it's behaving as intended:
// Function to cloud mask Landsat 8.
var maskL8SR = function(image) {
// Bits 3 and 5 are cloud shadow and cloud, respectively.
var cloudShadowBitMask = ee.Number(2).pow(3).int();
var cloudsBitMask = ee.Number(2).pow(5).int();
// Get the QA band.
var qa = image.select('pixel_qa');
// Both flags should be set to zero, indicating clear conditions.
var mask = qa.bitwiseAnd(cloudShadowBitMask).eq(0).and(
qa.bitwiseAnd(cloudsBitMask).eq(0));
return image
// Scale the data to reflectance and temperature.
.select(['B5', 'B7'], ['NIR', 'SWIR']).multiply(0.0001)
.addBands(image.select(['B11'], ['Thermal']).multiply(0.1))
.updateMask(mask);
};
// Function to cloud mask Landsats 5-7
var maskL57SR = function(image) {
var qa = image.select('pixel_qa');
// Second bit must be zero, meaning none to low cloud confidence.
var mask1 = qa.bitwiseAnd(ee.Number(2).pow(7).int()).eq(0).and(
qa.bitwiseAnd(ee.Number(2).pow(3).int()).lte(0)); // cloud shadow
// This gets rid of irritating fixed-pattern noise at the edge of the images.
var mask2 = image.select('B.*').gt(0).reduce('min');
return image
.select(['B4', 'B7'], ['NIR', 'SWIR']).multiply(0.0001)
.addBands(image.select(['B6'], ['Thermal']).multiply(0.1))
.updateMask(mask1.and(mask2));
};
// find all data and filter them by date
var lst5 = ee.ImageCollection('LANDSAT/LT05/C01/T1_SR')
.filterDate('1984-10-01', '2011-10-01')
.map(maskL57SR)
var lst7 = ee.ImageCollection('LANDSAT/LE07/C01/T1_SR')
.filterDate('2011-10-01', '2013-04-07')
.map(maskL57SR)
var lst8 = ee.ImageCollection('LANDSAT/LC08/C01/T1_SR')
.filterDate('2013-04-07', '2018-05-01')
.map(maskL8SR)
var lst7_08 = ee.ImageCollection('LANDSAT/LE07/C01/T1_SR')
.filterDate('2007-12-01', '2008-02-01')
.map(maskL57SR)
var lst7_92 = ee.ImageCollection('LANDSAT/LT04/C01/T1_SR')
.filterDate('1992-01-02', '1992-04-01')
.map(maskL57SR)
// Combine all landsat data, 1985 through 2015
var everything = ee.ImageCollection(lst5.merge(lst7));
everything = everything.merge(lst8);
everything = everything.merge(lst7_08);
everything = everything.merge(lst7_92);
// NBR:
var nbrFunction = function(image) {
image = ee.Image(image)
return image.addBands(image.expression(
'(nir - 0.0001 * swir * thermal) / ' +
'(nir + 0.0001 * swir * thermal)', {
nir: image.select(['NIR']),
swir: image.select(['SWIR']),
thermal: image.select(['Thermal'])
}).rename('NBR').clamp(-1, 1));
};
everything = everything.map(nbrFunction);
var check = ee.Image(everything.first());
Map.centerObject(check);
Map.addLayer(check);
The answer works great for SR imagery! Thanks! Sorry I can't just comment because I don't have 50 reputation yet, but I saw #Abhilash Singh Chauhan's question about why ee.Number(2).pow(3)... is used for the variables cloudshadow and clouds. I had the same question and I wanted to answer that it's because of the fact that the QA Pixel bands are Decimal integers that contain Binary information. So for example band 3 for surface reflectance LANDSAT products indicates the band for cloud shadow but the values are in binary. To get the values you need to convert the band to binary, hence 2^3 and similarly 2^5 for cloud values. I hope that clarifies the comment. you can check this here: https://www.usgs.gov/landsat-missions/landsat-4-7-surface-reflectance-quality-assessment

No output in second RenderTarget when blend state is set

I have a strange issue when using two RenderTargets in SharpDX, using DX11.
I am rendering a set of objects that can be layered, and am using blend modes to achieve partial transparency. Rendering is done to two render targets in a single pass, with the second render target being used as a colour picker - I simply render the object ID (integer) to this second target and retrieve the object ID from the texture under the mouse after rendering.
The issue I am getting is frustrating, as it does not happen on all computers. In fact, it doesn't happen on any of our development machines but has been reported in the wild - typically on machines with integrated Intel (HD) graphics. On these computers, no output is generated in the second render target. We have been able to reproduce the problem on a laptop here, and if we don't set the blend state, then the issue is resolved. Obviously this isn't a fix, since we need the blending.
The texture descriptions for the main render target (0) and the colour picking target look like this:
var desc = new Texture2DDescription
{
BindFlags = BindFlags.RenderTarget | BindFlags.ShaderResource,
Format = Format.B8G8R8A8_UNorm,
Width = width,
Height = height,
MipLevels = 1,
SampleDescription = sampleDesc,
Usage = ResourceUsage.Default,
OptionFlags = RenderTargetOptionFlags,
CpuAccessFlags = CpuAccessFlags.None,
ArraySize = 1
};
var colourPickerDesc = new Texture2DDescription
{
BindFlags = BindFlags.RenderTarget,
Format = Format.R32_SInt,
Width = width,
Height = height,
MipLevels = 1,
SampleDescription = new SampleDescription(1, 0),
Usage = ResourceUsage.Default,
OptionFlags = ResourceOptionFlags.None,
CpuAccessFlags = CpuAccessFlags.None,
ArraySize = 1,
};
The blend state is set like this:
var blendStateDescription = new BlendStateDescription { AlphaToCoverageEnable = false };
blendStateDescription.RenderTarget[0].IsBlendEnabled = true;
blendStateDescription.RenderTarget[0].SourceBlend = BlendOption.SourceAlpha;
blendStateDescription.RenderTarget[0].DestinationBlend = BlendOption.InverseSourceAlpha;
blendStateDescription.RenderTarget[0].BlendOperation = BlendOperation.Add;
blendStateDescription.RenderTarget[0].SourceAlphaBlend = BlendOption.SourceAlpha;
blendStateDescription.RenderTarget[0].DestinationAlphaBlend = BlendOption.InverseSourceAlpha;
blendStateDescription.RenderTarget[0].AlphaBlendOperation = BlendOperation.Add;
blendStateDescription.RenderTarget[0].RenderTargetWriteMask = ColorWriteMaskFlags.All;
_blendState = new BlendState(_device, blendStateDescription);
and is applied at the start of rendering. I have tried explicitly setting IsBlendEnabled to false for RenderTarget[1] but it makes no difference.
Any help on this would be most welcome - ultimately, we may have to resort to making two render passes but that would be annoying.
I have now resolved this issue, although exactly how or why the "fix" works in not entirely clear. Hat-tip to VTT for pointing me to the IndependentBlendEnable flag in the BlendStateDescription. Setting that flag on its own (to true), along with RenderTarget[1].IsBlendEnabled = false, was not enough. What worked in the end was filling a complete set of values for RenderTarget[1], along with the aforementioned flags. Presumably all other values in the second RenderTarget would be ignored, as blend is disabled, but for some reason they need to be populated. As mentioned before, this problem only appears on certain graphics cards so I have no idea if this is the correct behaviour or just a quirk of those cards.

MS Chart Control Range Bar

I am trying to somehow replicate the range bar chart here.
I've found this reference but I don't fully grasp the code.
What I have is a series of task (sometimes accomplished in different periods).
let d = [("task1", DateTime.Parse("11/01/2014 08:30"), DateTime.Parse("12/01/2014 10:30"));
("task2", DateTime.Parse("15/01/2014 09:30"), DateTime.Parse("16/01/2014 10:30"));
("task3", DateTime.Parse("11/01/2014 08:30"), DateTime.Parse("16/01/2014 10:30"))]
let chart = d |> FSharp.Charting.Chart.RangeBar
chart.ShowChart()
I am struggling to understand the logic of the API.
I have also tried:
let chart = new Windows.Forms.DataVisualization.Charting.Chart(Dock = DockStyle.Fill)
let area = new ChartArea("Main")
chart.ChartAreas.Add(area)
let mainForm = new Form(Visible = true, TopMost = true, Width = 700, Height = 500)
mainForm.Controls.Add(chart)
let seriesColumns = new Series("NameOfTheSerie")
seriesColumns.ChartType <- SeriesChartType.RangeBar
type SupportToChart(serieVals: Series) =
member this.addPointXY(lbl, [<ParamArray>] yVals: Object[]) =
serieVals.Points.AddXY(lbl, yVals) |> ignore
let supporter = SupportToChart(seriesColumns)
supporter.addPointXY("AAA", DateTime.Parse("11/01/2014 08:30"), DateTime.Parse("12/01/2014 10:30") )
which results in
System.ArgumentOutOfRangeException: You can only set 1 Y values for
this data point.
Has something changed in the API since then?
I'm not entirely sure that F# Charting is currently powerful enough to be able to reconstruct the above chart. However, one of the problems seems to be that it treats dates as float values (for some reason) and incorrectly guesses the ranges. You can at least see the chart if you use:
Chart.RangeBar(d)
|> Chart.WithYAxis(Min=41650.0, Max=41660.0)
Please submit this as an issue on GitHub. If you want to dig deeper into how F# Charting works and help us get this fixed, that would be amazing :-)
The trick is initializing the Series with
let serie = new Series("Range", yValues)
where yValues defines the max number of "Y-values".

Resources