Given the following query in Influxdb's Flux language:
from(bucket: "some-great-metrics")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|> aggregateWindow(every: 1mo, fn: sum)
|> yield()
Assuming my current timezone is PST. How to ensure that aggregateWindow respect the beginning and end of the 1mo duration in this specific timezone (PST)?
Searching in the documentation does not bring so much light to me, so far.
Seems like Influx 2.1 comes with a new Flux timezone package
You can try to upgrade to 2.1 and add this before your query:
import "timezone"
option location = timezone.location(name: "America/Los_Angeles")
Things are getting more and more interesting with Flux. Besides the option location that allows you to assign one time zone to an entire query (applicable to the original question, but not applicable in general), many functions that operate on time (including aggregateWindow()) now offer a location argument that allows you to operate with multiple time zones in a single streams of table. Unfortunately some of these are still undocumented.
In case, I've written an extensive review here.
Related
I'm changing from the old Influx query to the new Flux language and I'm wondering how to efficiently get the last value of something without knowing when this last value was. So far I can only get the last value by defining a range start time. See code:
from(bucket: "my_bucket")
|> range(start: -<some_value>s)
|> filter(fn: (r) => ...
|> keep(columns:["_time", "_value",])
|> last()
But the problem is that I don't know a priori when the last value was. So if I make <some_value> large it slows down the query for things that had many values in this time range and when I give it a too small value, it won't find the last value when it was too long ago. So my question is how to find the last value in the most efficient way, similar to SELECT LAST(value) in the old syntax.
Thanks for the help!
I can't find an example that doesn't define the range.start parameter.
You can use start parameter with value 0 to start at the very beginning of "all time".
If you experience query slow down with going too much back in time, I suggest to call keep() after last(), as keep() is not a push down op. Please have a look at performance hints in Optimizing Flux Performance. The list of push down functions and combinations is listed in Start queries with pushdowns
I am quite new with F# and still trying to decide what the best structure for my financial (back testing) program should be.
As data are immutable, I am thinking that "heavy"/all-in-one structures might not be ideal.
Here is what I try to achieve:
A backtesting engine which creates a strategy every business days.
The strategy consists of a few instruments with a seq/list of trades
(trading date / quantity / price)
I then run calculation (value, risks etc) daily on all those positions for each portfolio. I also add trades to each instruments each day as I adjust the position.
How I first constructed it:
List or seq of dates: [D1 .. Dn]
List or seq of Portfolio Pi [P1 .. Pn] (with several instruments). Each portoflio will start its life at a different Di
For each portfolio, I will have some daily trades on the instrusments
when I compute value, profit and losses, risks ...
What I have used for now (very simplified):
type Instrument1 = {
some specifications
}
type Instrument2 = {
some specifications
}
type Instrument =
| Inst1 of Instrument1
| Inst2 of Instrument2
type Trade = {
Dt ; DateTime
Qty : float
Price : float }
type Portfolio = {
InitDate : DateTime // one of the Di above
Inst : Instruments
Trades : Trade seq }
type BackTesting =
Dates : DateTime seq
Port : Portfolio seq }
And then I create a seq (Dates) of seq (Portfolio) of seq (Instrument) showing let's say P&L.
However, for each portfolio Pi I am iterating on all dates to check if I need to adjust the portfolio and then add a trade to the trade list, it means that every day, for every portfolio, for every instrument, I am creating a new BackTesting (non mutable). I believe this way of reasoning is way more OOP than FP but I am a bit lost on proper patterns to use (the F# books I have used are not very clear on the data structure that works best for FP - or I did not really understand them).
I might not be very clear but if anyone has a direction into which I should look at (or any useful documentation/support on the issue), please do not hesitate. Thanks a lot for your help.
Since you are starting with F#, my suggestion to you is to not worry too much about programming in a purely functional way. If you come from an imperative style of programming it may be too much of a change and you may be discouraged. The shift from imperative style to functional style takes time and it's gradual.
The good thing is F# lets you be imperative too!
So program like you would in other languages:
Use global mutable variables when it best suits you.
Use for and while
Did you know that array elements are mutable?
As you progress you will learn the functional way, some things are really easy to use
right away:
Definitely use option, never null
Try using map, filter, choose over list, array or seq.
In time you will naturally gravitate more towards the functional style but you don't have to jump all at once. One of the best resources to get started is https://fsharpforfunandprofit.com/ its full of very good articles, slides, videos conveyed in a clear way.
Good luck!
I have two code snippets that I assumed to both result in alerts. However the first results none while the second performs the alerts.
(map #(.alert js/window %) ["hey1" "hey2"])
This slight modification prints (nil nil) as expected, as well as fixing the alert issue. The question being WHY?
(print (map #(.alert js/window %) ["hey1" "hey2"]))
Another weird observation is that the first snippet works from a browser-repl, but not when typed into code.
Is the map function side effect free, but print is not? Maybe some core code optimization I do not know about?
Work-arounds and answers are both appreciated. If you need more info please let me know in a comment.
[org.clojure/clojurescript "1.8.51"]
BOOT_CLOJURE_VERSION=1.7.0
BOOT_VERSION=2.5.5
java version "1.8.0_101"
Description: Ubuntu 14.04.4 LTS
You don't want to use map for a side-effecty operation like alert. The issue you are seeing is a result of map being lazy, so it won't actually do the work until you consume the elements of the resulting sequence. If you really want to do side-effect sort of things, doseq might be a better option, especially if you don't need a sequence of results:
(doseq [msg ["hey1" "hey2"]]
(.alert js/window msg))
Or you can use doall to force the evaluation of the whole sequence:
(doall (map #(.alert js/window %) ["hey1" "hey2"]))
Elixir streams provide iterables, but I couldn't find any information on observables (Google was no help here). I'd greatly appreciate it if someone could point me to resources for the same.
You can combine Stream and Enum to write observable-style code. Here's an example of an echo server written in observable fashion:
IO.stream(:stdio, :line)
|> Stream.map(&String.upcase/1)
|> Enum.each(&IO.write(&1))
Basically, for each line you send to standard input, it will be converted to uppercase and then printed back to standard output. This is a simple example, but the point is that all you need to compose an observable is already available via Stream and Enum.
Streams in Elixir are abstractions over function composition. In the end, all you get is a function, calling which will loop over the input stream and transform it.
In order to build stateful streams like the example in Twitter4j (buffering new twitter statutes during one second and dispatching them all in one list), you'll need to use the building blocks that can have state. In Elixir, it is common to encapsulate state in processes.
The example might look like this
tweetsPerSecond =
twitterStream
|> SS.buffer({1, :second})
|> SS.map(&length(&1))
SS.subscribe(tweetsPerSecond, fn n -> IO.puts "Got #{n} tweets in the last second" end)
SS.subscribe(tweetsPerSecond, fn n -> IO.puts "Second subscriber" end)
SS is a new module we need to write to implement the observable functionality. The core idea (as far as I get it) is being able to subscribe to a stream without modifying it.
In order for this to work, the twitterStream itself should be a process emitting events for others to consume. You can't use Stream in this case because it has "blocking pull" semantics, i.e. you won't be able to interrupt waiting on the next element in a stream after some fixed amount of time has elapsed.
To achieve the equivalent functionality in Elixir, take a look at the GenEvent module. It provides the ability to emit and subscribe to events. There is no stream-like interface for it though, not that I'm aware of.
I have built a PoC of a Pub-Sub system where I have followed a kind of "Observable Pattern": http://mendrugory.weebly.com/blog/pub-sub-system-in-elixir.
In order keep the state (what process has to be informed) I have used an Agent.
A machine is turning on and off.
seqStartStop is a seq<DateTime*DateTime> that collects the start and end time of the execution of the machine tasks.
I would like to produce the sequence of periods where the machine is idle. In order to do so, I would like to build a sequence of tuples (beginIdle, endIdle).
beginIdle corresponds to the stopping time of the machine during
the previous cycle.
endIdle corresponds to the start time of the current production
cycle.
In practice, I have to build (beginIdle, endIdle) by taking the second element of the tuple for i-1 and the fist element of the following tuple i
I was wondering how I could get this task done without converting seqStartStop to an array and then looping through the array in an imperative fashion.
Another idea creating two copies of seqStartStop: one where the head is tail is removed, one where the head is removed (shifting backwards the elements); and then appying map2.
I could use skipand take as described here
All these seem rather cumbersome. Is there something more straightforward
In general, I was wondering how to execute calculations on elements with different lags in a sequence.
You can implement this pretty easily with Seq.pairwise and Seq.map:
let idleTimes (startStopTimes : seq<DateTime * DateTime>) =
startStopTimes
|> Seq.pairwise
|> Seq.map (fun (_, stop) (start, _) ->
stop, start)
As for the more general question of executing on sequences with different lag periods, you could implement that by using Seq.skip and Seq.zip to produce a combined sequence with whatever lag period you require.
The idea of using map2 with two copies of the sequence, one slightly shifted by taking the tail of the original sequence, is quite a standard one in functional programming, so I would recommend that route.
The Seq.map2 function is fine with working with lists with different lengths - it just stops when you reach the end of the shorter list - so you don't need to chop the last element of the original copy.
One thing to be careful of is how your original seq<DateTime*DateTime> is calculated. It will be recalculated each time it is enumerated, so with the map2 idea it will be calculated twice. If it's cheap to calculate and doesn't involve side-effects, this is fine. Otherwise, convert it to a list first with List.ofSeq.
You can still use Seq.map2 on lists as a list is an IEnumerable (i.e. a seq). Don't use List.map2 unless the lists are the same length though as it is more picky than Seq.map2.