I noticed the "gap" argument in sklearn.model_selection.TimeSeriesSplit and read an article about Blocked Time Series Split which introduces a gap between training and validation. There it is argued that this can be needed when a lagged variable is used as dependent and independent variable due to "data leakage concerns". I do not really see the problem when no gap is used. What is the exact rationale? Moreover, how large does the gap then needs to be?
Thanks in advance!
I'm playing with InfluxDB and trying to experiment it for a vehicle speed tracking usecase.
Every vehicle's speed at a given time is stored as a data point.
I'm modelling "vehicle_registration" as a tag and other values as fields. I'd want the where clause to be applied on the "vehicle_registration" and it got to be quick. Therefore I'm taking advantage of the indexing capabilities on a tag by default.
But the biggest stumbling block for me is that the tags need to have a lower cardinality.
What are the recommendations here? I want a high cardinal field to be applied in a "where" clause and the queries should be quick.
Any advice?
High cardinality means higher memory requirement. So it really depends what high cardinality means in your use case. 1k will be probably fine for 8GB memory, but 1M will be probably problem for 8GB. The best option is to try it. Simulate it and you will see real memory requirements. Then you will be able to configure proper sizing for InfluxDB based on that (and your budget of course).
Or you can try TSI https://docs.influxdata.com/influxdb/v1.8/concepts/tsi-details/
This question is related to my previous question
Is it possible to get a legit range info when using a SMT constraint with Z3
So it seems that "efficiently" finding the maximum range info is not proper, given typical 32-bit vectors and so on. But on the other hand, I am thinking whether it is feasible to find certain "sub-maximum" range info, which hopefully becomes more efficient. Another thing is that we may want to have certain "safe" guarantee, say for all elements in the sub-maximum range, they must satisfy the constraint, but there could exist some other solutions that would satisfy the constraint as well.
I am currently exploring whether model counting technique could make sense in this setting. Any thoughts would be appreciated very much. Thanks.
General case
This is not just a question of efficiency. Consider a problem where you have two variables a and b, and a single constraint:
a != b
What's the range of b? (maximum or otherwise?)
You can say all values are legitimate. But that would be wrong, as obviously the choice of a impacts the choice of b. The more variables you have around, the more complicated the problem will become. I don't think the problem is even well defined in this case, so searching for a solution (efficient or otherwise) doesn't make much sense.
Single variable assumption
Having said that, I think you can come up with a solution if you assume there's precisely one variable in the system. (Or, alternatively, if you fix all the other variables to some predefined constants.) If you're willing to go down this path, then you can implement a binary search algorithm to find a reasonably sized range by simply proving the quantified formula
Exists([b], And(b >= minBound, b <= maxBound, Not(constraints)))
Once you get unsat for this, you have your range. So long as you get sat, you can adjust your minBound/maxBound to search within smaller ranges. In the worst case, this can turn into a linear walk, but you can "cut-down" this search by making sure you go down a significant size in each step. That could be a parameter to the whole search, depending on how large you want your intervals to be. It'll have to be a choice between trying to find a maximal range, and how long you want to spend in this search. Of course, if you cut-down too much, you can miss a big interval, but that's the cost of efficiency.
Example1 (Good case) There's a single constraint that says b != 5. Then your search will be quick and depending on which branch you'll go, you'll either find [0, 4] or [6, 255] assuming 8-bit words.
Example2 (Bad case) There's a single constraint that says b is even. Then your search will exhibit worst-case behavior, and if your "cut-down" size is 1, you'll possibly iterate 255 times before you settle down on [0, 0]; assuming z3 gives you the maximum odd number in each call.
I hope that illustrates the point. In general, though, I'd assume you'd be closer to the "good case" for practical applications and even if your cut-down size is minimal you can most likely converge in a few iterations. Of course, this entirely depends on your problem domain, but I'd expect it to hold for software analysis in general.
Here This is My problem.
In this heatmap i did eliminate some variables.
This is after elimination of some variables
My Question Is:
Is There Any correlated variables there in 2nd image?
Is My process of Elimination of varibles is right?
do I Still need to eliminate variables from the second image?
Help Me Out This........
No, there are no co-related variables in the image.
As per the image, ur approach to eliminate variables seems to be correct.
No, u need not eliminate any more variables.
Additional Tip: If u want to reduce number of independent variables, study about 'Principal Component Analysis'. It significantly reduces number of variables by making new axes showing maximum variance.
Although it is easy to get the current step or increment number (variables KSTEP and KINC), I can't find an easy way to know the iteration number when inside the subroutine UMAT.
I know the following "if clause" will tell me if I'm in the first iteration of the first increment of the first step:
IF((KINC.EQ.1).AND.(SUM(STRAN+DSTRAN).EQ.0.0D0)) THEN
And I also know that I can get the iteration number writing to external files. However, is it possible to do it just inside the UMAT subroutine?
There is never really a reason to need to know the iteration number in a UMAT. If you think you need to know it, this is often a sign that you there is a better way to achieve what you want to know.
You can use a common block to track how often you enter a umat, and also which iteration you are on. But I really recommend against this. There is no good reason to know the iteration number. Unless your algorithm is perfect it will cause you more problems than it's worth.
Also in your code to check for the first increment - that will not tell you when you are in a real iteration, it will happen in PRE most likely.