I have a channel that I want stop animations from happening if running on a slower device like Roku Express and keep them on a faster device like Roku Premiere. Except I'm not sure what's the best way to go about it.
I wanted to filter by the amount of available ram, but I couldn't find an api that gives me available ram for the system that I could run in my code.
I could filter by model name, but I would then need to keep an update list of model names, which I prefer not to do.
Any help/insight appreciated.
Re graphic capabilities, try roDeviceInfo.getGraphicsPlatform() - if it returns opengl, that high performing engine that can do arbitrary rotations vs directfb being limited.
Re CPU, you can run a mini benchmark on start of your program, something like
ti = createObject("roTimeSpan"): s=""
for i = 1 to 1000: s = s + right((i^3).toStr(),2): end for
time = ti.totalMilliSeconds()
Have you considered using Animation.optional=true?
It won't stop them from happening on Roku Express (since it is a Littlefield) but it will "skip animations on lower performing Roku devices (Paolo, Giga, Jackson, Tyler, and Sugarland)".
Animation also contains an undocumented field called "willBeSkipped" which will be true on slower devices when "optional" is set to true.
I had the similar problem with the animations. Unfortunately, You must filter by model name. I didn't find another way.
You can store the list of devices in database so it would be easier for You to maintain.
You can set the optional field on the animation node to true. This is supposed to take care of that. I have set this field to true before and it does not seem to have an effect. I'm sure they'll get around to fixing it eventually.
The efficiency of the animations also depends on how many animation nodes you have. You should only need 1 animation node to handle all of your animations for a particular component. Add an interpolator for each individual type of animation you want to occur (i.e. scaling, rotating, color-shifting, translating).
Related
I am using opencv and openvino and am trying to figure out when I have a face detected, use the cv2.rectangle and have my coordinates sent but only on the first person bounded by the box so it can move the motors because when it sees multiple people it sends multiple coordinates and thus causing the servo and stepper motors to go crazy. Any help would be appreciated. Thank you
Generally, each code would run line by line. You'll need to create a proper function for each scenario so that the data could be handled and processed properly. In short, you'll need to implement error handling and data handling (probably more than these, depending on your software/hardware design). If you are trying to implement multiple threads of executions at the same time, it is better to use multithreading.
Besides, you are using 2 types of motors. Simply taking in all data is inefficient and prone to cause missing data. You'll need to be clear about what servo motor and stepper motor tasks are, the relations between coordinates, who will trigger what, if something fails or some sequence is missing then do task X, etc.
For example, the sequence of Data A should produce Result A but it is halted halfway because Data B went into the buffer and interfered with Result A and at the same time screwed Result B which was anticipated to happen. (This is what happened in your program)
It's good to review and design your whole process by creating a coding flowchart (a diagram that represents an algorithm). It will give you a clear idea of what should happen for each sequence of code. Then, design a proper handler for each situation.
Can you share more insights of your (pseudo-)code, please?
It sounds easy - you trigger a face-detection inference-request and you get a list/vector with all detected faces (the region-of-interest for each detected face) (including false-positive and false-positives, requiring some consistency-checks to filter those).
If you are interested in the first detected face only - then it could be to just process the first returned result from the list/vector.
However, you will see that sometimes the order of results might change, i.e. when 2 faces A and B were detected, in the next run it could still return faces, but B first and then A.
You could add object-tracking on top of face-detection to make sure you always process the same face.
(But even that could fail sometimes)
I'm trying to do trajectory optimization for a custom robot I've specified with an sdf file.
I'd like to use direct collocation, but when I try to create the MultibodyPlant with time_step=0.0 I fail with a segfault. It works just fine when I use discrete time (e.g. Multibodyplant(time_step=.005).
However, if I use discrete time, the state is no longer continuous so I can't use direct collocation. So I tried to use direct transcription and I get the error
SystemExit: Failure at bazel-out/k8-opt/bin/systems/framework/_virtual_includes/context/drake/systems/framework/context.h:111 in num_total_states(): condition 'num_abstract_states() == 0' failed.
I think the reason is that DirectTranscription does not have a assume_non_continuous_states_are_fixed, the same issue as in this question: direct transcription for compass gait. So maybe the easiest solution to my problem is to request this feature..
I recommended above that we do add that assume_non_continuous_states_are_fixed to DirectTranscription. But the reason that this option was not implemented already is a little subtle, so I’ll add it here.
It’s not actually MultibodyPlant that is adding the abstract state, but SceneGraph. For dynamics/planning, you only need SceneGraph if you are relying on contact forces in your dynamics. For acrobots / cart-poles, etc, you can already use a MultibodyPlant with DirectTranscription by passing the MBP only (no SceneGraph) to the optimization. And for systems that do make and break contact, I would have said that DirectTranscription might not be the algorithm you want; although there are no hard and fast rules saying it won’t work. It’s just that you’ll end up with stiff differential equations which are hard to transcribe in a reasonable trajectory optimization that doesn’t reason explicitly about the contact.
I think I know your application, which involves wheels that are in contact and stay in contact. That means that you do need SceneGraph. That might be a case were this currently missing combination makes perfect sense, and we should add it.
I've set up a bunch of sliders to manipulate the values of various GPUIImageFilters targeted by a GPUIImagePicture. My current chain order looks like this:
self.gpuImagePicture = [[GPUImagePicture alloc] initWithImage:self.image];
[self.gpuImagePicture addTarget:self.toneCurveFilter];
[self.toneCurveFilter addTarget:self.exposureFilter];
[self.exposureFilter addTarget:self.constrastFilter];
[self.constrastFilter addTarget:self.saturationFilter];
[self.saturationFilter addTarget:self.highlightShadowFilter];
[self.highlightShadowFilter addTarget:self.whiteBalanceFilter];
[self.whiteBalanceFilter addTarget:self.gpuImageView];
[self.whiteBalanceFilter setInputRotation:[self gpuImageRotationModeForImage:self.image] atIndex:0];
[self.gpuImagePicture processImage];
When I remove the tone curve filter everything works smoothly. If I use the tone curve filter alone I have no issues either. When I use the above implementation processing slows down tremendously.
Does the order of the filter-chaining matter when it comes to memory management and processing, or did adding the tone curve filter to the rest of the chain just push this setup over the edge?
EDIT:
I've realized it might be worth mentioning how the sliders change the filter values. If the exposure slider is moved, for example, it triggers this code:
[self.exposureFilter setExposure:sender.value];
[self.gpuImagePicture processImage];
Sometimes, filter order doesn't matter, but it usually does. Some color-adjustment operations work in such a way that they can be rearranged without changing the output, but most filter calculations will produce slightly to significantly different results if you rearrange them. Think about it like arithmetic where you change the order of operations or move some parentheses around.
Now, when it comes to performance or memory usage, order usually doesn't matter. Branching operations are the only case where this comes into play (having a filter that targets multiple outputs, that at some point are blended or combined into one output). You don't have that here, though.
You do have a number of steps in the above, and there is overhead in every filter you chain. However, I'm surprised you even see a difference in performance, because the bottleneck in the above should be the creation of the GPUImagePicture. Instantiating one of those is far slower than any filter you'd perform on it, due to the pass through Core Graphics needed to re-render and upload the picture as a texture.
If you are reusing your toneCurveFilter or others, make sure that they are fully disconnected from all their previous targets before using -addTarget: again. It's possible that you're switching out pictures while leaving all the filters connected to their previous targets, meaning that each new picture will keep adding targets. This will lead to tremendous slowdown.
I bet something like the above is what's slowing you down, but when in doubt fire up Time Profiler and / or the OpenGL profiler and see where you're really spending all your time.
If a technical indicator works very slow, and I wish to include it in an EA ( using iCustom() ), is there a some "wrapper" that could cache the indicator results to a file based on the particular indicator inputs?
This way I could get a better speed next time when I backtest it using the same set of parameters, since the "wrapper" could read the result from file rather than recalculate the result from the indicator.
I heard that some developers did that for their needs in order to speed up backtesting, but as far as i know, there's no publicly available solution.
If I had to solve this problem, I would create a class with two fields (datetime and indicator value, or N buffers of the indicator), and a collection class similar to CArrayObj.mqh but with an option to apply binary search, or to start looking for element from a specific index, not from the very beginning of the array.
Recent MT4 Builds added VERY restrictive conditions for Indicators
In early years of MT4, this was not so cruel as it is these days.
FACT#1: fileIO is 10.000x ~ 100.000x slower than memIO:
This means, there is no benefit from "pre-caching" values to disk.
FACT#2: Processing Performance has HARD CEILING:
All, yes ALL, Custom Indicators, that are being used in MetaTrader4 Terminal ( be it directly in GUI, or indirectly, via Template(s) or called via iCustom() calls & in Strategy Tester via .tpl + iCustom() ) ALL THESE SHARE A SINGLE THREAD ...
FACT#3: Strategy Tester has the most demanding needs for speed:
Thus - eliminate all, indeed ALL, non-core indicators from tester.tpl template and save it as "blank", to avoid any part of such non-core processing.
Next, re-design the Custom Indicator, where possible, so as to avoid any CPU-ops & MEM-allocation(s), that are not necessary.
I remember a Custom Indicatore designs with indeed deep-convolutions, which could have been re-engineered so as to keep just a triangular sparse-matrix with necessary updates, that has increased the speed of Indicator processing more than 10.000x, so code-revision is the way.
So, rather run a separate MetaTrader4 Terminal, just for BackTesting, than having to wait for many hours just due to un-compressible nature of numerical processing under a traffic-jam congestion in the shared use of the CustomIndicator-solo-Thread that none scheduling could improve.
FACT#4: O/S can increase a process priority:
Having got to the Devil's zone, it is a common practice to spin-up the PRIO for the StrategyTester MT4, up to the "RealTime PRIO" in the O/S tools.
One may even additionally "lock" this MT4-process onto a certain CPU-core(s) and setup all other processes with adjacent CPU-core-AFFINITY, so that these two distinct groups of processes do not jump one to the other group's CPU-core(s). Hard, but if squeezing the performance to the bleeding edge, this is a must.
I want to de-dupe a stream of data based on an ID in a windowed fashion. The stream we receive has and we want to remove data with matching within N-hour time windows. A straight-forward approach is to use an external key-store (BigTable or something similar) where we look-up for keys and write if required but our qps is extremely large making maintaining such a service pretty hard. The alternative approach I came up with was to groupBy within a timewindow so that all data for a user within a time-window falls within the same group and then, in each group, we use a separate key-store service where we look up for duplicates by the key. So, I have a few questions about this approach
[1] If I run a groupBy transform, is there any guarantee that each group will be processed in the same slave? If guaranteed, we can group by the userid and then within each group compare the sessionid for each user
[2] If it is feasible, my next question is to whether we can run such other services in each of the slave machines that run the job - in the example above, I would like to have a local Redis running which can then be used by each group to look up or write an ID too.
The idea seems off what Dataflow is supposed to do but I believe such use cases should be common - so if there is a better model to approach this problem, I am looking forward to that too. We essentially want to avoid external lookups as much as possible given the amount of data we have.
1) In the Dataflow model, there is no guarantee that the same machine will see all the groups across windows for the key. Imagine that a VM dies or new VMs are added and work is split across them for scaling.
2) Your welcome to run other services on the Dataflow VMs since they are general purpose but note that you will have to contend with resource requirements of the other applications on the host potentially causing out of memory issues.
Note that you may want to take a look at RemoveDuplicates and use that if it fits your usecase.
It also seems like you might want to be using session windows to dedupe elements. You would call:
PCollection<T> pc = ...;
PCollection<T> windowed_pc = pc.apply(
Window<T>into(Sessions.withGapDuration(Duration.standardMinutes(N hours))));
Each new element will keep extending the length of the window so it won't close until the gap closes. If you also apply an AfterCount speculative trigger of 1 with an AfterWatermark trigger on a downstream GroupByKey. The trigger would fire as soon as it could which would be once it has seen at least one element and then once more when the session closes. After the GroupByKey you would have a DoFn that filters out an element which isn't an early firing based upon the pane information ([3], [4]).
DoFn(T -> KV<session key, T>)
|
\|/
Window.into(Session window)
|
\|/
Group by key
|
\|/
DoFn(Filter based upon pane information)
It is sort of unclear from your description, can you provide more details?
Sorry for not being clear. I gave the setup you mentioned a try, except for the early and late firings part, and it is working on smaller samples. I have a couple of follow up questions, related to scaling this up. Also, I was hoping I could give you more information on what the exact scenario is.
So, we have incoming data stream, each item of which can be uniquely identified by their fields. We also know that duplicates occur pretty far apart and for now, we care about those within a 6 hour window. And regarding the volume of data, we have atleast 100K events every second, which span across a million different users - so within this 6 hour window, we could get a few billion events into the pipeline.
Given this background, my questions are
[1] For the sessioning to happen by key, I should run it on something like
PCollection<KV<key, T>> windowed_pc = pc.apply(
Window<KV<key,T>>into(Sessions.withGapDuration(Duration.standardMinutes(6 hours))));
where key is a combination of the 3 ids I had mentioned earlier. Based on the definition of Sessions, only if I run it on this KV would I be able to manage sessions per-key. This would mean that Dataflow would have too many open sessions at any given time waiting for them to close and I was worried if it would scale or I would run into any bottle-necks.
[2] Once I perform Sessioning as above, I have already removed the duplicates based on the firings since I will only care about the first firing in each session which already destroys duplicates. I no longer need the RemoveDuplicates transform which I found was a combination of (WithKeys, Combine.PerKey, Values) transforms in order, essentially performing the same operation. Is this the right assumption to make?
[3] If the solution in [1] going to be a problem, the alternative is to reduce the key for sessioning to be just user-id, session-id ignoring the sequence-id and then, running a RemoveDuplicates on top of each resulting window by sequence-id. This might reduce the number of open sessions but still would leave a lot of open sessions (#users * #sessions per user) which can easily run into millions. FWIW, I dont think we can session only by user-id since then the session might never close as different sessions for same user could keep coming in and also determining the session gap in this scenario becomes infeasible.
Hope my problem is a little more clear this time. Please let me know any of my approaches make the best use of Dataflow or if I am missing something.
Thanks
I tried out this solution at a larger scale and as long as I provide sufficient workers and disks, the pipeline scales well although I am seeing a different problem now.
After this sessionization, I run a Combine.perKey on the key and then perform a ParDo which looks into c.pane().getTiming() and only rejects anything other than an EARLY firing. I tried counting both EARLY and ONTIME firings in this ParDo and it looks like the ontime-panes are actually deduped more precisely than the early ones. I mean, the #early-firings still has some duplicates whereas the #ontime-firings is less than that and has more duplicates removed. Is there any reason this could happen? Also, is my approach towards deduping using a Combine+ParDo the right one or could I do something better?
events.apply(
WithKeys.<String, EventInfo>of(new SerializableFunction<EventInfo, String>() {
#Override
public java.lang.String apply(EventInfo input) {
return input.getUniqueKey();
}
})
)
.apply(
Window.named("sessioner").<KV<String, EventInfo>>into(
Sessions.withGapDuration(mSessionGap)
)
.triggering(
AfterWatermark.pastEndOfWindow()
.withEarlyFirings(AfterPane.elementCountAtLeast(1))
)
.withAllowedLateness(Duration.ZERO)
.accumulatingFiredPanes()
);