Could some one explain me the concept of Token bucket. Also need help in solving the below problem.
Given the token bucket size, b bytes; token rate, r bytes/sec; and maximum output rate M bytes/sec, what is the maximum burst time T?.
kindly elaborate on how to approach this problem
Have you read this? http://en.wikipedia.org/wiki/Token_bucket
Basically, token bucket could be used for throttling when you want to limit the rate of something. Think in this way, someone put 1 candy per second(r) in your bucket, then you can eat only 1 candy per sec. If your bucket can hold 10(b) candies and if you haven't eaten any of them for a while, your bucket will be full then you can eat 10 candies as fast as you can eat(M) at a time.
I guess the answer for your question about Maximum burst time T can be calculated in this way. Point is that while you're eating candies at M rate, it's also getting refilled at r rate.
t = b/m;
while(1) {
T = (b + (t * r)) / m
if (T == t) return T;
t = T;
}
So if b = 10, r = 1/sec, M = 2/sec
then T = 9.
Here's a good explain: https://www.coursera.org/lecture/packet-switching-networks-algorithms/traffic-shaping-by-token-bucket-V07oV
To your problem, we can make an equation base on the data IO: after time T, data output should equals to data input. That is:
b+rT=MT
T=b/(M-r)
Related
I have a problem where I need to do a linear interpolation on some data as it is acquired from a sensor (it's technically position data, but the nature of the data doesn't really matter). I'm doing this now in matlab, but since I will eventually migrate this code to other languages, I want to keep the code as simple as possible and not use any complicated matlab-specific/built-in functions.
My implementation initially seems OK, but when checking my work against matlab's built-in interp1 function, it seems my implementation isn't perfect, and I have no idea why. Below is the code I'm using on a dataset already fully collected, but as I loop through the data, I act as if I only have the current sample and the previous sample, which mirrors the problem I will eventually face.
%make some dummy data
np = 109; %number of data points for x and y
x_data = linspace(3,98,np) + (normrnd(0.4,0.2,[1,np]));
y_data = normrnd(2.5, 1.5, [1,np]);
%define the query points the data will be interpolated over
qp = [1:100];
kk=2; %indexes through the data
cc = 1; %indexes through the query points
qpi = qp(cc); %qpi is the current query point in the loop
y_interp = qp*nan; %this will hold our solution
while kk<=length(x_data)
kk = kk+1; %update the data counter
%perform online interpolation
if cc<length(qp)-1
if qpi>=y_data(kk-1) %the query point, of course, has to be in-between the current value and the next value of x_data
y_interp(cc) = myInterp(x_data(kk-1), x_data(kk), y_data(kk-1), y_data(kk), qpi);
end
if qpi>x_data(kk), %if the current query point is already larger than the current sample, update the sample
kk = kk+1;
else %otherwise, update the query point to ensure its in between the samples for the next iteration
cc = cc + 1;
qpi = qp(cc);
%It is possible that if the change in x_data is greater than the resolution of the query
%points, an update like the above wont work. In this case, we must lag the data
if qpi<x_data(kk),
kk=kk-1;
end
end
end
end
%get the correct interpolation
y_interp_correct = interp1(x_data, y_data, qp);
%plot both solutions to show the difference
figure;
plot(y_interp,'displayname','manual-solution'); hold on;
plot(y_interp_correct,'k--','displayname','matlab solution');
leg1 = legend('show');
set(leg1,'Location','Best');
ylabel('interpolated points');
xlabel('query points');
Note that the "myInterp" function is as follows:
function yi = myInterp(x1, x2, y1, y2, qp)
%linearly interpolate the function value y(x) over the query point qp
yi = y1 + (qp-x1) * ( (y2-y1)/(x2-x1) );
end
And here is the plot showing that my implementation isn't correct :-(
Can anyone help me find where the mistake is? And why? I suspect it has something to do with ensuring that the query point is in-between the previous and current x-samples, but I'm not sure.
The problem in your code is that you at times call myInterp with a value of qpi that is outside of the bounds x_data(kk-1) and x_data(kk). This leads to invalid extrapolation results.
Your logic of looping over kk rather than cc is very confusing to me. I would write a simple for loop over cc, which are the points at which you want to interpolate. For each of these points, advance kk, if necessary, such that qp(cc) is in between x_data(kk) and x_data(kk+1) (you can use kk-1 and kk instead if you prefer, just initialize kk=2 to ensure that kk-1 exists, I just find starting at kk=1 more intuitive).
To simplify the logic here, I'm limiting the values in qp to be inside the limits of x_data, so that we don't need to test to ensure that x_data(kk+1) exists, nor that x_data(1)<pq(cc). You can add those tests in if you wish.
Here's my code:
qp = [ceil(x_data(1)+0.1):floor(x_data(end)-0.1)];
y_interp = qp*nan; % this will hold our solution
kk=1; % indexes through the data
for cc=1:numel(qp)
% advance kk to where we can interpolate
% (this loop is guaranteed to not index out of bounds because x_data(end)>qp(end),
% but needs to be adjusted if this is not ensured prior to the loop)
while x_data(kk+1) < qp(cc)
kk = kk + 1;
end
% perform online interpolation
y_interp(cc) = myInterp(x_data(kk), x_data(kk+1), y_data(kk), y_data(kk+1), qp(cc));
end
As you can see, the logic is a lot simpler this way. The result is identical to y_interp_correct. The inner while x_data... loop serves the same purpose as your outer while loop, and would be the place where you read your data from wherever it's coming from.
I am currently having issues with figuring our some recurrence stuff and since I have midterms about it coming up soon I could really use some help and maybe an explanation on how it works.
So I basically have pseudocode for solving the Tower of Hanoi
TOWER_OF_HANOI ( n, FirstRod, SecondRod, ThirdRod)
if n == 1
move disk from FirstRod to ThirdRod
else
TOWER_OF_HANOI(n-1, FirstRod, ThirdRod, SecondRod)
move disk from FirstRod to ThirdRod
TOWER_OF_HANOI(n-1, SecondRod, FirstRod, ThirdRod)
And provided I understand how to write the relation (which, honestly I'm not sure I do...) it should be T(n) = 2T(n-1)+Ɵ(n), right? I sort of understand how to make a tree with fractional subproblems, but even then I don't fully understand the process that would give you the end solution of Ɵ(n) or Ɵ(n log n) or whatnot.
Thanks for any help, it would be greatly appreciated.
Assume the time complexity is T(n), it is supposed to be: T(n) = T(n-1) + T(n-1) + 1 = 2T(n-1) + 1. Why "+1" but not "+n"? Since "move disk from FirstRod to ThirdRod" costs you only one move.
For T(n) = 2T(n-1) + 1, its recursion tree will exactly look like this:
https://www.quora.com/What-is-the-complexity-of-T-n-2T-n-1-+-C (You might find it helpful, the image is neat.) C is a constant; it means the cost per operation. In the case of Tower of Hanoi, C = 1.
Calculate the sum of the cost each level, you will easily find out in this case, the total cost will be 2^n-1, which is exponential(expensive). Therefore, the answer of this recursion equation is Ɵ(2^n).
Ok, here it goes another Euler problem question.
I've started to learn Lua by solving Euler project problems and got stuck on Euler problem 12.
It looks to me very straightforward and I don't understand why is my result incorrect?
Here is my solution so far:
-- return triangular number of the specified number
function get_tri_num(num)
local n = 0
for i=1, num do
n = n + i
end
return n
end
-- return all factors of the specifeid number
function factors(num)
local factors = {}
for i=1, num/2 do
if num%i == 0 then
factors[#factors+1] = i
end
end
factors[#factors+1] = num
return factors
end
-- get the first triangle number with >500 divisors
function euler12()
local n = 0
local trinum = 1
while true do
n = n + 7
trinum = get_tri_num(n)
if #factors(trinum) > 500 then break end
end
print(trinum, n)
end
euler12()
This problem is computation intensive, well, at least the way I am solving it, so I use luajit.
time luajit euler12.lua
103672800 14399
real 3m14.971s
user 3m15.033s
sys 0m0.000s
First, I try this solution on the toy example provided in the problem description. Changing the line of euler12() to if #factors(trinum) > 5 then break end, I get:
28 7
Which corresponds to the results shown in the problem example.
Second, after I see that the toy example is working I run euler12() with >500 condition. According to my solution the answer is 103672800 and yes, if I separately check the number of divisors for this result is >500:
print(#factors(103672800))
648
But...
The problem is here:
while true do
n = n + 7
Why does n increaments 7 each time? That doesn't make sense, change it to 1, and you could get the correct answer.
However, the performance is still poor. Several places that could be improved:
Every time the function get_tri_num is called, it's calculating
from scratch, that's not necessary.
You don't need the factors of a number, you only need the number of
factors of a number, so why return a table in factors?
for i=1, num/2 do is not necessary. Iterating to the square root of
num is enough to get the number of factors.
Refer to my code for the same problem.
This is my code:
def is_prime(i)
j = 2
while j < i do
if i % j == 0
return false
end
j += 1
end
true
end
i = (600851475143 / 2)
while i >= 0 do
if (600851475143 % i == 0) && (is_prime(i) == true)
largest_prime = i
break
end
i -= 1
end
puts largest_prime
Why is it not returning anything? Is it too large of a calculation going through all the numbers? Is there a simple way of doing it without utilizing the Ruby prime library(defeats the purpose)?
All the solutions I found online were too advanced for me, does anyone have a solution that a beginner would be able to understand?
"premature optimization is (the root of all) evil". :)
Here you go right away for the (1) biggest, (2) prime, factor. How about finding all the factors, prime or not, and then taking the last (biggest) of them that is prime. When we solve that, we can start optimizing it.
A factor a of a number n is such that there exists some b (we assume a <= b to avoid duplication) that a * b = n. But that means that for a <= b it will also be a*a <= a*b == n.
So, for each b = n/2, n/2-1, ... the potential corresponding factor is known automatically as a = n / b, there's no need to test a for divisibility at all ... and perhaps you can figure out which of as don't have to be tested for primality as well.
Lastly, if p is the smallest prime factor of n, then the prime factors of n are p and all the prime factors of n / p. Right?
Now you can complete the task.
update: you can find more discussion and a pseudocode of sorts here. Also, search for "600851475143" here on Stack Overflow.
I'll address not so much the answer, but how YOU can pursue the answer.
The most elegant troubleshooting approach is to use a debugger to get insight as to what is actually happening: How do I debug Ruby scripts?
That said, I rarely use a debugger -- I just stick in puts here and there to see what's going on.
Start with adding puts "testing #{i}" as the first line inside the loop. While the screen I/O will be a million times slower than a silent calculation, it will at least give you confidence that it's doing what you think it's doing, and perhaps some insight into how long the whole problem will take. Or it may reveal an error, such as the counter not changing, incrementing in the wrong direction, overshooting the break conditional, etc. Basic sanity check stuff.
If that doesn't set off a lightbulb, go deeper and puts inside the if statement. No revelations yet? Next puts inside is_prime(), then inside is_prime()'s loop. You get the idea.
Also, there's no reason in the world to start with 600851475143 during development! 17, 51, 100 and 1024 will work just as well. (And don't forget edge cases like 0, 1, 2, -1 and such, just for fun.) These will all complete before your finger is off the enter key -- or demonstrate that your algorithm truly never returns and send you back to the drawing board.
Use these two approaches and I'm sure you'll find your answers in a minute or two. Good luck!
Do you know you can solve this with one line of code in Ruby?
Prime.prime_division(600851475143).flatten.max
=> 6857
EDIT: Based on additional experimentation, I'm fairly confident the slow-down occurs in response to many calls to the file open and close routines (h5open and close). I'm a bit short on time right now, but will come back and add more code/detail in the next few days.
Using the HDF5 package for Julia, I've noticed that the h5write function starts to slow down if one performs many iterations over calls to h5write and h5read. Interestingly, it appears that for the behaviour to be really obvious, one should be reading and writing to a large number of locations in a small number of files. A demonstration of the behaviour I'm talking about can be obtained by starting a Julia session and running the following subroutine (note, you will need the HDF5 package):
#Set parameters
numFile = 10;
numLocation = 10000;
writeDir = "/home/colin/Temp/";
FloatOut = 5.5;
#Import functions
using HDF5
#Loop over read/writes.
c1 = 1;
timeMat = Array(Float64, numFile * 2, 2);
for i in 1:numFile
filePath = string(writeDir, "f", string(i), ".h5");
for j in 1:numLocation
location = string("G1/L", string(j));
if j == 1 || j == numLocation; tic(); end;
h5write(filePath, location, FloatOut);
if j == 1 || j == numLocation; timeMat[c1, 1] = toc(); end;
if j == 1 || j == numLocation; tic(); end;
FloatIn = h5read(filePath, location);
if j == 1 || j == numLocation; timeMat[c1, 2] = toc(); end;
if j == 1 || j == numLocation; c1 = c1+1; end;
end
rm(filePath);
end
This code writes the floating point number 5.5 (chosen for no particular reason) to 10,000 locations in each of 10 files using h5write. Immediately after performing the write operation each time, the number is then read back in using h5read. For each file, I store the time taken to perform the write and read operation to the first and last location for each file in timeMat (note: initially, I stored the timing for every call but this level of detail is unnecessary to demonstrate the anomaly for the purposes of this question). The times are printed below:
h5write h5read
0.0007 0.0004
0.0020 0.0004
0.0020 0.0004
0.0031 0.0004
0.0034 0.0004
0.0049 0.0004
0.0050 0.0004
0.0064 0.0005
0.0068 0.0004
0.0082 0.0004
0.0084 0.0005
0.0106 0.0005
0.0114 0.0005
0.0114 0.0005
0.0120 0.0005
0.0131 0.0005
0.0135 0.0005
0.0146 0.0005
0.0151 0.0005
0.0163 0.0005
The timings for h5read are fairly consistent across the subroutine. However, the timings for h5write gradually get slower. By the end, a write is taking an order of magnitude longer than at the start. I understand that for each file, as we increase the number of locations, the time for a write (and read) might get slightly slower. But in this case, the slower performance persists even after we begin a new file. Perhaps strangest of all, we can run the subroutine a second time and time taken for a write will pick up where we left off on the previous run. The only way to get the time taken for a write back to the fastest speed is to completely restart Julia.
Final disclaimer: I am brand new to both Julia and hdf5, so I may have done something stupid or be overlooking something obvious.
The slowdown is indeed curious; profiling shows that almost all the time is spent in the close function, which is basically just a ccall. This suggests it may be a problem with the HDF5 C-library itself.
I think you'll be rather happier with the performance if you don't open and close the file each time you write a variable; instead, access the file through a file object. Here's an example:
filePath = string(writeDir, "f", string(i), ".h5")
h5open(filePath, "w") do file
global c1
for j in 1:numLocation
...
write(file, location, FloatOut)
...
FloatIn = read(file, location)
...
end
end
This way you're leaving the file open throughout the test. On my machine this is something like 100x faster.
If you want to pursue this further, please submit an issue.