More efficient way to parse a matrix in python? - parsing

I have some data in a text file that looks like this:
1895723957
8599325893
5723857831
5025852920
and I'd like to parse it into a list of lists in Python, so the output is
[[1, 8, 9, 5, 7, 2, 3, 9, 5, 7], [8, 5, ...
Right now, I have
data = open('file.txt')
rows = [str(line).strip() for line in data]
matrix=[]
for r in rows:
matrix.append(list(r))
but are there different ways to do this, such as using less lines of code or exploiting comprehensions?
I've tried looking around, but I'm not exactly sure what keywords to use here...
Thanks much!

I'd try something like this:
with open('file.txt', 'r') as handle:
matrix = [map(int, line.strip()) for line in handle]

I came up with the following way after playing around with comprehensions:
data = open('file.txt')
matrix = [[int(c) for c in row.rstrip()] for row in data]
rstrip is thanks to Blender above.

Related

Counting on NumberLine

I've been searching everywhere and I can't seem to find how one counts (with "bouncing curved arrows") on a NumberLine. How to do so?
This is my desired output:
I'm still quite new to manim. Thank you in advance.
You can use CurvedArrow for that.
Example :
class CountingNumberLine(Scene):
def construct(self):
n = NumberLine(x_range=[-5, 5, 1], include_numbers=True)
self.add(n)
for i in range(3):
self.add(CurvedArrow(n.number_to_point(i), n.number_to_point(i + 1), angle=- TAU / 4))
Result
Note that if you want to have only an arrow for the last count (like in your image) you can use ArcBetweenPoint instead of CurvedArrow.

highchartr: Synchronize point and line colors and legend

I'm using the highchartr package to create an interactive chart.
My chart has lines on it corresponding to different series. In addition, I would like to have shapes at certain points on the lines.
Its very easy to get the points in the right place. However, I would like to map the point color to the line it is associated with. And when the user clicks on the legend entry for the line, I'd like the associated points to be toggled as well.
The code looks like this:
highchart() %>%
hc_add_series(
type="line",
marker=list(enabled=F),
data=input_data,
mapping=hcaes(x=x, y=y, group=series_name)
) %>%
hc_add_series(
type="point",
data=input_data %>% filter(! is.na(marker)),
mapping=hcaes(x=x, y=y, color=series_name, fill=series_name, group=series_name, shape=marker)
)
The result gets the points in the right place. But the point color is on a different color mapping from the lines. Clicking on the entry for the line in the legend toggles only the line - the points show up as separate entries by series_name.
What
What can I do so:
- The points and lines share the same color mapping
- The points and lines can be toggled together by clicking on the line in the legend
- The points show up separately in the legend based on their shape rather than their color?
Thanks!
Generally, it can be achieved in at least few different ways. It all depends on your data which you haven't provided (I created a sample data).
Additionally, I will provide all the examples in jsFiddle (JavaScript) because it is faster to explain something that way with a quick online example.
The final answer will contain R code (maybe with some custom JavaScript if needed, but all will be reproducible in R.
First of all, your assumption that you need a separate series is wrong and causes problems. If you want markers on your line with the same color and you want to toggle them together on legend click, then you don't need separate series - one series with markers enabled on some points is enough, see this example: https://jsfiddle.net/BlackLabel/s24rk9x7/
In this case, the R data needs to be defined properly.
If you don't want to keep it simple as described above, you can keep lines and markers as separate series as in your original question.
In this case, you can use series.linkedTo property to connect your "point" series to line series (BTW in Highcharts there is no something like "point" series type, it is "scatter" series type. Another reason why your code is wrong and is not working and you got unvoted), but there is a problem with it in Highcharter - doesn't work, seems like a bug and should be reported on Highcharter GitHub repo.
This is a JavaScript version which works fine: https://jsfiddle.net/BlackLabel/3mtdfqLo/
In this example, if you want to keep markers and line series in the same color, you can define colors manually or you can write some custom code (like I did) that will change the color for you automatically.
And this is the same R version which should work, but is not:
library(highcharter)
highchart() %>%
hc_add_series(
data=list(4, 3, 5, 6, 2, 3)
) %>%
hc_add_series(
data=list(14, 13, 15, 16, 12, 13),
id="first"
) %>%
hc_add_series(
data=list(10, 8, 6, 2, 5, 12),
id="second"
) %>%
hc_add_series(
type="scatter",
linkedTo="first",
data=list(list(1, 3), list(2, 5))
) %>%
hc_add_series(
type="scatter",
linkedTo="second",
data=list(list(1, 13), list(2, 15), list(3, 16))
) %>%
hc_plotOptions(
line = list(marker=list(enabled=F))
)
There is probably something wrong with hc_add_series function.
As a workaround, you can write it all as a custom JavaScript code, which (again) works fine:
library(highcharter)
highchart() %>%
hc_plotOptions(
line = list(marker=list(enabled=F))
) %>%
hc_chart(
events = list(load = JS("function() {
this.addSeries({
data: [4, 3, 5, 6, 2, 3],
id: 'first'
});
this.addSeries({
data: [14, 13, 15, 16, 12, 13],
id: 'second'
});
this.addSeries({
data: [10, 8, 6, 2, 5, 12]
});
this.addSeries({
type: 'scatter',
linkedTo: 'first',
data: [[1, 3], [2, 5]]
});
this.addSeries({
type: 'scatter',
linkedTo: 'second',
data: [[1, 13], [2, 15], [3, 16]]
});
}")))
Of course, last examples don't contain functionality that changes colors - you can copy it from the jsFiddle above.

LSTM, pattern and noise gap

I want to find sequence patterns in a time series with random noise gap.
For example, this is the pattern I wan to find:
1, 2, 3, 4
But, my samples are:
*1*, 10, *2*, *3*, 11, 12, *4*
*1*, *2*, 10, 14, 15, *3*, 10, 13, *4*
10, *1*, 10, 10, 10, *2*, 11, 12, *3*, *4*
I don't know that the "good" elements are 1, 2, 3 and 4.
I started with a LSTM decoder, but "the noise" hide the good elements. For example, with the 3 samples, I get:
*1*, 10, 13, 10, ...
and 2, 3 and 4 are hidden
Have you an idea to find those patterns ?
Thanks.
Frédéric
As a starting point you can use a sequence-to-sequence (seq2seq) model. The linked repo has nice explanation how these models work and what type of problems they cover. The crucial point would be how to encode your sequence. Often they are encoded as one-hot vectors. So if you have a fixed upper bound on the number of distinct numbers/items in your sequence you can use it.
Instead of generating a new sequence without noise from the original one, you can also try to classify each point as noise or not and eliminate those classified as noise as your output. Something along the lines of:
seq = Input(shape=(timesteps, features))
hidden = LSTM(HIDDEN_UNITS, return_sequences=True)(seq)
out = TimeDistributed(Dense(1, activation='sigmoid'))(hidden)
You will have to know before hand whether each data point is noise or not.

Random Sampling without Replacement in Ruby

Is there some function in ruby that generates k random numbers in a range without replacement (repetition)?
Something like the random.sample function in Python as shown below
>>> import random
>>> random.sample(range(1, 100), 3)
[77, 52, 45]
You can create a range, convert the range to an array, and then call Array::sample, which can take an argument that specifies the number of samples.
(1..100).to_a.sample(3)
You could use this:
(1..100).to_a.shuffle[0..2] #=> [13, 36, 88]
Where:
(1..100).to_a creates an array with the possible values.
.shuffle sorts the array randomly.
[0..2] grabs the first 3 elements of shuffled array.

More compact Solution in Maxima

I have the following code:
for n:1 thru 11 do for j:1 thru 21 do v[n,j]:1/sqrt(dp)*
(sum(eigenfunctionsort[n,j]*exp(%i*2*%pi*m*x/dp),m,-10,10));
Where eigenfunctionsort is defined earlier,x is a variable I will integrate over later and I am summing over m.
When I print say v[1,1], I get a big long nasty equation. How can i have Maxima boil this down in to something meanigful so I can check my results.
Best,
Ben
Try the 'trigsimp' function or maybe map(trigsimp, your_expression). Not sure it will help, but it's worth a try. Also look at 'demoivre'.
I don't know what your vector eigenfunctionsort or your "big long nasty equation" looks like, but I often get complex eigenvalues and eigenvectors from Maxima even when I know they should be simple and real.
For example,
(%i1) A : matrix([1, 4, 1], [4, 1, 9], [1, 9, 1]);
(%i2) eigenvalues(A);
makes a mess. It can be simplified by applying rectform to transform the output to Cartesian form, followed by trigreduce to reduce the imaginary part of the result. Finally you can convert the
result into floating point:
(%i3) rectform(%)$
(%i4) trigreduce(%)$
(%i5) float(%);

Resources