KDB combine/join different table - join

How can i join two different table like
all_order_ask:([]ask:();ask_qty:();exchange_name_ask:())
all_order_bid:([]bid:();bid_qty:();exchange_name_bid:())
and get =====>
final_order:ask:();ask_qty:();exchange_name_ask:();bid:();bid_qty:();exchange_name_bid:()
the two table have the same number of rows

you can use uj:
https://code.kx.com/q/ref/uj/
all_order_ask uj all_order_bid
ask ask_qty exchange_name_ask bid bid_qty exchange_name_bid
-----------------------------------------------------------
q)
If your tables look similar like this:
all_order_ask
ask ask_qty exchange_name_ask
----------------------------------
7.051033 8 bjd
1.497004 3 lln
2.400771 0 edg
1.039355 7 lij
2.353326 6 hon
6.423479 4 ncp
5.778177 6 gee
2.193148 5 ijf
1.66486 4 bbf
4.784272 2 lmi
all_order_bid
bid bid_qty exchange_name_bid
----------------------------------
15.70605 2 pjbke
10.93533 17 epjak
7.040985 11 ekaaj
14.19316 19 mpnan
9.248942 17 nogel
1.615466 18 holpj
1.073589 16 kkfpn
19.85822 13 pegin
14.45499 8 jcgnm
16.47223 0 dlhep
You can try this:
all_order_ask^all_order_bid
ask ask_qty exchange_name_ask bid bid_qty exchange_name_bid
---------------------------------------------------------------------
7.051033 8 bjd 15.70605 2 pjbke
1.497004 3 lln 10.93533 17 epjak
2.400771 0 edg 7.040985 11 ekaaj
1.039355 7 lij 14.19316 19 mpnan
2.353326 6 hon 9.248942 17 nogel
6.423479 4 ncp 1.615466 18 holpj
5.778177 6 gee 1.073589 16 kkfpn
2.193148 5 ijf 19.85822 13 pegin
1.66486 4 bbf 14.45499 8 jcgnm
4.784272 2 lmi 16.47223 0 dlhep

Since your two tables have the same number of rows, you should also be able to join your two tables horizontally using ,' as follows:
q)final_order_ask:all_order_ask,'all_order_bid
q)final_order_ask
ask ask_qty exchange_name_ask bid bid_qty exchange_name_bid
-----------------------------------------------------------

Related

How To Skip Down by 1 Row/Cell The Formula Output and Remove The Last Sequential Output Before 1's Google Sheets?

I've got these 3 groups of data in range F2:G22 as below
(3 groups as minimal example, in reality many thousands of groups, and recurrent similar datasets expected in the future):
I need to number each group's rows sequentially, starting over at 1 at each new group.
The expected result would be like in range E1:E22.
I tried the following formula n cell C2 , then in cell D3:
=INDEX(IF(A2:A22="",COUNTIFS(B2:B22&A2:A22, B2:B22&A2:A22, ROW(B2:B22), "<="&ROW(B2:B22)),1))
In C2:
In D3:
That fixed partially the sequence issue, but there's still 2 issues I can't find remedy for.
1st remaining issue:
I'd prefer not having to manually do the C2 to D3 step each time I get new similar data (but would accomodate if there's no simple solution to this issue).
Is there a simple way to modify the formula to make it output the correct sequencing from C2 ?
2nd remaining issue:
At rows 7, 14 and 23 there still remain unecessary ending numbering for these intermediary rows in D7 , D14 , and D23:
I could only think of an extra manual step of filtering out the non-blank rows in Column A to fix this 2nd issue (i.e. Highlighting Column A > Data tab > Create Filter > Untick all > Tick Blanks > Copy All > Paste In new Sheet).
But would there be a way to do it in the same formula? I'm not seeing the way to add the proper filter or using another method in the formula.
Any help is greatly appreciated.
EDIT (Sorry for Forgotten Sample):
Formula Input A
Formula Input B
Formula Output 1
Formula Output 2
EXPECTED RESULT
rockinfreakshow
ztiaa
DATA
DATA BY GROUP
7
1
1
7
7
2
1
1
1
2
Element-1
Group-1
7
3
2
2
2
3
Element-2
Group-1
7
4
3
3
3
4
Element-3
Group-1
7
5
4
4
4
5
Element-4
Group-1
8
1
5
6
8
8
2
1
1
1
7
Element-1
Group-2
8
3
2
2
2
8
Element-2
Group-2
8
4
3
3
3
9
Element-3
Group-2
8
5
4
4
4
10
Element-4
Group-2
8
6
5
5
5
11
Element-5
Group-2
8
7
6
6
6
12
Element-6
Group-2
9
1
7
13
9
9
2
1
1
1
14
Element-1
Group-3
9
3
2
2
2
15
Element-2
Group-3
9
4
3
3
3
16
Element-3
Group-3
9
5
4
4
4
17
Element-4
Group-3
9
6
5
5
5
18
Element-5
Group-3
9
7
6
6
6
19
Element-6
Group-3
9
8
7
7
7
20
Element-7
Group-3
9
9
8
8
8
21
Element-8
Group-3
9
Can you try:
=INDEX(LAMBDA(y,z,
IF(LEN(z),COUNTIFS(y,y,ROW(z),"<="&ROW(z)),))
(LOOKUP(ROW(G2:G),FILTER(ROW(G2:G),BYROW(G2:G,LAMBDA(z,IF(z<>OFFSET(z,-1,0),row(z),0))))),G2:G))
You can simply use SCAN.
=SCAN(,G2:G,LAMBDA(a,c,IF(c="",,a+1)))
Sample sheet

Debugging APL code: how to use `#`(index) and `⊢` (right tack) together?

I am attempting to read Aaron Hsu's thesis on A data parallel compiler hosted on the GPU, where I have landed at some APL code I am unable to fix. I've attached both a screenshot of the offending page (page number 74 as per the thesis numbering on the bottom):
The transcribed code is as follows:
d ← 0 1 2 3 1 2 3 3 4 1 2 3 4 5 6 5 5 6 3 4 5 6 5 5 6 3 4
This makes sense: create an array named d.
⍳≢d
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
This too makes sense. Count the number of elements in d and create a sequence of
that length.
⍉↑d,¨⍳≢d
0 1 2 3 1 2 3 3 4 1 2 3 4 5 6 5 5 6 3 4 5 6 5 5 6 3 4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
This is slightly challenging, but let me break it down:
zip the sequence ⍳≢d = 1..27 with the d array using the ,¨ idiom, which zips the two arrays using a catenation.
Then, split into two rows using ↑ and transpose to get columns using ⍉
Now the biggie:
(⍳≢d)#(d,¨⍳≢d)⊢7 27⍴' '
INDEX ERROR
(⍳≢d)#(d,¨⍳≢d)⊢7 27⍴' '
Attempting to break it down:
⍳≢d counts number of elements in d
(d,¨⍳≢d) creates an array of pairs (d, index of d)
7 27⍴' ' creates a 7 x 27 grid: presumably 7 because that's the max value of d + 1, for indexing reasons.
Now I'm flummoxed about how the use of ⊢ works: as far as I know, it just ignores everything to the left! So I'm missing something about the parsing of this expression.
I presume it is parsed as:
(⍳≢d)#((d,¨⍳≢d)⊢(7 27⍴' '))
which according to me should be evaluated as:
(⍳≢d)#((d,¨⍳≢d)⊢(7 27⍴' '))
= (⍳≢d)#((7 27⍴' ')) [using a⊢b = b]
= not the right thing
As I was writing this down, I managed to fix the bug by sheer luck: if we increment d to be d + 1 so we are 1-indexed, the bug no longer manifests:
d ← d + 1
d
1 2 3 4 2 3 4 4 5 2 3 4 5 6 7 6 6 7 4 5 6 7 6 6 7 4 5
then:
(⍳≢d)#(d,¨⍳≢d)⊢7 27⍴' '
1
2 5 10
3 6 11
4 7 8 12 19 26
9 13 20 27
14 16 17 21 23 24
15 18 22 25
However, I still don't understand how this works! I presume the context will be useful
for others attempting to leave the thesis, so I'm going to leave the rest of it up.
Please explain what (⍳≢d)#(d,¨⍳≢d)⊢7 27⍴' ' does!
I've attached the raw screenshot to make sure I didn't miss something:
I'm happy to see that you found the the off-by-one error. It stems from Aaron Hsu working with index origin 0. If you set ⎕IO←0 then his code will work.
Some dyadic operators can take an array operand, giving the sequence OPERATOR operand argument, e.g. in -#(1 2 3)(4 5 6 7). This poses a problem because both the operand and the argument are arrays, and juxtaposition of arrays forms a new array with those arrays as elements by a process known as stranding. Compare:
(1 2 3)(4 5 6 7)
┌─────┬───┐
│1 2 3│4 5│
└─────┴───┘
However, in the case of the operator with its array operand, we want to "break" this strand so the left part can act as operand while the right part acts as argument. One way to break the stranding up is by applying a function to the argument, giving the sequence OPERATOR operand Function argument. Now, we don't actually need any transformation of the argument, so an identity function will do: -#(1 2 3)⊢(4 5 6 7).
As for what (⍳≢d)#(d,¨⍳≢d)⊢7 27⍴' ' actually does:
7 27⍴' ' creates a blank matrix.
(⍳≢d) are indices to insert into specified slots in the matrix.
#(d,¨⍳≢d) indicates at which locations in the matrix the above should replace the existing values
⊢ serves solely to separate (d,¨⍳≢d) from 7 27⍴' '. The code could also have been written as ((⍳≢d)#(d,¨⍳≢d))7 27⍴' ' with parentheses serving to "bind" the operand to the operator.

Return records in an unusual offset order

Currently I have a scope that pulls back my records in the following standard order:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
and it is converted in to a html block that's laid out as so:
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
I want a html block that's laid out like so:
1 6 11 16
2 7 12
3 8 13
4 9 14
5 10 15
So I think I need to pull the records back in the following order - (records offset by the number of rows)
1 6 11 16 2 7 12 3 8 13 4 9 14 5 10 15
Any idea what is the neatest way to do this in Rails / ActiveRecord?
You must know how many columns you want to eventually render. I think this must work for you:
columns = 5
MyModel.order(:id).in_groups_of(columns).transpose
And you get an array of lines, each one with an array of records.
If you want an unique array you can add .flatten at the end.
Can't do that with AR ordering! You will have to do that in your controller or the view.

how to join two pandas dataframe on specific column

I have 1st pandas dataframe which looks like this
order_id buyer_id caterer_id item_id qty_purchased
387 139 1 7 3
388 140 1 6 3
389 140 1 7 3
390 36 1 9 3
391 79 1 8 3
391 79 1 12 3
391 79 1 7 3
392 72 1 9 3
392 72 1 9 3
393 65 1 9 3
394 65 1 10 3
395 141 1 11 3
396 132 1 12 3
396 132 1 15 3
397 31 1 13 3
404 64 1 14 3
405 146 1 15 3
And the 2nd dataframe looks like this
item_id meal_type
6 Veg
7 Veg
8 Veg
9 NonVeg
10 Veg
11 Veg
12 Veg
13 NonVeg
14 Veg
15 NonVeg
16 NonVeg
17 Veg
18 Veg
19 NonVeg
20 Veg
21 Veg
I want to join this two data frames on item_id column. So that the final data frame should contain item_type where it has a match with item_id.
I am doing following in python
pd.merge(segments_data,meal_type,how='left',on='item_id')
But it gives me all nan values
You have to check types by dtypes of both columns (names) to join on.
If there are different, you can cast them, because you need same dtypes. Sometimes numeric columns are string columns, but looks like numbers.
If there are both same string types, maybe help cast both of them to int. Problem can be some whitespaces:
segments_data['item_id'] = segments_data['item_id'].astype(int)
meal_type['item_id'] = meal_type['item_id'].astype(int)
pd.merge(segments_data,meal_type,how='left',on='item_id')

QReport making a grid with 24 cells in a 3x8 layout

I'm using QuickReport 5 in Delphi 2007.
I want to make a grid layout on A4 paper like so:
+-----+-----+-----+
| 1 | 2 | 3 |
+-----+-----+-----+
| 4 | 5 | 6 |
...
| 22 | 23 | 24 |
+-----+-----+-----+
I only have detail bands, I need this to print address stickers in a Delphi application, with a different address on each sticker.
How do I do this in QuickReport?
You can set columns in your Quickreport.
Double click on the Quickreport and set the columns to 3.
This will set the columns like so:
1 9 17
2 10 18
3 11 19
4 12 20
5 13 21
6 14 22
7 15 23
8 16 24
If you want the columns to run like in your question you need to do some creative sorting in the query that feeds the report.
Order by rank div 24, if(rank mod 8 = 0,8,rank mod 8), rank
This will make things come out with 1,2,3 in the first row and 456 in the seconds etc.

Resources