I am trying to do some data reduction in my Google Sheets by using the following IMPORTRANGE formula:
=query(importrange("https://docs.google.com/a/ap.averydennison.com/spreadsheets/d/1xz1lXY-w5Ii_aWqVAhHgRCmeoes9ltSUtibE4kzhMHA/edit#gid=2051232966","SF_Flex_Rel!a:l"),
"select * where Col1 = '"&text(B1,"###")&"'",1)
The 'source' sheet has a whole lot of sales data records. What I am trying to do in the new sheet via this formula is only bring in the sales records from the source sheet that match the customer number specified in cell B1.
It seems to work OK if I limit the IMPORTRANGE to only query about 10,000 rows. Once I go over around 20,000 rows the screen will briefly flash up the records, then a small progress bar shows in the top right corner of the sheet and the records disappear. The cell with the formula just shows #ERROR! with no other comments to tell me why.
Is there something wrong with my formula syntax?
Is there a better way to achieve this data reduction?
Is there some undocumented data limitation on IMPORTRANGE function (I am using 'new' Google Sheets)?
try like my example :
=QUERY( // data
IMPORTRANGE(
"Spreadsheet Key", // spreadsheet key
"DATA!A:C" // datarange
),
"SELECT Col1 WHERE Col2=" & "'" & B2 & "'" // query
)
I had the same problem. This answer helped me find a workaround : https://productforums.google.com/forum/#!topic/docs/RxVUFGWQ2Y4
In my example :
1) In the spreadsheet where the data is I have added a few empty columns (E to H) in order to display 4 columns of data in 5 maximum rows.
=Query(Sheet1!A:D,"select * Where A contains 'KEYWORD' limit 5",1)
2) Then in the other spreadsheet:
=ImportRange("https://docs.google.com/spreadsheets/d/ss_key_here/", "'Sheet1'!E1:H5")
Related
so basically I have a searchbox in my sheet that searches and pulls up data. For a reference see this image: https://i.imgur.com/MVTUCSw.png. So basically in cell A4 I put the data that I am looking for, but my formula restricts me to only looking up stuff in 1 row. For example, data starting with the word MELD, but I would like to be able to also look up data based on for example the someone their name.
The formula I use for the searchbox: =QUERY({'Pallets & Locaties'!A2:G;Voorraadverschillen!A2:G}, "SELECT * WHERE Col1 "&Opzoeken!B4&" '"&A4&"'")
The data that I want to be able to look up is stored in 2 sheets: Pallets & Locaties - https://i.imgur.com/qV7h2tz.png and in Voorraadverschillen - https://i.imgur.com/foqLkKa.png.
The searchbox is only able to lookup data in row, but I just want to be able to search for any kind of stored data in any of the sheets.
Here is my sheet for reference: https://docs.google.com/spreadsheets/d/10wmnxV16JUiD_b_54abkiLPwTqLITWRcUw3gsqvXoBE/edit?usp=sharing
I'd recommend you add more rows for the lookup criteria and add a column for what column it would search for.
Sheet modification:
Formula:
=QUERY({'Pallets & Locaties'!A2:G;Voorraadverschillen!A2:G}, "SELECT * WHERE "&TEXTJOIN(" AND ", TRUE, ARRAYFORMULA(IF(ISBLANK(A4:A10), "", A4:A10&" "&B4:B10&" '"&C4:C10&"'"))))
Test Sheet
Note:
The above formula will allow you to search on other columns with their own words and criteria to search.
Only rows with Kolom values will be included in the criteria. If you only need Col1 criteria, make sure to leave other rows blank.
This does use an AND search, meaning all of the criteria should be true and match the row. Feel free to use OR in the TEXTJOIN function if you only want to search all rows matching any of the criteria.
This will only search on sheets Pallets & Locaties and Voorraadverschillen. Add the necessary sheets if you need them.
EDIT:
Cleaned up the formula to not be so repetitive.
=IF(A4<>"",(QUERY({'Pallets & Locaties'!A2:G;Voorraadverschillen!A2:G},"Select * WHERE "&textjoin(" OR ", true, arrayformula("Col"&ROW(1:7)&" "&B4&" '"&A4&"'")))),(QUERY({'Pallets & Locaties'!A2:G;Voorraadverschillen!A2:G},"Select * WHERE Col1 IS NOT NULL")))
This searches every column for the data, as long as data is not identical in two columns you won't have issues. An example would be the search criteria "MELD" being in both Column A and B. If that were the case, only the results from the first matching column would populate.
community!
I have a table of 3 columns in Google Sheets: Date, Name, Amount.
I want to process data from it, so for every date in the list, there will be a full list of participants and amounts.
If there is no amount for specific date and person, it will be "0", or just leave it blank.
In addition, would like to count the percent of the sum of the amount for each person till the date in the corresponding row.
And it should be dynamically updated, so if a new row added with new name or date or both, so new participant will be added to every date in the output table and new full list of participants for the new date created.
For better understanding, here is the example of the data and output tables
What I've tried?
In MS access query it could be done not so hard.
Here I tried to create 2 new lists of unique dates and participants and connect them through use of CONCATENATE, ARRAYFORMULA, then SPLIT and some QUERY. All this through use of helping column of text, which should be edited manually for every new row...
The code in helping column:
=concatenate(arrayformula(if(isblank($F$3:$F),,";"&$E3&","&$F$3:$F)))
Then split code:
=query(arrayformula(split(transpose(split(TEXTJOIN(";",true,$G$3:$G),";",true,true)),",",true,true)),"Select Col1,Col2 where Col2 is not null order by Col1 ASC",0)
But here I stuck...
Want to pull the amounts for every corresponding date and participant, but...
FILTER reaches first, not existent in the original table combination, and doesn't proceed.
QUERY doesn't fetch the data fully automatically for all the list in the output table.
VLOOKUP gives only one row or complete mess, when use it recursively, or error...
So, how can I do this (if it's possible at all) in google sheets, and so all the output will be fully automatically updated?
Thank you very much!
Update - based on comments
I could not figure out a way to make this through one formula. If this works, you can have a hidden column (say H here) that totals the amount for the person till that date using the formula =QUERY(FILTER($E$3:$G, $F$3:$F = F3, $E$3:$E <= E3), "select sum(Col3) label sum(Col3) ''", 0) in H3.
Then in I3 you can find the % of total till date using the formula =QUERY(FILTER($E$3:$G, $F$3:$F = F3, $E$3:$E <= E3), "select sum(Col3) label sum(Col3) ''", 0)/SUMIF($E$3:$E, E3, $H$3:$H).
Both of the formulae in H3 and I3 need to be dragged down unlike the one in E3. Perhaps someone will be able to offer a better solution.
Previous answer
You can try something like this:
In E3 you can have the formula =ArrayFormula({(FLATTEN(SPLIT(REPT(FILTER(UNIQUE(A3:A)&"✦", UNIQUE(A3:A)>0), COUNTA(UNIQUE(B3:B))), "✦"))), (TRANSPOSE(SPLIT(REPT(JOIN("", FILTER(UNIQUE(B3:B)&"◼︎", UNIQUE(B3:B)<>"")), (COUNTA(UNIQUE(A3:A)))), "◼︎"))), (IFNA(VLOOKUP({(FLATTEN(SPLIT(REPT(FILTER(UNIQUE(A3:A)&"✦", UNIQUE(A3:A)>0), COUNTA(UNIQUE(B3:B))), "✦")))&(TRANSPOSE(SPLIT(REPT(JOIN("", FILTER(UNIQUE(B3:B)&"◼︎", UNIQUE(B3:B)<>"")), (COUNTA(UNIQUE(A3:A)))), "◼︎")))}, {A3:A&B3:B, C3:C}, 2, 0), 0))}). It is a little long but fills Columns E through G dynamically.
Then in H3 you can have the formula =ArrayFormula(IFERROR(((IFNA(VLOOKUP({(FLATTEN(SPLIT(REPT(FILTER(UNIQUE(A3:A)&"✦", UNIQUE(A3:A)>0), COUNTA(UNIQUE(B3:B))), "✦")))&(TRANSPOSE(SPLIT(REPT(JOIN("", FILTER(UNIQUE(B3:B)&"◼︎", UNIQUE(B3:B)<>"")), 3), "◼︎")))}, {A3:A&B3:B, C3:C}, 2, 0), 0))/(SUMIF((FLATTEN(SPLIT(REPT(FILTER(UNIQUE(A3:A)&"✦", UNIQUE(A3:A)>0), COUNTA(UNIQUE(B3:B))), "✦"))), (FLATTEN(SPLIT(REPT(FILTER(UNIQUE(A3:A)&"✦", UNIQUE(A3:A)>0), COUNTA(UNIQUE(B3:B))), "✦"))), G3:G))), "")). This one is dynamic as well.
I tried with your data and added a row on my own and it works.
We have hit the dreaded 5 million rows limit which is so small for any semi-serious data.
We have an important ArrayFormula piece in one of our worksheets (tab) currently that summarizes the data from another worksheet in the same file where time series data is kept with dates. This is our current function:
=ArrayFormula(SUMIFS(DataSheet!$B:$B,
MONTH(DataSheet!$A:$A), 1,
YEAR(DataSheet!$A:$A), 2020)
)
Explanation: This basically summed all of column B in the DataSheet tab for the month of Jan 2020 based on date found in column A of that sheet.
However, this worksheet of data that is now running close to that row limit. We can move it to another Google Sheets file, and refer to the same data via IMPORTRANGE.
The question then is how to refer to that data instead of the DataSheet!$A:$A in the above old formula? Will this reference be replaced by the entire IMPORTRANGE function?
Old:
=ArrayFormula(SUMIFS(DataSheet!$B:$B,
MONTH(DataSheet!$A:$A), 1,
YEAR(DataSheet!$A:$A), 2020)
)
New:
=ArrayFormula(SUMIFS(IMPORTRANGE(filename, rows)!$B:$B,
MONTH(IMPORTRANGE(filename, rows)!$A:$A, 1,
YEAR(IMPORTRANGE(filename, rows)!$A:$A, 2020)
)
This does not work of course, because we cannot have the exclamation ! followed by the column in an importrange. Any other thoughts?
Try this in cell A1 on a fresh, brand new tab somewhere:
=ARRAYFORMULA(QUERY(1*TEXT(IMPORTRANGE("[spreadsheet key]","Sheet1!A:B"),{"mmmyyyy","0.00"}),"select Col1,SUM(Col2) where Col2<>0 group by Col1 order by Col1")
The "spreadsheet key" is the combination of letters and numbers after the "/d/" and before the "/edit..." in the URL of your source sheet.
Obviously, you'd also replace "Sheet1!A:B" with whatever the real tab/column reference is.
Then, select all of Column A and from the Menu choose Format>Number>More Formats>Custom Number Formatting, Then this in the dialog box:
mmmm yyyy
You want to IMPORTRANGE from two different sheets in a different spreadsheet.
While the following formula will import data from both sheets, it will also import the blank rows, so you might have to scroll down hundreds of rows in order to see the data from the second sheet (and this might give your the wrong impression that the second sheet is not getting imported):
{
IMPORTRANGE("SPREADSHEET_ID","CurrentMonth!$A:$J");
IMPORTRANGE("SPREADSHEET_ID","All2020!$A:$J")
}
You can use QUERY in order to filter out blank rows:
=QUERY(
{
IMPORTRANGE("SPREADSHEET_ID","CurrentMonth!$A:$J");
IMPORTRANGE("SPREADSHEET_ID","All2020!$A:$J")
},
"SELECT * WHERE Col1 IS NOT NULL ORDER BY Col1 DESC"
)
Note:
I thought you'd like to sort the data according to the date in column A, please remove ORDER BY Col1 DESC if that's not the case.
I'am working on some document which get data from Non Google form, and the output is 36 column but there will be always filled up 8 max 9 columns with empty columns in between, with our Form we CANT change way how data are outputed. Empty column are here cuz Client fill up specific type of order so data are put in specific column.
i need query for pulling data from the main sheet and output just specific 5 specific column ( those will be always in same place ), and 2 columns that can be anywhere in between.
writing down every single column is not perfect as i need to pull data row by row not all at once ( i will be adding data in between final query output) and with that it would drasticly slow down file with few hundreds inputs added per week.
I didnt found working way to combine filter and query together to do this job ( removing empty columns [ <>'' and is not null].
Could someone help me with this ?
providing link for file : https://docs.google.com/spreadsheets/d/1SDR939yUSq9trLcxBid9AQeZUn-lNNiRr7O7pDiu888/edit?usp=sharing
In cell M1 of your sheet 'Desired output' try entering this little monster
=Arrayformula(query({'Raw data'!A:Z\
{"Configurable list"; 'Raw data'!D2:D&'Raw data'!F2:F&'Raw data'!H2:H&'Raw data'!J2:J&'Raw data'!L2:L&'Raw data'!P2:P&'Raw data'!R2:R&'Raw data'!T2:T&'Raw data'!V2:V}\
{"Date"; ('Raw data'!E2:E&'Raw data'!G2:G&'Raw data'!I2:I&'Raw data'!M2:M&'Raw data'!O2:O&'Raw data'!Q2:Q&'Raw data'!S2:S&'Raw data'!U2:U&'Raw data'!W2:W)+0}};
"Select Col1, Col2, Col3, Col26, Col28, Col24, Col27 Where Col1 is not null format (Col28) 'dd.mm.yyyy'"))
See if that works for you?
Your answer is really in the heading. You can use =filter([Range], [Range]<>"") for each row to get to this in the desired output. The filter function is perfect for this application.
UPDATE:
Looked at your sheet to get a better view of the problem. You'll need some wildcard workaround as shown here as a second criteria to identify the Date & Configurable cells: =filter([Range], [Range]<>"", search("Date", [Top row]))
Testing Sheet:
Wondering if there is a witty way to add a Total to the last row +1 of
a Query result.
See Sheet 'Lookup' for a static example of what I am asking for.
I don't know if there is a way to have a hidden column that calculates
transposed only under the last row of a query, or if there is a smart
way to work Query for this answer.
All great answers. Each on very useful in its use case.
Макс Махров gets the answer with using a query statement.
Now I was not keen on having an extra sheet to hold the totals so I added a row at the top which I can simply hide and used this formula:
query({Orders!A:E;A1:E1},"select Col1, Col3, Col4 where Col2 = '"&C3&"' order by Col4",1)
Only problem I have is trying to figure out how to add TEXT to the bottom row, it seems to only want numerical input.
How do I fix this? What am I glitching?
Thanks !
Mars
The trick is to make second query and count totals for selected product.
Plan of actions:
add new sheet with query on it, something like this: =QUERY(Orders!A:E,"select B, 0, sum(D) where B like '"&Lookup!C2&"' Group by B",0)
Prepare arrayformula which combines data in Lookup sheet: = ArrayFormula({Importrange(1),Importrange(2)}) Note that number of columns must retain the same.
Edit query so it takes Col1, Col2, Col3... instead of A, B, C...
Make word 'total' visible instead of zero. Set number format: 0;0;total Set it for range B9:B on Lookup sheet
Make Conditional Formatting with formula =and($B4 =0,isnumber($B4)) for range A4:C on Lookup sheet.
That's seems have to complete the task.
Hope it Helps!
Your Example
Working example.
Here is one way:
Put TOTAL way down in row 1000
Select the range A3:C999. Select data > filter to create filters
Select C3, set the filter to hide all blanks
A second way is to limit the query result to show only the top 8 results:
Change your query to =query(Orders!A:E, "select A, C, D where B = '"&C2&"' order by D desc limit 8",1) It will reverse-order column D (largest first), and set row limit to 8.
Change the formula of your TOTAL to =sumif(Orders!B:B,C2,Orders!D:D)
Try this formula in the column adjacent to your query:
=ArrayFormula({$C$4:offset($C$4,count($C$4:$C),0,1,1);sum($C$4:offset($C$4,count($C$4:$C),0,1,1))})
It duplicates your column of values (I haven't figured out a way around that yet) and then adds a total to the bottom of that column, and changes dynamically with the range from your query.
Here's a working version.
Interesting challenge! It got the old grey matter turning... ;)
Thanks,
Ben