I have a spreadsheet with criteria, a start and end date, and a value. The goal is to find the lowest value for each unique criteria and start date without overlapping dates (exclusive of end date). I made a pivot table to make it easier for myself but I know there is probably a way to highlight all valid rows that meet the above requirements with some formula or conditional formatting.
I have attached a google drive link where the spreadsheet can be found here and I have some images of the sheet as well. I know that it might be possible with conditional formatting but I just don't know how to combine everything I want it to do in a single formula.
Example below:
Row 2 is a valid entry because it has the lowest value for Item 1 starting on 03-15-2021, same with row 9.
Row 5 is valid because the start date does not fall within the date range of row 2 (exclusive of end date)
Row 7 is not valid because the start date is between the start and end date of row 6
You may add a bounded script to your project. Then you can call it either with a picture/drawing that has the function assigned (button-like), or adding a menu to Google Sheets.
From what you said in the question and the comments, this seems to do what you are trying. Notice that this requires the V8 runtime (which should be the default).
function validate() {
// Get the correct sheet
const spreadsheet = SpreadsheetApp.getActiveSpreadsheet()
const sheet = spreadsheet.getSheetByName('Sheet1')
// Get the data
const length = sheet.getLastRow() - 1
const range = sheet.getRange(2, 1, length, 4)
const rows = range.getValues()
const data = Array.from(rows.entries(), ([index, [item, start, end, value]]) => {
/*
* Row Index
* 1 Criteria 1
* 2 Item 1 0
* 3 Item 1 1
* 4 Item 1 2
*
* row = index + 2
*/
return {
row: index + 2,
criteria: item,
start: start.getTime(),
end: end.getTime(),
value: value
}
})
// Sort the data by criteria (asc), start date (asc), value (asc) and end date (asc)
data.sort((a, b) => {
let order = a.criteria.localeCompare(b.criteria)
if (order !== 0) return order
order = a.start - b.start
if (order !== 0) return order
order = a.value - b.value
if (order !== 0) return order
order = a.end - b.end
return order
})
// Iterate elements and extract the valid ones
// Notice that because we sorted them, the first one of each criteria will always be valid
const valid = []
let currentCriteria
let currentValid = []
for (let row of data) {
if (row.criteria !== currentCriteria) {
// First of the criteria
valid.push(...currentValid) // Move the valids from the old criteria to the valid list
currentValid = [row] // The new list of valid rows is only the current one (for now)
currentCriteria = row.criteria // Set the criteria
} else {
const startDateCollision = currentValid.some(valid => {
row.start >= valid.start && row.start < valid.end
})
if (!startDateCollision) {
currentValid.push(row)
}
}
}
valid.push(...currentValid)
// Remove any old marks
sheet.getRange(2, 5, length).setValue('')
// Mark the valid rows
for (let row of valid) {
sheet.getRange(row.row, 5).setValue('Valid')
}
}
Algorithm rundown
We get the sheet that we have the data in. In this case we do it by name (remember to change it if it's not the default Sheet1)
We read the data and transform it in a more an array of objects, which for this case makes it easier to manage
We sort the data. This is similar to the transpose you made but in the code. It also forces a priority order and groups it by criteria
Iterate the rows, keeping only the valid:
We keep a list of all the valid ones (valid) and one for the current criteria only (currentValid) because we only have to check data collisions with the ones in the same criteria.
The first iteration will always enter the if block (because currentCriteria is undefined).
When changing criteria, we dump all the rows in currentValid into valid. We do the same after the loop with the last criteria
When changing criteria, the CurrentValid is an array with the current row as an element because the first row will always be valid (because of sorting)
For the other rows, we check if the starting date is between the starting and ending date of any of the valid rows for that criteria. If it's not, add it to this criteria's valid rows
We remove all the current "Valid" in the validity row and fill it out with the valids
The cornerstone of the algorithm is actually sorting the data. It allows us to not have to search for the best row, as it's always the next one. It also ensures things like that the first row of a criteria is always valid.
Learning resources
Javascript tutorial (W3Schools)
Google App Scripts
Overview of Google Apps Script
Extending Google Sheets
Custom Menus in Google Workspace
Code references
Class SpreadsheetApp
Class Sheet
Sheet.getRange (notice the 3 overloads)
let ... of (MDN)
Spread syntax (...) (MDN)
Arrow function expressions (MDN)
Array.from() (MDN)
Array.prototype.push() (MDN)
Array.prototype.sort() (MDN)
Date.prototype.getTime() (MDN)
String.prototype.localeCompare() (MDN)
I'm trying to make a formula that can recognize in Column A the name Brooke B for instance here, from there I'd like to SUM the values listed in Column I Cash Discounts for that specific user.
(Yes this user has no Cash Discounts, thus column I states "Non-Cash Payment").
There's about 80 users total here, so I'd prefer to automate the name recognition in Column A.
Sheet: https://docs.google.com/spreadsheets/d/1xzzHT7VjG24UJ4ZXaiZWsfzroTpn7jCJLexuTOf6SQs/edit?usp=sharing
Desired Results listed in Cash Discounts sheet, listed per user in column C.
You are trying to calculate the total amount of the Cash Discount per person given to people in a list. You have data that has been exported from a POS system to which that you have added a formula to calculate the amout of the discount on a line by line basis. You have speculated whether the discount totals could be calculated using SUMIFS formulae.
In my view, the layout of the spreadsheet and the format of the POS report do not lend themselves to isolating discrete data elements though Google sheets functions (though, no doubt, someone with greater skills than I will disprove this theory). Column A, containing names, also includes sub-groupings (and their sub-totals) as well as transaction dates. There are 83 unique persons and over 31,900 transaction lines.
This answer is a script-based solution which updates a sheet with the names and values of the discount totals. The elapsed execution time is #11 seconds.
function so5882893202() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
// get the Discounts sheet
var discsheetname = "Discounts";
var disc = ss.getSheetByName(discsheetname);
//get the Discounts data
var discStartrow = 3;
var discLR = disc.getLastRow();
var discRange = disc.getRange(discStartrow, 1, discLR-discStartrow+1, 9);
var discValues = discRange.getValues();
// isolate Column A
var discnameCol = discValues.map(function(e){return e[0];});//[[e],[e],[e]]=>[e,e,e]
//Logger.log(discnameCol); // DEBUG
// isolate Column I
var discDiscounts = discValues.map(function(e){return e[8];});//[[e],[e],[e]]=>[e,e,e]
//Logger.log(discDiscounts); // DEBUG
// create an array to build a names list
var names =[]
// get the number of rows on the Discounts sheet
var discNumrows = discLR-discStartrow+1;
// Logger.log("DEBUG: number of rows = "+discNumrows);
// identify search terms
var searchPercent = "%";
var searchTotal = "Total";
// loop through Column A
for (var i=0; i<discNumrows; i++){
//Logger.log("DEBUG: i="+i+", content = "+discnameCol[i]);
// test if value is a date
if (Object.prototype.toString.call(discnameCol[i]) != "[object Date]") {
//Logger.log("it isn't a date")
// test whether the value contains a % sign
if ( discnameCol[i].indexOf(searchPercent) === -1){
//Logger.log("it doesn't have a % character in the content");
// test whether the value contains the word Total
if ( discnameCol[i].indexOf(searchTotal) === -1){
//Logger.log("it doesn't have the word total in the content");
// test whether the value is a blank
if (discnameCol[i] != ""){
//Logger.log("it isn't empty");
// this is a name; add it to the list
names.push(discnameCol[i])
}// end test for empty
}// end test for Total
} // end for percentage
} // end test for date
}// end for
//Logger.log(names);
// get the number of names
var numnames = names.length;
//Logger.log("DEBUG: number of names = "+numnames)
// create an array for the discount details
var discounts=[];
// loop through the names
for (var i=0;i<numnames;i++){
// Logger.log("DEBUG: name = "+names[i]);
// get the first row and last rows for this name
var startrow = discnameCol.indexOf(names[i]);
var endrow = discnameCol.lastIndexOf(names[i]+" Total:");
var x = 0;
var value = 0;
// Logger.log("name = "+names[i]+", start row ="+ startrow+", end row = "+endrow);
// loop through the Cash Discounts Column (Column I) for this name
// from the start row to the end row
for (var r = startrow; r<endrow;r++){
// get the vaue of the cell
value = discDiscounts[r];
// test that it is a value
if (!isNaN(value)){
// increment x by the value
x = +x+value;
// Logger.log("DEBUG: r = "+r+", value = "+value+", x = "+x);
}
}
// push the name and the total discount onto the array
discounts.push([names[i],x]);
}
//Logger.log(discounts)
// get the reporting sheet
var reportsheet = "Sheet10";
var report = ss.getSheetByName(reportsheet);
// define the range (allow row 1 for headers)
var reportRange = report.getRange(2,1,numnames,2);
// clear any existing content
reportRange.clearContent();
//update the values
reportRange.setValues(discounts);
}
Report Sheet - extract
Not everyone wants a script solution to their problem. This answer seeks to supply a repeatable solution using common garden-variety formula/functions.
As noted elsewhere, the layout of the spreadsheet does not lend itself to a quick/simple solution, but it IS possible to break down the data to compile a non-script answer. Though it may "seem" as though the following formula are less than "simple, when taken one-at-a-time they are logical, very easy to create, and very easy to verify successful outcomes.
Note: It is important to know at the outset that the first row of data = row#3, and the last row of data = row#31916.
Step#1 - get Text values from ColumnA
Enter this formula in Cell J3, and copy to row 31916
=if(isdate(A3),"",A3):
evaluates Column A, if the content is a date, returns blank, otherwise, returns the context
Taking Customer "AJ" as an example, the content at this point includes:
AJ
10% BuildingDiscount
10% BuildingDiscount Total:
Northwestern 10%
Northwestern 10% Total:
AJ Total:
Step#2 - ignore the values that contain "10%" (this removes both headings and sub-subtotals
Enter this formula in Cell K3 and copy to row 31916
=iferror(if(search("10%",J3)>0,"",J3),J3): searches for "10%" in Column J. Returns all values except those that containing "10%".
Taking Customer "AJ" as an example, the content at this point includes:
AJ
AJ Total:
**Step#3 - ignore the values that contain the word "Total"
Enter this formula in Cell L3 and copy to row 31916.
=iferror(if(search("total",K3)>0,"",K3),K3)
Taking Customer "AJ" as an example, the content at this point includes:
AJ
Results after Step#3
You might wonder, "couldn't this be done in a single formula?" and/or "an array formula would be more efficent". Both those thoughts are true, but we're looking at simple and easy, and a single formula is NOT simple (as shown below); and given that, an array formula is out-of-the-question unless/until an expert can wave a magic wand over the data.
FWIW - Combining Steps#1, 2 & 3
each of the Steps#1, 2 and 3 build on each other. So it is possible to create a single formula that combines these steps.
enter this formula in Cell J3, and copy dow to row #31916.
=iferror(if(search("total",iferror(if(search("10%",if(isdate(A3),"",A3))>0,"",if(isdate(A3),"",A3)),if(isdate(A3),"",A3)))>0,"",iferror(if(search("10%",if(isdate(A3),"",A3))>0,"",if(isdate(A3),"",A3)),if(isdate(A3),"",A3))),iferror(if(search("10%",if(isdate(A3),"",A3))>0,"",if(isdate(A3),"",A3)),if(isdate(A3),"",A3)))
As the image showed, step#3 concludes with mainly empty cells in Column L; the only populated cell is the first instance of the customer name at the start of their transactions - such as "Alec" in this example. However (props to #Rubén) it is possible to populate the blank transaction Cells in Column L. An arrayformula to find the previous non-empty cell in another column on Webapps explains how.
Step#4 - Create a customer name for each transaction row.
Enter this formula in Cell M3, it will automatically populate the cells to row#31916
=ArrayFormula(vlookup(ROW(3:31916),{IF(LEN(L3:L31916)>0,ROW(3:31916),""),L3:L31916},2))
Step#5 - Get the discount amount for each transaction value
The discount values are already displayed in Column I. They are interspersed with text values, so the formula for tests if this is a total line by testing the value in Column D; only if there is a vale (Product item) does the formula then test of there is a value in column I.
Enter this formula in Cell N3, it will automatically populate the cells to row#31916
=ArrayFormula(if(len(D3:D31914)>0,if(ISNUMBER(I3:I31916),I3:I31916,0),""))
Screenshot after step#5
Reporting by Query
Reporting is done via queries. These can go anywhere, but it is probably more convenient to put it on a separate sheet.
Step#6.1 - query the results to create report showing total by ALL customers
=query(Discounts_analysis!$M$2:$N$31916,"select M, sum(N) where N is not null group by M label M 'Customer', sum(N) 'Total Discount' ",1)
Step#6.2 - query the results to create report showing total by customer where the customer received a discount
=query(Discounts_analysis!$M$2:$N$31916,"select M, sum(N) where N >0 group by M label M 'Customer', sum(N) 'Total Discount' ",1)
Step#6.3 - query the results to create report showing customers with no discount
- `=query(query(Discounts_analysis!$M$2:$N$31916,"select M, sum(N) where N is not null group by M label M 'Customer', sum(N) 'Total Discount' ",1),"select Col1 where Col2=0")`
Queries screenshot
There were a lot of questions with transpose but I have some specifics and can't create the right formula.
So I have near 100k rows in the following format
https://docs.google.com/spreadsheets/d/146-6YHv69DDOnPKheKeRFZIQeZYIgO6UwCMd7X9VeKU/edit?usp=sharing
I need ARRAYFORMULA or something like that to make this 100k rows in the format that you can see on the "Expected Outcome" tab.
=ARRAYFORMULA({UNIQUE(INDIRECT("source!A2:A"&COUNTA(source!A2:A))),
QUERY(SPLIT(TRANSPOSE(SPLIT(QUERY("♦"&INDEX(SPLIT(TRANSPOSE(SPLIT(
TRIM(QUERY(TRANSPOSE(QUERY(TRANSPOSE(
IF(source!B2:J<>"", "♦"&source!A2:A&"♠"&{"♥"&source!B2:B, source!C2:J}, ))
,,999^99)),,999^99)), "♦")), "♠"),,2),,999^99), "♥")), "♦"),
"where Col2 is not null", 0)})
You have a vast number of records but there are 19 data fields to each record spread over 10 columns & 12 rows. You suggested an arrayformula (and much more clever people than I can probably do that), but I suggest a script which takes the data in its current form on one sheet (say the 'source'), and which outputs the data to a new sheet (say, the 'target').
There are a couple of issues to be addressed in "mapping" the data:
how many records are represented on the input sheet - required to enable looping. I used the Javascript Math.floor method to calculate the number of products.
identify the correct row/column combination for each field. The data is in three segments
8 fields in contiguous columns in the first row,
2 fields in 2 rows in the ninth column, and
9 fields in nine contiguous rows in the tenth column.
Total = 12 rows/10 columns. To do this, in order to navigate the rows, I took the counter (i) multiplied by the number of rows, plus 1; the columns are more intuitive.
for efficiency,
get data once only at the beginning of the script;
use arrays to progressively build the output; and
update the output (setValues) once at the end of the script.
function myFunction() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sourcesheetname = "source";
var targetsheetname = "target";
var source = ss.getSheetByName(sourcesheetname);
var target = ss.getSheetByName(targetsheetname);
var sourcerange = source.getDataRange();
var sourcedata = sourcerange.getValues();
var sourceheaders = 1;
var rowsperproduct = 12;
var sourcelr = source.getLastRow();
var integerPart = Math.floor((sourcelr-sourceheaders)/rowsperproduct);
//Logger.log("DEBUG: number of products = "+integerPart);
var rowdata = [];
// look thought the number of products
for (var i = 0; i<integerPart; i++){
// use a temporary array to build the data for the row.
var temp = [];
// row 1, copy first eight columns (0-7)
temp.push(sourcedata[(i*12)+1][0]);//ID
temp.push(sourcedata[(i*12)+1][1]);//GID
temp.push(sourcedata[(i*12)+1][2]);//NAME
temp.push(sourcedata[(i*12)+1][3]);//PRICE
temp.push(sourcedata[(i*12)+1][4]);//BRAND
temp.push(sourcedata[(i*12)+1][5]);//URL
temp.push(sourcedata[(i*12)+1][6]);//country
temp.push(sourcedata[(i*12)+1][7]);//instock
// row 2 & 3 in Column I(8)
temp.push(sourcedata[(i*12)+2][8]);//url1
temp.push(sourcedata[(i*12)+3][8]);//url2
// rows 4 - 12 in Column J(9)
temp.push(sourcedata[(i*12)+4][9]);// tech name
temp.push(sourcedata[(i*12)+5][9]);// size
temp.push(sourcedata[(i*12)+6][9]);// inches
temp.push(sourcedata[(i*12)+7][9]);// mm
temp.push(sourcedata[(i*12)+8][9]);// depth
temp.push(sourcedata[(i*12)+9][9]);// BB
temp.push(sourcedata[(i*12)+10][9]);// ref1
temp.push(sourcedata[(i*12)+11][9]);// ref2
temp.push(sourcedata[(i*12)+12][9]);// ref3
// Update the rowdata for this row
rowdata.push(temp);
}
// update data to target
target.getRange(2,1,integerPart,19).setValues(rowdata);
}
Any idea how to convert form of sheet 1 to sheet 2?
Test Sheet: https://docs.google.com/spreadsheets/d/1MS4VvpS03Rnd-TMWohTlAS0vhfa4JxsX1TBpqorsi7A/edit?usp=sharing
Sheet 1
ID URL Price (from-to)
1 www.tseting.com 2-15
2 www.hello.com 1-10
Sheet 2
[1]
url = www.tseting.com
price = 2-15
[2]
url = www.hello.com
price = 1-10
ANSWER:
=ArrayFormula(TRIM(TRANSPOSE(SPLIT(QUERY(TRANSPOSE(QUERY(TRANSPOSE({"[","url = ","price = ","dontsearchkeywords = "}&FILTER(Sheet1!A2:D,LEN(Sheet1!A2:A))&{"]"&CHAR(9),CHAR(9),CHAR(9),CHAR(9)&" "&CHAR(9)}),,4)),,ROWS(Sheet1!A2:D)),CHAR(9)))))
You may start with smth like this:
=TRANSPOSE(SPLIT(ArrayFormula(CONCATENATE(Sheet1!A2:E7&"# ")),"#"))