Google Spreadsheet: Script to pick the right values from data which is html-imported and never in the right order - google-sheets

I have the following problem I can't solve:
There is data of a measuring station for water quality available online (https://hamburg.de/clp/hu/fischerhof/clp1/). It updates every day, but the data is nowhere stored or put into a chart. So my play was to have a script which imports the data automatically. Functions in my spreadsheet would do the rest and put e.g. the water temperature in a chart. So far so good, I found all the scripts and work arounds here.
Then I discovered a problem I can't solve: When the data is uploaded on the website, it is not in the same order. Sometimes water temperature is in the first row, sometimes in the last, sometimes anywhere between. This messes all my data up, as I use a function which filters e.g. every 12-th row, to filter all the water temperature values out and make a chart out of it. When that position changed because of the import, I get wrong values.
Here is my setup so far:
I import the data into Sheet 1 with a script
This script also gathers the data in Sheet 2, where I filter the Water temp values on the right side, the red value shows the problem I mentioned aboth
Have you guys an idea how I can solve this problem? It's my first "project" of this kind, it was really fun until I discovered the problem, but I would really love to make it work!

Use this query
=QUERY(A2:C40,"select A, C where B matches 'Wassertemperatur'
label A 'date', C 'Watertemp.'",1)

Related

Google Sheet Array Formula to automatically transpose responses from a Google Form into a single cell

I want to tweak an Array Formula that I'm trying to use to transpose a set of responses from a Google Form into a single 'list' in a cell in the linked Google Sheet.
Here is a simplified mock-up of the Google Sheet: https://docs.google.com/spreadsheets/d/1BKgjGK2RbXC5FkCgBOU53dLEb3fDeQADujenrqRgnT0/edit?usp=sharing
As you can see, I have a working Array Formula 'hidden' in the header for Column A, which takes the content from columns B:K in each row and transposes them with line breaks to make a nice neat list in the first cell at the start of each row (A2, A3, A4, etc.) Here is the Array Formula for ease of reference:
=ARRAYFORMULA({"Points to Develop";
transpose(query(transpose ($B$2:$B&CHAR(10)&$C$2:$C&CHAR(10)&$D$2:$D&CHAR(10)&$E$2:$E&CHAR(10)&$F$2:$F&CHAR(10)&$G$2:$G&CHAR(10)&$H$2:$H&CHAR(10)&$I$2:$I&CHAR(10)&$J$2:$J&CHAR(10)&$K$2:$K)
))})
The problem I'm having is that the formula stops working as soon as a new Google Form is submitted, as the new row of responses arrives at the top and 'pushes the formula down'/throws everything out of alignment, when all I want is for it to stay in place and run automatically every time a Form is submitted. As you can probably gauge by my limited technical terminology, I'm very much a novice at this!
How can I fix this formula, or is there a simpler or better alternative that will achieve the same results?
As often happens in life, the nature of the problem turned out to be the source of the solution. Here’s a condensed version of my thoughts as I slowly found my way to an answer (you can almost hear the hamster wheels turning in my head!):
The problem with formulas in a Google Sheet linked to a Google Form is that each new entry/submission from the form always arrives in Row 2 …
In fact, the only thing that stays ‘in place’ in the Google Sheet is the Header Row/Row 1 …
So, rather than using a formula that refers to a ‘Row 2’ that will become ‘Row 3/4/5/etc.’ as new Google Forms are submitted, I should use a formula that always refers to Row 1, that is, by using OFFSET
The obvious (if inelegant) formula that resulted from this is [I've also placed this in the final column in the Google Sheet that I linked to in my original question]:
= ARRAYFORMULA({"Points to Develop [Working OFFSET Formula]"; transpose(query(transpose(OFFSET(A1:A,1,1)&CHAR(10)&OFFSET(A1:A,1,2)&CHAR(10)&OFFSET(A1:A,1,3)&CHAR(10)&OFFSET(A1:A,1,4)&CHAR(10)&OFFSET(A1:A,1,5)&CHAR(10)&OFFSET(A1:A,1,6)&CHAR(10)&OFFSET(A1:A,1,7)&CHAR(10)&OFFSET(A1:A,1,8)&CHAR(10)&OFFSET(A1:A,1,9)&CHAR(10)&OFFSET(A1:A,1,10)))) })
I am sure that there are neater ways of getting to the same result, but this is one that works for me … and is within my levels of comprehension, so I can fine-tune and fix it as needed.

Google sheets residual formula values after source data removed

This happens to me quite a bit and I'm not really certain how to word it properly to search google for a solution, but whenever I populate my data sheet, my formula works fine. But then when I clear that data, the formula still works and shows N/A, or "" if I use IFERROR, in the FORMULA cell, but all of the previous values beyond that are still present. Like "ghost" values or something. They don't seem to be calculated in other formulas, but it is distracting and I'd just assume they not be there.
Any ideas? Is this something that happens regularly or am I just special? I don't recall if it is always a QUERY, but it happens to be in the screenshot below.
This sheet contains sensitive customer information, so I can't share it, but if necessary, I can make a copy and try to remove all of that information.

Creating Dynamic Sheet Cell Reference List for pulling numbers to SUM

I've been working on building a data analysis sheet, which is quite verbose at the moment and a bit more complicated than it should be as I've been trying to figure this out. Please note, I work doing student data in a school.
Basically, I have two sets of input data:
Data imported from a CSV file that includes test data and codes for Common Core Standards and the questions tied to those standards as a whole class summary
Data imported from a CSV file that includes individual scores by question
I am looking to construct 2 views:
A view that collates and displays data of individual standards per student that includes a dropdown to change the standard allowing a teacher to see class performance by standard in a broad view. The drop-down is populated dynamically from the input data (so staff could eventually dump data and go directly to reports)
A view that collates and displays data of individual students broken down by performance on each standard allowing a teachers to see the broader spectrum for each student. The student drop-down is populated from Source list 2.
I have been able to build the first view, but am struggling with the second. I've been able to separate the question codes and develop strings of cell references to the scoring data, including a dynamic reference to the row the selected student's score data appears on in the second source set from above.
I tried to pass through an indirect() formula into a sum() so as to process for a mean evaluation, and have encountered errors. I think SUM() doesn't process comma-separated cell reference lists from Indirect() [or in general] or there is something that I am missing to help parse it. Here is the formula I have tried:
=Sum(vlookup(D7,CCCodeManip!$A:$C,3,false))
CCCodeManip!C:C includes the created text (based on the dynamic standards and question codes, etc), here's an example of what would be found there:
'M-ADI'!M17, 'M-ADI'!N17, 'M-ADI'!O17, 'M-ADI'!P17, 'M-ADI'!Q17, 'M-ADI'!R17, 'M-ADI'!J17
I need these to be dynamic so that teachers can input different sets of standards, question, and student data and the sheet automatically collates and reports it in uniform ways (with an upward bound of 20 standards as I currently have it built)
Here is a link to the sheet I built, with names and ID anonymized. There's a CRAP TON of sub-tabs, and that's really just being able to split apart and re-combine data neatly without things error-ing out due to data overlapping, aside from a few different attempts and different approaches to parse the cell reference strings.
The first two tabs are the current status of the data views. I plan to hide a bunch of the functional stuff that is there to help pull data accurately.
The 3rd and 4th tab are the source data sets. 5th is a modified version of source data that allows me to reference things better, and I've tried to arrange the sheets most relevant towards the front of the set.
https://docs.google.com/spreadsheets/d/1fR_2n60lenxkvjZSzp2VDGyTUO6l-3wzwaV4P-IQ_5Y/edit?usp=sharing
Some have a different approach? I am aware that I might be as far as I cn go with this and perhaps should consider scripts - my coding experience is a bit out of date and my strength is more with the formulas, but I can dig into things with some direction, if anyone can help.
Ok so I noticed something.
It seems the failure is in the indirect reference:
=indirect(CCCodeManip!C3)
The string I am trying to parse via indirect is going to be generated into something like this, dynamic from reference to other data:
'M-ADI'!M17, 'M-ADI'!N17, 'M-ADI'!O17, 'M-ADI'!P17, 'M-ADI'!Q17, 'M-ADI'!R17, 'M-ADI'!J17
The indirect returns the error that the above string is not a cell reference with the #REF code.
Can someone give me a clue as to what is causing this? I am going to dig into the docs on Indirect() from google and will post anything that I find.
Perhaps it is that indirect() can't handle lists, but only specific references and arrays, which may require me a to build a sheet to do the SUM formula on for each question set (?)
So I think I figured it out, but i Ended up parsing the data differently, basically doing the sum based on individual cell references and a separate sum formula, bypassing the need to do it all at once, it jsut makes my sheets a lot dirtier! I am eventually going to see if code could do it better if I need to, but this is closed for now.
Basically, I did individual cell references to recall scores in a row, then used a separate SUM formula, and created references / structures to be able to pull those sum() results. Achieves the same end, but with extra crap on the sheet.

OnEdit or Time based triggers to check one cell then edit another

My scripting skills are not the greatest but I can not figure out how to accomplish what I need to with just functions inside of Google Sheets.
My sheet looks something like this: https://docs.google.com/spreadsheets/d/1R6YJIR37wtOt8ZknSdMQPFFp0Llf_LHpmGCV32iQY00/edit#gid=1213399830
I have form responses being brought into the second tab via the Indirect function and are then modified with a ArrayFormula function to convert everything to the correct unit of measure and the summed to keep everything on the same row. All of this could be altered if needed.
I then have a column that is totaling inputs in other columns to give me my on hand inventory. We'll call this TotalsColumn.
What I am attempting to achieve is to make a script that checks the form responses, then either with:
OnEdit (which if I understand correctly does not work for information being ported in) and then alter the corresponding cell in the TotalsColumn.
or if not possible, then
With a time based trigger checking the ported information to see if there is a new entry if I don't sum it with a last function or something, or if the information remains summed (which seems easier to me) then to check if the sum is greater than it was last time it was checked and alter TotalsColumn.
The alter to the TotalsColumn, regardless of which method used, will always be a static number and I am pretty sure I'll need to make a script for each of my different inventory items but that's fine if I can just figure out a way of making this happen.
Thank for your time and any information you can provide.
Please feel free to point me to another thread where I can start to piece together a solution, I just could not find anything that related closely enough to my question or find a good jumping off point.
What happens in this situation?

How do I make changes to a named range in google sheets propagate immediately?

I'm using Google Sheets to calculate my goals for weight lifting. Different workouts are based on different percentages of my high score for each lift.
I had been calculating each day's plan with something like "=CEILING((0.8*A3/5), 1)*5" to represent 80% of whatever the hell was in A3, rounded to something divisible by five. This was not maintainable.
What I'm working on now is using the script editor to provide custom functions. "=FIVES('bench')" gives me my bench press progression for five reps.
That works great, except for when I change values. I put a named range on each of my max lifts so I could call getRangeByName get the value by name instead of by cell coordinates. But the functions I made don't get called again so I keep seeing stale values, even after refreshing the page.
I found another post suggesting that getRangeByName had a second arg which could be set to now() to refresh the cache now. Tried that and it looks like that was out of date. I haven't found any other info on a named range being cached.
How can I tell my script that it needs to re-run when a named range gets updated? Is there some sort of observer/listener/watcher in sheets? If not, I'm open to techniques - named ranges were just the first thing I bumped into for naming my variables.
Google's script caching was never going to allow that to work without shenanigans. The expectation is that foo('bar') will be deterministic, always returning the same thing. I'd been passing the named range's name, hoping I could use getRangeByName and using the name as a label.
Instead I'm just using the range as an arg and passing a second string as a label: foo(bar, "Bar label"). It's a little more verbose, but much less so than the stuff I was trying to trick the script server with. Don't fight the framework, folks.

Resources