Happy new year, folks,
Currently, I'm accessing and loading a Google Sheets worksheet using the following, default way:
URL metafeedUrl = new URL(SPREADSHEET_URL);
SpreadsheetEntry spreadsheet = service.getEntry(metafeedUrl, SpreadsheetEntry.class);
URL cellFeedUrl = ((WorksheetEntry) spreadsheet.getWorksheets().get(0)).getCellFeedUrl();
// Get entries as cells
feed = (CellFeed) service.getFeed(cellFeedUrl, CellFeed.class);
Then I work with it, etc. Everyting works just fine.
The problem:
I'm about to deploy the application and have it work with a Worksheet that has several hundred, if not thousand rows of cells. To me, the only relevant rows are usually the 100-200 bottom ones.
Is there a way to partially load a CellFeed, preferrably from the bottom up? Does the API provide such a way?
Looking at the API itself, you can do it with cell feed or list feed.
in cell feed, look at https://developers.google.com/google-apps/spreadsheets/#fetching_specific_rows_or_columns
you can specify there the minimum/maximum row/columns to get, and there is also a java example in there.
a more efficient way to get your data, is the row feed as it sends less bytes in return:https://spreadsheets.google.com/feeds/list/
with the undocumented "start-index' parameter so it only reads starting at that row.
I use this and works for the "old" and "new" sheets.
The first time you will need to get all rows (or attempt some sort of binary lookup to find the last spreadsheet row).
I have not used the java api library, it probably does not allow for that undocumented parameter. You can always do a url "get" directly from java or any language and use the spreadsheet api directly by https.
I got this tip a long time ago from here:
https://groups.google.com/forum/#!topic/google-spreadsheets-api/dSniiF18xnM
and use it on this github project (javascript ajax call example)
https://github.com/zmandel/Plus-for-Trello/blob/master/source/sql.js
I don't know any CellFeed, but you can create an HTML feed that retrieves a number of ROWS from any Spreadsheet, then treat that HTML, would that work for you? What are the goals when retrieving the information?
Eg.
Code.gs
function doGet() {
return HtmlService.createTemplateFromFile("form").evaluate().setSandboxMode(HtmlService.SandboxMode.NATIVE);
}
function getLastLines( numLines, ssId, sheetName ){
var sheet = SpreadsheetApp.openById(ssId).getSheetByName(sheetName);
return JSON.stringify(sheet.getRange(sheet.getLastRow() - numLines, 1, numLines, sheet.getLastColumn()).getValues());
}
in form.html
<div id="arrayPlace"></div>
<script>
function changeDiv( res ){
document.getElementById("arrayPlace").innerHTML = res;
}
function getUrlParameter(sParam)
{
var sPageURL = window.location.search.substring(1);
var sURLVariables = sPageURL.split('&');
for (var i = 0; i < sURLVariables.length; i++)
{
var sParameterName = sURLVariables[i].split('=');
if (sParameterName[0] == sParam)
{
return sParameterName[1];
}
}
}
var numLines = getUrlParameter("numLines");
var ssId = getUrlParameter("ssId");
var sheetName = getUrlParameter("sheetName");
google.script.run.withSuccessHandler( changeDiv ).getLastLines( numLines, ssId, sheetName );
</script>
And the URL would have the additional parameters:
https://script.google.com/macros/s/AKfycbwFLN-qqTcXXAVgR-aDa9h61yTa39kVhE2MwX9htRbIm2NN5I4/exec?numLines=5&ssId=1dJlNmtvcsWixEDnUz7GxnyLMZKXHwA-9uopYPUC8I4E&sheetName=Sheet1
Related
I am trying to adapt Zack Akil's script to generate a Google Form from a Google Sheet using App Script, but one thing that I am struggling with is to make the sheet's input parsed as HTML. I generate a form based on my sheet, all the text on cells is placed in Forms as plain text, the HTML is not parsed (see figure below).
I pasted the script from Zack and I kindly ask you to point out where should I modify in order to have this parsed on the form.
function getSpreadsheetData(sheetName) {
// Return an list of objects (one for each row) containing the sheets data.
var arrayOfArrays = SpreadsheetApp.getActiveSpreadsheet().getSheetByName(sheetName || 'Sheet1').getDataRange().getValues();
var headers = arrayOfArrays.shift();
return arrayOfArrays.map(function (row) {
return row.reduce(function (memo, value, index) {
if (value) {
memo[headers[index]] = value;
}
return memo;
}, {});
});
}
function create_ranges_for_data(form, data, data_section_name){
// loop throughh each row
data.forEach(function (row) {
// create a new question page
form.addPageBreakItem()
// add page title
form.addSectionHeaderItem()
.setTitle(data_section_name);
// create number range input with the title being the document to be labeled
form.addScaleItem()
.setTitle(row[data_section_name])
.setBounds(1, 10)
.setRequired(true);
});
}
function make_form_using_column(column_name) {
// create a new Google Form document
var form = FormApp.create('Data labelling - ' + column_name)
desc = "Thank you for taking the time to label this data!";
form.setDescription(desc);
form.setProgressBar(true);
form.setShowLinkToRespondAgain(false)
var data = getSpreadsheetData();
create_ranges_for_data(form, data, column_name);
}
function gen_form(){
var COLUMN_TO_USE = 'Input text'
make_form_using_column(COLUMN_TO_USE);
}
You can't use HTML text formatting. most sites block it because it poses a security risk. You might need to install an add-on or, like fullfine said, use bold text.
I'm pretty new to excel or google sheets. The work place, that I work at does not have anything stream lined.
I'm trying to create my own work book that I can refresh everyday I log in so that I can have a list of things that I need to work on for that day.
One of the functions that I would like to have is, whenever a new sheet is shared with me on Google Sheets, I want the URL for that sheet to populate in one of the cells in my workbook automatically and arranged based on timestamp.
I was trying to search for this on Google, but I read that: shared with me docs are not stored anywhere specifically.
Any help or pointing me in the right direction is highly appreciated.
It is easy to fetch the files that have been shared with you. For that, you can simply call the Drive API's Files: list method specifying in the q parameter the sharedWithMe attribute.
Afterwards, you can use the SpreadsheetApp class to gather a spreadsheet and insert data into it. In this case, you can simply make several calls of apendRow() to insert the results.
Finally, properties can be used to store the status of the script (last date checked), so that it can know where to resume from. In this case below, we'll be saving the date in the LAST_DATE property.
Code.gs
var SPREADSHEET_ID = 'YOUR_SPREADSHEET_ID';
var SHEET_NAME = 'YOUR_SHEET_NAME';
function myFunction() {
var sheet = SpreadsheetApp.openById(SPREADSHEET_ID).getSheetByName(SHEET_NAME);
var lastDate = new Date(PropertiesService.getScriptProperties().getProperty('LAST_DATE'));
var currentDate = new Date();
var files = getFiles(lastDate);
for (var i=0; i<files.length; i++) {
var row = [
new Date(files[i].sharedWithMeDate),
files[i].title,
files[i].alternateLink,
files[i].sharingUser.emailAddress,
files[i].owners.map(getEmail).join(', ')];
sheet.appendRow(row);
}
console.log('lastDate: %s, currentDate: %s, events added: %d', lastDate, currentDate, files.length);
PropertiesService.getScriptProperties().setProperty('LAST_DATE', currentDate.toISOString());
}
function getEmail(user) {
return user.emailAddress;
}
function getFiles(lastSharedDate) {
var query = "sharedWithMe and mimeType = 'application/vnd.google-apps.spreadsheet'";
var res = Drive.Files.list({
q: query,
orderBy: "sharedWithMeDate desc",
fields: "*",
pageSize: 1000
});
// `query` parameter cannot compare sharedWithMeDate, so we do it afterwards
return res.items.filter(function (i) {
return (new Date(i.sharedWithMeDate) > lastSharedDate);
}).reverse();
}
You can set up the script to be ran periodically (i.e once a day, or more in case you'd need it) using Time-driven triggers.
I have a Google spreadsheet in which I record my freelance jobs. I have it set up that each line calculates whether it is paid for. (Payments are pulled from a separate sheet.)
What I would like to do is to generate an invoice, where I would select the customer and I get a listing of all unpaid entries for that customer.
Using a arrayed filter function does the job, but I can't use that as an invoice because I need the total line underneath, and would prefer the table format matching the count of entries.
Is it possible to insert such information into a Google Doc as a table, or within Sheets, to push the lines following an array down?
I thought this would be a simple enough concept but I can't find anything that does the full deal.
You could try this script. I'm not sure if the final results is what you are looking for. In case it is not, it can be easily modified:
function onEdit(e) {
//If you change the Customer in the Invoice sheet, it runs the code
if (e.range.getA1Notation() == 'A1' && e.source.getSheetName() == 'Invoice'){
var sprsheet = SpreadsheetApp.getActiveSpreadsheet();
var invoice = sprsheet.getSheetByName("Invoice");
var times = sprsheet.getSheetByName("Times");
var in_customer = invoice.getRange("A1").getValue(); //Name you selected in the dropdown menu
var data = times.getRange("A1:H").getValues(); //All the data from the Time sheet
var total = 0;
//Loops through all the data looking for unpaid subtotals from that customer
for (var i = 0; i < data.length; i++){
/*> "i" represents the row, the second number is the column
> The rows start at 0 since it is the first array position.
*/
if (in_customer == data[i][2]) {
if (data[i][7] == 'N'){
total += Number(data[i][5]); //Accumulates each subtotal into total
invoice.appendRow([data[i][0], data[i][1], data[i][3], data[i][5]]);
}
}
}
invoice.appendRow(["Total: ","","", total]);
}
}
This results in (I changed some values to test it):
As you see I added some headers.
References:
Range Class
onEdit Trigger
I have a lot of named ranges in my spreadsheet and time to time I need to delete all of them and create new one. In past I did this by deleting worksheet but recently I've discovered that this operation is not deleting named ranges. So Ive tried to use Class NamedRanges and list trough them by name on second row and to remove them one by one but it doesn't work here is what I tried:
function CellNamerRemove()
{
// the purpose of the CellNamerRemove function is to automatically remove all range names the complete vertical CellRanges
// for each column that has a 'header' in the sheet called 'RawData'. The Quality purpose is to force
// Named-Links only to exist in referencing the raw data from the Starccm+ files
// get a reference to the RawData Spreadsheet
var thisSheetString = "RawData";
var maxRows = SpreadsheetApp.getActiveSheet().getMaxRows();
var lastColumn = SpreadsheetApp.getActiveSheet().getLastColumn();
// loop from 1st column ie 1 to lastColumn.
for(thisColumn = 1; thisColumn <= lastColumn;thisColumn++)
{
var thisName = SpreadsheetApp.getActiveSpreadsheet().getSheetByName(thisSheetString).getRange(2, thisColumn, 1, 1).getValue();
var ss = SpreadsheetApp.getActiveSpreadsheet();
// ss.setNamedRange(trim(thisName), thisRange);
ss.getRangeByName(thisName).remove();
}
}
Can you help me with?
actually I prepared and ran this script which helped me remove ALL named ranges.
function removeNamedRanges(){
var namedRanges = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet().getNamedRanges();
for(i=0;i<namedRanges.length;i++){
namedRanges[i].remove();
}
}
The problem is in this line:
ss.getRangeByName(thisName).remove();
It should be:
ss.removeNamedRange(thisName);
SITUATION
I am using Trirand JQGrid for MVC[server side] in my proj.
I've got more than 5 hundred thousand records in a single table.
I load the data by calling this piece of code. this is what gives 500000 records collection.
IEnumerable<myIndexViewModel> myviewmodel= _allincidents.Select(x => new myIndexViewModel
{
IncidentRequestStatus = x.RequestStatus,
RequestByUserName = x.RequestByUserName,
Subject = x.Subject
});
gridModel.JqGrid.DataBind(myviewmodel.AsQueryable());
JQgrid handles the json based ajax requests very nicely for every next page i click.
PROBLEM
I dont want to load 5 hundred thousand records all together on the page load event as it kills jqgrid.
If i write a stored procedure in the DB for requesting a specific page to be displayed then its gonna load only that page in the myviewmodel collection.
How do i get pages on the fly from the DB when the next page is clicked. is this even possible in jqgrid?
SITUATION 2
Based on the answers from VIJAY and MARK the approach they have shown is absolutely correct but over here the JQGRID for MVC sets up the DATAURL property for making the method call. In this case its the IncidentGridRequest.
How do i send in the page number when the grid next page or previous page is clicked?
incidentModel.IncidentGrid.DataUrl = Url.Action("IncidentGridRequest")
public JsonResult IncidentGridRequest()
{
}
Your controller action that will provide your grid with results can accept some extra information from jqGrid.
public ActionResult GetGridData(string sidx, string sord, int page, int rows, bool _search, string filters)
The main parts you are interested in is the page, rows (sidx is for column sorting, sord for the sorting order, _search if there was a search done on the grid, and if so filters contains the search information)
When you generate your results you should be able to then
IEnumerable<myIndexViewModel> myviewmodel = allincidents.Select(x => new myIndexViewModel
{
IncidentRequestStatus = x.RequestStatus,
RequestByUserName = x.RequestByUserName,
Subject = x.Subject
}).Skip((page - 1) * rows).Take(rows)
PS. I'm not sure if you using IEnumberable will be moving a large amount of data from your DB but you might want to use IQueryable when you generate this subset of data for the jqGrid.
Edit: To deal with your paging issues, You should be calculating the number of total records in your query and passing that value to the grid, Ex
int totalRecords = myviewmodel.Count();
and then later you would pass that to your grid as a jSon value. Ex
var jsonData = new
{
total = (totalRecords + rows - 1) / rows,
page = page,
records = totalRecords,
userdata = new {SearchResultsFound = searchResultsFound},
rows = (
......
Yes, for example if you are accepting the page number you want to turn to in a variable named page and the have the size of page in a variable pageSize then:
IEnumerable<myIndexViewModel> myviewmodel = allincidents.Select(x => new myIndexViewModel
{
IncidentRequestStatus = x.RequestStatus,
RequestByUserName = x.RequestByUserName,
Subject = x.Subject
}).Skip((page-1)*pageSize).Take(pageSize));
will give you the records of size pageSize to you.
The Trirand jqGrid for ASP.NET MVC is using IQueryable interface inside the JqGrid.DataBind() method to implement pagin, sorting and filtering.
So the key here is to use datasource, which handle these types of operations at the database level (by crafting SQL queries to the database in such a way that only the data required is fetched). All major ORMs have this support, this includes: LINQ-2-SQL, Entity Framework, NHbiernate, LLBLGen.
You just need to use one of this technologies, and past the required context directly to JqGrid.DataBind() method (without extracting the data manually like you do it in your sample).
An easier approach by using PagedList library (from Nuget). There is a useful blog by Joseph Schrag
public JsonResult Users(int PageNo, int Rows)
{
var UserList = db.Users.Select(t => new
{
t.UserId,
t.Username,
t.Firstname,
t.Lastname,
t.Designation,
t.Country,
t.Email
}).OrderBy(t => t.UserId);
var pagedUserList = UserList.ToPagedList(PageNo, Rows);
var results = new
{
total = pagedUserList.PageCount, //number of pages
page = pagedUserList.PageNumber, //current page
records = UserList.Count(), //total items
rows = pagedUserList
};
return new JsonResult() { Data = results, JsonRequestBehavior = JsonRequestBehavior.AllowGet };
}