Capturing twitter status timeline updates/infinite scroll updates - twitter

I'm using KRL to inject elements into twitter timeline statuses similar to Jesse Stay's TwitterBook. The problem I have is that these elements are only associated with statuses that are currently visible when the bookmarklet is initiated. If a new status is added through the 'new tweet' updated via Ajax or through status updates via infinite scroll, these new statuses are untouched.
Is there a way to either poll for new statuses or sense a twitter status update event via KRL in order to inject elements only into those newly added statuses?

The example posted at
http://kynetxappaday.wordpress.com/2010/12/25/day-21-modifying-facebook-stream-with-kynetx/
works with the Facebook stream but the concept is the same
create setTimeout infinite loop to look for stream items
only select stream items not marked as processed
process stream items
rinse and repeat
Code example from post
ruleset a60x512 {
meta {
name "MikeGrace-status-update-translator"
description <<
MikeGrace-status-update-translator
>>
author "Mike Grace"
logging on
}
global {
datasource insult:HTML <- "http://www.pangloss.com/seidel/Shaker/index.html?" cachable for 1 second;
}
rule find_status_updates_by_mike_grace {
select when pageview ".*"
{
notify("Starting to look for status upates by Mike Grace","");
emit <|
// get app object to raise web events
app = KOBJ.get_application("a60x512");
// function that finds FB status updates by Mike Grace
function findMikeGrace() {
// loop through each stream item on the page that hasn't been processed already by the app
$K("li[id^=stream_story]:not(li[kfbt])").each(function() {
var currentStreamItem = this;
// grab the current stream item posters name
var name = $K(currentStreamItem).find(".actorName").text();
// mark the stream item as being processed to reduce future processing times
$K(currentStreamItem).attr("kfbt","y");
// is the stream item by the perpetrator?
if (name == "Michael Grace") {
// strikethrough the original update
$K(currentStreamItem).find(".messageBody").wrap("<strike />");
// get selector to return translation of status update
var returnSelector = $K(currentStreamItem).attr("id");
returnSelector = "li#"+returnSelector+" .messageBody";
// raise web event to get translation for non geeks
app.raise_event("get_insult", {"returnSelector":returnSelector});
} // end of checking name
}); // end of looping through unprocessed stream items
// call myself again later to process new items on the page
setTimeout(function() {
findMikeGrace();
}, 9000);
}
// start the process of finding the perpetrator
findMikeGrace();
|>;
}
}
rule get_insult {
select when web get_insult
pre {
selector = event:param("returnSelector");
insulter = datasource:insult("#{selector}");
foundInsult = insulter.query("font");
singleInsult = foundInsult[0];
}
{
emit <|
console.log(singleInsult);
$K(selector).parent().after("<br/>"+singleInsult);
|>;
}
}
}

Related

How to apply deduplication for an array returned in Zapier Code output

I have a Zapier Code block that does fetch for JSON array and the preprocess this data. I cannot use Zapier Webhook with polling, because I need to process the data a bit.
Zapier Webhook offers deduplication feature, by having id parameter associated with the items returned in an array from the URL endpoint. How can I achieve the same for Zapier Code? Currently, my zap is trying to process and trigger on the same data twice. This leads to the error that Zapier tries to send out the same tweet twice, every time the Code is triggered.
Here is mock data returned by my Code:
output = [{id: 1, name: "foo"}, {id: 2, name: "bar"}]
Currently, without deduplication, I am getting this email and having my zap disabled:
Your Zap named XXX was just stopped. This happened because our systems detected this Zap posted a duplicate tweet, which is against Twitter's Terms Of Service.
You can use storage by Zapier to achieve this. the ideal flow will be :
Trigger
Storage by Zapier [Get Value (use storage key = lastItemId) ]
Code By Zapier (Filter array return only those record that has id greater than the lastItemId)
Storage By Zapier (Set Value) : update lastItemId with the last item processed by Code By Zapier step
You can also use StoreClient in place of the Storage By zapier, but always update a existing key lastItemId and compare id of the record with ```lastItemId`` and at the end update StoreCLient key (lastItemId)
Based on the answer from Kishor Patidar, here is my code. I am adding the example code, here is too some time to figure it out. Specifically, in my case, the items cannot be processed in the order of appearance (no running counter primary keys) and also there are some limitations how far in the future Zapier can schedule actions (you can delay only up to one month).
The store also has a limitation of 500 keys.
// We need store for deduplication
// https://zapier.com/help/create/code-webhooks/store-data-from-code-steps-with-storeclient
// https://www.uuidgenerator.net/
var store = StoreClient('xxx');
// Where do we get our JSON data
const url = "https://xxx";
// Call FB public backend to get scheduled battles
const resp = await fetch(url);
const data = await resp.json();
let processed = [];
for(let entry of data.entries) {
console.log("Processing entry", entry.id);
// Filter out events with a non-wanted prize type
if(entry.prizes[0].type != "mytype") {
continue;
}
// Zapier can only delay tweets for one month
// As the code fires every 30 minutes,
// we are only interested scheduling tweets that happen
// very soon
const when = Date.parse(entry.startDate);
const now = Date.now();
if(!when) {
throw new Error("startDate missing");
}
if(when > now + 24 * 3600 * 1000) {
// Milliseconds not going to happen for next 24h
console.log("Too soon to schedule", entry.id, entry.startDate, now);
continue;
} else {
console.log("Starting to schedule", entry.id, entry.startDate, now);
}
const key = "myprefix_" + entry.id;
// Do manual deduplication
// https://stackoverflow.com/questions/64893057/how-to-apply-deduplication-for-an-array-returned-in-zapier-code-output
const existing = await store.get(key);
if(existing) {
// Already processed
console.log("Entry already processed", entry.id);
continue;
}
// Calculate some parameters on entry based on nested arrays
// and such
entry.startTimeFormat = "HH:mm";
// Generate URL for the tweet
entry.signUpURL = `https://xxx/${entry.id}`;
processed.push(entry);
// Do not tweet this entry twice,
// by setting a marker flag for it in store
await store.set(key, "deduplicated");
}
output = processed;

Graph Api Delta token is not returning correct data

I am trying to use delta query to get the changes in one of the rooms calendars, when i use the start date and end date to set the initial request, it returns the correct events data and then afterwards when I use the delta token next time to make request, but it returns event data with tag saying this particular event has been deleted and does not return any valuable info apart form ID.
Here is my code
private async Task<IEventDeltaCollectionPage> GetEventData(GraphServiceClient graphClient, object deltaLink)
{
IEventDeltaCollectionPage page;
if (lastPage == null)
{
var queryOptions = new List<QueryOption>();
if (deltaLink == null)
{
queryOptions = new List<QueryOption>()
{
new QueryOption("startdatetime", "2020-01-16T00:00:00Z"),
new QueryOption("enddatetime", "2020-01-24T00:00:00Z")
};
}
else
{
queryOptions = new List<QueryOption>()
{
new QueryOption("$deltatoken", deltaLink.ToString())
};
}
page = await graphClient
.Users["nitroom2#domain.onmicrosoft.com"].CalendarView
.Delta()
.Request(queryOptions)
.GetAsync();
}
else
{
lastPage.InitializeNextPageRequest(graphClient, deltaLink.ToString());
page = await lastPage.NextPageRequest.GetAsync();
}
lastPage = page;
return page;
}
AS you can see this code will either look for dates range or delta token. so what I do is I initially make a call without a delta token, then once I get the final response I get the delta token. I restart the application this time and I provide the hard-coded delta token. Before restarting the application I also make sure that there is either a new event created or updated in the rooms calendar and then I restart the application with the delta token. I get a response back with some event data but that's telling me the event has been deleted, but that's not true.
Not sure what i am misisng here. can any abody suggest?
when using delta query Microsoft Graph returns as deleted new/updated/deleted items that are not in start-end date range of the initial request.
please check if that was your case :)

HTPP POST to Google Forms or Alternative

I have a google form setup that emails me upon a manual submission when somebody fills it out (new lead) and transfers the information to a Google spreadsheet. Easy enough to figure that out.
However, now I'm trying to figure out how to send the same information information contained within a url string and automatically POST that information to the form. Or find a company who offers that ability, via an api or other means. So far I've tested out jotform and a few others. The information passed along fine, but it doesn't auto populate the fields. I assume it's because it doesn't know that x=y due to the fields being named differently. I've found a ton of documentation about pre-populating the forms, but not much about filling out a form every time a new POST url is generated.
URL looks like the following
VARhttps://script.google.com/macros/s/XXXXXXXXXXXXXXXX/exec?/--A--
first_name--B--/--A--last_name--B--/--A--address1--B--/--A--city--B--/--A--
state--B--/--A--postal_code--B--/--A--phone_number--B--/--A--date_of_birth--
B--/--A--email--B--
Information passed is as follows
https://website
here.com/Pamela/Urne/123+Test+Street/Henderson/TX/75652/281XXXXXX/1974-01-
01/test0101cw#test.com
The script I'm testing out
// original from: http://mashe.hawksey.info/2014/07/google-sheets-as-a-database-insert-with-apps-script-using-postget-methods-with-ajax-example/
// original gist: https://gist.github.com/willpatera/ee41ae374d3c9839c2d6
function doGet(e){
return handleResponse(e);
}
// Enter sheet name where data is to be written below
var SHEET_NAME = "Sheet1";
var SCRIPT_PROP = PropertiesService.getScriptProperties(); // new property service
function handleResponse(e) {
// shortly after my original solution Google announced the LockService[1]
// this prevents concurrent access overwritting data
// [1] http://googleappsdeveloper.blogspot.co.uk/2011/10/concurrency-and-google-apps-script.html
// we want a public lock, one that locks for all invocations
var lock = LockService.getPublicLock();
lock.waitLock(30000); // wait 30 seconds before conceding defeat.
try {
// next set where we write the data - you could write to multiple/alternate destinations
var doc = SpreadsheetApp.openById(SCRIPT_PROP.getProperty("key"));
var sheet = doc.getSheetByName(SHEET_NAME);
// we'll assume header is in row 1 but you can override with header_row in GET/POST data
var headRow = e.parameter.header_row || 1;
var headers = sheet.getRange(1, 1, 1, sheet.getLastColumn()).getValues()[0];
var nextRow = sheet.getLastRow()+1; // get next row
var row = [];
// loop through the header columns
for (i in headers){
if (headers[i] == "Timestamp"){ // special case if you include a 'Timestamp' column
row.push(new Date());
} else { // else use header name to get data
row.push(e.parameter[headers[i]]);
}
}
// more efficient to set values as [][] array than individually
sheet.getRange(nextRow, 1, 1, row.length).setValues([row]);
// return json success results
return ContentService
.createTextOutput(JSON.stringify({"result":"success", "row": nextRow}))
.setMimeType(ContentService.MimeType.JSON);
} catch(e){
// if error return this
return ContentService
.createTextOutput(JSON.stringify({"result":"error", "error": e}))
.setMimeType(ContentService.MimeType.JSON);
} finally { //release lock
lock.releaseLock();
}
}
function setup() {
var doc = SpreadsheetApp.getActiveSpreadsheet();
SCRIPT_PROP.setProperty("key", doc.getId());
}
I get a success message after accessing the url, but all information listed in the spreadsheet is "Undefined."
That's as far as I got so far. If somebody knows an easier solution or can point me in the right direction I'd appreciate it.

Rails 5: Send reload to set of clients on data update

I'm using Rails 5 to make a simple turn based game tracker for an in-person social game (via phones/tablets/etc..)
I want to have all the 'players' in the game (list of sessions/users/...) to reload their browsers automatically once a player has taken an action.
I know that there are live update capabilities such as AJAX and websockets, but they all seem far too weighty for what seems to be a simple problem. Furthermore, I want to update other clients pages, not just the client initiating the action.
Is there a simple solution to send a reload? Or do I need to code something up in one of the more complicated APIs?
For the simple trouble, you still can use AJAX to reload user client by making interval request for each XX seconds. The server can return the last action time which can be used for client to determine that it should reload itself or not.
For example, on the controller
# SomeController
def get_last_action_time
# Return the timestamp of the last action
render json: {last_action_time: "2017-12-29 10:00:42 UTC"}
end
on the client
function getLocalLastAction () {
/* return timestamp of the last action on local */
}
function setLocalLastAction (time) {
/* Store the `time` to somewhere, ex: localStorage */
}
function checkLastAction () {
$.getJSON("/get_last_action_time", function (data) {
if (getLocalLastAction() < data.last_action_time) {
/* destroy the interval */
setLocalLastAction(data.last_action_time)
/* do the reload page */
} else {
/* do nothing */
}
})
}
// Check every 1 second, shouldn't be too short due to performance
var checking = setInterval(checkLastAction, 1000)
Then when user A do an action, the server last_action_time will change, hence client of other users will be reloaded at most after 1 second.
This way is old but quite easy to do in some simple case, and when you implement together with actions caching, the performance of app still acceptable. In the more complicated cases, I suggest using WebSocket solution for
Full control
Low latency
Better performance for app
Thanks to #yeuem1vannam's answer, here is the final code I used that helps avoid the race condition of a page loading old information while the time is being updated and then the javascript updating the time and getting the new time, and hence missing the reload.
The javascript code:
var actionChecker;
function doneChecking () {
clearInterval(actionChecker);
}
function checkLastAction () {
// Get the game ID from the html access span
var dataId = document.getElementById('javascript_data_access');
if (!dataId) return doneChecking();
var initActionTime = dataId.getAttribute('init_last_action_time');
if (!initActionTime) return doneChecking();
dataId = dataId.getAttribute('game_number');
if (!dataId) return doneChecking();
// Get the last action time
var ret = $.getJSON("/get_last_action_time/"+dataId, function (data) {
var lastActionTime = data.last_action_time;
if (!lastActionTime) return doneChecking();
if (lastActionTime>initActionTime) {
location.reload();
}
})
}
window.onload = function() {
// Check every 1 second, shouldn't be too short due to performance
actionChecker = setInterval(checkLastAction, 1000);
}
The controller's action:
def get_last_action_time
last_time = nil
begin
#game = Game.find_by_id(params[:id])
# Return the timestamp of the last action
last_time = (#game && !#game.endTime) ? #game.reloadTime.to_i : 0
rescue ActiveRecord::RecordNotFound
last_time = 0
end
# Stop bugging us after 30m, we should have moved on from this page
last_time==0 if (last_time!=0 && (milliseconds - last_time)>30*60*1000)
render json: {last_action_time: last_time}
end
And then in the html.erb:
<span id='javascript_data_access' game_number=<%= params[:id] %> init_last_action_time=<%= #game.reloadTime %>></span>
Obviously you need to add reloadTime to your model and also endTime if there's a time you no longer want to check for reloads anymore.
Seems to be working fine so far, you have to make sure that you're careful about who is in charge of setting reloadTime. If two pages set reloadTime everytime they reload, you'll be stuck in a reload loop battle between the two pages.

run script when xpages saving document

The xpages contain SAVE button. The xpages also contain InternetAddres field.
When user click SAVE button, need to check first on names.nsf
- Save success if InternetAddress value NOT found in names.nsf view "($Users)"
- Save fail if InternetAddress value found in names.nsf view "($Users)"
How to write the script to do that?
This is the LotusScript version of script:
Set namesview = namesdb.GetView( "($Users)" )
Set namesdoc = namesview.GetDocumentByKey( Lcase(doc.CurrentInternetAddress( 0 ) ), True )
If ( namesdoc Is Nothing ) Then '-- Create New Doc
How to move on xpages?
The latest release of the OpenNTF Domino API adds a checkUnique() method to the View class. It takes two parameters, the first being a key to check against the view (e.g. a String or List of Strings), the second being the current document. After all, if you're checking for a pre-existing document, you don't want to fail just because it finds this document in the view.
So assuming CurrentInternetAddress is a single value field, the code would be:
function continueWithValidUser(namesDB, doc) {
var success = false;
try {
var view = namesDB.getView("($Users)");
success = view.checkUnique(doc.getItemValue("CurrentInternetAddress"),doc);
} catch (e) {
print(e.message);
}
return success;
}
OpenNTF Domino API recycles all handles to Domino objects, so the recycle() calls aren't needed.
In your datasource is a querySave event. You write JS there. It is almost the same code. Just with { } and ;
Remarks:
your app will break when there is more than one address book, so you you would want to use #NameLookup which is quite fast and checks all addressbooks.
unless you need the document getEntry is faster than getDocument
In SSJS your function would look like this:
function continueWithValidUser(namesDB, addressCandidate) {
var success = false;
try {
var view = namesDB.getView("($Users)");
var doc = view.getDocumentByKey(addressCandidate);
success = (doc != null);
doc.recycle();
view.recycle();
} catch (e) {
print(e.message);
}
return success;
}
That should do the trick

Resources