Is there a way to decode tinyURL links in R so that I can see which web pages they actually refer to?
Below is a quick and dirty solution, but should get the job done:
library(RCurl)
decode.short.url <- function(u) {
x <- try( getURL(u, header = TRUE, nobody = TRUE, followlocation = FALSE) )
if(class(x) == 'try-error') {
return(u)
} else {
x <- strsplit(x, "Location: ")[[1]][2]
return(strsplit(x, "\r")[[1]][1])
}
}
The variable 'u' below contains one shortend url, and one regular url.
u <- c("http://tinyurl.com/adcd", "http://www.google.com")
You can then get the expanded results by doing the following.
sapply(u, decode.short.url)
The above should work for most services which shorten the URL, not just tinyURL. I think.
HTH
Tony Breyal
I don't know R but in general you need to make a http request to the tinyurl-url. You should get back a 301 response with the actual url.
I used Tony Breyal's code, but the function returned NA values for those URLs where there was no URL redirection. Even though Tony listed "google.com" in his example, I think Google redirects you in any case to some sort of localized version of google.com.
Here is how I modified Tony's code to deal with that:
decode.short.url <- function(u) {
x <- try( getURL(u, header = TRUE, nobody = TRUE, followlocation = FALSE) )
if(class(x) == 'try-error') {
print(paste("***", u, "--> ERORR!!!!"))
return(u)
} else {
x <- strsplit(x, "Location: ")[[1]][2]
x.2 <- strsplit(x, "\r")[[1]][1]
if (is.na(x.2)){
print(paste("***", u, "--> No change."))
return(u)
}else{
print(paste("***", x.2, "--> resolved in -->", x.2))
return(x.2)
}
}
}
u <- list("http://www.amazon.com", "http://tinyurl.com/adcd")
urls <- sapply(u, decode.short.url)
library(RCurl)
decode.short.url <- function(u) {
x <- try( getURL(u, header = TRUE, nobody = TRUE, followlocation = FALSE) )
if(class(x) == 'try-error') {
return(u)
} else {
x <- strsplit(x, "Location: ")[[1]][2]
return(strsplit(x, "\r")[[1]][1])
}
}
( u <- c("http://tinyurl.com/adcd", "http://tinyurl.com/fnqsh") )
( sapply(u, decode.short.url) )
Related
How can we delete incoming or outgoing links for a particular objects from a particular modules through dxl script.
this code is for deleting every incoming and outgoing links for all objects from current module. It is not tested, but you can start from here.
Object o
Link lk
for o in current Module do
{
for lk in o -> "*" do // for outcoming links
{
delete(lk)
}
for lk in o <- "*" do // for incoming links
{
delete(lk)
}
}
Advice: When you ask something post what you tried to do in order to solve the problem.
EDIT:
This code is tested and deletes outcoming and incoming links from a specified object from a specified module.
Folder f = current
Module m
string object_id = <hardcoded_id_or_from_somewhere>
string module_name = <same_as_object_id>
for m in f do
{
if(name(m) == module_name)
{
Object o
for o in m do
{
if(identifier(o) == object_id)
{
Link lk
for lk in o -> "*" do // for outcoming links
{
delete(lk)
}
for lk in o <- "*" do // for incoming links
{
delete(lk)
}
}
}
}
}
Result example:
{collisions=0, rx_bytes=258, rx_crc_err=0, rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=3, tx_bytes=648, tx_dropped=0, tx_errors=0, tx_packets=8}
This format is like JSON, but not JSON.
Is there an easy way to parse this into map[string]int? Like json.Unmarshal(data, &value).
If that transport format is not recursively defined, i.e. a key cannot start a sub-structure, then its language is regular. As such, you can soundly parse it with Go's standard regexp package:
Playground link.
package main
import (
"fmt"
"regexp"
"strconv"
)
const data = `{collisions=0, rx_bytes=258, rx_crc_err=0, rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=3, tx_bytes=648, tx_dropped=0, tx_errors=0, tx_packets=8}`
const regex = `([a-z_]+)=([0-9]+)`
func main() {
ms := regexp.MustCompile(regex).FindAllStringSubmatch(data, -1)
vs := make(map[string]int)
for _, m := range ms {
v, _ := strconv.Atoi(m[2])
vs[m[1]] = v
}
fmt.Printf("%#v\n", vs)
}
Regexp by #thwd is an elegant solution.
You can get a more efficient solution by using strings.Split() to split by comma-space (", ") to get the pairs, and then split again by the equal sign ("=") to get the key-value pairs. After that you just need to put these into a map:
func Parse(s string) (m map[string]int, err error) {
if len(s) < 2 || s[0] != '{' || s[len(s)-1] != '}' {
return nil, fmt.Errorf("Invalid input, no wrapping brackets!")
}
m = make(map[string]int)
for _, v := range strings.Split(s[1:len(s)-1], ", ") {
parts := strings.Split(v, "=")
if len(parts) != 2 {
return nil, fmt.Errorf("Equal sign not found in: %s", v)
}
if m[parts[0]], err = strconv.Atoi(parts[1]); err != nil {
return nil, err
}
}
return
}
Using it:
s := "{collisions=0, rx_bytes=258, ...}"
fmt.Println(Parse(s))
Try it on the Go Playground.
Note: If performance is important, this can be improved by not using strings.Split() in the outer loop, but instead searching for the comma "manually" and maintaining indices, and only "take out" substrings that represent actual keys and values (but this solution would be more complex).
Another solution...
...but this option is much slower so it is only viable if performance is not a key requirement: you can turn your input string into a valid JSON format and after that you can use json.Unmarshal(). Error checks omitted:
s := "{collisions=0, rx_bytes=258, ...}"
// Turn into valid JSON:
s = strings.Replace(s, `=`, `":`, -1)
s = strings.Replace(s, `, `, `, "`, -1)
s = strings.Replace(s, `{`, `{"`, -1)
// And now simply unmarshal:
m := make(map[string]int)
json.Unmarshal([]byte(s), &m)
fmt.Println(m)
Advantage of this solution is that this also works if the destination value you unmarhsal into is a struct:
// Unmarshal into a struct (you don't have to care about all fields)
st := struct {
Collisions int `json:"collisions"`
Rx_bytes int `json:"rx_bytes"`
}{}
json.Unmarshal([]byte(s), &st)
fmt.Printf("%+v\n", st)
Try these on the Go Playground.
Below is a simplified version of my code. I'm simply trying to get Shiny to pass an input value to an rmongodb query, run the query based on the user input, and then plot the mean of a variable. The code below includes everything needed to replicate my issue, including insertion of documents into the collection.
I'd be very grateful for any help! I'm pulling my hair out (and there wasn't much left to begin with). I suspect that I'm placing the reactive() command inappropriately or something along those lines.
Many thanks to whoever can provide assistance.
#Install required packages and call each
library(devtools)
install_github(repo = "mongosoup/rmongodb")
library(rmongodb)
library(shiny)
#Establish connection with mongodb, check status, name database and collection, insert some documents, return one document
mongo <- mongo.create()
mongo.insert(mongo, "simpledb.main",'{"user":"Joe", "age":34}')
mongo.insert(mongo, "simpledb.main",'{"user":"Joe", "age":31}')
mongo.insert(mongo, "simpledb.main",'{"user":"Joe", "age":53}')
mongo.insert(mongo, "simpledb.main",'{"user":"Kate", "age":29}')
mongo.insert(mongo, "simpledb.main",'{"user":"Lisa", "age":21}')
mongo.insert(mongo, "simpledb.main",'{"user":"Henry", "age":34}')
mongo.insert(mongo, "simpledb.main",'{"user":"David", "age":43}')
if(mongo.is.connected(mongo) == TRUE) {
help("mongo.count")
mongo.count(mongo, "simpledb.main")
}
if(mongo.is.connected(mongo) == TRUE) {
mongo.find.one(mongo, "simpledb.main")
}
#Code needed for Shiny UI
ui <- fluidPage(
fluidRow(
column(2, textInput(inputId = "userName", label = "", value = "Enter name here"))),
mainPanel(plotOutput(outputId = "main_plot"))
)
#Code needed for Shiny server
server <- function(input, output) {
queryReactive <- reactive({
nameFinal <- paste0(input$userName)
query = mongo.bson.buffer.create()
mongo.bson.buffer.append(query, "user", nameFinal)
query = mongo.bson.from.buffer(query)
})
#Run the query and store results as an R list object
queryresults <- mongo.find.all(mongo=mongo, ns = coll, query=queryReactive)
#Convert the R list object into a data frame
resultsdf <- data.frame(matrix(unlist(queryresults), nrow=length(queryresults), byrow=T), stringsAsFactors=FALSE)
output$main_plot <- renderPlot({boxplot(as.numeric(resultsdf$X3))})
}
#Code needed to call Shiny UI and server
shinyApp(ui = ui, server = server)
There is no need for a reactive command in your server function. I have simplified and corrected your function below:
server <- function(input, output) {
output$main_plot <- renderPlot({
nameFinal <- paste0(input$userName)
query = mongo.bson.buffer.create()
mongo.bson.buffer.append(query, "user", nameFinal)
query = mongo.bson.from.buffer(query)
queryresults <- mongo.find.all(mongo=mongo, ns = "simpledb.main", query=query)
if (length(queryresults) > 0) {
resultsdf <- data.frame(matrix(unlist(queryresults), nrow=length(queryresults), byrow=T), stringsAsFactors=FALSE)
boxplot(as.numeric(resultsdf$X3))
}
else boxplot(c(0))
})
}
I was wondering how to set the width of an rChart using % as opposed to px. I noticed in the source code that it defaults to pixels. I'm trying to use it in a shiny app, and fixed with charts seem to be an issue as they don't scale with the rest of the user interface. Is there a way around this?
This should be considered a hack, since neither rCharts or most of the libraries have built in responsive behavior. However, one change would be to define your own renderChart function with this behavior. Something like this might work if you define in the server piece of shiny.
renderChart_pct <- function(expr, env = parent.frame(), quoted = FALSE) {
func <- shiny::exprToFunction(expr, env, quoted)
function() {
rChart_ <- func()
cht_style <- sprintf("<style>.rChart {width: %s; height: %s} </style>",
#### change these here to desired %
"100%", "100%")
cht <- paste(capture.output(rChart_$print()), collapse = '\n')
HTML(paste(c(cht_style, cht), collapse = '\n'))
}
}
Then use renderChart_pct instead of renderChart2. You'll probably notice that the result will not be truly responsive. Altogether, here would be an example with dPlot.
library(shiny)
library(rCharts)
renderChart_pct <- function(expr, env = parent.frame(), quoted = FALSE) {
func <- shiny::exprToFunction(expr, env, quoted)
function() {
rChart_ <- func()
cht_style <- sprintf("<style>.rChart {width: %s; height: %s} </style>",
"100%", "100%")
cht <- paste(capture.output(rChart_$print()), collapse = '\n')
HTML(paste(c(cht_style, cht), collapse = '\n'))
}
}
ui = shinyUI(fluidPage(
fluidRow(
column(4,
h1("Filler Header")
)
,column(8,
div (
showOutput("myChart", "dimple")
)
)
)
))
server = function(input,output){
output$myChart <- renderChart_pct({
d = dPlot(
x~x
, data = data.frame(x=1:10)
, type = "line"
, height = 500
, width = NULL
)
})
}
runApp( list(ui=ui,server=server))
I'm reading email headers (in Node.js, for those keeping score) and they are VARY varied. E-mail addresses in the to field look like:
"Jake Smart" <jake#smart.com>, jack#smart.com, "Development, Business" <bizdev#smart.com>
and a variety of other formats. Is there any way to parse all of this out?
Here's my first stab:
Run a split() on - to break up the different people into an array
For each item, see if there's a < or ".
If there's a <, then parse out the email
If there's a ", then parse out the name
For the name, if there's a ,, then split to get Last, First names.
If I first do a split on the ,, then the Development, Business will cause a split error. Spaces are also inconsistent. Plus, there may be more e-mail address formats that come through in headers that I haven't seen before. Is there any way (or maybe an awesome Node.js library) that will do all of this for me?
There's a npm module for this - mimelib (or mimelib-noiconv if you are on windows or don't want to compile node-iconv)
npm install mimelib-noiconv
And the usage would be:
var mimelib = require("mimelib-noiconv");
var addressStr = 'jack#smart.com, "Development, Business" <bizdev#smart.com>';
var addresses = mimelib.parseAddresses(addressStr);
console.log(addresses);
// [{ address: 'jack#smart.com', name: '' },
// { address: 'bizdev#smart.com', name: 'Development, Business' }]
The actual formatting for that is pretty complicated, but here is a regex that works. I can't promise it always will work though. https://www.rfc-editor.org/rfc/rfc2822#page-15
const str = "...";
const pat = /(?:"([^"]+)")? ?<?(.*?#[^>,]+)>?,? ?/g;
let m;
while (m = pat.exec(str)) {
const name = m[1];
const mail = m[2];
// Do whatever you need.
}
I'd try and do it all in one iteration (performance). Just threw it together (limited testing):
var header = "\"Jake Smart\" <jake#smart.com>, jack#smart.com, \"Development, Business\" <bizdev#smart.com>";
alert (header);
var info = [];
var current = [];
var state = -1;
var temp = "";
for (var i = 0; i < header.length + 1; i++) {
var c = header[i];
if (state == 0) {
if (c == "\"") {
current.push(temp);
temp = "";
state = -1;
} else {
temp += c;
}
} else if (state == 1) {
if (c == ">") {
current.push(temp);
info.push (current);
current = [];
temp = "";
state = -1;
} else {
temp += c;
}
} else {
if (c == "<"){
state = 1;
} else if (c == "\"") {
state = 0;
}
}
}
alert ("INFO: \n" + info);
For something complete, you should port this to JS: http://cpansearch.perl.org/src/RJBS/Email-Address-1.895/lib/Email/Address.pm
It gives you all the parts you need. The tricky bit is just the set of regexps at the start.