I am trying to use the rdf distiller by Gregg Kollegg:
which works nicely via the web ui:
http://rdf.greggkellogg.net/distiller?command=serialize&url=http:%2F%2Fceur-ws.org%2FVol-2549%2F
now i'd like to get the result via curl and kind find an example/documentation for it.
I have also filed an issue for this at:
https://github.com/gkellogg/rdf-distiller/issues/35
How would the curl command look like to get one of the offered triple formats?
Thanks for your interest in the Ruby RDF Distiller. It's been a useful tool for many years, and has grown to allow access to more formats and functionality from the various Linked Data Ruby gems.
I provided an answer on the Github Issue. If you're using the web interface, as you selection options from the UI, it will update the URL in the "Alternative access to the Distiller" section. To get "raw" output, that bypasses the HTML text box, use the "raw" option, although this seems to work better using "curl".
Also noted in the issue response, is that the entire Distiller service is powered using the "rdf" CLI, which is installed when you install the "rdf" or "linkeddata" ruby gems. (The "linkeddata" gem depends on all of the various gems in the Ruby RDF ecosystem, so provides the fullest functionality).
PRs against the repo to improve documentation or functionality are welcome.
Related
I'm building a rails (rails 6) app that facilitates collaboration on math papers. As the papers are usually written in tex but produce PDFs there may be pictures of blackboards, text note files and sketches I'd like to avoid resolving the merge/source control issues and simply serve a git repository to members of that group of users who want to clone it. But not everyone wants to use the gitcli and the point is to create a single page to manage the collaboration so I also need to be able to display and edit (eg issue commits) directly from the webapp (likely restricted to only fast forwards to master...at least at first).
I suspect there is a gem or engine that allows this and it might even have been answered before here but all my attempts to search just return discussions about using git to version your rails app not using it in the app.
Heck, I'm even willing (at least to hack together a quick working version) to insist that one of the group members setup a GitHub repo and grant some kind of permission to the app (and other users who want direct repo access) if that's a quicker and easier way to hack together a quick solution but I need the rails app to be able to make commits and query the head.
Any suggestion for existing utilities that allow this? If not suggestions on the best way to approach coding that functionality?
This answer (thanks to Int'l man of coding mystery for the pointer) suggests using the library Ruby/Git or Rugged. I haven't had the chance to look closely at the libraries but that solves the problem I had of being unable to find them with search thanks to all the results about putting your ruby code in a git repo.
I have a REST interface.
Now I want to have a client for ruby, javascript, python and java (and maybe more in the future)
Currently I am maintaining 4 different projects and their tests.
How can I improve this and what tools can I use?
currently I am starting to describe the API in a file (e.g. yaml/json) and then generate sources for each language by some logic specific to language, but that is a lot of work, and I am certain I am not the first to run into this issue.
I searched the web for generate source from yaml and such but couldn't find anything
edit: someone suggested I'd use yeoman. so currently I am investigating that.
I believe Postman as a tool is good for that. You can specify an API endpoint and generate clients for multiple languages.
I'm working on a Rails project which will need to interface with multiple third-party APIs. I'm pretty new to Rails, and I've never done this before, so I'm lacking some basic information here. Specifically, What is the preferred Rails way of simply querying an external URL?
In the PHP world, it was cURL. You take whatever the resource URL is, throw cURL at it, and start processing the response, whether it be XML, JSON, etc.
So, what's the cURL equivalent in Rails? While we're at it, what is the preferred method of parsing XML and JSON responses? My instincts are to Google around for some Ruby gems to get the job done, but this is such a practical problem that I wouldn't be surprised if the Rails community had already worked out a tried-and-true solution to this kind of problem.
If it's of any contextual value, I plan to run these third-party API interactions as nightly cronjobs, probably all packaged up as custom rake tasks.
Thanks for sharing your expertise.
In a perfect world, a gem already exists for the API you want to use, and you would just use that. Otherwise, you have a few options:
ActiveResource might make sense for you depending on the complexity of the API you want to use. For example, here's an old (and no longer functional) example of using ActiveResource to connect to the Twitter API
Net::Http is lower-level, but certainly does the trick
open-uri is a wrapper for net/http
Curb uses libcurl to get things done
Parsing JSON is generally very straightforward. For XML, as stated in another answer, Nokogiri is probably the way to go.
for opening urls you can use open-uri
just
require 'open-uri'
file_handle = open("http://google.com/blah.xml")
to parse xml you can use Nokogiri
$ gem install nokogiri
document = Nokogiri::XML(file_handle)
document/"xpath/search"
very powerful library, can do all kinds of searching and modifying for both XML and HTML
same for html Nokogiri::HTML
there is also lots of JSOM support out there too
checkout Nokogiri also Hpricot is good for XML/HTML
for JSON in rails
parsed_json = ActiveSupport::JSON.decode(your_json_string)
parsed_json["results"].each do |longUrl, convertedUrl|
site = Site.find_by_long_url(longUrl)
site.short_url = convertedUrl["shortUrl"]
site.save
end
see this question:
How do I parse JSON with Ruby on Rails?
I want to copy some specific content from a website using ruby/rails.
The content I need is inside a marquee html tag, divided by divs.
How can I get access to this content using ruby?
To be more precise - I want to use some kind of ruby gui (Preferably shoes).
How do I do it?
This isn't really a Rails question. It's something you'd do using Ruby, then possibly display using Rails, or Sinatra or Padrino - pick your poison.
There are several different HTTP clients you can use:
Open-URI comes with Ruby and is the easiest. Net::HTTP comes with Ruby and is the standard toolbox, but it's lower-level so you'd have to do more work. HTTPClient and Typhoeus+Hydra are capable of threading and have both high-level and low-level interfaces.
I recommend using Nokogiri to parse the returned HTML. It's very full-featured and robust.
require 'nokogiri'
require 'open-uri'
doc = Nokogiri::HTML(open('http://www.example.com'))
puts doc.to_html
If you need to navigate through login screens or fill in forms before you get to the page you need to parse, then I'd recommend looking at Mechanize. It relies on Nokogiri internally so you can ask it for a Nokogiri document and parse away once Mechanize retrieves the desired URL.
If you need to deal with Dynamic HTML, then look into the various WATIR tools. They drive various web browsers then let you access the content as seen by the browser.
Once you have the content or data you want, you can "repurpose" it into text inside a Rails page.
If I'm to understand correctly, you want a GUI interface to a website scraper. If that's so, you might have to build one yourself.
The easiest way to scrape a website is using nokogiri or mechanize gems. Basically, you will give those libraries the address of the website and then use their XPath capabilities to select the text out of the DOM.
https://github.com/sparklemotion/nokogiri
https://github.com/sparklemotion/mechanize (for the documentation)
Ive seen mentions of curl here and there on rails blogs and ive scanned some posts here on stackoverflow but im still a bit in the dark as to its use especially when it comes to rails development.
Is it useful for testing? Im currently learning the ins and outs of testing and one of the things i need to do is test a before filter that only allows an action to be called if the user came from a certain external site. Is this an occasion where curl would be used?
curl is a command line program which can retrieve urls, it's quite flexible, for example it can use limited regexes to download a range of files.
You might want to use it for testing, by seeing what happens to your application under certain circumstances, if you want to test what happens if your user comes from a certain site then use the -e option on curl.
The man page is online Here
No, I don't think you'd use cURL for TDD. Using cURL would imply communicating with a remote service, which would be an integration test.
Are you talking about this? http://www.viget.com/extend/curl-and-your-rails-2-app/
In that post, cURL isn't being used in the rails app at all. It's being used on the terminal to test/demo the app's REST service, not in a TDD fashion.
cURL is good for this purpose because its many options allow you to simulate different headers from user agents, such as language acceptance, characterset acceptance, cache usage, etc...
Have a look at hurl—which is written in Rails—and then imagine a more powerful command-line version. That's pretty much cURL.
Ruby (Rail?) hackers might also be interested in one of the many Ruby bindings for libcurl, the library that powers the curl tool. Like them mentioned here: http://curl.haxx.se/libcurl/ruby/