I want to get the top repositories on Github to display on my Rails App. I think I'm half way through.
This is the code I'm using on Google BigQuery:
SELECT repository_name, repository_url, MAX(repository_forks) as total_forks, MAX(repository_watchers) as total_watchers, repository_language
FROM [publicdata:samples.github_timeline]
GROUP BY repository_url, repository_language, repository_name
ORDER BY total_watchers DESC
LIMIT 100
Is there any way I can get this query's results as a JSON so I can parse them in my Rails app?
Thanks in advance.
There's a Ruby client library for BigQuery here:
https://developers.google.com/bigquery/client-libraries
When you run a query, your results are stored in a temporary table (which you can copy to a permanent table if you'd like to keep them for more than 24 hours). You can read the contents of this table using the tabledata.list API:
https://developers.google.com/bigquery/docs/reference/v2/tabledata/list
Related
I am trying to extract a large amount of details out of our Eloqua system using it's API and got this API to work perfectly for single IDs: https://docs.oracle.com/en/cloud/saas/marketing/eloqua-rest-api/op-api-rest-1.0-data-contact-id-get.html
The problem is that I need to run this for a large number of IDs and it will require alot in order to run it for the entire population. Is there any bulk APIs that can extract all of the following details out of Eloqua/Contact for the entire population? I don't see any on that pages documentation that meet this need under the Bulk section.
contactid, company, employees, company_revenue, business_phone, email_address, web_domain, date_created, date_modified, address_1, address_2, city, state_or_province, zip_or_postal_code, mobile_phone, first_name, last_name, title
It's a multi-step process with the Bulk API, typically in the following fashion:
Get a list of the current internal field names - useful for creating your export definition
Create an export definition and post it here. There is a useful example on the page, you do not need a filter criteria. Store the export ID somewhere
Using your export definition id, create a sync. It will gather the data in the background and prepare it for you. Take note of the sync ID provided in the initial response.
Check on the sync status with your sync ID here. It should only take a couple of minutes - and there is a callback url option as well in the previous step, if you don't want to keep polling.
Once your data is ready, use that sync id and request the data. Depending on how many rows were retrieved, you might need to paginate through the results using the offset query param. By default it will give you JSON, but I usually choose CSV (specify in the header).
If you need updated data, feel free to create a new sync using the same export definition id. You do not need to create a new export definition each time.
I have libraries containing way more than 5000 files
When using:
https://graph.microsoft.com/v1.0/sites/3a8180fe-ca21-40a8-81e6-0950f5966f07/lists/DPC Documents/Items?top=499&$expand=fields&$filter=fields/FileLeafRef eq '0100.009_Client Name_Physician Name_FMV Exhibits_466689.XLSX'
I Get error "The request is unprocessable because it uses too many resources"
I can I ask for it to be paged?
BTW equivalent SharePoint REST call returns "The attempted operation is prohibited because it exceeds the list view threshold."
So there is no way to look for items in lists containing more than 5000 items.
I noticed I can't publish major version programmatically in such libraries.
I MS not supporting any API to work with this?
Thanks
This is most probably due to the reason that you have more than 5000 files in the doc library and the filter column FileLeafRef is not indexed (you cannot add index this column either). I encountered same error when filtering with a custom metadata column but fortunately after adding the index to the column, the error went away. The only solution to your problem is not to use FileLeafRef as your filter column but use the Title column in the filter condition, after ensuring that Title column is populated with name of the document and the column is indexed.
I am trying to use the below API to get the list of docker images so that I could populate the dropdown on Jenkins build. Is there a way that this could be listed in a reverse chronological order rather than alphanumerical so that the newest image is at the top ? Thanks.
/artifactory/api/docker/repo/v2/image/tags/list
You will have to take help of Artifactory AQL query language.
An example AQL fragment is...
items.find({"repo":{"$eq":"<repositoryname>"}, "name":{"$eq" : "<artifactoryItemName>"}}) .sort({"$desc" : ["created"]})
The descending sort order is specified with $desc sort operator on the timestamp field created.
You can also limit the number of results returned by adding an extra limit to the above query...
items.find({"repo":{"$eq":"<repositoryname>"}, "name":{"$eq" : "<artifactoryItemName>"}}) .sort({"$desc" : ["created"]}).limit(10)
The AQL needs to be submitted at /artifactory/api/search/aql.
The same can be done via REST API as well with a POST request. The content should be posted not as a JSON but directly in the way query is specified as-is as text. The header for content type is Content-Type:text/plain. You can use the Basic authentication or other supported authentication methods.
There are a ton of things you can do with AQL. The syntax can look a bit confusing to begin with.
I had a project in Redmine with more than 600 issues. I moved all the issues to a different project. I had no idea that the move deletes all the data for the custom fields!
So all the custom field values are now lost. I did not backup the database before this action as I really did not think that I was going to do any harm by moving issues as moving is a native function in the UI.
What I noticed is though that the production.log contains events for all creation and updates. All my 600 issues are in order in the production log. How can I use these log statements to repeat the actions? If I can import all the log actions, I can migrate the custom fields that it writes to the original Redmine instance and restore my values.
Entries look like this:
Processing IssuesController#update (for XX.XX.XX.X at 2013-02-07 11:19:54) [PUT]
Parameters: {"_method"=>"put", "authenticity_token"=>"nWNSSRYjHhN0BGb+Ya8M4pYWPPgsfdM=", "issue"=>{"assigned_to_id"=>"", "custom_field_values"=>{"10"=>"", "5"=>"Not translated", "1"=>"fi", "8"=>"http://screencast.com/t/ODknR8K", "9"=>"", "3"=>"", "4"=>""}, "done_ratio"=>"0", "due_date"=>"", "priority_id"=>"4", "estimated_hours"=>"", "start_date"=>"2013-02-07", "subject"=>"1\tInstallation in English", "tracker_id"=>"1", "lock_version"=>"0", "description"=>"Steps:\r\nOpen Nitro\r\n\r\nProblem:\r\nNot localized"}, "controller"=>"issues", "time_entry"=>{"hours"=>"", "activity_id"=>"", "comments"=>""}, "attachments"=>{"1"=>{"description"=>""}}, "id"=>"3876", "action"=>"update", "commit"=>"Submit", "notes"=>""}
I am really hoping that there is a way, any help will be greatly appreciated
You could use a decent text editor and/or spreadsheet application and do a massive find and replace and construct a series of UPDATE SQL commands and run them directly on the database (TEST FIRST!!)
Extract from log
Remove unnessary information
Copy into spreadsheet
Split text into columns
Add in columns with necessary SQL commands "UPDATE SET etc" copy into all rows of this column etc.
Join columns to make one text command per row
Export joined data to a text file
Run against test database as sql
If all goes well run against production database as sql
The log entry, following "Parameters:", looks like a regular Ruby hash definition. I'd parse that out and eval it back into a hash variable.
From there you will need to peel off elements and insert them into a database. I'd do that using Sequel, but use what works for you.
Talk to the RedMine support people and get the schema for their tables so you can figure out what data goes where and the database driver needed.
i am trying to get access to whitepages using YQL. Unfortunately i don't have much experience with opentables.
I was directed to the whitepages xml file at:
http://github.com/spullara/yql-tables/blob/c63212b2ac9db6feb77ae3cecace51ed52e08c01/whitepages/whitepages.search.xml
Does anyone know how to use this table to extract meaningful information using YQL?
Specifically, I'm not sure how to make a query in YQL using this table to search for a person's name.
Help?
Go to the YQL console http://developer.yahoo.com/yql/console/ and click Show Community Tables. That should make the list on the right much bigger, and will include your table.
have you tried to put
&env=http%3A%2F%2Fdatatables.org%2Falltables.env
at the end of your request?