On a scheduled basis, I would like to export Volusion orders by a date range:
select * from orders o where o.OrderDate >= '7/20/2015' AND o.OrderDate <= '7/23/2015'
Is this possible? It appears my URL can only do an equals sign:
https://www.XX.net/net/WebService.aspx?Login=shopxperts#yahoo.com&EncryptedPassword=XX&EDI_Name=Generic\Orders&SELECT_Columns=*&WHERE_Column=o.orderdate&WHERE_Value=7/18/2015 10:58:09 AM
I looked at the SQL saved query feature. Is there a way to save a query with parameters, then fill them in?
Yes you can insert parameters into the query and execute it from the Generic folder.
Using a custom ASP page, this is a summary of what you would have to do
1- Read the querystring(s) and sanitizing them.
2- Construct the SQL query with sanitized parameters
3- Write the SQL and XSD files to the "Generic" folder
4- execute the now written file in the generic folder by making a http request
https://www.XX.net/net/WebService.aspx?Login=shopxperts#yahoo.com&EncryptedPassword=XX&EDI_Name=Generic\xyz
5- Delete the files since it's no longer needed once the query is complete.
6- Return data from the query to the page requesting the data
Obviously, this is an very abridged version of what you would need to do but it is certainly possible.
Related
I am quite new in ADF so thats why i am asking you for any suggestions.
The use case:
I have a csv file which contains unique id and url's (see image below). i would like to use this file in order to export the value from various url's. In the second image you can see a example of the data from a url.
So in the current situation i am using each url and insert this manually as a source from the ADF Copy Activity task to export the data to a SQL DB. This is very time consuming method.
How can i create a ADF pipeline to use the csv file as a source, and that a copy activity use each row of the url and copy the data to Azure SQL DB? Do i need to add GetMetaData activity for example? so how?
Many thanks.
use a look up activity that reads all the data,Then use a foreach loop which reads line by line.Inside foreach use a copy activity where u can able to copy response to the sink.
In order to copy XML response of URL, we can use HTTP linked service with XML dataset. As #BeingReal said, Lookup activity should be used to refer the table which contains all the URLs and inside for each activity, Give the copy activity with HTTP as source and sink as per the requirement. I tried to repro the same in my environment. Below are the steps.
Lookup table with 3 URLs are taken as in below image.
For-each activity is added in sequence with Lookup activity.
Inside For-each, Copy activity is added. Source is given as HTTP linked service.
In HTTP linked service, base URL, #item().name is given. name is the column that stored URLs in the lookup table. Replace the name with the column name that you gave in lookup table.
In Sink, azure database is given. (Any sink of your requirement is to be used). Data is copied to SQL database.
this is the dataset HTTP inside the copy activity
This is the input of the Copy Activity inside the for each
this is the output of the Copy Activity
My sink is A Azure SQL Database without any tables yet. i would like to create auto table on the fly from ADF. Dont understand why this error came up
I am trying to extract a large amount of details out of our Eloqua system using it's API and got this API to work perfectly for single IDs: https://docs.oracle.com/en/cloud/saas/marketing/eloqua-rest-api/op-api-rest-1.0-data-contact-id-get.html
The problem is that I need to run this for a large number of IDs and it will require alot in order to run it for the entire population. Is there any bulk APIs that can extract all of the following details out of Eloqua/Contact for the entire population? I don't see any on that pages documentation that meet this need under the Bulk section.
contactid, company, employees, company_revenue, business_phone, email_address, web_domain, date_created, date_modified, address_1, address_2, city, state_or_province, zip_or_postal_code, mobile_phone, first_name, last_name, title
It's a multi-step process with the Bulk API, typically in the following fashion:
Get a list of the current internal field names - useful for creating your export definition
Create an export definition and post it here. There is a useful example on the page, you do not need a filter criteria. Store the export ID somewhere
Using your export definition id, create a sync. It will gather the data in the background and prepare it for you. Take note of the sync ID provided in the initial response.
Check on the sync status with your sync ID here. It should only take a couple of minutes - and there is a callback url option as well in the previous step, if you don't want to keep polling.
Once your data is ready, use that sync id and request the data. Depending on how many rows were retrieved, you might need to paginate through the results using the offset query param. By default it will give you JSON, but I usually choose CSV (specify in the header).
If you need updated data, feel free to create a new sync using the same export definition id. You do not need to create a new export definition each time.
I'm on a project in which I need to get data from a ServiceNow instance and treat all data with PowerBI. I'm able to retrieve a big amount of data (Snow collect a lot of data), but I still need a way to filter data correctly. I'm calling to this URL:
Besides, I want also to apply a filter to retrieve just some specific registries from the table Requested Items. For that, I use the sysparm_query parameter to filter the field "cmdb_ci" and more specifically it's "name", something like:
&sysparm_query=cmdb_ci=What I need to Filter
Apart from this, I have also tried:
&sysparm_query=cmdb_ci.value=What I need to Filter
&sysparm_query=cmdb_ci.display_value=What I need to Filter
&sysparm_query=cmdb_ci.sys_id=What I need to Filter
&sysparm_query=cmdb_ci.name=What I need to Filter
But still not found the solution... as all these does not respond the solution needed.
Does someone know how I can manage this?
Thanks!!
JLG
There are two "Configuration item" fields in sc_req_item: cmdb_ci and configuration_item. Make sure that you are using the correct one. Either cmdb_ci.name=value or configuration_item.name=value should work, depending on which of the two fields you are using. cmdb_ci.value and cmdb_ci.display_value will not work as there are no fields on the record with these names. cmdb_ci.sys_id should work if you are supplying a sys_id (although it is redundant to type .sys_id for a reference field).
You should first verify your query through the ServiceNow UI before attempting to use it in an API call.
Type sc_req_item.list in the Filter navigator
On the filter list select "Show related fields"
Get your filter to work correctly
Right-click on the filter and select "Copy query"
The next step is to test it using the REST API Explorer.
Final step is to configure your client tool (PowerBI).
There are APIs to query using an existing wiq file.
I want my app to create actual wiq files in a shared folder - which can be accessed outside my app too. But I cannot find any API to do that.
What's that mean for wiql files here?
WIQL is Work Item Query Language, if you mean create a query with WIQL and save the query as a file, then no any API to do that, however you can try below steps:
Create a query with the Query Editor, save the query.
Open the query in Visual Studio --> Edit Query
File -> Save {Query}[Editor] as...
Then you can open and edit the saved *.wiq file with text editor, then save/copy to a shared folder...
Reference below articles:
Export a query
Quickly generate Work Item Query for use in the TFS API
Yes, there is an API, see Create a query in the REST API.
The POST body should contains the WIQL Query and the name for it.
{
"name": "All Bugs",
"wiql": "Select [System.Id], [System.Title], [System.State] From WorkItems Where [System.WorkItemType] = 'Bug' order by [Microsoft.VSTS.Common.Priority] asc, [System.CreatedDate] desc"
}
To invoke the REST API, I suggest you to use a Microsoft library instead of writing all the code yourself, e.g. https://learn.microsoft.com/en-us/vsts/integrate/concepts/dotnet-client-libraries.
My iOS application fetches some photos, tags and comments from web server. I want it to fetch only changed or new added data. I don't want it to fetch repeated data again and again.
I use SDWebImage for pictures. But text are based on SQL text.
How could I understand the result of the SQL is changed or not? What kind of technique
should I use?
Is there a third party library for client side SQL catching?
I think it is not iOS related question technically. You should query always with the last queried timestamp.
Like:
SELECT * FROM comments WHERE last_modified > last_queried_timestamp;
last_modified field should store the timestamp of the last modification date or the timestamp of creation and the last_queried_timestamp parameter is the timestamp of the last date when you queried from the server.
This way you will not get twice the same changes.
(Unless you want it)