Azure Data Factory V2 : Error using the REST connector - "Failed to get schema from" - restful-url

I am doing a POC to call restFul API using ADF CopyData.
The restFul API details are available on this link
http://www.groupkt.com/post/f2129b88/free-restful-web-services-to-consume-and-test.htm
from this page, i want to rest below service
http://services.groupkt.com/country/get/iso3code/IND
The linked service to base URL is successfully created in ADF
http://services.groupkt.com/country/get
in the next steps when I try to lookup data or do next steps i get error
Error using the REST connector - "Failed to get schema from"
Let me know if you have come across this issue in ADF V2

I was able to make this POC work by changing the relative URL which i was initially giving "/iso3code/IND"
Below is how the URLs should be
Absolute URL should be http://services.groupkt.com/country/get : this is used for creating the Linked Service of the CopyData
Relative URL should be http://services.groupkt.com/country/get/iso3code/IND
This is used for getting the mapping details of the Restful service

Related

Configuring Mule Microsoft Power BI connector using OAUTH

I'm trying to connect to Microsoft Power BI using the Mule Power BI connector but the connector configuration is very confusing with respect to OAuth. Has anyone successfully connected to Power BI with Mule? Here is my config:
<power-bi-rest:config name="Power_Bi_Rest_Connector_Config" doc:name="Power Bi Rest Connector Config" doc:id="d9dbee0f-50fe-46ca-977e-2b3c6ce1ff3b" >
<power-bi-rest:oauth2-authorization-code-connection >
<power-bi-rest:oauth-authorization-code consumerKey="xxx" consumerSecret="yyy" authorizationUrl="https://login.microsoftonline.com/69f2560d-6d2a-4768-989e-d35ed85aeab2/oauth2/v2.0/authorize" accessTokenUrl="https://login.microsoftonline.com/69f2560d-6d2a-4768-989e-d35ed85aeab2/oauth2/v2.0/token" scopes="api://96eb9d1c-6dfb-4acf-9950-d4af04038693/Mule"/>
<power-bi-rest:oauth-callback-config listenerConfig="HTTP_Listener_config" callbackPath="https://login.microsoftonline.com/common/oauth2/nativeclient" authorizePath="https://login.microsoftonline.com/common/oauth2/nativeclient" />
</power-bi-rest:oauth2-authorization-code-connection>
</power-bi-rest:config>
I have not specifically used Power BI but I can tell the what the issue probably is.
The oAUthCallbackConfig is where you will need to make the changes.
External Callback URL: While configuring the OAuth credentials in your MS account, you must have provided a callback URL that should be pointing to your Mule application. You need to put that URL here.
Callback Path: Here you need to mention where the Mule App should be expecting to get the call back with the authorization code. For example of you have configured external path as https://localhost:8082/oauth/callbackthis parameter could be oauth/callback. (Note that you will need to consider the base path that you may have configured in the http listener)
Authorize path: This should not be a URL pointing to a Microsoft api, but it is the path that you will use to hit you mule app to start the oAuth dance. You can put something like /oauth/authorize. Then when you try https://localhost:8082/oauth/authorize the mule app will trigger the oAuth dance and will redirect you to the login screen of everything else is setup correctly.

OData in Datafactory

I have a task toget some data from an external supplier.
They have a Rest OData API. I have to connect using a subscription-key(APIKey).
When creating the OData LService, I add an Auth Header: "subscription-key" and in the Value field, I enter my key. After saving, I create a new dataset, and the OData LinkedService, provides me with the remote tables. I can choose the table I want and after that I create a pipeline to copy data from that table to my Azure SQL server.
This works fantastic :-)
However, after closing my browser and re-open it, the subscription key that I have entered earlier on the linked service, is now replaced with stars as it is a securestring. When I now run my pipeline, it will think that my key is the ten stars that have replaced my real key.
What am I doing wrong here ?
Also I would prefer to get my value from the KeyVault, but it seems that this is not possible on ODat connections....
Hope someone is able to provide some insight here :-)
BR Tom
From my testing I did not get any error on re-running. However coming to dynamic keys - I was not able to achieve it using the ODATA linked service.
Alternatively, if you can hit the ODATA endpoint with REST / HTTP Connector
You could - have a Web Activity to get the keys from the Key Vault and Set in the Variable.
WEB Activity URL : https://<your-keyvalut-name>.vault.azure.net/secrets/<your-secret-name>;
You could access the output of the web Activity using : #activity('Web1').output.value & Store in a variable.
You can reference this variable as the SUBSCRIPTION KEY for the subsequent steps in the REST/HTTP dataset.
You could pass it along the additional headers

Getting 'unauthorized_client' with Power Platform Custom Connector using OAuth2

I am trying to build a Custom Connector in the Power Platform to connect to the BMC Helix (formerly Remedy) system to create work orders and such. I am using OAuth2 and was given a callback URL, auth URL, token URL, client ID and client secret.
I went to create a connector from scratch. I populated the fields, but I wasn't sure what to put for the 'Refresh URL', so I used the token URL there too.
I am trying to accomplish testing this connector and my successful test would be to get a JWT from doing a POST to the /api/jwt/login endpoint of BMC Helix. It should return a JWT which I can use to make subsequent calls.
Upon testing this, I go to create a connection, but a window opens (which I believe should be a prompt for authentication), but instead it contains an error saying 'unauthorized_client' coming back from the BMC Helix system at the /rsso/oauth2/authorize endpoint. It also contains a property within the URL of redirect_uri = https://global.consent.azure-apim.net/redirect.
Is there something on the Helix side I need to further configure? Not sure why I am getting this....
It sounds like you need TWO METHODS in your connector. A POST to call the token server, a GET (or another POST) to call the API (using the token received from Call 1).
One approach I've successfully used in the past is:
Use Postman to get your token server call working with OAUTH
Then use Postman to get your subsequent API calls working with the token appended
Save both requests to a single Postman collection
Export the Postman collection (as a V1 (deprecated) if I recall correctly)
Import this collection into PowerApps Custom Connector (create new/import from Postman Collection)
You'll have to massage it a bit after import, but it will give you a good headstart and you're starting from a known-good place (working Postman calls)
Good luck!

Trying to get detailed data dumps from new google nest e thermostat

I bought a nest thermostat as I thought it would be able to give me detailed data to showing the target temp and the actual as well as time etc. I needed this for various reasons.
However, it seems the official API "Works with Nest" was closed by Google. I've been able to get postman to ping the same location that the Google Nest Webapp hits and get back the data I need. I want to create a simple webapp to keep polling and save the data locally. However, I'm unable to find the OAuth Client Secret that the Nest Webapp uses to get the authorization code. I had to login via the webapp to get the code in one of the request and then test it out using postman.
Is there any other API that will allow my to poll this data for my Nest easier?
If there isn't another API, is there a way to get the Client ID and Client Secret form the Nest Webapp so I can drop that in mine to use? (I know its hacky, but am I think I'm out of options)

Breeze.js - + Facing issue while accessing service metata

Breeze.js - + Facing problem while accessing service metata.
Request for metadata is sent to
http://mylocalmachine:4625/WcfDataService1.svc/Metadata
but metadata is getting published at http://mylocalmachine:4625/WcfDataService1.svc/$Metadata
Your Breeze client must be configured to communicate with your remote service. As I read your question, it appears that you're trying to reach an OData data source whose data service name is "http://mylocalmachine:4625/WcfDataService1.svc".
If so, you'll have to tell Breeze to switch from its default Web Api dataService adapter to the OData dataService adapter. Do that during your application bootstrapping with a line such as:
breeze.config.initializeAdapterInstance("dataService", "OData");
Now, when you run your app and look at the network traffic, you should see that Breeze requests metadata with something like
http://mylocalmachine:4625/WcfDataService1.svc/$Metadata
There is a prerequisite JavaScript library that you will need to include on the page in order to use the OData adapater.
It's mentioned in the documentation:- dataJS.
Hope this helps.

Resources