Query string with OData linked service in Azure Data Factory - odata

Setting up a OData linked service on public OData like at:
https://www.odata.org/odata-services/
works fine in the Azure Data Factory. Hard coding a url in the OData linked service to:
https://services.odata.org/TripPinRESTierService
from an associated data set shows the entities: People, Airlines, Airports. Previewing is helpful when setting up linked services and data sources. Would be great if that was doable with Azure Search OData endpoints.
The Azure Search requires that the api-version exist in the query string. A query string can not be added until either a Copy or LookUp activity (see documentation). Previewing in the LookUp seems not to take with it the query and the preview fails with:
Invalid or missing api-version query string parameter
Debug doesn't show the url or header with the api-key from linked service nore the path from data source. But the output does show the query (with the api-version):
{
"source": {
"type": "ODataSource",
"query": "api-version=2021-04-30-Preview",
"httpRequestTimeout": "00:05:00"
},
"dataset": {
"referenceName": "AzureODataSearch",
"type": "DatasetReference",
"parameters": {}
}
}
Anybody?

Related

Documenting error codes definition in Swagger API contract

Imagine you are working under following circumstances:
You have REST API modules with API documentation generated into swagger-ui.html
Possible HTTP status codes for endpoints are documented well via io.swagger.annotations.* on controller method level for each endpoint.
In case of some error state (4XX or 5XX) application replies with ErrorResponseDTO with structure like
"ErrorResponseDTO": {
"type": "object",
"properties": {
"errorCode": {
"type": "integer",
"format": "int32"
},
"message": {
"type": "string"
}
}
}
Application have tens of errorCodes (within range like 1 - XX and please consider them as a legacy definiton without an easy way to change them).
List of errorCodes is relevant for the whole application so you prefer to keep their definiton list/table in overall API documentation rather then maintaining errorCodes definiton for each API endpoint separately.
Now you are looking for an effective way how to document these application error codes into API contract.
The approach of including a list of errorCodes with codes description into generated swagger-ui.html seems like a better way to keep API documentation up to date instead of having static and handwritten codes definition table attached in Markdown format in some Readme file.
Would you please have any recommendation how to document various codes which your applications produce?
Do you follow some specific good practice in this topic?
Thank you in advance.
Meanwhile within a small internal dev team and with frequent API extensions there can be used generated swagger-ui with included error codes:
#Bean
public Docket swaggerSettings() {
ApiSelectorBuilder builder = new Docket(DocumentationType.SWAGGER_2)
.apiInfo(
new ApiInfoBuilder()
.title("Response errorCodes table")
.description(RestResponse.generateErrorsDocumentation().toString())
.build()
)
...
.select();
return builder.apis(RequestHandlerSelectors.withClassAnnotation(Api.class))
.paths(PathSelectors.any())
.build()
.useDefaultResponseMessages(false);
}

Retrieve parameter from a Jenkins REST query

The following REST query will return parameters of the last successful build of a job:
https://localhost/job/test1/lastSuccessfulBuild/api/json
I'd be interested to retrieve one of the parameters of this build, the BUILD_VERSION:
{
"_class": "org.jenkinsci.plugins.workflow.job.WorkflowRun",
"actions": [
{
"_class": "hudson.model.CauseAction",
"causes": [
{
"_class": "hudson.model.Cause$UpstreamCause",
"shortDescription": "Started by upstream project \"continuous-testing-pipeline-for-nightly\" build number 114",
"upstreamBuild": 114,
"upstreamProject": "continuous-testing-pipeline-for-nightly",
"upstreamUrl": "job/continuous-testing-pipeline-for-nightly/"
}
]
},
{ },
{
"_class": "hudson.model.ParametersAction",
"parameters": [
{
"_class": "hudson.model.StringParameterValue",
"name": "BUILD_VERSION",
"value": "1.1.15"
Is there a way to retrieve the BUILD_VERSION (1.1.15) directly using the REST Api or do I have to parse manually the json string ?
Thanks
Yeah you can get the value,But it will only work for XML API :(
The JSON API will return a simplified json object using Tree :)
So Jenkins provides you with api (XML,JSON,PYTHON) from which you can read the Jenkins related data of any project. Documentation in detail is provide in https://localhost/job/test1/lastSuccessfulBuild/api
In that it clearly states that
XML API - Use XPath to control the fragment you want.For example, ../api/xml?xpath=//[0]
JSON API - Use tree
Python API - Use st.literal_eval(urllib.urlopen("...").read())
All the above can be used to get a specific fragment/piece from the entire messy data that you get from the API.
In your case, we will use tree for obvious reasons :)
Syntax : tree=keyname[field1,field2,subkeyname[subfield1]]
In order to retrieve BUILD_VERSION i.e. value
//jenkins/job/myjob/../api/json?tree=lastSuccessfulBuild[parameters[value]]
The above should get you what you want, but a bit of trail and error is required :)
You can also refer here for a better understanding of how to use Tree in JSON API
https://www.cloudbees.com/blog/taming-jenkins-json-api-depth-and-tree
Hope it helps :)
Short answer: No.
Easiest way to programmatically access any attribute exposed via the JSON API is to take the JSON from one of Jenkins supported JSON APIs (in your case: https://localhost/job/<jobname>/lastSuccessfulBuild/api/json)
Copy the resultant JSON into http://json2csharp.com
Generate the corresponding C# code. Don't forget to create a meaningful name for top level class.
Call RestAPI programmatically from C# using RestSharp.
Deserialise the json to the C# class you defined in 2 above.
Wammo, you have access to the entire object tree and all its values.
I used this approach to write an MVC5 ASP.NET site I called "BuildDashboard" to provide all the information a development team could want and answered every question they had.
Here is an example with a public jenkins instance and one of its builds in order to get "candidate_revision" parameter for "lastSuccessfulBuild" build:
https://jenkins.qa.ubuntu.com/view/All/job/account-plugins-vivid-i386-ci/lastSuccessfulBuild/parameters/
https://jenkins.qa.ubuntu.com/view/All/job/account-plugins-vivid-i386-ci/lastSuccessfulBuild/api/xml?xpath=/freeStyleBuild/action/parameter[name=%22candidate_revision%22]/value

how to use odata services to create model dynamically from manifest.json

I am very new to UI5, I am working on an application which requires me to create models based on the request made from the browser (client).
If I consume all the odata services beforehand & use them according to the request made, it will become too heavy unnecessarily.
is there any way, this can be done dynamically?
I think your question title and the question content might be contradictory so I am placing my suggestions separately.
how to use odata services to create model dynamically from manifest.json
In your manifest.json file, locate the "sap.app" section/property and then add a datasource as follows:
"dataSources": { //used data sources -> ui5-related information stored in sap.ui5 namespace (unique inside the app)
"modelalias": { //key is alias which is used below in e.g. sap.ui5 ...
"uri": "/sap/opu/odata/snce/PO_S_SRV;v=2/" , //mandatory; version is part of uri, e.g. ";v=2", default is 1
"type": "OData" , //OData (default)|ODataAnnotation|INA|XML|JSON
"settings": { //data-source-type-specific attributes (key, value pairs)
"odataVersion": "2.0" , //possible values: 2.0 (default), 4.0
"annotations": [ "equipmentanno" ], //filled e.g. for Smart Template
"localUri": "model/metadata.xml" //relative url to local metadata
"maxAge": 360 //time in seconds
}
}
To instantiate this model with the alias "mymodel", you can add an entry into the manifest.json under "sap.ui5" as follows:
"models": {
...
"mymodel": { //empty string "" is the default model
"preload": true; //indicator that the model will be created immediately after the manifest is loaded by component factory and before the component instance is created
"dataSource": "modelalias", //reference of dataSource under sap.app - only enhance it with more settings for UI5 if needed
"settings": {
}
}
},
Now the manifest file will instantiate "mymodel" based on your odata uri in "datasources" and then set the model onto your Component.js. So when your application starts, you can access the model in any controller using:
this.getOwnerComponent().getModel("mymodel")
If I consume all the odata services beforehand & use them according to
the request made, it will become too heavy unnecessarily. is there any
way, this can be done dynamically?
Your assumption is that creating a model will slow the app startup. This may not always be true since:
The model creation is very quick
Reading data is what takes time and model instantiation
ODataModels work asyncronously be default so calling .read or .write are operations that can be managed asynchronously
Special case: If you wish to pre-fetch all your data in advance (at startup) I would advise that you ensure that you utilize filters like $select, $top and $skip on your Gateway service to implement growing list like behavior.
Hope that helps you.
More information on manifest.json: Link
Growing List: Link
ODataModel Examples: Link

Contains filter on Microsoft Graph query not working

I'm trying to user the contains filter on a /users query, like this for example:
https://graph.microsoft.com/v1.0/users?$filter=contains(displayName, 'Garth')
However, this results in a BadRequest response saying "An unknown function with name 'contains' was found. This may also be a key lookup on a navigation property, which is not allowed."
According to the OData 4.0 specs, the contains filter should be available though. Is there a way to use a contains filter on a list of users?
The contains function is not available for users. startsWith is available though.
e.g. https://graph.microsoft.com/v1.0/users?$filter=startswith(displayName,'Garth').
Additionally you can try the people API (this is only available on /beta). This supports $search AND will do fuzzy and phonetic matching.
https://graph.microsoft.com/beta/me/people?$search=Garth
$search can replace $contains
for example to show all the users that their names may contain the string "sa"
https://graph.microsoft.com/v1.0/users?$count=true&$search="givenName:sa"
"value": [
{....
"givenName": "Sammy",
.....
},
{
.....
"givenName": "Sabrina",
.......
},
.....

Getting Issue Type Scheme and Workflow Scheme for JIRA project using REST API

I'm doing an integration for JIRA using REST API 6.2.6. One thing that I need to do is to get Issue Type Scheme and Workflow Scheme for a project.
What I tried:
Issue Type Scheme
The only thing that I can get right now is a list of issue types using /rest/api/2/project/{projectIdOrKey}. I can't see any way of getting an ID of Issue Type Scheme. Looking at API there is no any endpoints for issue type schemes, so I guess it's not possible.
Workflow Scheme
/rest/api/2/project/{projectIdOrKey} doesn't return any information about Workflow Scheme. But there is an endpoint /rest/api/2/workflowscheme/{id}, so that means that it's possible to get ID somehow... At the end I want to get a list of workflows for a project to check transitions for an issue type.
Question
Is there any way to get the data I want? Maybe there is some hidden not documented API?
Note: I'm using only JIRA REST API.
This is what you want.
/rest/projectconfig/1/workflowscheme/{projectIdOrKey}
Latest Jira documentation provides information about the APIs which can be used to fetch details for issuetype scheme and workflow scheme. Below are the APIs which can be used for the same,
Fetching Issue type Scheme for a project Issuetype Scheme API
Rest URL:GET https://your-domain.atlassian.com/rest/api/2/issuetypescheme/project?projectId={projectId}'
Sample Response:
{
"maxResults": 100,
"startAt": 0,
"total": 3,
"isLast": true,
"values": [
{
"issueTypeScheme": {
"id": "10000",
"name": "Default Issue Type Scheme",
"description": "Default issue type scheme is the list of global issue types. All newly created issue types will automatically be added to this scheme.",
"defaultIssueTypeId": "10003",
"isDefault": true
},
"projectIds": [
"10000",
"10001"
]
}
]
}
Fetching workflow scheme configured for a project Workflow Scheme API
REST URl: GET https://your-domain.atlassian.com/rest/api/2/workflowscheme/{id}
Sample Response:
{
"id": 101010,
"name": "Example workflow scheme",
"description": "The description of the example workflow scheme.",
"defaultWorkflow": "jira",
"issueTypeMappings": {
"10000": "scrum workflow",
"10001": "builds workflow"
},
"draft": false,
"self": "https://your-domain.atlassian.net/rest/api/2/workflowscheme/101010"
}
As far as I am aware you can get the correct XML or JSON response from the
REST API:
/rest/api/2/project/{projectIdOrKey}.
Then if you want to find out the information about the workflowscheme you can do this programmatically by using the following information.
If you have an issue that you want to use then you can use it to get the workflowscheme id by doing this this:
ComponentAccessor.getWorkflowSchemeManager().getWorkflowScheme(issue.getProject()).get("id");
Then once you get the id of the workflow scheme e.g. 10, you then can get the scheme generic value as follows:
GernericValue scheme = ComponentAccessor.getWorkflowSchemeManager().getScheme(10);
Now that you have the scheme you then can get all of the workflows that are referenced in the scheme by doing this:
Collection<JiraWorkflow> workflows = ComponentAccessor.getWorkflowManager().getWorkflowsFromScheme(scheme);
Then if you want to get one workflow you will have to use:
workflows.iterator().next():
Also note, that workflows are identified by their name in JIRA as there are no id's in JiraWorkflow.
So that would be the approach I would use if I wanted to find out the workflowscheme information so I could use the id to then use the REST API:
But the main reason that you might not be able to find a workflowscheme is because it is not present in the JIRA's Issue.
Using this HTTP and inputting it into "Postman" as a get request will return all the JSON information.
Using this https://jira.atlassian.com/browse/JRA-25121/project/23 will return all the information for that project. Then using this REST API:
/rest/api/2/workflowscheme/{id}
Using this HTTP get request https://jira.atlassian.com/browse/JRA-25121/workflow/45 will get you the returned XML or JSON workflow information too.

Resources