ksql rest api creation and getting output in json - ksqldb

I am unable to fetch the data from a katable to a pojo class ouput
I have tried to call rest end point of confluent kafka but I am uanble to put the output in a pojo class.
#PostMapping("/QueryForEquipment")
#Consumes("application/json")
public String getEquipments() {
String walletBalanceUrl = "http://172.21.79.18:8088/query";
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.set("Content-Type", "application/json");
httpHeaders.add("Accept", MediaType.APPLICATION_JSON.toString());
JSONObject json = new JSONObject();
json.put("ksql", "Select * from EQP_STREAM limit 10;");
JSONObject jsonSub = new JSONObject();
jsonSub.put("ksql.streams.auto.offset.reset", "earliest");
json.put("streamsProperties", jsonSub);
/* String body=new String("{
"ksql": "Select ROWTIME,ROWKEY,TRN_ID_KEY from TABLE_EQP_LOCATION limit 10;",
"streamsProperties": {"ksql.streams.auto.offset.reset": "earliest"} }");
*/
System.out.println(walletBalanceUrl);
HttpEntity httpEntity = new HttpEntity(body, httpHeaders);
RestTemplate restTemplate = new RestTemplate();
System.out.println("json text====" + body);
ResponseEntity<String> uri = restTemplate.postForEntity(walletBalanceUrl, httpEntity, String.class);
return uri.getBody().toString();
Expected ouput must be in json format
At present it is coming as
{"row":{"columns":[1556458915675,"CDAU603054","Q1214320190418","TRAIN","Arrived","BELLEVILLE","Q","1214","4",1555545600000," ",1555618200000,"SRS_TRAIN","-77.374856","44.179657",null,null,"CDAU603054","CDAU","603054","INTERMODAL","L","KC7","7","DTTX720029","DTTX","720029","C","T","TOP",1555621693617]},"errorMessage":null,"finalMessage":null}
{"row":{"columns":[1556458915680,"CDAU603066","Q1214320190418","TRAIN","Arrived","BELLEVILLE","Q","1214","4",1555545600000," ",1555618200000,"SRS_TRAIN","-77.374856","44.179657",null,null,"CDAU603066","CDAU","603066","INTERMODAL","L","KC7","8","DTTX720030","DTTX","720030","D","T","TOP",1555621693617]},"errorMessage":null,"finalMessage":null}
{"row":{"columns":[1556458915680,"CDAU603070","Q1214320190418","TRAIN","Arrived","BELLEVILLE","Q","1214","4",1555545600000," ",1555618200000,"SRS_TRAIN","-77.374856","44.179657",null,null,"CDAU603070","CDAU","603070","INTERMODAL","L","KC7","8","DTTX720030","DTTX","720030","A","T","TOP",1555621693617]},"errorMessage":null,"finalMessage":null}


Here, Content-Type will be application/vnd.ksql.v1+json and also set charset to utf-8.
HttpHeaders headers = new HttpHeaders();
headers.set("Content-Type", "application/vnd.ksql.v1+json");
headers.set("charset", "utf-8");
headers.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
And the ksql query will be SELECT * from EQP_STREAM EMIT CHANGES limit 10;
json.put("ksql", "SELECT * from EQP_STREAM EMIT CHANGES limit 10;")
For reference check this link : https://docs.ksqldb.io/en/latest/developer-guide/ksqldb-rest-api/ksql-endpoint/

Related

How to upload a small file plus metadata with GraphServiceClient to OneDrive with a single POST request?

I would like to upload small files with metadata (DriveItem) attached so that the LastModifiedDateTime property is set properly.
First, my current workaround is this:
var graphFileSystemInfo = new Microsoft.Graph.FileSystemInfo()
{
CreatedDateTime = fileSystemInfo.CreationTimeUtc,
LastAccessedDateTime = fileSystemInfo.LastAccessTimeUtc,
LastModifiedDateTime = fileSystemInfo.LastWriteTimeUtc
};
using (var stream = new System.IO.File.OpenRead(localPath))
{
if (fileSystemInfo.Length <= 4 * 1024 * 1024) // file.Length <= 4 MB
{
var driveItem = new DriveItem()
{
File = new File(),
FileSystemInfo = graphFileSystemInfo,
Name = Path.GetFileName(item.Path)
};
try
{
var newDriveItem = await graphClient.Me.Drive.Root.ItemWithPath(item.Path).Content.Request().PutAsync<DriveItem>(stream);
await graphClient.Me.Drive.Items[newDriveItem.Id].Request().UpdateAsync(driveItem);
}
catch (Exception ex)
{
throw;
}
}
else
{
// large file upload
}
}
This code works by first uploading the content via PutAsync and then updating the metadata via UpdateAsync. I tried to do it vice versa (as suggested here) but then I get the error that no file without content can be created. If I then add content to the DriveItem.Content property, the next error is that the stream's ReadTimeout and WriteTimeout properties cannot be read. With a wrapper class for the FileStream, I can overcome this but then I get the next error: A stream property 'content' has a value in the payload. In OData, stream property must not have a value, it must only use property annotations.
By googling, I found that there is another way to upload data, called multipart upload (link). With this description I tried to use the GraphServiceClient to create such a request. But it seems that this is only fully implemented for OneNote items. I took this code as template and created the following function to mimic the OneNote behavior:
public static async Task UploadSmallFile(GraphServiceClient graphClient, DriveItem driveItem, Stream stream)
{
var jsondata = JsonConvert.SerializeObject(driveItem);
// Create the metadata part.
StringContent stringContent = new StringContent(jsondata, Encoding.UTF8, "application/json");
stringContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("related");
stringContent.Headers.ContentDisposition.Name = "Metadata";
stringContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
// Create the data part.
var streamContent = new StreamContent(stream);
streamContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("related");
streamContent.Headers.ContentDisposition.Name = "Data";
streamContent.Headers.ContentType = new MediaTypeHeaderValue("text/plain");
// Put the multiparts together
string boundary = "MultiPartBoundary32541";
MultipartContent multiPartContent = new MultipartContent("related", boundary);
multiPartContent.Add(stringContent);
multiPartContent.Add(streamContent);
var requestUrl = graphClient.Me.Drive.Items["F4C4DC6C33B9D421!103"].Children.Request().RequestUrl;
// Create the request message and add the content.
HttpRequestMessage hrm = new HttpRequestMessage(HttpMethod.Post, requestUrl);
hrm.Content = multiPartContent;
// Send the request and get the response.
var response = await graphClient.HttpProvider.SendAsync(hrm);
}
With this code, I get the error Entity only allows writes with a JSON Content-Type header.
What am I doing wrong?

Not sure why the provided error occurs, your example appears to be a valid and corresponds to Request body example
But the alternative option could be considered for this matter, since Microsoft Graph supports JSON batching, the folowing example demonstrates how to upload a file and update its metadata within a single request:
POST https://graph.microsoft.com/v1.0/$batch
Accept: application/json
Content-Type: application/json
{
"requests": [
{
"id":"1",
"method":"PUT",
"url":"/me/drive/root:/Sample.docx:/content",
"headers":{
"Content-Type":"application/octet-stream"
},
},
{
"id":"2",
"method":"PATCH",
"url":"/me/drive/root:/Sample.docx:",
"headers":{
"Content-Type":"application/json; charset=utf-8"
},
"body":{
"fileSystemInfo":{
"lastModifiedDateTime":"2019-08-09T00:49:37.7758742+03:00"
}
},
"dependsOn":["1"]
}
]
}
Here is a C# example
var bytes = System.IO.File.ReadAllBytes(path);
var stream = new MemoryStream(bytes);
var batchRequest = new BatchRequest();
//1.1 construct upload file query
var uploadRequest = graphClient.Me
.Drive
.Root
.ItemWithPath(System.IO.Path.GetFileName(path))
.Content
.Request();
batchRequest.AddQuery(uploadRequest, HttpMethod.Put, new StreamContent(stream));
//1.2 construct update driveItem query
var updateRequest = graphClient.Me
.Drive
.Root
.ItemWithPath(System.IO.Path.GetFileName(path))
.Request();
var driveItem = new DriveItem()
{
FileSystemInfo = new FileSystemInfo()
{
LastModifiedDateTime = DateTimeOffset.UtcNow.AddDays(-1)
}
};
var jsonPayload = new StringContent(graphClient.HttpProvider.Serializer.SerializeObject(driveItem), Encoding.UTF8, "application/json");
batchRequest.AddQuery(updateRequest, new HttpMethod("PATCH"), jsonPayload, true, typeof(Microsoft.Graph.DriveItem));
//2. execute Batch request
var result = await graphClient.SendBatchAsync(batchRequest);
var updatedDriveItem = result[1] as DriveItem;
Console.WriteLine(updatedDriveItem.LastModifiedDateTime);
where SendBatchAsync is an extension method which implements JSON Batching support for Microsoft Graph .NET Client Library

How to set security/Authorisation Token in Post request of Rest Assured using selenium

Please suggest how i can set Security token for below code
RestAssured.baseURI ="http://qa.bridge2capital.com";
RestAssured.port = 9099;
RequestSpecification request = RestAssured.given();
JSONObject requestParams = new JSONObject();
requestParams.put("UserName", "10111");
requestParams.put("Password", "123568");
request.header("Content-Type", "application/json");
request.body(requestParams.toString());
Response response = request.post("/entrepreneur/consolidatedEodTrigger");
int statusCode = response.getStatusCode();
Assert.assertEquals(statusCode, "200");
String successCode = response.jsonPath().get("SuccessCode");
Assert.assertEquals( "Correct Success code was returned", successCode, "OPERATION_SUCCESS");

You can use below code to set basic token:
request.header("Authorization", "Basic " + yourToken);

According to your answer you should try
RequestSpecification request = RestAssured.given();
JSONObject requestParams = new JSONObject();
requestParams.put("UserName", "10111");
requestParams.put("Password", "123568");
request.header("Content-Type", "application/json");
request.header("Authorization", "Basic " + base64encodedToken); //This is the important line
request.body(requestParams.toString());
Response response = request.post("/entrepreneur/consolidatedEodTrigger");
int statusCode = response.getStatusCode();
Assert.assertEquals(statusCode, "200");
String successCode = response.jsonPath().get("SuccessCode");
Assert.assertEquals( "Correct Success code was returned", successCode, "OPERATION_SUCCESS");

Call WebApi method, Error 404 Not Found

Im having a problem with a WebMethod MVC
This is the method in the controller
[HttpPost]
public IHttpActionResult GetData(int Company,
int FromYear,int ToYear, int language ,
int DATA_SERIES_TYPE, bool PERIOD_Q1)
{
var ds = new DataSet();
try
{
ds = Actions.GetValues(Company, FromYear, ToYear,DATA_SERIES_TYPE);
return Ok(ds);
}
catch(Exception ex)
{
return InternalServerError(ex);
}
}
I have this method in an API, when i call the api from another project using webrequest
Dim Request As WebRequest = WebRequest.Create("http://localhost/PBWebApi/api/GetData")
With strJson
.Append("{")
.Append($"Company:{_dynaSolver.CompanyCode.ToString()}" & ",")
.Append($"FromYear:{_dynaSolver.FromYear.ToString}" & ",")
.Append($"ToYear:{_dynaSolver.ToYear.ToString}" & ",")
.Append($"language:{CType(_dynaSolver.DataLanguage, Integer).ToString}" & ",")
.Append($"DATA_SERIES_TYPE:{ CType(_dynaSolver.Fundamentals, Integer).ToString}")
.Append("}")
End With
Dim data = Encoding.UTF8.GetBytes(strJson.ToString)
With Request
.Method = "POST"
.ContentType = "application/json; charset=utf-8"
.ContentLength = data.Length
End With
Dim stream = Request.GetRequestStream()
stream.Write(data, 0, data.Length)
stream.Close()
Dim response = Request.GetResponse().GetResponseStream
Dim reader As StreamReader = New StreamReader(response)
Dim res = reader.ReadToEnd()
dsRes = JsonConvert.DeserializeObject(Of DataSet)(res)
reader.Close()
response.Close()
When i execute this, the response gives this error.
No HTTP resource was found that matches the request URI
I tried using just one parameter in another function but i got the same result.
I tried using fiddler but i got the same error.

I fixed the problem creating and object called id and adding all the properties to that object.
After that i send the object as json in the content and it works.
Regards

Oauth2 Yahoo Gemini API

I am having trouble calling Yahoo Gemini API to access Yahoo Gemini Advertising from my C# console (desktop) application.
Here are steps I used:
Create an installed application on https://developer.yahoo.com/apps/create/. This gave me both {Client ID} and {Client Secret}.
https://api.login.yahoo.com/oauth2/request_auth?client_id={Client ID} &redirect_uri=oob&response_type=code&language=en-us. This will take me to the yahoo login screen where I sign in. Press the Agree button and the next screen shows the seven-letter authorization code (say nzbcns9). I write down this authorization code.
Then I use the following code to try to get the access token:
class Program
{
static void Main(string[] args)
{
string clientId = {Client ID};
string secret = {Client Secret};
var request = WebRequest.Create(#"https://api.login.yahoo.com/oauth2/get_token");
request.Method = "POST";
SetBasicAuthHeader(request, clientId, secret);
string postData = "grant_type=authorization_code&redirect_uri=oob&code=nzbcns9";
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] byte1 = encoding.GetBytes(postData);
request.ContentLength = byte1.Length;
Stream dataStream = request.GetRequestStream();
dataStream.Write(byte1, 0, byte1.Length);
dataStream.Close();
request.ContentType = "application/x-www-form-urlencoded";
var response = request.GetResponse();
Console.WriteLine(((HttpWebResponse)response).StatusDescription);
}
static void SetBasicAuthHeader(WebRequest request, String userName, String userPassword)
{
string authInfo = userName + ":" + userPassword;
authInfo = Convert.ToBase64String(Encoding.Default.GetBytes(authInfo));
request.Headers["Authorization"] = "Basic " + authInfo;
}
}
Then I get
Unhandled Exception: System.Net.WebException: The remote server returned an error: (401) Unauthorized. at System.Net.HttpWebRequest.GetResponse().
What did I do wrong?
I also try to post the same message using Fiddler, I get
{"error":"invalid_request"}

I tried your code and what worked for me was to put the line request.ContentType = "application/x-www-form-urlencoded"; BEFORE Stream dataStream = request.GetRequestStream();
So this worked:
string postData = "grant_type=authorization_code&redirect_uri=oob&code=nzbcns9";
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] byte1 = encoding.GetBytes(postData);
request.ContentLength = byte1.Length;
request.ContentType = "application/x-www-form-urlencoded";
Stream dataStream = request.GetRequestStream();
dataStream.Write(byte1, 0, byte1.Length);
dataStream.Close();

Neither of these worked for me, but it did work once I changed the SetBasicAuthHeader to use ISO-8859-1 encoding:
static void SetBasicAuthHeader( WebRequest request, String userName, String userPassword )
{
string authInfo = userName + ":" + userPassword;
authInfo = Convert.ToBase64String( Encoding.GetEncoding( "ISO-8859-1" ).GetBytes( authInfo ) );
request.Headers[ "Authorization" ] = "Basic " + authInfo;
}

WIndows Phone: POST request doesn't work

i'm tryng to send the notification link and deviceID to a script on the web via a POST request. the problem is that the script receive only empty params.
i followed another example and my code looks like this:
WebClient wc = new WebClient();
System.Diagnostics.Debug.WriteLine("sending rquest");
var URI = new Uri("http://www.jack-prove.comuv.com/update_link.php");
wc.UploadStringCompleted += new UploadStringCompletedEventHandler(wc_UploadStringCompleted);
byte[] myDeviceID = (byte[])Microsoft.Phone.Info.DeviceExtendedProperties.GetValue("DeviceUniqueId");
string ID = Convert.ToBase64String(myDeviceID);
String par = "?link=" + e.ChannelUri.ToString() + "&ID=" + ID;
MessageBox.Show(par);
wc.UploadStringAsync(URI, "POST", par);
NOTE: the ID string contains a '=' charater could this be the problem? i've tried to call the script from the web and it works even with the '='
any idea?

Solved, thanks to this question
using the Http client Library
using System.Net.Http;
var values = new List<KeyValuePair<string, string>>
{
new KeyValuePair<string, string>("link", e.ChannelUri.ToString()),
new KeyValuePair<string, string>("ID", ID)
};
var httpClient = new HttpClient(new HttpClientHandler());
HttpResponseMessage response = await httpClient.PostAsync(URI, new FormUrlEncodedContent(values));
response.EnsureSuccessStatusCode();
var responseString = await response.Content.ReadAsStringAsync();

Resources