I want to fetch data from a KSQL query result from a java service. Can you please suggest the way? Is it to be done via KSQL REST API?
Your wording is a bit ambiguous, so I give you two answers:
You want to query KSQL (has the information) from an external service (needs the information). Here, you'd use the KSQL REST API as you suggested.
You want to query an external service (has the information) from KSQL (needs the information). Here, you could write a KSQL User Defined Function (UDF), which are implemented in Java, and call your external Java service through it: https://docs.confluent.io/current/ksql/docs/developer-guide/udf.html.
Related
I have been trying to build a pipeline in Google Cloud Data Fusion where data source is a 3rd party API endpoint. I have been unable to successfully use the HTTP Plugin, but it has been suggested that I use Pub/Sub for the data ingest.
I've been trying to follow this tutorial as a starting point, but it doesn't help me out with the very first step of the process: ingesting data from API endpoint.
Can anyone provide examples of using Pub/Sub -- or any other viable method -- to ingest data from an API endpoint and send that data down to Data Fusion for transformation and ultimately to BigQuery?
I will also need to be able to dynamically modify the URI (e.g., date filter parameters) in the GET request in this pipeline.
In order to achieve the first step in the tutorial you are following
Ingest CSV (Comma-separated values) data to BigQuery using Cloud Data Fusion.
You need to set up a functioning pub/sub system. This can be done via the command line, the console, or in your case the best would be to use, one of the client libraries. If you follow this tutorial you should have a functioning pub/sub system.
At that point you should be able to follow the original tutorial
At the moment i still only have about 2 months of experience in UI5. i developed a little sample-app, used sap gateway builder to pass my requests to sap backend.
Now my employer asked me to research the possibility to access two different backends (one sap, one nonsap) via odata from the same app. After a little reading and thinking i came to the conclusion that it would be best to access both backends from a single gateway.
Since ive already worked with sap gateway, i wonder if there is a way to access nonsap backends with sap gateway? Are the better options?
Or is my current approach complete wrong and i should think about a whole other way?
It depends on your approach and the non-sap-system:
Is the non-sap-system accesible via Webservices? Then use second data model (e.g. JSON/ODATA) within SAPUI5 by loading data via webservices after initial loadup of your application.
Is the non-sap-system connected to SAP? E.g. via RFC or another technology, then you can read data from the other system during calling your initial Gateway service and simply call your RFC function module in your method.
From my opinion you will not achieve an 'easy' way to read both via one single SAP NetWeaver Gateway.
Not sure why you would want to access a non-SAP oData service via SAP Gateway. On the other hand you may want a router of some sort so that all services are exposed on the same network location and then incoming requests are routed to the appropriate backend for action.
You may also want to "mash-up" the SAP and non-SAP services into some sort of new service. In that case maybe look to some of the API management tools like Apigee to help you achieve that.
Looking at data access for a multitenant SaaS database for a UI5 app I have two choices - JSON or OData. OData has some features of interest. However, I have concluded that I cannot use OData because of security issues.
In particular I am concerned that OData is a form of query language. As a very rough comparison this is like a SQL statement being composed at the client and sent to an ODBC server for processing. In a multitenant DB I will separate data between owners using, say, and org_id. If it were a SQL query I would have to include select data from table where org_id=this_tennant and .... If I can discover another tenants org_id then I can use an OData explorer to modify the query and pass into the OData source.
In a JSON data interface I would call a REST method that would mask how the data is accessed, and I would not need to expose secret details such as the org_id token. Granted the JSON / REST approach involves more work.
Or do I misunderstand OData ?
Another similar question was asked some time ago but some time has passed so I shall ask again.
The approach of OData is to define how RESTful APIs can be build. It adds a lot of concepts and conventions (e.g. entity types and sets, URL conventions, common query parameters, request format, batch support...) to plain REST and therefore provides you a standard for building your API.
However, it does not define where your data is coming from, but in most cases it will be database. Of course it would be possible to expose your database tables via an OData service in a generic way, but I would expect that the provider of the framework clearly describes how to handle and protect different tenants.
A more specific answer would require more information about the framework you will use.
I need some conceptual understanding of Neo4j REST api's usage. I went through the tutorial of neo4j RESt api and got some idea. So is it like these REST api's are used only to CREATE nodes and Relationships and also to fetch data from the neo4j graph database? So can you give me one example/scenario where and how these neo4j REST api's are used?
Thanks,
Shree
Historically the very granular REST API calls for e.g. creating nodes, setting properties were there before Cypher was established. Cypher is a way more concise and compact way to query and modify the graph compared to the granular REST calls. These days you typically use Cypher over the transactional http endpoint.
For some usage examples refer to the manual as well.
I would like to clarify how data exactly flows from on-premis or internet data source to GoodData platform during local execution of the graphs. Case with local data source I think is obvious, but data source outside LAN is not so when executed locally.
Yuya,
This depends on the reader and connection type. If using a REST or SOAP connector it uses that protocol over HTTP to read the data for example. Certain technology ones are hard coded like SFDC using their REST API or ExactTarget using their SOAP API. Besides the HTTP protocol we support FTP/SFTP, WebDAV, and JDBC for connections.
http://developer.gooddata.com/docs/data-loading provides an overview of data loading with a bunch of related articles / tutorials linked to further down the page. There is also a CloudConnect User Guide and Data Loading Admin Guide that might help.
Is there a specific type of data pull you are interested in understanding further?