There does not appear to be any method in the Python client API for Google's storage transfer service that checks the status of an ongoing transfer operation. There is get_transfer_job, which shows the status of a transfer job itself (and gives the latest operation name). But I can't find any way of getting the status of an actual operation, which is critical.
I know other languages' client APIs (including at least Go and node.js) have this functionality. It may be possible to use a naked REST API request, but we're running into authentication issues. Is there any other way that I'm missing? Any way to call the TransferOperation type directly (such as client.TransferOperation(<transfer_operation_name>)?
There is method available for the same which you can use in the following manner.
GetTransferJobRequest(mapping=None, *, ignore_unknown_fields=False, **kwargs)
Reference Link - Class GetTransferJobRequest
Related
I have a large JAVA application which connects to hundreds of cloud based systems using their REST API's and fetch the data from those systems.
To connects those different cloud systems we have different modules and each one have different approach to call REST API's like some modules using apache rest client some module using googles rest client.
So there is no centralise place where the REST api is getting called.
I have to track performance of the application e.g. to fetch accounts info from test system takes 1 hour. and this process need
4 api calls for https://test/api/v2/accounts -- (this will return all account id's)
8000 api calls for https://test/api/v2/accounts/{accountId}. --- (this will return deaths of each account)
I need to track what is the time taken by each api to responds and based on that calculate time taken by application to process that data.
Important part here is deatiled api analysis and make graphical data if possible e.g.
4 api calls for https://test/api/v2/accounts --- taken 3 minutes
8000 api calls for https://test/api/v2/accounts/{accountId} -- taken
48 minutes
I need any any pointer how can I achieve that something like intercept all rest api made to https://test/api/v2
As you've probably already discovered, without some extra tweaking, wireshark just shows you the connections at the FQDN level: you can't see which individual endpoint is called (because TLS, by design, hides the content of the connection). You have a few options though:
if you control the APIs that are being connected to, you can load the
TLS keys into wireshark, and it'll let you decrypt the TLS
connection;
if you can force your app to use a proxy, you can use a Man-In-The-Middle (MITM) proxy (like Burp) to intercept the traffic; or
you can instrument your app to log destination and duration for all the API requests.
I'm using Google Tink for encryption across network calls and wondering about when the proper time is to register my keyset. AeadConfig.register()
Since this is a backend application, is it correct to call this just once at server start up. Or is it correct to call it each time before I make my API call that encrypts the request?
I see the documentation calling out to make this call, but no information about when it should be made, or if it should/can be called multiple times.
I have spent my afternoon getting very excited about the container-native serverless platform 'fn project' - http://fnproject.io/.
I love the idea of the FaaS model but have no intention of locking myself into a particular cloud vendor for most of the lifetime of an app - and several other reasons including the desire to spin up the entire app on a small server anywhere if I choose.
fn project seems great for my needs until I finish perusing the documentation and all the relevant blog posts and suddenly think 'what? Wait....what??? Where are the http operations?'.
I cannot find a single reference anywhere that states if it is even possible to to have http triggers for different http operations (ie POST, PUT, PATCH, DELETE), let alone how I would do it.
I want to build REST api's (or certainly at the very least json-serving http-based RPC apis - if it doesn't have hypermedia links it isn't REST ;) but let's not get into that one in this thread)
Am I missing something here (certainly the correct bit of documentation)??
Can anybody please enlighten me as to how I would do this, or even tell me if I have totally misunderstood what I should use this for?
My excitement has gone soft for now but I'm hoping someone that will change with the right information.
It feels odd that I can't find anyone else complaining about this, so I think that indicates my misunderstanding perhaps.
Other solutions such as OpenFaaS look interesting but I dont wan't to have to learn how to deploy kubernetes and docker swarms if I can avoid it :)
I'm not an expert, but as of now it seems not possible to specify the http method inside the trigger. Check latest trigger spec : as you can see, there is no notion of http method here.
However, handling different HTTP methods can be done inside the function itself.
For example, in Java (with fdk-java v1.0.80), you can use com.fnproject.fn.api.httpgateway.HTTPGatewayContext as the first parameter of the function, as described in the section "Accessing HTTP Information From Functions" of the documentation :
In Fn for Java, when your function is being served by an HTTP trigger (or another compatible HTTP gateway) you can get access to both the incoming request headers for your function by adding a 'com.fnproject.fn.api.httpgateway.HTTPGatewayContext' parameter to your function's parameters.
Using this allows you to :
...
Access the method and request URL for the trigger
...
You can then retrieve the HTTP method by calling getMethod() on the HTTPGatewayContext passed as parameter.
In other languages (with others fdk), it's possible to do the same :
in Go : example calling RequestMethod() on context
in Ruby : class HTTPContext
in Python : class HTTPGatewayContext
in Node : class HTTPGatewayContext
From this different contexts, you'll then be able to get method parameter passed when fn invoke --method=[GET|POST|...] (via fn-http-method header).
The main drawback here is that all HTTP methods should be handled in the same function. Nonetheless, you can structure your code to have only one class per method.
After some further thought it seems fairly clear now what my actual misunderstanding was....
When I have built Serverless framework services in the past (or built and deployed Lambda functions using terraform) I have been deploying to AWS and so have been using AWS's API Gateway offering (their product is actually called API gateway but its important to recognise that API Gateway is a distributed systems / micro-sevices design pattern).
API gateway makes it possible to route specific http request types including the method (GET,POST,PUT,DELETE) to the desired functions.
Platforms such as Fn project and OpenFaaS do not provide an out of the box api gateway solution and it seems we would need to take care of this ourselves.
These above mentioned platforms are about deployment of functions. We find the other bits via our product of choice.
I am completely dependent on Rest Kit for my app for network calls. I want to see the logs of how much
1) Time taken by each API to get a response
2) Size of Request/Response Payload
3) URL of the API
Is there any way I can enable such logging in Restkit. My app is calling like 50-60 API an dI don't want to dig into entire code base an add manual logs. Also I don't want to use network profiling tool since I will be tracking this data when an actual user is using the application.
Cant also use any third party paid tool so want to log these values in application database.
RestKit does have a log you can enable, but that isn't what you want to do if you plan to actually release this. It also writes to the log, not a value you can actually process and save.
Your likely best option is to subclass the RKObjectManager and intercept the requests that are being placed and the NSURLRequests which are being generated.
When I make API calls to the server, I'm getting 404 errors for various data -- grades, role IDs, terms -- that I won't get on the next time I call it. The data's there on the server, viewable by the same user, and is often returned successfully, but not every time. The same user context will return data successfully for other calls.
Any ideas what could be causing this?
I'm using the Valence API with the Python client library and our 9.4.1 SP18 instance of Desire2Learn in a non-interactive script.
more detail: the text it returns on the bad 404s is " ErrorThe system cannot find the path specified."
It would help enormously to gather data about your case: packet traces that can show successful calls from your client alongside unsuccessful calls, in particular, would be very useful to see. If you are quite certain (and I see no reason you shouldn't be from your description) that you're forming the calls in the right way each time you make them, then the kind of behaviour you're noticing would seem to speak to some wider network or configuration issue: sometimes your calls are properly getting through the web service layer, and sometimes they are not -- this would seem therefore not to be down to the way you're using the API but in the way the service is able to receive that request.
I would encourage you, especially if you can gather data to provide showing this behaviour, to open a support incident with Desire2Learn's help desk in conjunction with your Approved Support Contact, or your Partner Manager (depending on whether you're a D2L client or a D2L partner).