One of my Vapor app endpoints needs to be able to receive arbitrary JSON and output it to the logs. Once I have the logs, I can go back through and set up Codables structs and do the typical Vapor workflow.
In your route handler do print("\(req)") and you'll see the data
Doing this requires solving two problems that Vapor apps don't normally contend with:
Where to get the HTTP body (and how to decode it)?
How to return a future outside the context of the usual helpers?
The simplest solution I found gets the body from req.http.body.data, converts the data to JSON using JSONSerialization.jsonObject(with:options), and returns a Future using req.eventLoop.newSucceededFuture:
router.put("printanyjson") { req -> Future<HTTPStatus> in
if let data = req.http.body.data {
if let json = try? JSONSerialization.jsonObject(with: data, options: []) {
print("\(json)")
}
}
return req.eventLoop.newSucceededFuture(result: .ok)
}
Note #1: This solution does not work if body is streaming. I'd like to see a solution that incorporates this idea.
Note #2: It's also possible to make recursive Codables that can decode any structure, allowing you to stay within the rails of typical Vapor usage. I'd like to see a solution that incorporates this idea.
Related
I use gatling to send data to an ActiveMQ. The payload is generated in a separate method. The response should also be validated. However, how can I access the session data within the checks
check(bodyString.is()) or simpleCheck(...)? I have also thought about storing the current payload in a separate global variable, but I don't know if this is the right approach. My code's setup looks like this at the moment:
val scn = scenario("Example ActiveMQ Scenario")
.exec(jms("Test").requestReply
.queue(...)
.textMessage{ session => val message = createPayload(); session.set("payload", payload); message}
.check(simpleCheck{message => customCheck(message, ?????? )})) //access stored payload value, alternative: check(bodystring.is(?????)
def customCheck(m: Message, string: String) = {
// check logic goes here
}
Disclaimer: providing example in Java as you don't seem to be a Scala developper, so Java would be a better fit for you (supported since Gatling 3.7).
The way you want to do things can't possibly work.
.textMessage(session -> {
String message = createPayload();
session.set("payload", payload);
return message;
}
)
As explained in the documentation, Session is immutable, so in a function that's supposed to return the payload, you can't also return a new Session.
What you would have to do it first store the payload in the session, then fetch it:
.exec(session -> session.set("payload", createPayload()))
...
.textMessage("#{payload}")
Regarding writing your check, simpleCheck doesn't have access to the Session. You have to use check(bodyString.is()) and pass a function to is, again as explained in the documentation.
I am trying integrate the new Twitter API specifically the streaming tweets part in my android app, I am using Retrofit for my http calls.
When I try to make the call to get the streaming tweets it just hangs and does not return anything.
this is my retrofit call
#Streaming
#GET("tweets/search/stream")
suspend fun getFilteredStream(#Header("Authorization") token:String)
I then tried making a call with just OkHttp as shown in the documentation I get a successful response but I dont know how to stream the data.
I can make the call successfully via a curl call and see the data no problem.
How do I stream the data via retrofit or OkHttp
Update:
With OkHttp I was able to get data by doing this
val client: OkHttpClient = OkHttpClient().newBuilder()
.build()
val request: Request = Request.Builder()
.url("https://api.twitter.com/2/tweets/search/stream")
.method("GET", null)
.build()
val response: Response = client.newCall(request).execute()
val source = response.body?.source()
val buffer = Buffer()
while(!source!!.exhausted()){
response.body?.source()?.read(buffer, 8192)
val data = buffer.readString(Charset.defaultCharset())
}
data holds the string data representation of multiple tweet objects but how do I read one tweet at a time, or parse the response like this?
From the docs, I think you'll need to combine the two examples you have.
https://github.com/square/retrofit/blob/108fe23964b986107aed352ba467cd2007d15208/retrofit/src/main/java/retrofit2/http/Streaming.java
Treat the response body on methods returning {#link ResponseBody ResponseBody} as is, i.e. without converting the body to {#code byte[]}.
And example calling code
https://github.com/square/retrofit/blob/108fe23964b986107aed352ba467cd2007d15208/retrofit/src/test/java/retrofit2/CallTest.java#L599
I suspect, dependending on the JSON API you use you may be able to use a Streaming API from the response body. But if not you could split on either just newline or newline followed by { on next line and parse individually. Sorry I can't help here.
See https://en.wikipedia.org/wiki/JSON_streaming#Concatenated_JSON
I am trying to create websocket server and client in my iOS app, which i successfully managed to do with the help of sample implementation here. (https://github.com/apple/swift-nio/tree/master/Sources/NIOWebSocketServer) - so current working situation is, i run the websocket server when app launches and then I load the client in a webview which can connect to it.
Now my problem is I want my server to secured websocket server (Basically connect to the websocket server from a HTTPS html page)
I am new to network programming and Swift-nio documentation is lacking to say the least. As far as I understand I could use (https://github.com/apple/swift-nio-transport-services)
I found this thread which is exactly what I need - https://github.com/apple/swift-nio-transport-services/issues/39 - I could disable the TLS authentication as I dont care in my usecase as long as I could get the websocket connected.
So my question is how to I extend my client (https://github.com/apple/swift-nio/tree/master/Sources/NIOWebSocketClient) and server (https://github.com/apple/swift-nio/tree/master/Sources/NIOWebSocketServer) to use swift-nio-transport-service.
I could add the NIOSSLContext and stuff but I think I need to add the EventLoopGroup and new bootstrap methods. I know the answers is right there.... but I just cannot seem to pinpoint it.
Any pointer would be appreciated.
Thanks.
To translate a simple NIO Server to a NIOTransportServices one, you need to make the following changes:
Add a dependency on NIOTransportServices to your server.
Change MultiThreadedEventLoopGroup to NIOTSEventLoopGroup.
Change ClientBootstrap to NIOTSConnectionBootstrap.
Change ServerBootstrap to NIOTSListenerBootstrap.
Build and run your code.
Some ChannelOptions don’t work in NIOTransportServices, but most do: the easiest way to confirm that things are behaving properly is to quickly test the common flow.
This doesn’t add any extra functionality to your application, but it does give you the same functionality using the iOS APIs.
To add TLS to either NIOTSConnectionBootstrap or NIOTSListenerBootstrap, you use the .tlsOptions function. For example:
NIOTSListenerBootstrap(group: group)
.tlsOptions(myTLSOptions())
Configuring a NWProtocolTLS.Options is a somewhat tricky thing to do. You need to obtain a SecIdentity, which requires interacting with the keychain. Quinn has discussed this somewhat here.
Once you have a SecIdentity, you can use it like so:
func myTLSOptions() -> NWProtocolTLS.Options {
let options = NWProtocolTLS.Options()
let yourSecIdentity = // you have to implement something here
sec_protocol_options_set_local_identity(options.securityProtocolOptions, sec_identity_create(yourSecIdentity)
return options
}
Once you have that code written, everything should go smoothly!
As an extension, if you wanted to secure a NIO server on Linux, you can do so using swift-nio-ssl. This has separate configuration as the keychain APIs are not available, and so you do a lot more loading of keys and certificates from files.
I needed a secure websocket without using SecIdentity or NIOTransportServices, so based on #Lukasa's hint about swift-nio-ssl I cobbled together an example that appears to work correctly.
I dunno if it's correct, but I'm putting it here in case someone else can benefit. Error-handling and aborting when the try's fail is left out for brevity.
let configuration = TLSConfiguration.forServer(certificateChain: try! NIOSSLCertificate.fromPEMFile("/path/to/your/tlsCert.pem").map { .certificate($0) }, privateKey: .file("/path/to/your/tlsKey.pem"))
let sslContext = try! NIOSSLContext(configuration: configuration)
let upgradePipelineHandler: (Channel, HTTPRequestHead) -> EventLoopFuture<Void> = { channel, req in
WebSocket.server(on: channel) { ws in
ws.send("You have connected to WebSocket")
ws.onText { ws, string in
print("Received text: \(string)")
}
ws.onBinary { ws, buffer in
// We don't accept any Binary data
}
ws.onClose.whenSuccess { value in
print("onClose")
}
}
}
self.eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: 2)
let port: Int = 5759
let promise = self.eventLoopGroup!.next().makePromise(of: String.self)
_ = try? ServerBootstrap(group: self.eventLoopGroup!)
// Specify backlog and enable SO_REUSEADDR for the server itself
.serverChannelOption(ChannelOptions.backlog, value: 256)
.serverChannelOption(ChannelOptions.socketOption(.so_reuseaddr), value: 1)
.childChannelInitializer { channel in
let handler = NIOSSLServerHandler(context: sslContext)
_ = channel.pipeline.addHandler(handler)
let webSocket = NIOWebSocketServerUpgrader(
shouldUpgrade: { channel, req in
return channel.eventLoop.makeSucceededFuture([:])
},
upgradePipelineHandler: upgradePipelineHandler
)
return channel.pipeline.configureHTTPServerPipeline(
withServerUpgrade: (
upgraders: [webSocket],
completionHandler: { ctx in
// complete
})
)
}.bind(host: "0.0.0.0", port: port).wait()
_ = try! promise.futureResult.wait()
try! server.close(mode: .all).wait()
I'm building an app that needs to make a GET request to the API endpoint https://thecountedapi.com/api/counted using the Siesta framework. The endpoint returns a JSON array, just like an endpoint like https://api.github.com/users/ebelinski/repos, which is used in the Siesta example Github Browser. As a result, I'm trying to make my app use Siesta in the say way that one does. I create a service:
let API = Service(baseURL: "https://thecountedapi.com/api")
Then a transformer for my endpoint in application:didFinishLaunchingWithOptions:
API.configureTransformer("/counted") {
($0.content as JSON).arrayValue.map(Incident.init)
}
Where Incident is a struct with an initializer that takes in a JSON object.
Then in my view controller, I create a resource:
let resource = API.resource("/counted")
and in viewDidLoad:
resource.addObserver(self)
and in viewWillAppear:
resource.loadIfNeeded()
Then I have the following function in my VC to listen to changes:
func resourceChanged(resource: Resource, event: ResourceEvent) {
print(resource.jsonArray)
if let error = resource.latestError {
print(error.userMessage)
return
}
if let content: [Incident] = resource.typedContent() {
print("content exists")
incidents = content
}
print(incidents.count)
}
But when I run my app, I get mixed results. print(resource.jsonArray) just prints [], I have an error message Cannot parse server response, and if I set Siesta.enabledLogCategories = LogCategory.detailed, I can see the error mesage [Siesta:StateChanges] Siesta.Resource(https://thecountedapi.com/api/counted)[] received error: Error(userMessage: "Cannot parse server response", httpStatusCode: nil, entity: nil, cause: Optional(Siesta.Error.Cause.WrongTypeInTranformerPipeline(expectedType: "JSON", actualType: "__NSCFArray", transformer: Siesta.ResponseContentTransformer<SwiftyJSON.JSON, Swift.Array<TheCountedViewer.Incident….
If I comment out the whole transformer, I have some success in that print(resource.jsonArray) prints out the correct array from the endpoint. So my transformer must be wrong in some way, but I think I'm using basically the same transformer as in Github Browser:
service.configureTransformer("/users/*/repos") {
($0.content as JSON).arrayValue.map(Repository.init)
}
Am I missing something?
The key clue to your problem is buried in that (perhaps not ideally helfpul) log message:
Siesta.Error.Cause.WrongTypeInTranformerPipeline
expectedType: "JSON"
actualType: "__NSCFArray"
It’s saying that your transformer expected an input type of JSON, which makes sense — you said as much with ($0.content as JSON). However, it got the type __NSCFArray, which is the secret internal backing type for NSArray. In other words, it expected a SwiftyJSON value, but it got the raw output of NSJSONSerialization instead.
Why? The GithubBrowser project includes an NSDict/NSArray → SwiftyJSON transformer which it configures in the parsing stage. The model transformers in that project all depend on it.
To use SwiftyJSON in the same way in your project, you’ll need to include that transformer from the example project in yours:
private let SwiftyJSONTransformer =
ResponseContentTransformer
{ JSON($0.content as AnyObject) }
And then when setting up your service:
service.configure {
$0.config.pipeline[.parsing].add(SwiftyJSONTransformer, contentTypes: ["*/json"])
}
(Note that you might want to create the ResponseContentTransformer with transformErrors: true if you are interested in errors.)
An alternative way to use SwiftyJSON, which is not quite as pretty but requires less setup, is to manually wrap things in JSON in each individual response transformer:
service.configureTransformer("/users/*/repos") {
JSON($0.content as AnyObject).arrayValue.map(Incident.init)
}
I cannot consistently successfully send form variables that may/may not include special characters e.g. ? & #
Depending on where I try to escape the chars I encounter different errors when reading the data server-side.
I am aware that an update is due for React Native 0.7 to include formdata but wondered if I could safely post objects without needing this.
Someone has already posted a similar issue but no example code was posted to illustrate the POST working:
How to post a form using fetch in react native?
I have tried - amongst other things :
fetch(APIURL, {
method: 'POST',
body: JSON.stringify({
object1: {
param1a: "value 1a",
param1b: "value 1b - with bad chars & # ?",
},
object2:
{
param2a: "value 2a",
param2b: 0,
}
})
})
but it groups the data into a single unnamed parameter (changing the API to accept this is not an option).
also this:
fetch(APIURL, {
method: 'GET',
accessPackage: JSON.stringify({
accessToken: "abc123",
tokenType: 2,
}),
taggData: JSON.stringify({
title: "test",
wishlistID: 0,
anotherVar: "anotherVal"
})
})
I wish to receive the data as two strings that can be parsed as json objects at the other end.
Looking at the the fetch repo https://github.com/github/fetch hasn't helped as this assumes the post with be a full JSON post (which it isn't) or uses FormData which isn't available to React Native yet.
Another solution may be to safely encode/serialise all of the data to URL parameters but this has also proven inconsistent so far especially with the # char.
What's the best way to do this?
"it groups the data into a single unnamed parameter (changing the API
to accept this is not an option)."
It would, because you've set the post body. This is how it's supposed to work.
I wish to receive the data as two strings that can be parsed as json objects at the other end.
You can do whatever you want, but there's no magic happening here. You will receive a single string, the string you set body to. Likewise, the post body can contain anything but shouldn't get confused with "special" characters. Most likely it is your server-side that is causing the problems.
If you want to use FormData then I think it'll be in v0.7.0 which should be out any day now, or you could probably just include the JS file in your own project. You can find it here. Usage examples are in the UIExplorer demo.