While wokring with gprc in dart, if the response type of the very first rpc call is a streaming response the client app fails to connect to the server when stream handler is envoked. I found the issue while building upon the helloworld example from the package.
Is there any way to ensure that the connection is established? Or is there is anything I am doing incorrectly?
I have tried it with await channel.getConnection(); but it makes no difference.
grpc version: 3.0.2
helloworld.proto:
// Copyright 2015 gRPC authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";
option java_multiple_files = true;
option java_package = "io.grpc.examples.helloworld";
option java_outer_classname = "HelloWorldProto";
option objc_class_prefix = "HLW";
package helloworld;
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
rpc SayHelloStream (HelloRequest) returns (stream HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
server.dart:
// Copyright (c) 2018, the gRPC project authors. Please see the AUTHORS file
// for details. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/// Dart implementation of the gRPC helloworld.Greeter server.
import 'package:grpc/grpc.dart';
import 'package:helloworld/src/generated/helloworld.pbgrpc.dart';
class GreeterService extends GreeterServiceBase {
#override
Stream<HelloReply> sayHelloStream(
ServiceCall call, HelloRequest request) async* {
for (var i = 0; i < 10; i++) {
yield HelloReply()..message = 'Hello, ${request.name}!';
await Future.delayed(Duration(seconds: 1));
}
}
#override
Future<HelloReply> sayHello(ServiceCall call, HelloRequest request) async {
return HelloReply()..message = 'Hello, ${request.name}!';
}
}
Future<void> main(List<String> args) async {
final server = Server(
[GreeterService()],
const <Interceptor>[],
CodecRegistry(codecs: const [GzipCodec(), IdentityCodec()]),
);
await server.serve(port: 50051);
print('Server listening on port ${server.port}...');
}
client.dart
// Copyright (c) 2018, the gRPC project authors. Please see the AUTHORS file
// for details. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/// Dart implementation of the gRPC helloworld.Greeter client.
import 'package:grpc/grpc.dart';
import 'package:helloworld/src/generated/helloworld.pbgrpc.dart';
Future<void> main(List<String> args) async {
final channel = ClientChannel(
'localhost',
port: 50051,
options: ChannelOptions(
credentials: ChannelCredentials.insecure(),
codecRegistry:
CodecRegistry(codecs: const [GzipCodec(), IdentityCodec()]),
),
);
final stub = GreeterClient(channel);
final name = args.isNotEmpty ? args[0] : 'world';
try {
// works if this call is made first
// final response = await stub.sayHello(HelloRequest()..name = name);
// print('Greeter client received: ${response.message}');
// this has no effect
// await channel.getConnection();
final responseStream = stub.sayHelloStream(
HelloRequest()..name = name,
);
// This doesn't work standalone.
responseStream
.listen((value) => print('Greeter client received: ${value.message}'));
// Works when using await for
// await for (var value in responseStream) {
// print('Greeter client received: ${value.message}');
// }
} catch (e) {
print('Caught error: $e');
}
await channel.shutdown();
}
Expected result: It should have worked correctly and printed Greeter client received: ${value.message}' 10 times at 1 second interval.
Actual result: On running client.dart the following error is recieved.
gRPC Error (code: 14, codeName: UNAVAILABLE, message: Error connecting: Connection shutting down., details: null, rawResponse: null, trailers: {})
On adding the following lines (as shown in comments) there are no issues and the result is printed 1 + 10 times as expected.
// final response = await stub.sayHello(HelloRequest()..name = name);
// print('Greeter client received: ${response.message}');
You should only shutdown the channel when you're done with the stream. In your case you shutdown the channel immediately so there's no way gRPC would keep updating the stream as you've already shutdown the connection.
Related
I'm trying to send multiple packets at once to a server, but the socket keeps "merging" all sync calls to write as a single call, I did a minimal reproducible example:
import 'dart:io';
void main() async {
// <Server-side> Create server in the local network at port <any available port>.
final ServerSocket server =
await ServerSocket.bind(InternetAddress.anyIPv4, 0);
server.listen((Socket client) {
int i = 1;
client.map(String.fromCharCodes).listen((String message) {
print('Got a new message (${i++}): $message');
});
});
// <Client-side> Connects to the server.
final Socket socket = await Socket.connect('localhost', server.port);
socket.write('Hi World');
socket.write('Hello World');
}
The result is:
> dart example.dart
> Got a new message (1): Hi WorldHello World
What I expect is:
> dart example.dart
> Got a new message (1): Hi World
> Got a new message (2): Hello World
Unfortunately dart.dev doesn't support dart:io library, so you need to run in your machine to see it working.
But in summary:
It creates a new tcp server at a random port.
Then creates a socket that connects to the previous created server.
The socket makes 2 synchronous calls to the write method.
The server only receives 1 call, which is the 2 messages concatenated.
Do we have some way to receive each synchronous write call in the server as separated packets instead buffering all sync calls into a single packet?
What I've already tried:
Using socket.setOption(SocketOption.tcpNoDelay, true); right after Socket.connect instantiation, this does modify the result:
final Socket socket = await Socket.connect('localhost', server.port);
socket.setOption(SocketOption.tcpNoDelay, true);
// ...
Using socket.add('Hi World'.codeUnits); instead of socket.write(...), also does not modify the result as expected, because write(...) seems to be just a short version add(...):
socket.add('Hi World'.codeUnits);
socket.add('Hello World'.codeUnits);
Side note:
Adding an async delay to avoid calling write synchronously:
socket.add('Hi World'.codeUnits);
await Future<void>.delayed(const Duration(milliseconds: 100));
socket.add('Hello World'.codeUnits);
make it works, but I am pretty sure this is not the right solution, and this isn't what I wanted.
Environment:
Dart SDK version: 2.18.4 (stable) (Tue Nov 1 15:15:07 2022 +0000) on "windows_x64"
This is a Dart-only environment, there is no Flutter attached to the workspace.
As Jeremy said:
Programmers coding directly to the TCP API have to implement this logic themselves (e.g. by prepending a fixed-length message-byte-count field to each of their application-level messages, and adding logic to the receiving program to parse these byte-count fields, read in that many additional bytes, and then present those bytes together to the next level of logic).
So I chose to:
Prefix each message with a - and suffix with ..
Use base64 to encode the real message to avoid conflict between the message and the previously defined separators.
And using this approach, I got this implementation:
// Send packets:
socket.write('-${base64Encode("Hi World".codeUnits)}.');
socket.write('-${base64Encode("Hello World".codeUnits)}.');
And to parse the packets:
// Cache the previous parsed packet data.
String parsed = '';
void _handleCompletePacket(String rawPacket) {
// Decode the original message from base64 using [base64Decode].
// And convert the [List<int>] to [String].
final String message = String.fromCharCodes(base64Decode(rawPacket));
print(message);
}
void _handleServerPacket(List<int> rawPacket) {
final String packet = String.fromCharCodes(rawPacket);
final String next = parsed + packet;
final List<String> items = <String>[];
final List<String> tokens = next.split('');
for (int i = 0; i < tokens.length; i++) {
final String char = tokens[i];
if (char == '-') {
if (items.isNotEmpty) {
// malformatted packet.
items.clear();
continue;
}
items.add('');
continue;
} else if (char == '.') {
if (items.isEmpty) {
// malformatted packet.
items.clear();
continue;
}
_handleCompletePacket(items.removeLast());
continue;
} else {
if (items.isEmpty) {
// malformatted packet.
items.clear();
continue;
}
items.last = items.last + char;
continue;
}
}
if (items.isNotEmpty) {
// the last data of this packet was left incomplete.
// cache it to complete with the next packet.
parsed = items.last;
}
}
client.listen(_handleServerPacket);
There are certainly more optimized solutions/approaches, but I got this just for chatting messages within [100-500] characters, so that's fine for now.
I'm getting this error when I call my gRPC Golang server from Dart:
Caught error: gRPC Error (code: 12, codeName: UNIMPLEMENTED, message: grpc: Decompressor is not installed for grpc-encoding "gzip", details: [], rawResponse: null, trailers: {})
I have read https://github.com/bradleyjkemp/grpc-tools/issues/19, and it doesn't appear to apply to my issue.
The server is running 1.19.2 on Gcloud Ubuntu.
Dart is running 2.18.2 on Mac Monterey
I have a Dart client calling a Go server. Both appear to be using GZIP for compression.
Dart proto
syntax = "proto3";
option java_multiple_files = true;
option java_package = "io.grpc.examples.helloworld";
option java_outer_classname = "HelloWorldProto";
option objc_class_prefix = "HLW";
package helloworld;
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
GO proto:
syntax = "proto3";
option go_package = "google.golang.org/grpc/examples/helloworld/helloworld";
option java_multiple_files = true;
option java_package = "io.grpc.examples.helloworld";
option java_outer_classname = "HelloWorldProto";
package helloworld;
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
Dart Client code:
import 'package:grpc/grpc.dart';
import 'package:helloworld/src/generated/helloworld.pbgrpc.dart';
Future<void> main(List<String> args) async {
final channel = ClientChannel(
'ps-dev1.savup.com',
port: 54320,
options: ChannelOptions(
credentials: ChannelCredentials.insecure(),
codecRegistry:
CodecRegistry(codecs: const [GzipCodec(), IdentityCodec()]),
),
);
final stub = GreeterClient(channel);
final name = args.isNotEmpty ? args[0] : 'world';
try {
final response = await stub.sayHello(
HelloRequest()..name = name,
options: CallOptions(compression: const GzipCodec()),
);
print('Greeter client received: ${response.message}');
} catch (e) {
print('Caught error: $e');
}
await channel.shutdown();
}
The Go gRPC server works fine with a Go gRPC client and BloomRPC.
I'm new to gRPC in general and very new to Dart.
Thanks in advance for any help resolving this issue.
That error that you shared shows that your server doesn't support gzip compression.
The quickest fix is to not use gzip compression in the client's call options, by removing the line:
options: CallOptions(compression: const GzipCodec()),
from your Dart code.
The go-grpc library has an implementation of a gzip compression encoding in package github.com/grpc/grpc-go/encoding/gzip, but it's experimental, so likely not wise to use it in production; or at least you should pay close attention to it:
// Package gzip implements and registers the gzip compressor
// during the initialization.
//
// Experimental
//
// Notice: This package is EXPERIMENTAL and may be changed or removed in a
// later release.
If you want to use it in your server, you just need to import the package; there is no user-facing code in the package:
import (
_ "github.com/grpc/grpc-go/encoding/gzip"
)
The documentation about compression for grpc-go mentions this above package as an example of how your implement such a compressor.
So you may also want to copy the code to a more stable location and take responsibility for maintaining it yourself, until there is a stable supported version of it.
I am following GitHub code on how to implement push notification based on realtime database triggers.
Here is the code and the link:
https://github.com/firebase/functions-samples/blob/master/fcm-notifications/functions/index.js
/**
* Copyright 2016 Google Inc. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
'use strict';
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
/**
* Triggers when a user gets a new follower and sends a notification.
*
* Followers add a flag to `/followers/{followedUid}/{followerUid}`.
* Users save their device notification tokens to `/users/{followedUid}/notificationTokens/{notificationToken}`.
*/
exports.sendFollowerNotification = functions.database.ref('/followers/{followedUid}/{followerUid}').onWrite(event => {
const followerUid = event.params.followerUid;
const followedUid = event.params.followedUid;
// If un-follow we exit the function.
if (!event.data.val()) {
return console.log('User ', followerUid, 'un-followed user', followedUid);
}
console.log('We have a new follower UID:', followerUid, 'for user:', followerUid);
// Get the list of device notification tokens.
const getDeviceTokensPromise = admin.database().ref(`/users/${followedUid}/notificationTokens`).once('value');
// Get the follower profile.
const getFollowerProfilePromise = admin.auth().getUser(followerUid);
return Promise.all([getDeviceTokensPromise, getFollowerProfilePromise]).then(results => {
const tokensSnapshot = results[0];
const follower = results[1];
// Check if there are any device tokens.
if (!tokensSnapshot.hasChildren()) {
return console.log('There are no notification tokens to send to.');
}
console.log('There are', tokensSnapshot.numChildren(), 'tokens to send notifications to.');
console.log('Fetched follower profile', follower);
// Notification details.
const payload = {
notification: {
title: 'You have a new follower!',
body: `${follower.displayName} is now following you.`,
icon: follower.photoURL
}
};
// Listing all tokens.
const tokens = Object.keys(tokensSnapshot.val());
// Send notifications to all tokens.
return admin.messaging().sendToDevice(tokens, payload).then(response => {
// For each message check if there was an error.
const tokensToRemove = [];
response.results.forEach((result, index) => {
const error = result.error;
if (error) {
console.error('Failure sending notification to', tokens[index], error);
// Cleanup the tokens who are not registered anymore.
if (error.code === 'messaging/invalid-registration-token' ||
error.code === 'messaging/registration-token-not-registered') {
tokensToRemove.push(tokensSnapshot.ref.child(tokens[index]).remove());
}
}
});
return Promise.all(tokensToRemove);
});
});
});
My silly question, new to Functions and Node, is in this code notifications are sent to all users who tokens are saved, is that correct? and if it is how can I let's say send just to one particular person instead all?
I was thinking of saving token of each user in different nodes (children) so I can pick the one I want to send notification to. Does it work?
Thanks All
This code will send notification to just one user (follower in this example). This user can have multiple tokens, representing multiple devices, and hence the variable name: tokensSnapshot.
What you intend to do is very doable with Cloud Functions. You just have to be careful with paths of your nodes where you save users, or tokens, for instance. Also as Frank van Puffelen suggested, having some acquaintance with Admin SDK (Realtime Database and FCM) will really help you out.
I am presently working on program on Android Things for connecting to Google Cloud IoT Core. I used to sample maven code provided by Google and modified it for Gradle(with all the imports and stuff). After doing every kind of check, whenever I am trying to run the program on a Raspberry Pi3 running Android Things it keeps giving this error
W/System.err: java.io.FileNotFoundException: com/example/adityaprakash/test/rsa_private.pem (No such file or directory)
telling me that the private key file that I am supposed to use for the JWT doesn't exist despite the fact it does and I have given the path for the pem file.Here are my java codes
package com.example.adityaprakash.test;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.util.Log;
public class MainActivity extends AppCompatActivity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
//setContentView(R.layout.activity_main);
Log.i("#########","######");
MqttExample mqtt = new MqttExample();
try {
mqtt.Start();
} catch (Exception e) {
e.printStackTrace();
}
}
}
The MqttExample.java
package com.example.adityaprakash.test;
// [END cloudiotcore_mqtt_imports]
import org.eclipse.paho.client.mqttv3.MqttClient;
import org.eclipse.paho.client.mqttv3.MqttConnectOptions;
import org.eclipse.paho.client.mqttv3.MqttMessage;
import org.eclipse.paho.client.mqttv3.persist.MemoryPersistence;
import org.joda.time.DateTime;
import java.io.BufferedReader;
import java.io.FileReader;
import java.security.KeyFactory;
import java.security.spec.PKCS8EncodedKeySpec;
import android.util.Base64;
import io.jsonwebtoken.JwtBuilder;
import io.jsonwebtoken.Jwts;
import io.jsonwebtoken.SignatureAlgorithm;
public class MqttExample {
// [START cloudiotcore_mqtt_createjwt]
/** Create a Cloud IoT Core JWT for the given project id, signed with the given RSA key. */
public static String createJwtRsa(String projectId, String privateKeyFile) throws Exception {
DateTime now = new DateTime();
String strKeyPEM = "";
BufferedReader br = new BufferedReader(new FileReader(privateKeyFile));
String line;
while ((line = br.readLine()) != null) {
strKeyPEM += line + "\n";
}
br.close();
// Create a JWT to authenticate this device. The device will be disconnected after the token
// expires, and will have to reconnect with a new token. The audience field should always be set
// to the GCP project id.
JwtBuilder jwtBuilder =
Jwts.builder()
.setIssuedAt(now.toDate())
.setExpiration(now.plusMinutes(20).toDate())
.setAudience(projectId);
String privateKeyPEM = strKeyPEM;
privateKeyPEM = privateKeyPEM.replace("-----BEGIN PRIVATE KEY-----\n", "");
privateKeyPEM = privateKeyPEM.replace("-----END PRIVATE KEY-----", "");
byte[] encoded = Base64.decode(privateKeyPEM,Base64.DEFAULT);
PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(encoded);
KeyFactory kf = KeyFactory.getInstance("RSA");
return jwtBuilder.signWith(SignatureAlgorithm.RS256, kf.generatePrivate(spec)).compact();
}
/** Parse arguments, configure MQTT, and publish messages. */
public void Start() throws Exception {
// [START cloudiotcore_mqtt_configuremqtt]
MqttExampleOptions options = MqttExampleOptions.values();
if (options == null) {
// Could not parse.
System.exit(1);
}
// Build the connection string for Google's Cloud IoT Core MQTT server. Only SSL
// connections are accepted. For server authentication, the JVM's root certificates
// are used.
final String mqttServerAddress =
String.format("ssl://%s:%s", options.mqttBridgeHostname, options.mqttBridgePort);
// Create our MQTT client. The mqttClientId is a unique string that identifies this device. For
// Google Cloud IoT Core, it must be in the format below.
final String mqttClientId =
String.format(
"projects/%s/locations/%s/registries/%s/devices/%s",
options.projectId, options.cloudRegion, options.registryId, options.deviceId);
MqttConnectOptions connectOptions = new MqttConnectOptions();
// Note that the the Google Cloud IoT Core only supports MQTT 3.1.1, and Paho requires that we
// explictly set this. If you don't set MQTT version, the server will immediately close its
// connection to your device.
connectOptions.setMqttVersion(MqttConnectOptions.MQTT_VERSION_3_1_1);
// With Google Cloud IoT Core, the username field is ignored, however it must be set for the
// Paho client library to send the password field. The password field is used to transmit a JWT
// to authorize the device.
connectOptions.setUserName("unused");
System.out.println(options.algorithm);
if (options.algorithm.equals("RS256")) {
connectOptions.setPassword(
createJwtRsa(options.projectId, options.privateKeyFile).toCharArray());
}else {
throw new IllegalArgumentException(
"Invalid algorithm " + options.algorithm + ". Should be one of 'RS256' or 'ES256'.");
}
// [END cloudiotcore_mqtt_configuremqtt]
// [START cloudiotcore_mqtt_publish]
// Create a client, and connect to the Google MQTT bridge.
MqttClient client = new MqttClient(mqttServerAddress, mqttClientId, new MemoryPersistence());
try {
client.connect(connectOptions);
// Publish to the events or state topic based on the flag.
String subTopic = options.messageType.equals("event") ? "events" : options.messageType;
// The MQTT topic that this device will publish telemetry data to. The MQTT topic name is
// required to be in the format below. Note that this is not the same as the device registry's
// Cloud Pub/Sub topic.
String mqttTopic = String.format("/devices/%s/%s", options.deviceId, subTopic);
// Publish numMessages messages to the MQTT bridge, at a rate of 1 per second.
for (int i = 1; i <= options.numMessages; ++i) {
String payload = String.format("%s/%s-payload number-%d", options.registryId, options.deviceId, i);
System.out.format(
"Publishing %s message %d/%d: '%s'\n",
options.messageType, i, options.numMessages, payload);
// Publish "payload" to the MQTT topic. qos=1 means at least once delivery. Cloud IoT Core
// also supports qos=0 for at most once delivery.
MqttMessage message = new MqttMessage(payload.getBytes());
message.setQos(1);
client.publish(mqttTopic, message);
if (options.messageType.equals("event")) {
// Send telemetry events every second
Thread.sleep(1000);
}
else {
// Note: Update Device state less frequently than with telemetry events
Thread.sleep(5000);
}
}
} finally {
// Disconnect the client and finish the run.
client.disconnect();
}
System.out.println("Finished loop successfully. Goodbye!");
// [END cloudiotcore_mqtt_publish]
}
}
and the MqttExampleOptions.java code:
package com.example.adityaprakash.test;
public class MqttExampleOptions {
String projectId;
String registryId;
String deviceId;
String privateKeyFile;
String algorithm;
String cloudRegion;
int numMessages;
String mqttBridgeHostname;
short mqttBridgePort;
String messageType;
/** Construct an MqttExampleOptions class. */
public static MqttExampleOptions values() {
try {
MqttExampleOptions res = new MqttExampleOptions();
res.projectId = "_";
res.registryId = "_";
res.deviceId = "_";
res.privateKeyFile = "com/example/adityaprakash/test/rsa_private.pem";
res.algorithm = "RS256";
res.cloudRegion = "asia-east1";
res.numMessages = 100;
res.mqttBridgeHostname = "mqtt.googleapis.com";
res.mqttBridgePort = 8883;
res.messageType = "event";
return res;
} catch (Exception e) {
System.err.println(e.getMessage());
return null;
}
}
}
Please can anyone give a solution to this problem.
P.S. I know the code looks totally crappy.I don't have experience with Android programming,so please let it go.
The example you are following is not designed for Android.
res.privateKeyFile = "com/example/adityaprakash/test/rsa_private.pem";
Will not relate to the same directory on the Android file system.
I wrote up an AndroidThings explanation of how to talk to Cloud IoT Core here: http://blog.blundellapps.co.uk/tut-google-cloud-iot-core-mqtt-on-android/
You can setup communication like this (with your pem file going into the /raw directory)
// Setup the communication with your Google IoT Core details
communicator = new IotCoreCommunicator.Builder()
.withContext(this)
.withCloudRegion("your-region") // ex: europe-west1
.withProjectId("your-project-id") // ex: supercoolproject23236
.withRegistryId("your-registry-id") // ex: my-devices
.withDeviceId("a-device-id") // ex: my-test-raspberry-pi
.withPrivateKeyRawFileId(R.raw.rsa_private)
.build();
Source code is here: https://github.com/blundell/CloudIoTCoreMQTTExample
Note that the above is good enough for a secure environment or for testing that the end to end works. However if you wanted to release a production IoT device, you would look at embedding the PEM into the ROM and using private file storage access. https://developer.android.com/training/articles/keystore.html
An example of this can be found here: https://github.com/androidthings/sensorhub-cloud-iot
Specifically this class:
https://github.com/androidthings/sensorhub-cloud-iot/blob/e50bde0100fa81818ebbadb54561b3b68ccb64b8/app/src/main/java/com/example/androidthings/sensorhub/cloud/cloudiot/MqttAuthentication.java
You can then generate and use the PEM on the device:
public Certificate getCertificate() {
KeyStore ks = KeyStore.getInstance("AndroidKeyStore");
ks.load(null);
certificate = ks.getCertificate("Cloud IoT Authentication");
if (certificate == null) {
Log.w(TAG, "No IoT Auth Certificate found, generating new cert");
generateAuthenticationKey();
certificate = ks.getCertificate(keyAlias);
}
Log.i(TAG, "loaded certificate: " + keyAlias);
}
and
private void generateAuthenticationKey() throws GeneralSecurityException {
KeyPairGenerator kpg = KeyPairGenerator.getInstance(KeyProperties.KEY_ALGORITHM_RSA, "AndroidKeyStore");
kpg.initialize(new KeyGenParameterSpec.Builder("Cloud IoT Authentication",KeyProperties.PURPOSE_SIGN)
.setKeySize(2048)
.setCertificateSubject(new X500Principal("CN=unused"))
.setDigests(KeyProperties.DIGEST_SHA256)
.setSignaturePaddings(KeyProperties.SIGNATURE_PADDING_RSA_PKCS1)
.build());
kpg.generateKeyPair();
}
I'm pretty sure you're not doing the file I/O correctly. Your file, "com/example/adityaprakash/test/rsa_private.pem", doesn't correspond to an actual filepath on the device. The location of files on the device may be different than in your project. You will have to determine where on the device your file actually is.
On AndroidThings, it is easier to provide the authentication credentials in an Android Resource. See my fork of the WeatherStation sample to see how this works.
First, copy the private key file (e.g. rsa_private_pkcs8) to app/src/main/res/raw/privatekey.txt
Next, you can load the key used to calculate your JWT as:
Context mContext;
int resIdPk = getResources().getIdentifier("privatekey", "raw", getPackageName());
...
InputStream privateKey = mContext.getResources().openRawResource(resIdPk);
byte[] keyBytes = inputStreamToBytes(privateKey);
PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(keyBytes);
KeyFactory kf = KeyFactory.getInstance("EC");
One final note, it appears that you're referencing a file that is not in pkcs8 format, which will cause issues with Java. Make sure to use a key that is packaged in PKCS8 when opening credentials on Android (Java).
Due to Heartbleed, our Gateway Server was updated and this problem presented itself.
Due to POODLE, SSLv3 is no longer supported.
Note, the problem is only present on Win7+ boxes; WinXP boxes work without issue (same code, different OS = problem); granted WinXP is no longer a valid OS, just wanted to make note of functionality.
Client application (.NET 2.0) sits on a Windows 7 (or 8) box. Server runs within a DMZ behind a Gateway Server. Just to note, I found that this problem is no longer present on .NET 4.0+ - however due to legacy code, I do not have the luxury of updating.
Gateway Server is a pass through box on which Apache HTTP Server with SSL run. Its location is outside the DMZ, and it is used to access the Server which is inside the DMZ. Versions of software running on the Gateway server are Apache/2.2.25 (Win32), mod_jk/1.2.39, mod_ssl/2.2.25, OpenSSL/1.0.1g
Here is the code used on the Client application (with an exorbitant amount of logging added) ... note, 'serverName' typically contains a value such as "https://some.url.com"
private bool ConnectAndAuthenicate(string serverName, out TcpClient client, out SslStream sslStream)
{
client = null;
sslStream = null;
try
{
client = new TcpClient(serverName, 443); // Create a TCP/IP client; ctor attempts connection
Log("ConnectAndAuthenicate: Client CONNECTED"));
sslStream = new SslStream(client.GetStream(), false, ValidateServerCertificate, null);
Log("ConnectAndAuthenicate: SSL Stream CREATED"));
}
catch (Exception x)
{
Log("ConnectAndAuthenicate: EXCEPTION >> CONNECTING to server: {0}", x.ToString()));
if (x is SocketException)
{
SocketException s = x as SocketException;
Log("ConnectAndAuthenicate: EXCEPTION >> CONNECTING to server: Socket.ErrorCode: {0}", s.ErrorCode));
}
if (client != null) { client.Close(); client = null; }
if (sslStream != null) { sslStream.Close(); sslStream = null; }
}
if (sslStream == null) return false;
try
{
sslStream.ReadTimeout = 10000; // wait 10 seconds for a response ...
Log("ConnectAndAuthenicate: AuthenticateAsClient CALLED ({0})", serverName));
sslStream.AuthenticateAsClient(serverName);
Log("ConnectAndAuthenicate: AuthenticateAsClient COMPLETED SUCCESSFULLY"));
return true;
}
catch (Exception x)
{
Log("ConnectAndAuthenicate: EXCEPTION >> AuthenticateAsClient: {0}", x.ToString()));
client.Close(); client = null;
sslStream.Close(); sslStream = null;
}
return false;
}
Note - answers posted pertaining to ServicePointManager have absolutely no effect on the outcome of this application.
Every time that AuthenicateAsClient() is called when application is run on Win 7+ box, the exception occurs - if application is run on WinXP box, code works properly without exceptions.
Any ideas for solutions are very welcome.
Following the trail of setting the ServicePointManager.SecurityProtocol static ctor with a SecurityProtocolType, I found mention of another enum called SslPolicy -- further research found that AuthenicateAsClient has an overload that takes SslPolicy as an argument.
Changing this line in the above code fixed this problem:
sslStream.AuthenticateAsClient(serverName, null, SslPolicy.Tls, false);