Unable to filter SDK logs in Azure Function instances - serilog

We are seeing few issues with the latest (5.x) version SDKs:
Below lock renew happens constantly with 5.x sdk which we didn't see with the previous 4.x sdk. Request [a0294b3b-3aa4-4b65-9b35-20c2e0493ed3] PUT https://<>.blob.core.windows.net/azure-webjobs-hosts/locks/<>/host?comp=metadata
image
We are seeing 409 Conflict (Container already exists) Request [b4948ae8-cc71-4239-828f-bc7657391d0b] PUT https://dataqualityfnstorage.blob.core.windows.net/azure-webjobs-hosts?restype=container
This event is flooding our logs every 10 seconds for ServiceBus trigger function. Receiver: ReceiveBatchAsync start. MessageCount = 1.
Below changes works fine in my local machine but not in Azure post deployment:
host.json file change
{
"version": "2.0",
"functionTimeout": "00:55:00",
"Serilog": {
"MinimumLevel": {
"Default": "Information",
"Override": {
"Azure.Messaging.ServiceBus": "Error",
"Azure.Storage.Blobs": "Error",
"Azure.Core": "Error",
"Microsoft.Azure.WebJobs.Extensions.Storage": "Error",
"Microsoft.Azure.WebJobs.Extensions.ServiceBus": "Error"
}
}
}
}
Corresponding Startup.cs change to load this settings:
var logger = new LoggerConfiguration()
.ReadFrom.Configuration(hostConfig, sectionName: "AzureFunctionsJobHost:Serilog")
.WriteTo.Console()
.WriteTo.ApplicationInsights(telemetryConfiguration, TelemetryConverter.Traces)
.CreateLogger();
services.AddLogging(lb => lb.AddSerilog(logger));

Related

Error Domain=NSURLErrorDomain Code=-1202 using dotnet core webapi on macos to test xcode iphone simulator

I am getting this error:
Error Domain=NSURLErrorDomain Code=-1202 "The certificate for this
server is invalid. You might be connecting to a server that is
pretending to be “x.x.x.x” which could put your confidential
information at risk."
When trying to test my dotnet core webapi on my local machine from a xcode iPhone 13 simulator.
The ip is a static from the dhcp server. I am developing APIs using dotnet core using vs code on a mac, so Kestrel is hosting the APIs. I have a developer cert in the keychain ("Developer CA") and can hit the APIs from Postman and an Android emulator using Android Studio. I got that working with a .pfx file, but clearly that's not ok for iPhone testing.
This is the code I'm using in xCode to test it out.
Button("Call API") {
print("Calling API")
Task {
let (data, _) = try await URLSession.shared.data(from: URL(string: "https://localhost:5001/api/v1.0/testapi/1")!)
let decodedResponse = try? JSONDecoder().decode(ApiData.self, from: data)
apiResponse = decodedResponse?.value ?? ""
}
}
In the ContentView I have:
#State private var apiResponse: String = ""
And there is also a struct:
struct ApiData: Codable {
let value: String
}
I have replaced localhost with the ip (as referenced in the error message above).
I created this post when I was trying to connect to the api through the android emulator, and eventually got that working. It references this post with a Kestrel configuration:
{
"Certificates": {
"HTTPS": {
"Source": "Store",
"StoreLocation": "CurrentUser",
"StoreName": "My",
"Subject": "CN=localhost",
"AllowInvalid": true
}
}
}
I tried a variety of settings because I do have a cert following these instructions from developer.apple.com. And I did export it and drag it onto the Simulator.
I also looked at this post, but looks like a different problem as it was working fine on the simulator. I looked at this post as well, but this was 8 years ago and I'm not sure coding specific domains into the session is a good approach.
I have been through a lot of documentation on certificates (like this, this, and this), dotnet/kestrel specific ssl links (like this one, this one, and this one), but getting that right combination of recent, with kestrel, on a mac, for both android emulator and ios simulator has been challenging.
Does anybody have the specific steps? I think the kestrel configuration for the Developer CA that I have is the current path and seems correct, but I don't really know.
UPDATE
I tried with this configuration:
"Kestrel": {
"Certificates": {
"Default": {
"Source": "Store",
"StoreLocation": "CurrentUser",
"StoreName": "My",
"Subject": "CN=Dev CA",
"AllowInvalid": true
}
},
"EndPoints": {
"Https": {
"Url": "https://*:5001"
}
}
},
and got this error:
Error:
Exception has occurred: CLR/System.InvalidOperationException An
unhandled exception of type 'System.InvalidOperationException'
occurred in System.Private.CoreLib.dll: 'The requested certificate
CN=Dev CA could not be found in CurrentUser/My with AllowInvalid
setting: True.' at
Microsoft.AspNetCore.Server.Kestrel.Https.CertificateLoader.LoadFromStoreCert(String
subject, String storeName, StoreLocation storeLocation, Boolean
allowInvalid) at
Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Certificates.CertificateConfigLoader.LoadFromStoreCert(CertificateConfig
certInfo) at
Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Certificates.CertificateConfigLoader.LoadCertificate(CertificateConfig
At least I'm getting an error, so if I figure this out I will know. I am trying to follow this and this.

AzureBlobStorageOnIoTEdge: Error Target container connection not specified, upload turned off

My local blob storage is not uploading blobs to my cloud storage account. It reports back
"configurationValidation": {
"deviceAutoDeleteProperties": {
"deleteOn": {
"Status": "Success"
},
"deleteAfterMinutes": {
"Status": "Warning",
"Message": "Auto Delete after minutes value not specified, auto deletion turned off."
},
"retainWhileUploading": {
"Status": "Success"
}
},
"deviceToCloudUploadProperties": {
"uploadOn": {
"Status": "Success"
},
"cloudStorageAccountName": {
"Status": "Error",
"Message": "Target container connection not specified, upload turned off."
},
"cloudStorageAccountKey": {
"Status": "Error",
"Message": "Target container connection not specified, upload turned off."
},
"uploadOrder": {
"Status": "Success"
},
"deleteAfterUpload": {
"Status": "Success"
}
}
},
I am pretty sure that it should work. My desired properties are
"deviceToCloudUploadProperties": {
"uploadOn": true,
"uploadOrder": "OldestFirst",
"cloudStorageConnectionString": "DefaultEndpointsProtocol=https;AccountName=*****;AccountKey=******;EndpointSuffix=core.windows.net",
"storageContainersForUpload": {
"***": {
"target": "***"
}
},
"deleteAfterUpload": true
}
The container exists locally and on the cloud site. I copied the primary connection string from my local storage account into the configuration. The local storage is working, I can see that my container was created and contains data but it doesn't want to synchronize with the cloud. Why is it saying "Target container connection not specified, upload turned off."? It sounds like this part is missing
"storageContainersForUpload": {
"***": {
"target": "***"
}
},
but obviously it is not.
I'm using the latest docker image of this service. Is there any chance to use an older version? Some months ago I could make it work already. I tried to use a different version like mcr.microsoft.com/azure-blob-storage:1.4.0 but it doesn't accept any other tags than latest.
Thx!
The difference between my working version of the local blob storage module and my non working version was that the non working version was deployed by a deployment plan. In the deployment plan you cannot just paste the module twin settings of the documentation of the blob storage on IoT edge like https://learn.microsoft.com/en-us/azure/iot-edge/how-to-deploy-blob?view=iotedge-2020-11
You need to split the configuration into two parts where the first part looks like this
and the second part looks like that
And that totally makes sense. If you want to update your modules you probably want to keep your configuration because there might have been some changes which were made e.g. by a customer. This gives you the possibilty to add some properties to your inital configuration later without changing anything what was already configured. In fact every device can keep its individual configuration at any time.
My wrongly configured reported proterties were hidden in the suggested default path "properties.desired.settings" and thus the edge runtime could not find it.

iOS Push notifications using cordova. The notification is never received

I created a sample Cordova app which is using "phonegap-push-plugin".
That app doesn't have any complexity. On "deviceready" I run the plugin initialization code as shown here:
var push = PushNotification.init({android: {}, ios: {
sound: true,
alert: true,
badge: true,
categories: {
invite: {
yes: {
callback: 'accept',
title: 'Accept',
foreground: true,
destructive: false
},
no: {
callback: 'reject',
title: 'Reject',
foreground: true,
destructive: false
},
maybe: {
callback: 'maybe',
title: 'Maybe',
foreground: true,
destructive: false
}
},
delete: {
yes: {
callback: 'doDelete',
title: 'Delete',
foreground: true,
destructive: true
},
no: {
callback: 'cancel',
title: 'Cancel',
foreground: true,
destructive: false
}
}
}
}})
push.on('notification', data => {
console.log(data.message);
console.log(data.title);
console.log(data.count);
console.log(data.sound);
console.log(data.image);
console.log(data.additionalData);
})
push.on('emailGuests', data => {
console.log('I should email my guests');
});
push.on('snooze', data => {
console.log('Remind me later');
});
push.on('registration', data => {
console.log(data.registrationId);
console.log(data.registrationType);
});
push.subscribe('xx', console.log)
And this is the log output to console:
=> Successfully subscribe to topic xx
// The first run (after app install) will ask for permissions. If I click allow the lines below are printed to console.
=> dCAtjhCFBcU:APA91bG90c8VhNl_BzZ-2e9fmq_9fN6jfrRNJ1LPCRIpKnZ-AG-eLY4xtX84oJRZBh2D....KtNNQ35GM8ubPF5zr8HqeB6jffs
=> FCM
In order to push I'm sending the following payload to the Legacy Server https://fcm.googleapis.com/fcm/send.
{
"priority": "high",
"to": "/topics/xx", // I tried this but I also tried to specify the device token received upon "registration" event. I did this using to:<device_token> and also using registration_ids: [<device_token>].
"notification": {
"title": "My Message",
"body": "My Message Body",
"badge": 1,
"content-available": "1", // I tried with and without
"category": "identifier", // I tried with and without
"thread-id": "id", // I tried with and without
"sound": "default",
"icon": "default"
},
"data": {
"title": "A short string describing the purpose of the notification",
"body": "The text of the alert message",
"clubId": 1000
},
"notId": 1,
"custom_key1": "value1",
"custom_key2": "value2"
}
Note: I tried every combination possible in what concerns the app state: App in background; app closed; app in foreground; The event "notification" has never fired and the push notification was never received.
The request sent to the FCM server returns a message id when I use the topic (which is understandable since other devices subscribe the topic). For that reason my android that has subscribed to the same topic receives the message. The iOS in the other hand receives nothing!
{
"message_id": 5059997308576486332
}
If I try to specify the token that I received upon registration, I will get a slightly different message. Most of the time the token received upon registration works and the results will contain a string id. But this is temporary since a few minutes later the token become "NotRegistered".
{
"multicast_id": 88880398234xxxxx7,
"success": 0,
"failure": 1,
"canonical_ids": 0,
"results": [
{
"error": "NotRegistered" // <-- This happens after a few minutes. I have to delete the app and reinstall it in order to get a new token.
}
]
}
This is the build configuration
Notifications are correctly enabled on my iOS device. What am I missing?
Updated:
Accessing Apple's APN directly (yup... no FCM!)
I would like to send my push notifications through FCM but in order to determine the cause of the issues described above, I decided to try APN directly. To do this, I had to remove the from the app's config.xml, so phonegap-push-plugin can obtain a token from APN and not from FCM.
Now, using the new token and a server that uses node-apn module to communicate with APN server, I'm able to send push notifications to my iOS app. The downside of this, is that I lose the ability to push to topics since this is a FCM only feature.
The only thing that I still don't know is how to use the topic to target devices in the APN network, that are subscribed by the push.subscribe() method.
Checkout my issue here.
Any help on this too?
So, turns out there's an issue with the push plugin.
This issue only impacts users on iOS device apps that use the FCM.
If you use the APNs it works.
You can check my answer in here: https://github.com/phonegap/phonegap-plugin-push/issues/2644#issuecomment-445346335
And my initial issue reporting in here:
https://github.com/phonegap/phonegap-plugin-push/issues/2613

Serilog cannot show output logs aspnetcore 2.1 vs 2017

I am writing a few webapis using aspnetcore core 2.1
I would like to use serilog and I cannot find a simple example out there that works that gives me a starting point and shows me what I need to do.
I would like to do the following
Setup Serilog in appsetting.json file
Write to VisualStudio output windows
Write to File location eg "C:\Logs"
Inject logger in various controllers
Can somebody either correct my code or point me to a link where it shows how it is used
I have done as follows: (Copy and paste from internet)but they do not show in vs output windows
Not sure if need all of these nugets
Serilog
Serilog.Extensions.Logging
Serilog.Settings.Configuration
Serilog.Sinks.ColoredConsole
Serilog.Sinks.File
Serilog.Sinks.RollingFile
serilog.Sinks.Trace
Log.Logger = new LoggerConfiguration()
.ReadFrom.Configuration(Configuration)
.Enrich.FromLogContext()
.WriteTo.ColoredConsole()
.WriteTo.Console()
.CreateLogger();
Log.Verbose("This is a verbose statement");
Log.Debug("This is a debug statement");
Log.Information("This is a info statement");
Log.Warning("This is a warning statement");
Log.Error(new IndexOutOfRangeException(), "This is an error statement");
Log.Fatal(new AggregateException(), "This is an fatal statement");
Appsetting file set up serilog
{
"Serilog": {
"MinimumLevel": {
"Default": "Information",
"Override": {
"Microsoft": "Warning",
"System": "Warning"
}
},
"WriteTo": [
{
"Name": "[ColoredConsole]",
"Args": {"outputTemplate": "[{Timestamp:HH:mm:ss.fff}] {Level:u3} - {Message}{NewLine}{Exception}"}
},
"WriteTo": [
{
"Name": "console",
"Args": {"outputTemplate": "[{Timestamp:HH:mm:ss.fff}] {Level:u3} - {Message}{NewLine}{Exception}"}
},
{
"Name": "RollingFile",
"Args": {
"pathFormat": "logs\\log-{Date}.log",
"outputTemplate": "[{Timestamp:dd/MM/yy HH:mm:ss.fff z}] {Level:u3} {Message}{NewLine}{Exception}"
}
}
]
},
Nothing shows in VS OUTPUT WINDOWS
no file is written
Where Am I going wrong?
thanks
You can always start by copying the sample application found here. It will get you a part of your way, and then what you will have to do is add the specific sinks you required.

Netflix Conductor SQS

Has anyone successfully integrated Netflix Conductor with AWS SQS?
I have tried below steps but the workflow is not triggered.
Create SQS queue
Added AWS creds to environment
Registered tasks, workflows and the event listener below
{
"name": "sqs_event_listener",
"event": "sqs:name_of_sqs_queue",
"condition": "true",
"active": true,
"actions": [{
"action": "start_workflow",
"start_workflow": {
"name": "mywf"
}
}]
}
I know this is too late to help the original poster, but adding a response to improve the hive mind of SO:
In your Conductor application.properties file, make sure you have the following values
conductor.default-event-queue.type=sqs
conductor.event-queues.sqs.enabled=true
conductor.event-queues.sqs.authorized-accounts=(your AWS account number)
We need to update annotations-processor/awssqs-event-queue/src/main/java/com/netflix/conductor/SQSEventQueueConfiguration.java
#Bean
AWSCredentialsProvider createAWSCredentialsProvider() {
return new DefaultAWSCredentialsProviderChain();
}
With this configuration in Conductor, you can now restart your instance, and your event should receive events from the SQS message queue.
For a full post with workflow and tasks SENDING ans RECEIVING SQS messages - check out: https://orkes.io/content/docs/how-tos/Tasks/SQS-event-task

Resources