I'm running an A/B test in Firebase. The client is getting values and responding appropriately. I can see "current users" in the live view:
But even after 48 hours I still see "Running (0 users exposed)" in the headline of the experiment.
How can these be "active experiement users" yet not show up as part of the experiment?
For comparison, here's the staging build of the app, which has assigned users with the same code, just building with a different scheme and hitting a different Firebase app:
We call fetchAndActivate on launch:
FirebaseApp.configure()
RemoteConfig.remoteConfig().fetchAndActivate { _, error in
print(error ?? "Fetched!")
}
For now, it happens (no data in Firebase remote config A/B test experiment) if you have an activation event configured for A/B test experiment.
Check these:
If you have 2 different experiments, both will fail to get results even if you have "activation event" configured only in 1 of them.
Additionally, remote config will not work as well, you'll be able to get only default values.
it takes many hours before you can see any data in your experiment, wait for at least 24 hours. It will show 0 for many hours after the start.
make sure your users have the version of your app with the latest SDK.
Since your experiment is with Remote Config: When activateFetched() is called, all events from that point on will be tagged with the experiment. If you have a goal or activation event that happens before activateFetched(), such as automatic events like first_open, session_start, etc., the experiment setup might be wrong.
Make sure to call fetch() and activateFetched() before the activation event occurs.
The good way to check if your experiment is working now is to set it to a specific version you didn't publish yet and check logs from remote config with the fresh app install(or erase all app data & restart). It should show different variant every time you reinstall the app, since your Firebase Instance ID changes after app reinstall/app data erase.
If you see variants change - then A/B test is running well.
Related
I've built an app (with flask, flask-login and dash) on GCP Cloud Run. The app allows users to login, look at some fancy dashboards and leave comments on certain pages. It works great performance-wise: instances spin up quickly for users with minimal lag, the BigQuery interface I built works great and pub/sub messages sent from user interactions do exactly what they're supposed to do.
The only issue I'm having right now is that there's something weird about which instance of a container a user connects to. What will often happen is a user will login to my app via their browser successfully, and then when navigating to another password-protected page will receive a 401 error (seemingly randomly).
My belief is that this behavior is happening because the navigation request (clicking a link to another password protected page) from the user to another password protected page spins up another Cloud Run instance. Is there any way to force Cloud Run to maintain a specific instance of my container for a given request? So that if a user logs in and then navigates GCP doesn't take the next request and decide to autoscale?
I've experimented with setting the maximum number of requests for the app's frontend container to 1 but it doesn't seem to improve this behavior which happens sporadically throughout a given user's session.
To clarify, the frontend part of the app is still usable, but it is an annoying user experience to constantly have to login again.
Any help or guidance is appreciated!
The answer was as simple as turning on session affinity per #DazWilkin 's comment.
What I did:
Went to the Cloud Run dashboard on GCP and selected the service of interest
Clicked "Edit and Deploy New Revision"
Went to the "Connections"
Checked the box next to the "Session affinity" preview feature
Clicked deploy
This ended up completely solving the problem!
In Hyperstack every state change enters a message queue through a websocket mechanism to inform every application client for model/app state changes. So if you update a model from my browser sessions, everyone else connected at the time can see it in their session (if there are the necessary permissions).
This is even done from console 'sessions'. You change a model from Rails console and changes automatically propagate to all connected web clients.
For this to be done the web application part has to be operational (i.e. rails server, must be up and running).
The problem is that there are two situations where you might not want console updates to propagate to the client:
when rails server is not operational, for any reason and you want to interact with the application through it's console (until rails server is up again)
You want to perform batch updates through console or rake tasks and you don't want the overhead of keeping clients informed.
Is there a way to to quickly turn of messaging from the console or some kind of toggle method for that purpose?
If the rails server is not up it will not try to send messages (however see note at end)
But the case of a rake task that you want to run while the server IS up, is interesting. I don't think there is any published way to turn off the "remote process -> server" push, but this patch will accomplish the same:
module Hyperstack
def self.send_to_server(*args)
# drop the message on the floor
end
end
Just stick that in the rake task.
Regarding the server "not being up" the one case that does not work is if the server is in fact "up" but simply never responds. See https://github.com/hyperstack-org/hyperstack/issues/144 for details. If you are trying to debug a server problem then the same patch above will help until that issue is fixed.
Our system has two servers (S1) one is running processesing and data storage (basically DB) and the other one is a webserver (WS).
There are two types of even that can happen in the system:
User A pings User B. In this case we check if user B is logged in and we push a notification to User B client throw SignalR. It works.
Services constantly running on S1 and generating new data that concenrs multiple users. My goal is as soon as a new data important for user A is generated I immediately want to dispatch a signalR notification to user A client provided he/she is logged in.
This part 2 is not quite clear for me how to design. My thought right now is to start an indefinite process on webserves that monitors our DataBase and checks if new records are generated fpr this user and then push a SignalR message.
That would be fine, but now we have 10k users logged in and I don't think the right decision would be run 10k threads monitoring activities.
Basically, my question is what would a proper way do design signalR based notification mechanism that is based on events that are not originated on our webserver.
I would use a service bus or mq, for example this Free MQ https://www.rabbitmq.com/
You can proxy the messages direcly to the Clients using this proxy library (I'm the author).
Doc's here https://github.com/AndersMalmgren/SignalR.EventAggregatorProxy/wiki
Demo https://github.com/AndersMalmgren/SignalR.EventAggregatorProxy/tree/master/SignalR.EventAggregatorProxy.Demo.MVC4
You can also set up a sql dependency that triggers a message to your signalr clients,
http://techbrij.com/database-change-notifications-asp-net-signalr-sqldependency
This link is the one that I based my code on.
couple of things to watch for, the setup of the table. You cannot use 3 part table names
"SELECT [CMRID],
[SolutionID],
[CreateDT],
[ModifyDT]
**FROM [dbo].[Case]**
WHERE [ModifyDT] > " + LastExecutionDateTime;
Also, and this is very important, you MUST reset the event handler every time the dependency triggers, if not it will work the first time and then stop working.
I hope this helps you.
Is it possible to setup TFS/Test Manager so that it sends out an email after a test fails?
Yes, it is possible but it requires quite a lot of changes/additions to the process template and possibly a custom-made activity.
After tests have run, we check if BuildDetail.BuildPhaseStatus has status failed
We send mail to everyone who has changesets committed to this build, so the build goes through BuildDetail.AssociatedChangesets (you need to have AssociateChangesetsAndWorkItems on) and get the committer username.
Unfortunately for us, there's no good correlation between TFS username and email address at our place, so we had to create a custom activity that looks that up in the AD.
The actual email is sent with the BuildReport action from Community TFS Build Extensions. We modified the xslt, but that's not really necessary. We also wanted to include a listing of the failed tests, and that required modification of the action itself (test data isn't included by default).
Looking at this description and all the work made to get this working, I'm beginning to wonder if it was worth it ;).
Does anybody have any quick and clever ways to flip an MVC app running on Windows Azure into a "maintenace mode"
I don't have a huge need for this because I use the azure staging environment a lot but occasionally I do have the need to make sure there are no users in the production instance of the application (mainly database updates).
I'd like to be able to do this on the fly without uploading new code or swapping deployment slots. Any suggestions?
The friendliest way to do it is on login. When a user authenticates, check a maintenance mode flag in the database and don't let them log in. Let active users continue to use the application until they log out or their session times out. Keep an activity log so you can know when all users have expired.
Of course this means it will take time from when you put the app into maintenance mode and when it is effectively ready, but it's not nice to boot out an active user.
If the usage pattern of your app makes it so this methodology will not ensure no activity in a reasonable time, you can add a timeout on top of this. Check the same maintenance flag for a request every so often. Doesn't have to be every request but every five minutes or so. If necessary you can also cache the maintenance mode value locally for a reasonable period of time (a few minutes).
I would use routing for this. Have the flag be inspected during routing configuration. If it is on, route to "Maintenance" screens
I would suggest adding a Global Action Filter that respects you maintenance mode Flag.