Can an Azure Function be Executed for Multiple Environments - dependency-injection

I've encountered a dependency injection scenario which I cannot find a way through.
We currently have an Azure function.
We are using dependency injection via the FunctionsStartup attribute.
That all works fine, until I get asked to make it work for multiple environments.
The tester found it too onerous to deploy to 7 different environments, so I was asked to re-jig the function so that it runs (in a loop) for those environments.
That means 7 different IConfigurations and somehow having 7 separate compartmentalised IOC registrations of services.
I can't think of a way of doing that, without significantly re-structuring the way abstractions are being resolved. Even if you set up registrations in a loop and inject an IEnumerable of a service, when it goes to resolve a child dependency, it just pulls the last one registered, rather than the one which was meant to correlate with the current item being iterated.
So, something like this (using Autofac):
Registration
foreach (var configuration in configurations)
{
containerBuilder.Register<ICosmosDbService<AccountUsage>>(sp =>
{
var dBConfig = CosmosDBHelper.GetProjectDatabaseConfig(configuration.Value, Project.Jupiter);
return CosmosClientInitializer<AccountUsage>.Initialize(dBConfig);
}).As<ICosmosDbService<AccountUsage>>();
}
Usage
private readonly IEnumerable<IAccountUsageService> _accountUsageService;
public JobScheduler(IEnumerable<IAccountUsageService> accountUsageService)
{
_accountUsageService = accountUsageService;
}
[FunctionName("JobScheduler")]
public async Task Run([TimerTrigger("0 */2 * * * *")] TimerInfo myTimer, ILogger log)
{
log.LogInformation($"Job Scheduler Timer trigger function executed at: {DateTime.Now}");
try
{
foreach (var usageService in _accountUsageService)
{
var logs = await usageService.GetCurrentAccountUsage("gfkjdsasjfa");
// ...
}
}
I realise this kind of DI usage is not ideal (and does not even work).
Is there a way to structure an Azure Function such that it can execute for different configurations in a compartmentalised manner? Or is this really just fighting against the technology?

You've got a couple of ways to do this - either inject the right dependencies into the function constructor, or resolve them dynamically using a service-locater type approach with a named instance.
Let's consider the second approach and what it would mean for your implementation. As you demonstrated, you'd be looping through your instances and resolving the dependency you want to use, then invoking it
foreach (var usageService in _accountUsageService)
{
var logs = await usageService.GetCurrentAccountUsage("named-instance");
logs.DoSomething();
}
This is technically possible, but now you're doing batch processing - you're doing more than once piece of work that's been triggered by a single event (the timer object), which means you have to deal with a couple of extra problems. What should you do if there's a failure with one of the instances, and what to do if one of the instances is running slowly?
Ideally, you want functions to do the smallest bit of work they can, and complete quickly - You don't want failure or slowness with one particular instance impacting the other instances. By breaking it down to the smallest piece of work (think, one event trigger does one piece of work) then you can take advantage of the functions runtime for things like retries on failures, and threading and concurrency is now being done for you by the runtime.
You could then think about a couple of ways you could do this. a) multiple function signatures and a service resolver approach, e.g.
public class JobScheduler
{
public JobScheduler(IEnumerable<IAccountUsageService> accountUsageService)
{
_accountUsageService = accountUsageService;
}
[FunctionName("FirstInstance")]
public Task FirstInstance([TimerTrigger("%MetricPoller:Schedule%")] TimerInfo myTimer)
{
var logs = await _accountUsageService.GetNamedInstance("instance-a");
logs.DoSomething();
}
[FunctionName("SecondInstance")]
public Task SecondInstance([TimerTrigger("%MetricPoller:Schedule%")] TimerInfo myTimer)
{
var logs = _accountUsageService.GetNamedInstance("instance-b");
logs.DoSomething();
}
}
or b), multiple classes with the necessary dependencies injected
public class JobSchedulerFirstInstance
{
public JobSchedulerFirstInstance(ILogs logs)
{
_logs = logs;
}
[FunctionName("FirstInstance")]
public Task FirstInstance([TimerTrigger("%MetricPoller:Schedule%")] TimerInfo myTimer)
{
_logs.DoSomething();
}
}
I'd personally lean towards multiple classes approach, and register named instances with my container. A bit of extra wire up work needed, but you'll end up with lots of small classes that all look very similar that are basically jus t plumbing that the functions runtime executes.

Related

Amazon SWF queries

Over the last couple of years, I have done a fair amount of work on Amazon SWF, but the following points are still unclear to me and I am not able to find any straight forward answers on any forums yet.
These are pretty basic requirements I suppose, sure others might have come across too. Would be great if someone can clarify these.
Is there a simple way to return a workflow execution result (maybe just something as simple as boolean) back to workflow starter?
Is there a way to catch Activity timeout exception, so that we can do run customised actions in such scenarios?
Why doesn't WorkflowExecutionHistory contains Activities, why just Events?
Why there is no simple way of restarting a workflow from the point it failed?
I am considering to use SWF for more business processes at my workplace, but these limitations/doubts are holding me back!
FINAL WORKING SOLUTION
public class ReturnResultActivityImpl implements ReturnResultActivity {
SettableFuture future;
public ReturnResultActivityImpl() {
}
public ReturnResultActivityImpl(SettableFuture future) {
this.future = future;
}
public void returnResult(WorkflowResult workflowResult) {
System.out.print("Marking future as Completed");
future.set(workflowResult);
}
}
public class WorkflowResult {
public WorkflowResult(boolean s, String n) {
this.success = s;
this.note = n;
}
private boolean success;
private String note;
}
public class WorkflowStarter {
#Autowired
ReturnResultActivityClient returnResultActivityClient;
#Autowired
DummyWorkflowClientExternalFactory dummyWorkflowClientExternalFactory;
#Autowired
AmazonSimpleWorkflowClient swfClient;
String domain = "test-domain;
boolean isRegister = true;
int days = 7;
int terminationTimeoutSeconds = 5000;
int threadPollCount = 2;
int taskExecutorThreadCount = 4;
public String testWorkflow() throws Exception {
SettableFuture<WorkflowResult> workflowResultFuture = SettableFuture.create();
String taskListName = "testTaskList-" + RandomStringUtils.randomAlphabetic(8);
ReturnResultActivity activity = new ReturnResultActivityImpl(workflowResultFuture);
SpringActivityWorker activityWorker = buildReturnResultActivityWorker(taskListName, Arrays.asList(activity));
DummyWorkflowClientExternalFactory factory = new DummyWorkflowClientExternalFactoryImpl(swfClient, domain);
factory.getClient().doSomething(taskListName)
WorkflowResult result = workflowResultSettableFuture.get(20, TimeUnit.SECONDS);
return "Call result note - " + result.getNote();
}
public SpringActivityWorker buildReturnResultActivityWorker(String taskListName, List activityImplementations)
throws Exception {
return setupActivityWorker(swfClient, domain, taskListName, isRegister, days, activityImplementations,
terminationTimeoutSeconds, threadPollCount, taskExecutorThreadCount);
}
}
public class Workflow {
#Autowired
private DummyActivityClient dummyActivityClient;
#Autowired
private ReturnResultActivityClient returnResultActivityClient;
#Override
public void doSomething(final String resultActivityTaskListName) {
Promise<Void> activityPromise = dummyActivityClient.dummyActivity();
returnResult(resultActivityTaskListName, activityPromise);
}
#Asynchronous
private void returnResult(final String taskListname, Promise waitFor) {
ActivitySchedulingOptions schedulingOptions = new ActivitySchedulingOptions();
schedulingOptions.setTaskList(taskListname);
WorkflowResult result = new WorkflowResult(true,"All successful");
returnResultActivityClient.returnResult(result, schedulingOptions);
}
}
The standard pattern is to host a special activity in the workflow starter process that is used to deliver the result. Use a process specific task list to make sure that it is routed to a correct instance of the starter. Here are the steps to implement it:
Define an activity to receive the result. For example "returnResultActivity". Make this activity implementation to complete the Future passed to its constructor upon execution.
When the workflow is started it receives "resultActivityTaskList" as an input argument. At the end the workflow calls this activity with a workflow result. The activity is scheduled on the passed task list.
The workflow starter creates an ActivityWorker and an instance of a Future. Then it creates an instance of "returnResultActivity" with that future as a constructor parameter.
Then it registers the activity instance with the activity worker and configures it to poll on a randomly generated task list name. Then it calls "start workflow execution" passing the generated task list name as an input argument.
Then it wait on the Future to complete. The future.get() is going to return the workflow result.
Yes, if you are using the AWS Flow Framework a timeout exception is thrown when activity is timed out. If you are not using the Flow framework than you are making your life 100 times harder. BTW the workflow timeout is thrown into a parent workflow as a timeout exception as well. It is not possible to catch a workflow timeout exception from within the timing out instance itself. In this case it is recommended to not rely on workflow timeout, but just create a timer that would fire and notify workflow logic that some business event has timed out.
Because a single activity execution has multiple events associated to it. It should be pretty easy to write code that converts history to whatever representation of activities you like. Such code would just match the events that relate to each activities. Each event always has a reference to the related events, so it is easy to roll them up into higher level representation.
Unfortunately there is no easy answer to this one. Ideally SWF would support restarting workflow by copying its history up to the failure point. But it is not supported. I personally believe that workflow should be written in a way that it never fails but always deals with failures without failing. Obviously it doesn't work in case of failures due to unexpected conditions. In this case writing workflow in a way that it can be restarted from the beginning is the simplest approach.

breeze: creating inheritance in client-side model

I'm having a weird issue with the configureMetadataStore.
My model:
class SourceMaterial {
List<Job> Jobs {get; set;}
}
class Job {
public SourceMaterial SourceMaterial {get; set;}
}
class JobEditing : Job {}
class JobTranslation: Job {}
Module for configuring Job entities:
angular.module('cdt.request.model').factory('jobModel', ['breeze', 'dataService', 'entityService', modelFunc]);
function modelFunc(breeze, dataService, entityService) {
function Ctor() {
}
Ctor.extend = function (modelCtor) {
modelCtor.prototype = new Ctor();
modelCtor.prototype.constructor = modelCtor;
};
Ctor.prototype._configureMetadataStore = _configureMetadataStore;
return Ctor;
// constructor
function jobCtor() {
this.isScreenDeleted = null;
}
function _configureMetadataStore(entityName, metadataStore) {
metadataStore.registerEntityTypeCtor(entityName, jobCtor, jobInitializer);
}
function jobInitializer(job) { /* do stuff here */ }
}
Module for configuring JobEditing entities:
angular.module('cdt.request.model').factory(jobEditingModel, ['jobModel', modelFunc]);
function modelFunc(jobModel) {
function Ctor() {
this.configureMetadataStore = configureMetadataStore;
}
jobModel.extend(Ctor);
return Ctor;
function configureMetadataStore(metadataStore) {
return this._configureMetadataStore('JobEditing', metadataStore)
}
}
Module for configuring JobTranslation entities:
angular.module('cdt.request.model').factory(jobTranslationModel, ['jobModel', modelFunc]);
function modelFunc(jobModel) {
function Ctor() {
this.configureMetadataStore = configureMetadataStore;
}
jobModel.extend(Ctor);
return Ctor;
function configureMetadataStore(metadataStore) {
return this._configureMetadataStore('JobTranslation', metadataStore)
}
}
Then Models are configured like this :
JobEditingModel.configureMetadataStore(dataService.manager.metadataStore);
JobTranslationModel.configureMetadataStore(dataService.manager.metadataStore);
Now when I call createEntity for a JobEditing, the instance is created and at some point, breeze calls setNpValue and adds the newly created Job to the np SourceMaterial.
That's all fine, except that it is added twice !
It happens when rawAccessorFn(newValue); is called. In fact it is called twice.
And if I add a new type of job (hence I register a new type with the metadataStore), then the new Job is added three times to the np.
I can't see what I'm doing wrong. Can anyone help ?
EDIT
I've noticed that if I change:
metadataStore.registerEntityTypeCtor(entityName, jobCtor, jobInitializer);
to
metadataStore.registerEntityTypeCtor(entityName, null, jobInitializer);
Then everything works fine again ! So the problem is registering the same jobCtor function. Should that not be possible ?
Our Bad
Let's start with a Breeze bug, recently discovered, in the Breeze "backingStore" model library adapter.
There's a part of that adapter which is responsible for rewriting data properties of the entity constructor so that they become observable and self-validating and it kicks in when register a type with registerEntityTypeCtor.
It tries to keep track of which properties it has rewritten. The bug is that it records the fact of rewrite on the EntityType rather than on the constructor function. Consequently, every time you registered a new type, it failed to realize that it had already rewritten the properties of the base Job type and re-wrapped the property.
This was happening to you. Every derived type that you registered re-wrapped/re-wrote the properties of the base type (and of its base type, etc).
In your example, a base class Job property would be re-written 3 times and its inner logic executed 3 times if you registered three of its sub-types. And the problem disappeared when you stopped registering constructors of sub-types.
We're working on a revised Breeze "backingStore" model library adapter that won't have this problem and, coincidentally, will behave better in test scenarios (that's how we found the bug in the first place).
Your Bad?
Wow that's some hairy code you've got there. Why so complicated? In particular, why are you adding a one-time MetadataStore configuration to the prototypes of entity constructor functions?
I must be missing something. The code to register types is usually much smaller and simpler. I get that you want to put each type in its own file and have it self-register. The cost of that (as you've written it) is enormous bulk and complexity. Please reconsider your approach. Take a look at other Breeze samples, Zza-Node-Mongo for example.
Thanks for reporting the issue. Hang in there with us. A fix should be arriving soon ... I hope in the next release.

Creating objects with dependencies - dependency injection

Let's say we have class:
public class WithDependencies
{
public WithDependencies(IAmDependencyOne first, IAmDependencyTwo second)
// ...
}
Now the question. How do you create objects of WithDependencies class in an application?
I know there are many ways.
new WithDependencies(new DependencyOne(), new DependencyTwo());
new WithDependencies(IoC.Resolve(IDependencyOne), IoC.Resolve(IDependencyTwo());
// register IDependencyOne, IDependencyTwo implementations at app start
IoC.Resolve(WithDependencies);
// register IDependencyOne, IDependencyTwo implementations at app start
// isolate ourselves from concrete IoC Container
MyCustomWithDependenciesFactory.Create();
and so on...
What do you think is the way to do it?
Edit:
Because I don't get answers or I don't understand them I'll try to ask again. Let's say that on some event (button, timer, whatever) I need new object WithDependencies(). How do I create it? Assume IoC container is already configured.
It depends on the context, so it's impossible to provide a single answer. Conceptually you'd be doing something like this from the Composition Root:
var wd = new WithDependencies(new DependencyOne(), new DependencyTwo());
However, even in the absence of a DI Container, the above code isn't always unambiguously the correct answer. In some cases, you might want to share the same dependency among several consumers, like this:
var dep1 = new DependencyOne();
var wd = new WithDependencies(dep1, new DependencyTwo());
var another = AnotherWithDependencies(dep1, new DependencyThree());
In other cases, you might not want to share dependencies, in which case the first option is more correct.
This is just a small glimpse of an entire dimension of DI concerned with Lifetime Management. Many DI Containers can take care of that for you, which is one excellent argument to prefer a DI Container over Poor Man's DI.
Once you start using a DI Container, you should follow the Register Resolve Release pattern when resolving types, letting Auto-wiring take care of the actual composition:
var wd = container.Resolve<WithDependencies>();
The above example assumes that the container is already correctly configured.
If you need to create a dependency which has its own dependencies, you can either A) do it yourself, or B) ask something else to do it for you. Option A negates the benefits of dependency injection (decoupling, etc.), so I would say option B is a better starting point. Now, we have chosen to use the factory pattern, no matter whether it takes the form of a service locator (i.e. IoC.Resolve), a static factory, or an instance factory. The point is that we have delegated that responsibility to an external authority.
There are a number of trade-offs required for static accessors. (I went over them in another answer, so I won't repeat them here.) In order to avoid introducing a dependency on the infrastructure or the container, a solid option is to accept a factory for creating WithDependencies when we need an instance somewhere else:
public class NeedsWithDependencies
{
private readonly IWithDependenciesFactory _withDependenciesFactory;
public NeedsWithDependencies(IWithDependenciesFactory withDependenciesFactory)
{
_withDependenciesFactory = withDependenciesFactory;
}
public void Foo()
{
var withDependencies = _withDependenciesFactory.Create();
...Use the instance...
}
}
Next, we can create a container-specific implementation of the factory:
public class WithDependenciesFactory : IWithDependenciesFactory
{
private readonly IContainer _container;
public WithDependenciesFactory(IContainer container)
{
_container = container
}
public WithDependencies Create()
{
return _container.Resolve<WithDependencies>();
}
}
Now NeedsWithDependencies is completely isolated from any knowledge of how WithDependencies gets created; it also exposes all its dependencies in its constructor, instead of hiding dependencies on static accessors, making it easy to reuse and test.
Defining all those factories can get a little cumbersome, though. I like Autofac's factory relationship type, which will detect parameters of the form Func<TDependency> and automatically inject a function which serves the same purpose as the hand-coded factory above:
public class NeedsWithDependencies
{
private readonly Func<WithDependencies> _withDependenciesFactory;
public NeedsWithDependencies(Func<WithDependencies> withDependenciesFactory)
{
_withDependenciesFactory = withDependenciesFactory;
}
public void Foo()
{
var withDependencies = _withDependenciesFactory();
...Use the instance...
}
}
It also works great with runtime parameters:
public class NeedsWithDependencies
{
private readonly Func<int, WithDependencies> _withDependenciesFactory;
public NeedsWithDependencies(Func<int, WithDependencies> withDependenciesFactory)
{
_withDependenciesFactory = withDependenciesFactory;
}
public void Foo(int x)
{
var withDependencies = _withDependenciesFactory(x);
...Use the instance...
}
}
Sometimes I try to get rid of factories or at least not depend directly on them, so Dependency Injection (without factories) is useful of course.
Therefore I use Google Juice, cause its a small little framework using Java Annotations and you can quickly change your injections / dependencies. Just take a look at it:
http://code.google.com/p/google-guice/

File IO within an ASP.NET MVC Action

Is it possible to use some kind of 'critical section' so that it is safe to do something like the following within an action...
public ActionResult GenerateTasks()
{
string someDir = ....
if (!Directory.Exists(someDir))
{
Directory.CreateDirectory(someDir);
}
...
}
You can do this only by using a system-wide mutex. Process or app-domain locking primitives will fail to work under certain conditions (for instance when an application pool is recycled).
However, for the specific case here that's not necessary: Directory.CreateDirectory already does implement an existence check on its own, so that you shouldn't need to do anything in this regard.
I'm assuming by your question that the concurrent safety you're interested in is whether or not the directory is created between the Directory.Exists and the Directory.CreateDirectory on a different thread. (If you're concerned about Directory.CreateDirectory throwing an exception if the directory already exists, it won't.) If so, and this is the point in your code that will have the potential to do that, then you can simply use a lock object to make these set of operations safe across multiple threads:
private static object lockObject = new object();
public ActionResult GenerateTasks()
{
string someDir = ....
lock(lockObject)
{
if (!Directory.Exists(someDir))
{
Directory.CreateDirectory(someDir);
}
}
...
}
This does not however make any garauntees that the directory isn't being interacted with outside of your control, say, in another application process.

Handling dependencies with IoC that change within a single function call

We are trying to figure out how to setup Dependency Injection for situations where service classes can have different dependencies based on how they are used. In our specific case, we have a web app where 95% of the time the connection string is the same for the entire Request (this is a web application), but sometimes it can change.
For example, we might have 2 classes with the following dependencies (simplified version - service actually has 4 dependencies):
public LoginService (IUserRepository userRep)
{
}
public UserRepository (IContext dbContext)
{
}
In our IoC container, most of our dependencies are auto-wired except the Context for which I have something like this (not actual code, it's from memory ... this is StructureMap):
x.ForRequestedType().Use()
.WithCtorArg("connectionString").EqualTo(Session["ConnString"]);
For 95% of our web application, this works perfectly. However, we have some admin-type functions that must operate across thousands of databases (one per client). Basically, we'd want to do this:
public CreateUserList(IList<string> connStrings)
{
foreach (connString in connStrings)
{
//first create dependency graph using new connection string
????
//then call service method on new database
_loginService.GetReportDataForAllUsers();
}
}
My question is: How do we create that new dependency graph for each time through the loop, while maintaining something that can easily be tested?
To defer the creation of an object until runtime, you can use a factory:
public interface ILoginServiceFactory
{
ILoginService CreateLoginService(string connectionString);
}
Usage:
public void CreateUserList(IList<string> connStrings)
{
foreach(connString in connStrings)
{
var loginService = _loginServiceFactory.CreateLoginService(connString);
loginService.GetReportDataForAllUsers();
}
}
Within the loop, do:
container.With("connectionString").EqualTo(connString).GetInstance<ILoginService>()
where "connectionString" is the name of a string constructor parameter on the concrete implementation of ILoginService.
So most UserRepository methods use a single connection string obtained from session, but several methods need to operate against a list of connection strings?
You can solve this problem by promoting the connection string dependency from IContext to the repository and adding two additional dependencies - a context factory and a list of all the possible connections strings the repository might need to do its work:
public UserRepository(IContextFactory contextFactory,
string defaultConnectionString,
List<string> allConnectionStrings)
Then each of its methods can build as many IContext instances as they need:
// In UserRepository
public CreateUserList() {
foreach (string connString in allConnectionStrings) {
IContext context = contextFactory.CreateInstance(connString);
// Build the rest of the dependency graph, etc.
_loginService.GetReportDataForAllUsers();
}
}
public LoginUser() {
IContext context = contextFactory.CreateInstance(defaultConnectionString);
// Build the rest of the dependency graph, etc.
}
We ended up just creating a concrete context and injecting that, then changing creating a wrapper class that changed the context's connection string. Seemed to work fine.

Resources