Possible to initialise Consul Key Value store at startup? - docker

I can initialise the Consul services at startup using the /consul/config directory. I would like to be able to initialise my application settings into the Consul kv store when I start the Consul container. Is this possible?

There are several projects which might be interesting for this scenario:
https://github.com/zerotens/consul-kv-bootstrap
https://github.com/cimpress-mcp/git2consul
https://github.com/cimpress-mcp/fsconsul

I wanted a simple approach to set application settings in Consul for a microservice project. I'm sure there are a number of existing tools but I wanted something that was lightweight and suits my development process. So rather than try to work with some of #mgyongyosi's suggestions I wrote a quick dotnet console application to do the (not so) heavy lifting. Under the project root directory I've created a directory structure Consul/kv. The child directories under that path represent keys in the Consul KV store and a json file at a leaf represents the application settings. Note there is only expected to be a single json file per leaf directory. The application uses the Consul.NET nuget package for communicating with Consul.
using System;
using System.IO;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using Consul;
namespace ConfigureConsulKv
{
class Program
{
private static readonly string _basePath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments),
"Projects", "MyProject", "Consul", "kv");
static void Main(string[] args)
{
RecurseDirectory(_basePath);
}
static void RecurseDirectory(string path)
{
foreach (var dir in Directory.EnumerateDirectories(path)) RecurseDirectory(dir);
var file = Directory.EnumerateFiles(path, "*.json").FirstOrDefault();
if (!string.IsNullOrWhiteSpace(file))
{
var key = path.Substring(_basePath.Length + 1);
var json = File.ReadAllBytes(file);
Console.WriteLine($"key {key} file {file}");
using (var client = new ConsulClient())
{
var attempt = client.KV.Put(new KVPair(key) { Value = json }).Result;
Console.WriteLine($"status {attempt.StatusCode}");
}
}
}
}
}

Related

Unable to resolve service for type Microsoft Extensions Configuration IConfiguration

I am getting this error, could not understand for the life of me.
Unable to resolve service for type 'Microsoft.Extensions.Configuration.IConfiguration' while attempting to activate 'Microsoft.FeatureManagement.ConfigurationFeatureSettingsProvider'.
This is a simple .net core 2.2 console app, with the following nuget packages added.
Microsoft.Extensions.Configuration.Json
Microsoft.Extensions.DependencyInjection
Microsoft.FeatureManagement
using System;
using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.FeatureManagement;
using Microsoft.FeatureManagement.FeatureFilters;
namespace ConfigurationConsoleApp
{
class Program
{
static async Task Main(string[] args)
{
const string FeatureName = "Beta";
var configuration = new ConfigurationBuilder().AddJsonFile("appsettings.json").Build();
var services = new ServiceCollection();
services.AddSingleton(configuration).AddFeatureManagement().AddFeatureFilter<PercentageFilter>().AddFeatureFilter<AccountIdFilter>();
var serviceProvider = services.BuildServiceProvider();
var featureManager = serviceProvider.GetRequiredService<IFeatureManager>();
var enabled = await featureManager.IsEnabledAsync(FeatureName);
Console.WriteLine($"The {FeatureName} feature is {(enabled ? "enabled" : "disabled")} ");
}
}
}
// The following are the command for the packages.
dotnet add package Microsoft.Extensions.Configuration.Json --version 2.1.1
dotnet add package Microsoft.Extensions.DependencyInjection --version 2.1.1
dotnet add package Microsoft.FeatureManagement --version 2.0.0-preview-010610001-1263
Ok, here it is after hours of hair pulling.
services.AddSingleton(configuration).AddFeatureManagement().AddFeatureFilter<PercentageFilter>().AddFeatureFilter<AccountIdFilter>();
should be
services.AddSingleton<IConfiguration>(configuration).AddFeatureManagement().AddFeatureFilter<PercentageFilter>().AddFeatureFilter<AccountIdFilter>();
Note the generic <IConfiguration>
Also I have noted that declaring configuration object as IConfiguration will also do the trick. Using var to declare configuration is giving the problem. Instead of var use IConfiguration. Then again the problem goes away.
Sorry for commenting this as an answer, but the above code be careful if you are using it inside a new project. It goes still into the root to search for it, might cause issues when you have 2 projects with the same application.Environment.json
var configuration = new ConfigurationBuilder().AddJsonFile("appsettings.json").Build();

How do I setup Windsor container on a console application to inject to external library

I have a console app and web API both referencing the same data layer which is a separate project.
In that data layer, I have a class that requires a repository that we are grabbing from the container when that class is instantiated.
In that class, it has a base class which we are doing the following in the constructor to setup the Repository:
IContainerAccessor containerAccessor = HttpContext.Current.ApplicationInstance as IContainerAccessor;
Repository = containerAccessor.Container.Resolve<IRepository>();
What would be the best way to set this up? This is obviously a problem for our console application as it has no HttpContext.
If I'm correct you want to setup your console app so it can inject classes from the shared data layer.
To do so, you need to create an installer for the console app and tell it to run the installers in the shared library, but to modify the life style from 'PerWebRequest' to 'Singleton' or 'Transient'.
For more information read this article:
http://blog.ploeh.dk/2010/04/26/ChangingWindsorlifestylesafterthefact/
Be aware that changing this may cause problems.
I.e.: If multiple components configured as "perWebRequest" require a 'Unit-Of-Work' to be injected, then this uow will be different for all components if you change the life style to transient.
Changing it to Singleton causes the same but opposite problem. Objects that are created now will have the same object for different requests ...
If you are okay with the problems this code should get you starting
public class ConsoleAppInstaller: IWindsorInstaller
{
public void Install(IWindsorContainer container, IConfigurationStore store)
{
// 1) make sure we do not use PerWebRequest life style types
var convertWebToTransient = new WebToTransientConvertor();
container.Kernel.ComponentModelBuilder.AddContributor(convertWebToTransient);
// 2) call installers on all libraries we use ...
container.Install(FromAssembly.Containing<SharedDataLayerInstaller>());
// 3) link internal services ...
container.Register(Component.For<IXxxxFactory>().AsFactory());
container.Register(Component.For<IYyyyFactory>().AsFactory());
container.Register(Classes.FromThisAssembly().Where(c => typeof(Form).IsAssignableFrom(c)).LifestyleTransient());
}
public static IWindsorContainer Bootstrap()
{
return new WindsorContainer().Install(FromAssembly.This());
}
}
/// <summary>
/// This class allows to intercept installers using PerWebRequest lifestyles and replaces them with Transient life styles.
/// <code>container.Kernel.ComponentModelBuilder.AddContributor(new WebToTransientConvertor())</code>
/// </summary>
public class WebToTransientConvertor : IContributeComponentModelConstruction
{
//http://blog.ploeh.dk/2010/04/26/ChangingWindsorlifestylesafterthefact/
public void ProcessModel(IKernel kernel, ComponentModel model)
{
if (model.LifestyleType == LifestyleType.PerWebRequest)
//model.LifestyleType = LifestyleType.Transient;
model.LifestyleType = LifestyleType.Singleton;
}
}

How to read file from an imported library

I have two packages: webserver and utils which provides assets to webserver.
The webserver needs access to static files inside utils. So I have this setup:
utils/
lib/
static.html
How can I access the static.html file in one of my dart scripts in webserver?
EDIT: What I tried so far, is to use mirrors to get the path of the library, and read it from there. The problem with that approach is, that if utils is included with package:, the url returned by currentMirrorSystem().findLibrary(#utils).uri is a package uri, that can't be transformed to an actual file entity.
Use the Resource class, a new class in Dart SDK 1.12.
Usage example:
var resource = new Resource('package:myapp/myfile.txt');
var contents = await resource.loadAsString();
print(contents);
This works on the VM, as of 1.12.
However, this doesn't directly address your need to get to the actual File entity, from a package: URI. Given the Resource class today, you'd have to route the bytes from loadAsString() into the HTTP server's Response object.
I tend to use Platform.script or mirrors to find the main package top folder (i.e. where pubspec.yaml is present) and find imported packages exported assets. I agree this is not a perfect solution but it works
import 'dart:io';
import 'package:path/path.dart';
String getProjectTopPath(String resolverPath) {
String dirPath = normalize(absolute(resolverPath));
while (true) {
// Find the project root path
if (new File(join(dirPath, "pubspec.yaml")).existsSync()) {
return dirPath;
}
String newDirPath = dirname(dirPath);
if (newDirPath == dirPath) {
throw new Exception("No project found for path '$resolverPath");
}
dirPath = newDirPath;
}
}
String getPackagesPath(String resolverPath) {
return join(getProjectTopPath(resolverPath), 'packages');
}
class _TestUtils {}
main(List<String> arguments) {
// User Platform.script - does not work in unit test
String currentScriptPath = Platform.script.toFilePath();
String packagesPath = getPackagesPath(currentScriptPath);
// Get your file using the package name and its relative path from the lib folder
String filePath = join(packagesPath, "utils", "static.html");
print(filePath);
// use mirror to find this file path
String thisFilePath = (reflectClass(_TestUtils).owner as LibraryMirror).uri.toString();
packagesPath = getPackagesPath(thisFilePath);
filePath = join(packagesPath, "utils", "static.html");
print(filePath);
}
To note that since recently Platform.script is not reliable in unit test when using the new test package so you might use the mirror tricks that I propose above and explained here: https://github.com/dart-lang/test/issues/110

StructureMap/Autofac: how to create objects based on web.config settings

I've got a ConfigurationReader class that I'm trying to wire up using StructureMap or AutoFac (I haven't settled on which container I'm using).
public class ConfigurationReader {
private string _filePath;
public ConfigurationReader(string filePath){
this._filePath = filePath;
}
public IList<Baz> ListStuff(){
//do something with _filePath;
}
}
There will be 1..n to instances of this class based on how the app is configured (web.config will contain a delimited list of files). I'm looking for an extension point in either IoC container that would allow me to leverage them to create instances of ConfigurationReader.
Well, in AutoFac you can just register each one in the Container (during Application_Start for example).
Whenever you need to read all configurations you can add a dependency to IEnumerable<ConfigurationReader> (or IConfigurationReader if you decide to extract an interface) and it will provide you with all of them.
Something like this:
var builder = new ContainerBuilder();
foreach(var file in ConfigurationManager.AppSettings[yourKey].Split(','))
{
var fileName = file;
builder.Register(c => new ConfigurationReader(fileName));
}
DependencyResolver.SetResolver(new AutofacDependencyResolver(builder.Build()));
If you extract interfaces, then you may want to register by adding the .AsImplementedInterfaces() or .As<IConfigurationReader>() at end as well.

Storing binary objects in Neo4j

Neo4j doesn't seem to allow me to store binary objects. Does this mean I must used Neo4j in conjuntion with another data store, such as the filesystem., Oracle, etc?
Daniel already answered that it's possible to store binary objects in Neo4J.
But i would suggest you not to do so. You can do nothing interesting with binary objects in database. You cannot search them. The only thing you will achieve by storing binary objects - grow the file size of your database. Mind you, Neo4J is not scalable horizontally. It does not have automatic sharding. So if your db grows too big, you are in trouble. By storing binaries in a file system or external distributed key-value store like riak, cassandra, hadoop etc, you are keeping your database small, which is good for performance, backups and avoiding horizontal scaling problems.
If you look in the API here: http://api.neo4j.org/1.2/org/neo4j/graphdb/PropertyContainer.html#setProperty(java.lang.String, java.lang.Object), you see that byte arrays are allowed.
Using byte-arrays you can store your binary objects. When you store binary objects (using Java) in Oracle, you load in the data as byte[] as well.
You can store binary objects as byte[] or encoded in a String, but I would recommend to store larger (e.g. > 1,000 bytes) blobs as separate files, and only keep a reference to the file in your database.
We do this in Structr (http://structr.org) as well.
As mentioned doing this is highly disadvantageous.
However if you decide to do so, you could do it like this in C#:
using Neo4jClient;
using Neo4jClient.Cypher;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Neo4JBlob
{
class Program
{
static void Main(string[] args)
{
try
{
GraphClient client = new GraphClient(new Uri("http://localhost:7474/db/data"));
client.Connect();
byte[] image = File.ReadAllBytes("image.jpg");
BlobNode blob = new BlobNode(){Blob = image, name = "An image: " + DateTime.Now.ToShortDateString()};
client.Cypher.Create("(blob:Blob {category})").WithParam("category", blob).ExecuteWithoutResults();
var res = client.Cypher.Match("(b:Blob)").Return<BlobNode>(b => b.As<BlobNode>()).Limit(1).Results;
BlobNode BlobReturned = res.First();
File.WriteAllBytes("image_returned.jpg", BlobReturned.Blob);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
Console.WriteLine(ex.StackTrace);
}
Console.ReadKey();
}
class BlobNode
{
public byte[] Blob
{
get;
set;
}
public string name
{
get;
set;
}
}
}
}

Resources