Migrating from monolith to a microservice architecture having a single API gateway to access multiple services i.e., cars, shoes, mobiles etc. I'm using NET 6, Docker, Docker-Compose, Ocelot for my project. I'd highly appreciate your feedbacks on my question below based on two scenarios.
Scenario 1
Number of solutions [ApiGateway.sln, Cars.sln, Shoes.sln, Mobiles.sln, ...]
Docker Container [ApiGateway, Cars, Shoes, Mobiles, ...]
Docker Sub Containers for Cars [Hyundai, Honda, ...]
Ocelot used for [ApiGateway, Cars, Shoes, Mobiles]
Sub-ApiGateways: used for all services. MasterAPIGateway will interact with the SubApiGateways of each services.
Details: For an instance, a call for getting all cars of Hyundai is made. So the MasterApiGateway calls the cars service. Now the car serivce uses its own ApiGateways configured using Ocelot to call the required project i.e., Hyundai.csproj methods.
Yes this can be simplied by removing the ocelot from Cars and converting projects into methods.
Scenario 2
Number of solutions [ApiGateway.sln, Services.sln]
Docker Container [ApiGateway, Services]
Docker Sub Containers for Services [Cars, Mobiles, Shoes, ...]
Ocelot used for [ApiGateway]
Details: This is too mainstream but what if each services cars is a big project in itself. Due to which I've tried to separate the services i.e., cars.service, mobile.services hosted in differnt ports as seen in the above image. Again what if services has a huge module i.e., cars.services.honda has over 1000 methods. Due to which I've created sub projects within Cars again hosted in different ports. However, I am trying to encapsulate these sub projects as a single service i.e., for cars only 5000 port will be used by the MasterApiGateway.
Please do suggest me best way to achieve this. Again each service and sub projects within each services is a huge project. So having all these in one solution is something I'm trying to avoid. Thank you for your feedbacks.
this is a design problem and it is highly abstract and depends on business requirements so there is no absolute solution.
the scenario that you have car service and has api for each car may looks proper one BUT as you said each one of them is huge. THIS IS MY OPONION AND NOT A SOLUTION:
if it is just HUGE in data dont bother your self its better go fore one service of car
if all type of cars share same sort of functionality (methods , process..etc) then one service is ok.
if each car type has its own methods and process (not just getting data) then you have complexity in business logic then go for services for each car or one main service of car with similar functionality which can be support by services specific to cars type which contains specific functionality to the car type. here the car service may be play role as aggregator service.
if the car service become very very huge in code size in such a way that the maintenance require more than 5 colleagues (the number may be vary depend on organization size etc) then it should break in pieces.
also look at ubiquitous language in Domain Driven Design. at least it helps your architecture to be more appropriate by hard communication with domain experts.
your problem is the very challenging part of microservices (true microservices) and its out of my experience (i am still studying the microservice architecture and always i find the huge mistakes on my previous works). so please discuss and study and just dont rely what i said.
these two articles are very very useful
decompose-by-subdomain decompose-by-business-capability
The first question you should ask yourself is why do you need microservices, and usually it's better to start with a modular monolith and then break out a service at a time when needed...
You really need to have a clear understanding of the reason why you do it and not just for the fun of creating a solution like this.
I agree what Rouzbeh says about domain driven design, start there! and find your true bounded contexts using the ubiquitous language as a guide.
Related
I know that the question seems odd, but I'll try to explain it as well as I can.
I have a docker-compose file with 4 services: zookeeper, kafka, schema registry and redis. Besides that, I also have 2 SpringBoot microservices which use the dockerized services. The first microservice receives HTTP requests from Postman and then processes those requests before sending them to a topic. The second microservice reads from that topic, processes the messages and sends them to, you've guessed it, another topic.
How would a UML use-case diagram for this situation look like? I have no idea how to start because I've always identified roles first and took it from there. In this case, there are no people actually using it because it's just meant to collect some data and manipulate it.
My professor told me I had to have a use-case diagram, so I suppose it has to be possible to draw it, I'm just confused in how to start.
Also, if you have any recommendations on which diagrams can explain microservices nicely, let me know.
You are already focusing on a technical solution, i.e. how should it work. You have described here in details a system made of several microservices, and using some identified technologies for their implementation.
But the systems have a purpose. They are meant to interact with some actors and help themto achieve some goals. This is what use-cases are about: what's the purpose of the system for which actor. And whether it’s 12 microservices or 1 monolith, does absolutely not influence the use-cases, since these are independent of the inner structure of the system.
Unfortunately, nothing in the narrative tells us what this system is supposed to to nor for whom. This is where you should start.
Finally actors of the UC diagram can be human users or technical systems (i.e. independent systems).
You can use feature in use case diagram called include. Basically, include means a use-case will call other specific use-case. (see https://www.uml-diagrams.org/use-case-diagrams.html for reference). If you have chain of activity on multiple microservices, you can connect them using include like so:
Note that use case diagram doesn't care about technology used to build an app. It doesn't care what's under the include between your services. It was intended as pure abstraction to allow non-developers to understand it.
I'm in the process of designing a web-service hosted with Google App Engine comprised of three parts, a client website (or more), a simple CMS I designed to edit and view the content of that website, and lastly a server component to communicate between these two services and the database. I am new to docker and currently doing research to figure out how exactly to set up my containers along with the structure of my project.
I would like each of these to be a separate service, and therefor put them in different containers. From my research it seems perfectly possible to put them in separate containers and still have them communicate, but is this the optimal solution? Also given that in the future I might want to scale up so that my backed can supply multiple different frontends all managed from the same CMS.
tldr:
How should I best structure my web-service with docker, as well as assuming my back-end supplies more than one front end managed from a CMS.
Any suggestion for tools, or design patterns that make my life easier are welcome!
Personally, I don't like to think of designing whatever in terms of containers. Containers should be good for deployment process, for their main goal.
If you keep your logic in separate components/services you'll be able to combine them within containers in many different ways.
Once you have criteria what suits your product requirements (performance, price, security etc) you'll configure your docker images in the way you prefer.
So my advise is focus on design of your application first. Start from the number of solutions you have, provide a dockerfile for each one and then see what you will have to change.
I'm in the early stages of prototyping a Rails 3 application that will expose a public API. The site has three separate concerns which I am planning to split across three subdomains.
api.mysite.com
The publicly exposed API.
admin.mysite.com
The admin portal for creating blogs (using the public API).
x.mysite.com
The public blog site created at admin.mysite.com where x is the name of the blog. This too will make use of the public API.
All three will share domain objects. For example, you should be able to login to admin.mysite.com using an account you created on api.mysite.com or x.mysite.com.
Questions
Should I attempt to build one rails application to handle all three concerns or should I split this in multiple applications each handling a specific concern?
What are the Pros & Cons of each?
Does anyone have any insight into how some of the larger sites (basecamp, github, shopify) are organized?
Your question is fairly general so I'll try and answer in general terms. And the fact that you mention "larger sites" leads me to the conclusion that you're concerned about scaling.
In the beginning it is definitely going to be easier to build one application - especially since the domain is shared. You can do separate controllers for the various interfaces (api, html, etc) but with a shared back-end. This will reduce code duplication and the complexity of keeping 3 apps in sync. Also remember that you might change your mind about features based on user feedback and you want to be nimble enough to respond quickly.
The main benefit I can see of separating out three different deployables is that you can have an independent deploy schedule for each. For example, a bug fix in the api won't have to wait for admin to be ready to deploy. Or that you can have separate teams working in parallel.
If you're careful about what you keep in your session you'll be able to deploy multiple instances of your application on multiple servers, pointing at the same database (a.k.a. horizontal scaling). Each of these instances are identical to the others and a load balancer (either dedicated hardware or virtual) directs traffic between them. Eventually this approach runs out of steam when your database can't handle the load. At that point you can look at more caching, sharding, no-sql and all sorts of clever scaling techniques.
Most (but not all) larger sites end up doing some sort of horizontal scaling with some sharding of data.
All told, focus on getting a useful application to your users. If things take off you can worry about scaling. More applications fail because the user experience is awful rather than not being able to scale.
This is a general question about design. What's the best way to communicate between your business layer and presentation layer? We currently have a object that get pass into our business layer and the services reads the info from the object and sets the result into the object. When the service are finish, we'll have a object populated with result from business layer and then the UI can display according to the result of the object.
Is this the best approach? What other approach are out there?
Domain Driven Design books (the quickly version is freely avaible here) can give you insights into this.
In a nutshell, they suggest the following approach: the model objects transverse from model tier to view tier seamlessly (this can be tricky if you are using static typed languages or different languages on clinet/server, but it is trivial on dynamic ones). Also, services should only be used to perform action that do not belong to the model objects themselves (or when you have an action that involves lots of model objects).
Also, business logic should be put into model tier (entities, services, values objects), in order to prevent the famous anemic domain model anti pattern.
This is another approach. If it suits you, it depends a lot on the team, how much was code written, how much test coverage you have, how long the project is, if your team is agile or not, and so on. Domain Driven Design quickly discusses it even further, and any decision would be far less risky if you at least skim over it first (getting the original book from Eric Evans will help if you choose to delve further).
We use a listener pattern, and have events in the business layer send information to the presentation layer.
It depends on your architecture.
Some people structure their code all in the same exe or dll and follow a standard n-tier architecture.
Others might split it out so that their services are all web services instead of just standard classes. The benefit to this is re-usable business logic installed in one place within your physical infrastructure. So single changes apply accross all applications.
Software as a service and cloud computing are becoming the platform where things are moving towards. Amazons Elastic cloud, Microsofts Azure and other cloud providers are all offering numerous services which may affect your decisions for architecture.
One I'm about to use is
Silverlight UI
WCF Services - business logic here
NHibernate data access
Sql Server Database
We're only going to allow the layers of the application to talk via interfaces so that we can progress upto Azure cloud services once it becomes more mature.
Does anyone have advice or tips on using a web service as the model in an ASP.Net MVC application? I haven't seen anyone writing about doing this. I'd like to build an MVC app, but not tie it to using a specific database, nor limit the database to the single MVC app. I feel a web service (RESTful, most likely ADO.Net Data Services) is the way to go.
How likely, or useful, is it for your MVC app to be decoupled from your database? How often have you seen, in your application lifetime, a change from SQL Server to Oracle? From the last 10 years of projects I've delivered, it's never happened.
Architectures are like onions, they have layers of abstractions above things they depend on. And if you're going to use an RDBMS for storage, that's at the core of your architecture. Abstracting yourself from the DB so you can swap it around is very much a fallacy.
Now you can decouple your database access from your domain, and the repository pattern is one of the ways to do that. Most mature solutions use an ORM these days, so you may want to have a look at NHibernate if you want a mature technology, or ActiveRecord / linq2sql for a simpler active record pattern on top of your data.
Now that you have your data strategy in place, you have a domain of some sort. When you expose data to your client, you can choose to do so through an MVC pattern, where you'll usually send DTOs generated from your domain for rendering, or you can decide to leverage an architecture style like REST to provide more loosely coupled systems, by providing links and custom representations.
You go from tight coupling to looser coupling as you go towards the external layers of your solution.
If your question however was to build an MVC app on top of a REST architecture or web services, and use that as a model... Why bother? If you're going to have a domain model, why not reuse it in your system and your services where it makes sense?
Generating a UI from an MVC app and generating documents needed for a RESTful architecture are two completely different contexts, basing one on top of each other is just going to cause much more pain than needed. And you're sacrificing performance.
Depends on your exact scenario, but remote XML-based service as the model in MVC, from experience, not a good idea, it's probably over-engineering and disregarding the need for a domain to start with.
Edit 2010-11-27; clarified my thoughts, which was really needed.
A web service exposes functionality across different types of applications, not for abstraction in one single application, most often. You are probably thinking more of a way of encapsulating commands and reads in a way that doesn't interfere with your controller/view programming.
Use a service from a service bus if you're after the decoupling and do an async pattern in your async pages. You can see Rhino.ServiceBus, nServiceBus and MassTransit for .Net native implementations and RabbitMQ for something different http://blogs.digitar.com/jjww/2009/01/rabbits-and-warrens/.
Edit: I've had some time to try rabbit out in a way that pushed messages to my service which in turn pushed updates to the book keeping app. RabbitMQ is a message broker, aka a MOM (message oriented middle-ware) and you could use it to send messages to your application server.
You can also simply provide service interfaces. Read Eric Evan's Domain Driven Design for a more detailed description.
REST-ful service interfaces deal a lot with data, and more specifically with addressable resources. It can greatly simplify your programming model and allows great control over output through the HTTP protocol. WCF's upcoming programming model uses true rest as defined in the original thesis, where each document should to some extent provide URIs for continued navigation. Have a look at this.
(In my first version of this post, I lamented REST for being 'slow', whatever that means) REST-based APIs are also pretty much what CouchDB and Riak uses.
ADO.Net is rather crap (!) [N+1 problems with lazy collection because of code-to-implementation, data-access leakage - you always need your db context where your query code is etc] in comparison to for example LightSpeed (commercial) or NHibernate. Spring.Net also allows you to wrap service interfaces in their contain with a web service facade, but (without having browsed it for a while) I think it's a bit too xmly in its configuration.
Edit 1: With ADO.Net here I mean the default "best practice" with DataSets, DataAdapter and iterating lots of rows from a DataReader; it breeds rather ugly and hard-to-debug code. The N+1 stuff, yes, that is about the entity framework.
(Edit 2: EntityFramework doesn't impress me either!)
Edit 1: Create your domain layer in a separate assembly [aka. Core] and provide all domain and application services there, then import this assembly from your specific MVC application. Wrap data access in some DAO/Repository, through an interface in your core assembly, which your Data assembly then references and implements. Wire up interface and implementation with IoC. You can even program something for dynamic service discovery with the above mentioned service buses, to solve for the interfaces. WCF uses interfaces like this and so do most of the above service busses; you can provide a subcomponentresolver in your IoC container to do this automatically.
Edit 2:
A great combo for the above would be CQRS+EventSourcing+ReactiveExtensions. Your write-model would take commands, your domain model would decide whether to accept them, it would push events to the reactive-extensions pipeline, perhaps also over RabbitMQ, which your read-model would consume.
Update 2010-01-02 (edit 1)
The jest of my idea has been codified by something called MindTouch Dream. They have made a screencast where they treat almost all parts of a web application as a (web)-service, which also is exposed with REST.
They have created a highly parallel framework using co-routines to handle this, including their own elastic thread pool.
To all the nay-sayers in this question, in ur face :p! Listen to this screen-cast, especially at 12 minutes.
The actual framework is here.
If you are into this sort of programming, have a look at how monads work and their implementations in C#. You can also read up on CoRoutines.
Happy new year!
Update 2010-11-27 (edit 2)
It turned out CoRoutines got productized with the task parallel library from Microsoft. Your Task now implement the same features, as it implements IAsyncResult. Caliburn is a cool framework that uses them.
Reactive Extensions took the monad comprehensions to the next level of asynchronocity.
The ALT.Net world seems to be moving in the direction I talked about when I wrote this answer the first time, albeit with new types of architectures I knew little of.
You should define your models in a data access agnostic way, e.g. using Repository pattern. Then you can create concrete implementations backed by specific data access technologies (Web Service, SQL, etc).
It really depends on the size of this mvc project. I would say keep the UI and Domain in same running environment if the website is going to be used by a small number of users ( < 5000).
On the other side, if you are planning on a site that is going to be accessed by millions, you have to think distributed and that means you need to build your website in a way that it can scale up/out. That means you might need to use extra servers (Web, application and database).
For this to work nicely, you need to decouple your mvc UI site from the application. The application layer would usually contain your domain model and might be exposed through WCF or a service bus. I would prefer a Service Bus because it is more reliable and might use persistent queues like msmq.
I hope this helps