How can I design a database for storing custom integrations for each client in an application where different clients can have integrations with different services (e.g. ATS, HRIS), the catch is Client Alpha can have integration with ATS service X where as client Beta can have integration with service HRIS Y or ATS Z.? I have a single database with a company_id in most of the tables.
Looking for a solution to manage the integration information efficiently. I really appreciate it if someone can shed some light on how I can store the integration information for each client. Thanks in advance.
What you describe is a many-to-many (M:M) relationship Clients and Services. This typically results in 3 tables:
clients: information about the individual client.
services: information about each service offered.
client_services: a resolution table having clients and their selected services and any attribute(s) requiring the combination.
It also permits, but does not require, multiple clients to have the same service.
Demo here.
Related
An ex-employee planned a Microservice Architecture which is being implemented now. I've few question regarding the design and I'd highly appreciate your feedbacks.
Explanation
Dematerialized UI has a matching dematerialized API.
Dematerailized API validates the user and generates token via SSO Library.
Flight API does the I/O validation & validate the request via validate request microservice
Flight API calls Booking API to get some bookings based on the UserId
Flight API calls Print Booking API to generate Messages using Generate Message Microservice
Print Booking API must call Data Access API to get data and then call Generate PDF microservices.
Data Access API calls the database for data.
My Project Structure
FlightBookingsMicroserice.V1 //solution
ApiGatways //folder
DMZ.API/DMZ.API.csproj //Folder/project
BuildingBlocks
EventBus/EventBus.csproj
EventBus/EventBusRabbitMQ
Services
SSO
SSO.API/SSO.csproj
SSO.UnitTests
Flight
Flight.API/Flight.API.csproj
Flight.UnitTets
//Similar for all
ValidationRequest
Booking
PrintBooking
PrintBooking.API.csproj
DataAccess
DataAccess.API.csproj
GeneratePDF
GenerateMessage
UI
UI
Docker-compose
Questions
Should I be using ocelot in DMZ.API.csproj, Flight API and Print Booking API.
Is my project structure a Microservice way of development
Should I continue to use ASP.NET Core Web API with .NET 6 for Dematerialized API in orange, Function API in blue and Microservice in purple projects.
For validation, since the SSO is passed from Dematerialized UI what if the token expires while CRUD operations
is already performed for some stages [rolling back changes is a hassle].
Should each API access to an identidy server and validate the user passed and generate its own token for its
services in purple.
Thank you in advance.
The core question is if you really need all those services and if you perhaps are making things too complicated. I think the important thing is to really consider and really make sure you justify why you want to go through this route.
If you do synchronous API calls between the services, that creates coupling and in the long run a distributed monolith.
For question #4, you typically use one access token for the user to access the public service, and then you use a different set of internal tokens (machine-to-machine also called client credentials in OpenID Connect parlor) between services that have a totally different lifetime.
q1: ocelot is an API GATEWAY which is the entry point for your requests. so it should be the first layer/service meet by user request in front of your services and it forwards the request to the service according to its configuration. so it is lay in the front for all services you have. some arch provide another api gateway for different reasons like specific api gateway for mobiles request for example.
q2: as looking separate services (i cant understand function api but i assume they are services also ) yes but the microservices development is not just about separating things, its about design and identifying the services from business context (Domain Driven Design).its very challenging to identify services and their size and the way they are communicate to each other (asynchronous communication and synchronous communication).
q3: microservices is not about languages and frameworks.one of benefits of microservices architecture is its not language or framework dependent. the may be multiple languages used in microservices. choosing languages it depends on organization policy or your own reasons. if you are .net developer then go for .net.
q4: all the services are registered with identity server and they validate the given token by it. the identity server generate token (there may be multiple tokens) with scopes . the request from identified users always has the token in the headers and the services validate incoming token by referring identity server. this tokens has lifetime and also identity server generates refresh tokens in case of expiry of current token. please look at Oauth docs and rfc. also this https://www.youtube.com/watch?v=Fhfvbl_KbWo&list=PLOeFnOV9YBa7dnrjpOG6lMpcyd7Wn7E8V may helped. you can skip the basic topics. i learned a lot from this series.
i have several databases running in RDS Service.
I'd like to know the best pratice to grant access to developers to these DB.
I tought a solution using jenkins but i dont think this is the best option.
I am trying to avoid give some password to developers.
Hope you can help me.
As #ceejayoz mentioned you can create a few users with restricted privileges, for example an user who only can run selects on few schemas, another user who can update registers in a few tables.
I can share what we do and what I've seen. We do A and use B where it is easy.
A) Standard Users
For all databases, we have 3 standard users with the following suffixes (_dba, _rw, _ro). Those all have their own passwords using a strong password generator.
_dba is used to deploy scheme and has all rights
_rw is used by the application (CRUD on all tables, but can't modify scheme)
_ro only has R on all tables and generally given to developers
Note: Developers have access to a bastion used for port forwarding and proxycap. They can query the RDS endpoints from their own machines (DB Tools) going through socks proxy and bastion.
This is lazy method - since creation of users is done programmatically and we feel comfortable giving some developers read only access. They could write a bad query and slow down system, but they could do that with a specific user so not much different and the bastion logs tell me who really was in if I had to investigate.
B) UI
Simple web app with login (ideally MFA) - that provides a way to run queries. If only for reporting, ideally against R/O copy of system. Stackoverflow offers one themselves (https://data.stackexchange.com/).
What would be nice is if RDS offered this themselves (linked to your IAM roles). They offer this on RDS Serverless (https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/query-editor.html) and it may be a feature in other RDS versions. That allows fine control or even lazy control (IAM groups).
I'm working for a non-profit that is trying to create a collection of services that will allow them to do a few things:
Create/Manage Users
Create/Manger Competitions
Create/Manager Events(a Competition is made up of many Events)
Logistics
Etc.
Here are some of the requirements:
Host on Azure
Accounts are created using the user's own email address (can be any domain)
Each service must be independent of each other
System should be accessible from anywhere(browser, mobile app, etc.)
Once a user logs in, access to other systems should be available(if needed or depending on permissions)
-services can talk to each other(we've successfully done a POC on this using Azure Active Directory)
I've spent some time researching the possible ways to tackle this, including looking at articles like this:
http://bitoftech.net/2014/10/27/json-web-token-asp-net-web-api-2-jwt-owin-authorization-server/
It sounds like JWT is the way to go, but I want to make sure that this architecture approach lends itself to flexibility down the road. I'm willing to learn/user any technology as long as it plays along .Net, Web Api, and MVC.
My initial idea was to set each "system" as a Web API, which is pretty straightforward. My concern is authentication/authorization. The million dollar question then becomes:
How can I authenticate a user on a browser/mobile/desktop app and then make use of the other services(APIs), where each service can verify if the user is authenticated independent of other services.
For example, lets say I am using the web app(browser) and register as a user(using User service) in order to register for a competition(Competition service). what kind of technologies/architecture would need to be used for both services to use the same authentication mechanism?
I am not new to MVC or Web API but I am to a scenario like this, so all help or advice is greatly appreciate it. For any that are interested, this is the non-profit: http://worldjumprope.org/
They are doing some cool stuff in terms of outreach and spreading the love for jump rope. Their goal is to be able to help people all around the world and provide a way to for them to come together and compete. They've been doing it for years, growing each year, all for free and out of pure passion for the sport. Help me help them!
Azure Active Directory can help you with your need.
About the security/Auth mechanism, simply put, it is like you get a token from Azure Active Directory to be able to use it to do authentication for a certain Audiences/Tenants.
And in your architecture, when you acquire a Token, you can specify if you are going to use it against a certain audience or it will work for a list of audiences or all audiences in a specific tenant or in a multi tenant scenario.
Here is a link to a video about AAD Single Sign on:
https://azure.microsoft.com/en-us/documentation/videos/overview-of-single-sign-on/
You can download the AAD Solutions arcutecture from here:
http://www.microsoft.com/en-us/download/details.aspx?id=45909
Also maybe worth looking at Identity Server - https://github.com/IdentityServer/IdentityServer3.
Same concept as Azure AD in terms of tokens, but perhaps with greater options for what you choose as a data store for your user information.
There are lots of demos and source code on the site, particularly around the different types of authentication flow \ service to service authentication
In a DELPHI 2007 application that I am developing some prospect clients just found interesting to be able to share data and information with each other.
They all have the same application.
All have independent Databases
But all have the same installed application and there are some data types that they might want to share (replicate) between their databases.
How can I enable them to share data with other users of the same application program, but not to everybody on the whole internet.
I would like this to be as automatic as possible, as I already have considered approaches that involve manually sending emails.
I know Datasnap is an option, is there any other.
UPDATE:
The idea is to enable companies that have the same application to be able to share data.
They should be able to select what partner and what to send.
I have been investigating datasnap, but would like to know if there is another way to do this
Another standard way to connect distributed applications and share data and information is through some Message-oriented middleware (MOM). There are many open source middleware products (message brokers) available, which can be used over Delphi client libraries, even in multithreaded Delphi server applications. (Disclaimer: I am the author of message broker client libraries for Delphi and Free Pascal)
There are many essential differences between web services and message brokers, like peer-to-peer and publish/subscribe communication models. They also play a key role in enterprise application integration patterns.
One standard way to connect applications to other applications is to make a web-service, and make a client that consumes that web-service, called a web-client. Technologies like SOAP and REST refer to such web service and web clients.
Your question is vague, perhaps due to english not being your language, but you should probably edit it and be more specific.
If all your applications are going to talk directly to each other that is called "peer to peer networking" and there are huge problems with enabling that kind of communication directly over the internet. It is much easier if you build a server that all these applications will connect to.
As a sample, consider the IRC Chat service, and consider writing a Web Service that will be the Chat Server, and consider all your clients to be "Chat clients". Sharing data could be the same idea as creating "rooms" or "channels" on a chat server.
I get the idea that you want something like a Peer to Peer Data Replication Service. I think that the closest you're going to get to that is something like "RSS Feeds" (used by blog syndication services). You subscribe to them via a simple web service, and pull down the new content on some periodic basis. Since that data has to be published to a central server, that means, that a peer to peer approach is out of the question. If you don't have your own web server running on a web hosting service, or on a "cloud", and you need a truly peer to peer solution, I am not aware of any way to do that, at least not without an incredible custom engineering effort.
I am about to be writing a Ruby on Rails app which will use sub-domains to authenticate users. We will have two types of accounts:
user accounts
domain accounts
Users will thus be able to belong to multiple domain accounts using the same credentials. I hope to have the ability for a domain account administrator to be able to search for particular users and add them to their domain.
In addition to simply creating a domain account in the database, I want to setup an actual account on the machine (linux-based) so that users can drop files into a special directory and we can run some scripts to import that new data. Alternatively, I may write a client/server script to make this process easier.
All of this I believe I can do, however, as soon as the project attains a certain number of domain accounts, it will be necessary to figure out how to cluster the domain accounts appropriately so that we can have multiple machines.
From a database standpoint, this is fairly easy and there are lots of tutorials on how to cluster MySQL or whichever SQL server I decide to use. So my question really pertains more to machine accounts as well as how to cluster a Rails app.
If you want a comparison, think of this project like GitHub or Beanstalk but with data that isn't source control related.
Does anybody have any experience with this or know of any really good articles/books to get me started?
Thanks very much!
I suggest you look at using one of the PAM modules that lets you do account authentication against a SQL database. That way you just add the domain account to the SQL database and you get UNIX accounts (on all your servers) automagically, for free. So the clustering should just happen for free too...