Spring Cloud Data Flow Server (Local) does not have any dynamic way to set up users and roles either through dashboard UI or shell, ie. there is no way to add or delete users with roles while the server is running.
I have been able to get both single user or file based authentication and authorization working but both of them I had to set up the docker-compose.yml file like so:
spring.cloud.dataflow.security.authentication.file.enabled=true
spring.cloud.dataflow.security.authentication.file.users.bob=bobpass, ROLE_MANAGE
spring.cloud.dataflow.security.authentication.file.users.alice=alicepass, ROLE_VIEW, ROLE_CREATE
spring.cloud.dataflow.security.authentication.file.users.hare=harepass, ROLE_VIEW
However, if I have to add new users with roles, I will have to docker-compose down, edit the docker-compose.yml and then do docker-compose up, for the new user authentication authorization to work.
Is there any work around this?
There isn't any other approach to dynamically add/update users and then have it reflect at runtime in SCDF.
However, in SCDF 2.0, we have redesigned/rewritten the security architecture. In this baseline, we rely on Cloud Foundry's UAA component, which is a standalone application that can work in Local, CF or K8s.
Here, you can directly interact with UAA outside of SCDF. You can add, update, and delete users, too. Of course, you can centrally manage the OAuth token-credentials such as remote renewals and revocations. Check out the end-to-end sample demonstration of the new design with SCDF + OAuth + LDAP, all in action.
The recent 2.0 M1 release already include this improvement - see blog. Try it out and let us know if you have any questions/feedback.
UPDATE:
I recently also bumped into a UAA Web-UI from the community. Perhaps UAA team could consider adding it to the official stack eventually.
Related
I've got Artifactory set up to allow SSO via an OIDC client in Keycloak. Keycloak talks to Active Directory in the background. I've also got the same Active Directory configured within Artifactory. I've also configured Keycloak to provide user groups in the userinfo structure.
What I'm trying to do is to get OAuth users to automatically be added to the imported LDAP groups within Artifactory. I don't mind if this is done via the userinfo structure or via a separate LDAP lookup when a user logs in. However I can't seem to figure out how to achieve this.
I know that Artifactory provides a plugin called synchronizeLdapGroups.groovy, which seems to advertise doing what I need, however it seems like the plugin is not actually taking effect. That is to say, users do not end up with the permissions that being in the LDAP groups would provide.
I've attempted to write a plugin myself to do what I need, but when I make the API call to add the groups to the user, the plugin crashes. It's unclear why at this point.
It seems like others have used the SCIM feature in Artifactory for something akin to this (mostly via SAML rather than OIDC though), however Keycloak doesn't support SCIM out of the box and the SCIM plugin I've tried using has similarly given me no results.
Has anyone done something similar to this, and has a working solution I could follow?
If it's a specific group you want all users to be in you could try:
Under Security - OAuth SSO settings tab - check "Auto Create Users"
Under Identity and Access - Groups - select the specific group and check "Automatically Join New Users To The Group"
I'm guessing you want to automatically sync user-group association between Keycloak and Artifactory. SCIM is what you're looking for but there's a known issue specifically with Keycloak SCIM plugin.
We're looking into the SCIM plugin for Keycloak (can't commit on a specific timeline for a fix but it should be sometime this quarter).
If you only need the association in the UI you could try SAML with "Auto associate groups" set. It won't apply the groups association for APIKey/Token calls but it would work for the UI.
EDIT:
after further investigation github.com/Captain-P-Goldfish/scim-for-keycloak isn't relevant here - it makes keycloak a SCIM client, not a SCIM server. There's no official support of SCIM in keycloak, see issues.redhat.com/browse/KEYCLOAK-2537 . and there's no working plugin for keycloak that makes it a SCIM server that I could find (tried a few, all broken). For now Artifactory can't support SCIM with Keycloak
I have tried the SCDF Security with UAA + LDAP example (link).
However, Im looking for some more Security approaches while using SCDF local. It 'll be great if having example (source, document)
It is not an official sample, but we have another full-blown UAA based end-to-end solution that works with local, and as well the other platform choices that we support.
See: jvalkeal/randomstuff/dataflow-uaa.
With this, you can run UAA as a standalone process served at 8080, and SCDF+Skipper running separately it can negotiate with and use the UAA backend for authentication/authorization flows. Feel free to customize the uaa.yaml with desired users, roles, and the associated scopes. Once again, all this is outside of SCDF, and it comes down to how much you'd want to rely on UAA like identity providers.
Keycloak is another excellent option for a local deployment experience. I believe there's a Docker image for Keycloak.
I am trying to setup Spring Cloud Data Flow (SCDF) to run in Local mode and how few questions which may help me decide if its a suitable platform for my requirements.
Even though the recommendation is to use Cloud Foundry, Kubernetes etc as task execution environment my preference to run things on production is local mode mainly because I don't have a lot of workload and cant deal with all the additional complexity. Now in local mode will I be able to run all types of SCDF apps, namely Streams, Jobs and tasks with no limitations? Some parts of the document mentions that only Jobs can be run in local mode.
Security - I am looking to put controls in place around deployment of apps and operational access to the tool (dashboard) and do see the support for LDAP with roles as an option but the whole concept of using Cloudfoundry UAA, another product to drive the user managements seems like an overkill. Is there no way to configure the tool with an existing LDAP server? Found the following in one of the LDAP issues in Github but its not clear whether it uses UAA in its docker image. Worst case I wont mind if the dashboard can be run in a view/read only mode.
https://github.com/spring-cloud/spring-cloud-dataflow/issues/2871
If Spring Cloud Data Flow were a purely monolithic application, integrating all aspects of security directly into the app is definitely easier to to wrap one's mind around. This is how Spring Cloud Data Flow originally started out from a security perspective and thus, versions of Spring Cloud Dataflow <2.0.0 supported what we labelled traditional security.
However, even before 2.0.x Spring Cloud Data Flow:
Had to integrate with external platforms such as Cloud Foundry
Became more and more microservices oriented (e.g. by using Skipper)
As a result 2 parallel security architectures had emerged, one using traditional security and the other one driven by OAuth2/OpenID Connect.
This started to become increasingly harder to maintain and for 2.0.x we decided to exclusively focus on OAuth2/OpenID Connect. However, we still had to support a rich set of enterprise features such as Roles, LDAP integration etc. As such, we find that the open-source, production-ready CloudFoundry User Account and Authentication (UAA) Server is an excelling choice. Its LDAP support and features actually exceeds the features offered by Spring Cloud Dataflow <2.0.0.
So yes, in order to setup security for Spring Cloud Data Flow locally, you need to run the UAA. And the UAA would also provide the LDAP support. Technically, Spring Cloud Data Flow has no awareness of the LDAP setup at all.
I hope this provides some background regarding how the Spring Cloud Data Flow security architecture emerged. Please have a look at the reference documentation and the aforementioned SCDF Security with UAA + LDAP example. Don't hesitate to reach out in case of further questions!
Disclaimer: I am a committer on the project.
Starting from v2.0, we delegate to UAA for authentication and authorization. There are a variety of write-ups on this matter; a more comprehensive one to look at is the end-to-end sample on how all this could be put together locally. You do not need CF or K8s, all this can run locally also. We rely on UAA as the gateway to standardize on end-to-end SSO across all the client tools, including shell, dashboard, RESTful APIs, CTR, etc.,
Sample: SCDF Security with UAA + LDAP. For further reading, please refer to the security section in the ref. guide.
Lastly, we do not recommend Local for a production install, but I understand that resiliency and/or restartability of apps under failure condition is not a requirement for some workloads.
SCDF Server for Cloudfoundry: 1.2.4.RELEASE
Configuring the security properties for LDAP authentication, and I have the authentication part working, but authorization is proving a little strange.
SCDF's security implementation appears to be looking for some roles like this:
ROLE_CREATE, ROLE_MANAGE, ROLE_VIEW.
But for me, the standard group names require some specific naming convention in AD similar to the following: app_myapplication_authz_CREATE, app_myapplication_authz_MANAGE, and app_myapplication_authz_VIEW
When I debug through the SCDF authentication output, I can see that my authenticated principal's group memberships are being retrieved correctly. They show up in the DEBUG output as: ROLE_APP_MYAPPLICATION_AUTHZ_CREATE, ROLE_APP_MYAPPLICATION_AUTHZ_MANAGE, ROLE_APP_MYAPPLICATION_AUTHZ_VIEW
Now, I include a YML security configuration that looks like this:
spring:
cloud:
dataflow:
security:
authorization:
enabled: true
rules:
- GET /metrics/streams => hasRole('ROLE_APP_MYAPPLICATION_AUTHZ_VIEW')
- POST /apps/** => hasRole('ROLE_APP_MYAPPLICATION_AUTHZ_CREATE')
- etc, etc, etc
And so on, for all the endpoint authorizations.
However, I'm still receiving a message after successful authentication that I don't have the appropriate roles and I need to talk to my administrator.
What am I misconfiguring, or what am I missing in this setup?
Update
I downloaded the source code for the 1.2.1.RELEASE version of the spring cloud dataflow ui from here: GitHub spring-cloud-dataflow-ui
And discovered that in all the .html view files, the role names are hard-coded for ROLE_VIEW, ROLE_CREATE, ROLE_MANAGE. Thus, it looks like my configuration will allow me to customize the authorization on the REST endpoints based on my LDAP group names, but I will not be able to do the same with the actual UI views. I think I have one option here, which would be to build/generate my own custom version of the UI, and bundle that with the spring-cloud-dataflow-server JAR instead of using the OOTB ui.
I'll have to weigh whether I really want to do that.
We don't yet have the direct mapping of LDAP AD Groups <-> SCDF Roles. We haven't had anyone from the community or customers' ask for this integration, though. UAA backed OAuth turns out to be the popular choice in PCF so far.
That said, I created spring-cloud/spring-cloud-dataflow#2084 to track the support for group mapping. It could be trivial to implement it (Group vs. ROLE mapping in YAML and parsing logic in the backend code); I marked it for 1.5, but we may pick it up sooner for the 1.4 release next week.
I'd recommend not venturing into adjusting the UI code directly, though. Too much on the local fork and you'll have to maintain it.
I am learning Jenkins and i am using version 1.605. I am learning about setting security options in jenkins. I found below options in jenkins configure global security page under the security realms:-
Delegate to servlet container
Jenkins'own user database
LDAP
I want to understand in which scenario, which option should be used?Please help me understand.
The "servlet container" refers to the web server that you are using to host Jenkins, typically Tomcat. The configuration file $CATALINA_BASE/conf/tomcat-users.xml may already be set up and managed by your organization. In this case: "why reinvent the wheel?". Use this if your organization already has a process that manages the Tomcat users configuration. This is also probably the most archaic solution.
The "own user database" is just what it sound like. Instead of relying on something else, Jenkins keeps it's own database of users. You can create and delete users through the Jenkins UI. You can even let new users sign up right from UI. If you don't know what to use, use this one. It's simple and self-contained.
The "LDAP" provides integration with LDAP/ Windows Active Directory. If you are in a corporate/small business environment that already utilizes LDAP for maintaining users and groups, it will be very beneficial to hook into that and off-load user management to the IT team that manages LDAP/AD. Note that unless you are that IT admin, you will need to contact the said admins for connection information/credentials to the LDAP/AD
Slav did a very clear answer.
If you choose the option 2 or 3, you can use the Role Stragegy plugin to create application roles and assign roles to Jenkins users, LDAP users or LDAP static/virtual groups.
If you have a lot of users, this plugin can save your life to assign permissions :)