I am using SCDF deployment on k8s and trying to add a new Task Application from our internal Maven repo. By default, SCDF seems to only lookup in the [springRepo] repository. I followed the documentation to add a new maven repo here .
Since the documentation only talks about CloudFoundry example, I added these lines to application.yaml section based on my understanding.
spring:
cloud:
dataflow:
task:
platform:
local:
accounts:
localDev:
********
datasource:
uri: xxx
*********
maven:
remote-repositories:
repo1:
url: https://repo1
auth:
username: user1
password: pass1
snapshot-policy:
update-policy: daily
checksum-policy: warn
release-policy:
update-policy: never
checksum-policy: fail
While adding the app I used the syntax : maven://:[:[:]]:. However, when I launch the task, it fails with error : Failed to resolve maven Resource XXX at configured Remote Repository : [springRepo]
How can I override it to search in my newly added repo.. why SCDF still only searching in default [springRepo]? Appreciate any help.
The property prefix is maven.remote-repositories but what you have is spring.maven.remote-repositories.
You need to specify:
spring:
cloud:
dataflow:
task:
platform:
local:
accounts:
localDev:
********
datasource:
uri: xxx
*********
maven:
remote-repositories:
repo1:
url: https://repo1
...
Please note that the Kubernetes deployment works with containers rather than maven jar artifacts and hence, you need to have your apps registered with the app's URI using docker: prefix.
Related
I am using Ansible (v 2.8) as the provisioner behind a Packer template to build an AMI for a Jenkins master node. For previous versions, the playbook passed successfully. However, as of Jenkins version 2.176.3, the jenkins_plugin module has been throwing:
HTTP Error 403: No valid crumb was included in the request
I have retrieved the crumb and registered it in a variable. I have tried passing it to jenkins_plugin with the http_agent field, but that doesn't work. I tried using attributes, but that didn't help either. Unless I am missing something incredibly basic, I am at the end of my tether.
- name: Get Jenkins Crumb
uri:
force_basic_auth: yes
url_username: ****
url_password: ****
url: http://localhost:8080/crumbIssuer/api/json
return_content: yes
register: jenkins_crumb
until: jenkins_crumb.content.find('Please wait while Jenkins is getting ready') == -1
retries: 10
delay: 5
- name: Install plugin
jenkins_plugin:
name: "{{ item }}"
version: latest
force_basic_auth: yes
url_username: ****
url_password: ****
http_agent: "Jenkins-Crumb:{{ jenkins_crumb.json.crumb }}"
with_items: "{{ jenkins_plugins }}"
I expected installed plugins and a happily built AMI. What I got was "HTTP Error 403: No valid crumb was included in the request" and the Packer build failed.
Looks like a change to the crumb issuer in the 2.176 LTS release forces the inclusion of the web session id of the initial token generation call along with the crumb in subsequent calls that use said crumb.
CSRF tokens (crumbs) are now only valid for the web session they were created in to limit the impact of attackers obtaining them. Scripts that obtain a crumb using the /crumbIssuer/api URL will now fail to perform actions protected from CSRF unless the scripts retain the web session ID in subsequent requests.
In addition to the suggestion that you temporarily disable CSRF, the same doc suggests that you could only disable the new functionality, rather than CSRF as a whole, which should allow your packer/ansible to complete as it previously did, as-written.
To disable this improvement you can set the system property hudson.security.csrf.DefaultCrumbIssuer.EXCLUDE_SESSION_ID to true.
EDIT :
Adding the following line in /etc/default/jenkins cleared the CSRF issues in my own playbook (Ansible 2.8.4, Ubuntu 18.04, OpenJDK 11.0.4)
JAVA_ARGS="$JAVA_ARGS -Dhudson.security.csrf.DefaultCrumbIssuer.EXCLUDE_SESSION_ID=true"
Might be a good-enough crutch until tool maintainers catch up with the API changes.
It's exactly the cause #runningEagle mentioned. You need to propagate the initial session cookie value to all subsequent requests along with the crumb.
Required new Ansible code modifications:
...
# Requesting the crumb
uri:
url: "<crumb_URL>"
register: response
...
# Actual action request
uri:
url: "<action_URL>"
headers: '{ ... , "Cookie": "{{ response.set_cookie }}", ... }'
...
I was facing this issue too and given the pointer the work needed to be done in a session I opened a PR for ansible:
https://github.com/ansible/ansible/issues/61672
https://github.com/ansible/ansible/issues/61673
It is a small change and it should be possible to patch your local installation.
UPDATE: The patch was applied in Ansbile v2.8.9 and v2.9.1 be sure to upgrade ansible if you have an older version.
The solution I ended up applying was to disable CSRF using a handy piece of Groovy, and then re-enable it at the end of the play.
Thanks all for your help and recommendations.
Had tried setting the below per the documentation for /etc/default/jenkins (Most Linux) and /etc/sysconfig/jenkins (RHEL):
hudson.security.csrf.DefaultCrumbIssuer.EXCLUDE_SESSION_ID to true
Thus adding:
JAVA_ARGS="-Dhudson.security.csrf.DefaultCrumbIssuer.EXCLUDE_SESSION_ID=true
But to no avail. Using #Yuri answer fixed it for me. See below:
Requested the crumb the same as I did before.
Crumb:
- uri:
url: "http://localhost:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,\":\",//crumb)"
return_content: yes
register: crumb
Action request:
uri:
method: POST
url: "http://localhost:8080/credentials/store/system/domain/_/createCredentials"
headers:
Jenkins-Crumb: "{{ crumb.content.split(':')[1] }}"
Cookie: "{{ crumb.set_cookie }}"
body: |
json={
"": "0",
"credentials": {
"scope": "GLOBAL",
"id": "identification",
"username": "manu",
"password": "bar",
"description": "linda",
"$class": "com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl"
}
}
status_code: 302
Resolution:
Install a plugin named Strict Crumb Issue
Go to Manage Jenkins -> Configure Global Security -> CSRF Protection.
Select Strict Crumb Issuer.
Click on Advanced.
Uncheck the Check the session ID box.
Save it.
I am having issues deploying an app of mine to Google AppEngine. It is currently live on Heroku with no issues, so I dont think the issue is with the app. I emailed back and forth with Google support for a while, and one of the things they had me try was to deploy one of their starter apps. For simplicity sake, I am asking this question from the point of view that I am simply trying to deploy their test application, not my real app.
I have downloaded the hello world app from Github, per Google's instructions.
When I go to deploy, it sticks on a certain step and will not go any further.
I have done a fresh install of the google cloud tools and have made sure to run gcloud init.
I downloaded the app from that URL and am using the included app.yaml
Here is my entire CLI output. It will sit at the same status for HOURS.
Thanks for the help.
--Tim
Timothys-MBP:2-hello-world tgwright$ gcloud auth login
Your browser has been opened to visit:
https://accounts.google.com/o/oauth2/auth?redirect_uri=xxxxxx
Saved Application Default Credentials.
You are now logged in as [timothy.xxxxx#gmail.com].
Your current project is [xxx-gae-02]. You can change this setting by running:
$ gcloud config set project PROJECT_ID
Timothys-MBP:2-hello-world tgwright$ gcloud init
Welcome! This command will take you through the configuration of gcloud.
Settings from your current configuration [default] are:
Your active configuration is: [default]
[app]
suppress_change_warning = true
[core]
account = timothy.xxxxx#gmail.com
disable_usage_reporting = False
project = xxx-gae-02
Pick configuration to use:
[1] Re-initialize this configuration [default] with new settings
[2] Create a new configuration
Please enter your numeric choice: 1
Your current configuration has been set to: [default]
Pick credentials to use:
[1] timothy.xxxxx#gmail.com
[2] Log in with new credentials
Please enter your numeric choice: 1
You are now logged in as: [timothy.g.wright#gmail.com]
Pick cloud project to use:
[1] [xxx-gae-01]
[2] [xxxx-cloud01]
Please enter your numeric choice: 1
Your current project has been set to: [xxx-gae-01].
Which compute zone would you like to use as project default?
[1] [asia-east1-b]
[2] [asia-east1-a]
[3] [asia-east1-c]
[4] [europe-west1-c]
[5] [europe-west1-d]
[6] [europe-west1-b]
[7] [us-central1-c]
[8] [us-central1-b]
[9] [us-central1-f]
[10] [us-central1-a]
[11] [us-east1-d]
[12] [us-east1-b]
[13] [us-east1-c]
[14] Do not set default zone
Please enter your numeric choice: 9
Your project default compute zone has been set to [us-central1-f].
You can change it by running [gcloud config set compute/zone NAME].
Your project default compute region has been set to [us-central1].
You can change it by running [gcloud config set compute/region NAME].
Do you want to use Google's source hosting (see
"https://cloud.google.com/source-repositories/docs/") (Y/n)? n
gcloud has now been configured!
You can use [gcloud config] to change more gcloud settings.
Your active configuration is: [default]
[app]
suppress_change_warning = true
[compute]
region = us-central1
zone = us-central1-f
[core]
account = timothy.xxxx#gmail.com
disable_usage_reporting = False
project = xxx-gae-01
Timothys-MBP:2-hello-world tgwright$ gcloud auth list
Credentialed accounts:
- timothy.xxx#gmail.com (active)
To set the active account, run:
$ gcloud config set account ``ACCOUNT''
Timothys-MBP:2-hello-world tgwright$ gcloud config list
Your active configuration is: [default]
[app]
suppress_change_warning = true
[compute]
region = us-central1
zone = us-central1-f
[core]
account = timothy.xxxx#gmail.com
disable_usage_reporting = False
project = xxx-gae-01
Timothys-MBP:2-hello-world tgwright$ gcloud preview app deploy --verbosity debug
DEBUG: Running gcloud.preview.app.deploy with Namespace(__calliope_internal_deepest_parser=ArgumentParser(prog='gcloud.preview.app.deploy', usage=None, description="*(BETA)* This command is used to deploy both code and configuration to the App Engine\nserver. As an input it takes one or more ``DEPLOYABLES'' that should be\nuploaded. A ``DEPLOYABLE'' can be a module's .yaml file or a configuration's\n.yaml file.", version=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=False), account=None, authority_selector=None, authorization_token_file=None, bucket=None, cmd_func=<bound method Command.Run of <googlecloudsdk.calliope.backend.Command object at 0x10e936590>>, command_path=['gcloud', 'preview', 'app', 'deploy'], configuration=None, credential_file_override=None, deployables=[], docker_build=None, document=None, force=False, format=None, h=None, help=None, http_timeout=None, ignore_bad_certs=False, image_url=None, log_http=None, project=None, promote=None, quiet=None, repo_info_file=None, server=None, stop_previous_version=None, trace_email=None, trace_log=False, trace_token=None, user_output_enabled=None, verbosity='debug', version=None).
DEBUG: API endpoint: [https://appengine.googleapis.com/], API version: [v1beta4]
You are about to deploy the following modules:
- xxx-gae-01/default (from [/Users/tgwright/Google Drive/Sites/2-hello-world/app.yaml])
Deployed URL: [https://xxx-gae-01.appspot.com]
Do you want to continue (Y/n)? y
Beginning deployment...
DEBUG: Host: appengine.google.com
DEBUG: _Authenticate configuring auth; needs_auth=False
DEBUG: Sending request to https://appengine.google.com/api/vms/prepare?app_id=ebt-gae-01 headers={'X-appcfg-api-version': '1', 'content-length': '0', 'Content-Type': 'application/octet-stream'} body=
INFO: Attempting refresh to obtain initial access_token
INFO: Refreshing access_token
If this is your first deployment, this may take a while...\DEBUG: Got response: {bucket: vm-containers.xxx-gae-01.appspot.com, path: /containers}
If this is your first deployment, this may take a while...done.
INFO: Not checking for [go] because runtime is [ruby]
I am using GitLab Omnibus 7.10.0 on RHEL 6.6. I have enabled LDAP using the following configuration:
gitlab_rails['ldap_enabled'] = true
gitlab_rails['ldap_servers'] = YAML.load <<-'EOS' # remember to close this block with 'EOS' below
main: # 'main' is the GitLab 'provider ID' of this LDAP server
label: 'FOO COM Active Directory (LDAP)'
host: 'ad.server.foo.com'
port: 3268
uid: 'someuser'
method: 'plain' # "tls" or "ssl" or "plain"
bind_dn: 'CN=My Whole. Name,OU=Some Users,DC=ad,DC=server,DC=foo,DC=com'
password: 'thepassword'
active_directory: true
allow_username_or_email_login: false
block_auto_created_users: false
base: 'DC=ad,DC=server,DC=foo,DC=com'
user_filter: ''
# ## EE only
# group_base: ''
# admin_group: ''
# sync_ssh_keys: false
#
# secondary: # NOT FILLED OUT
EOS
My problem is that I can't get users to authenticate via LDAP. I'm not sure if the configuration is wrong, or I need to do something on the server side (which I have no direct access to). When I run
gitlab-rake gitlab:ldap:check RAILS_ENV=production
I get this
Checking LDAP ...
LDAP users with access to your GitLab server (only showing the first 100 results)
Server: ldapmain
Checking LDAP ... Finished
I can search for individual users using java with this account (my personal account) or another account for a different application, but can't get AD working with gitlab. I got the bind_dn "My Whole. Name" by running this command on a Windows box.
gpresult -r
I have also tried a bind_dn of:
uid=myADaccountname,OU=Some Users,DC=ad,DC=server,DC=foo,DC=com
and
myADaccountname#ad.server.foo.com
but I still have the same problem.
For Active Directory, the uid should be:
uid: 'sAMAccountName'
Gitlab should connect using the user specified in the bind_dn, with the given password.
Since GitLab 9.5.1 the uid now requires [ ]
See this issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/37120
This might just be a bug which will be fixed.
I had to update the value for Active Directory from the answer above to:
uid: ['sAMAccountName']
I'm using GitLab CE Omnibus package (gitlab_7.7.2-omnibus.5.4.2.ci-1_amd64) on a clean Debian (debian-7.8.0-amd64) installation.
I followed the installation process on https://about.gitlab.com/downloads/ and everything works fine.
I modified /etc/gitlab/gitlab.rb to use a single LDAP server for authentification.
Which worked also as expected.
But when I tried to use a secondary LDAP connection "gitlab-ctl reconfigure" gives me the output:
---- Begin output of /opt/gitlab/bin/gitlab-rake cache:clear ----
STDOUT:
STDERR: rake aborted!
Devise::OmniAuth::StrategyNotFound: Could not find a strategy with name `Ldapsecondary'. Please ensure it is required or explicitly set it using the :strategy_class option .
Tasks: TOP => cache:clear => environment
(See full trace by running task with --trace)
---- End output of /opt/gitlab/bin/gitlab-rake cache:clear ----
So, the problem is that I can use the LDAP connection 'main' but I cannot use the connection 'secondary'.
Is there any possibility to use two different LDAP connection in the CE edition at once?
I'm new to ruby [on rails]. I found something in /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/ldap/config.rb but I'm not able to debug anything.
Here are my settings in /etc/gitlab/gitlab.rb
gitlab_rails['ldap_enabled'] = true
gitlab_rails['ldap_servers'] = YAML.load <<-EOS # remember to close this block with 'EOS' below
main: # 'main' is the GitLab 'provider ID' of this LDAP server
label: 'First Company'
host: '192.168.100.1'
port: 389
uid: 'sAMAccountName'
method: 'tls' # "tls" or "ssl" or "plain"
bind_dn: 'debian#firstcompany.local'
password: 'Passw0rd'
active_directory: true
allow_username_or_email_login: false
base: 'dc=firstcompany,dc=local'
user_filter: '(&(objectClass=person)(objectClass=user)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))'
## EE only
group_base: ''
admin_group: ''
sync_ssh_keys: false
secondary: # 'secondary' is the GitLab 'provider ID' of second LDAP server
label: 'Second Company'
host: '192.168.200.1'
port: 389
uid: 'sAMAccountName'
method: 'tls' # "tls" or "ssl" or "plain"
bind_dn: 'debian#secondcompany.local'
password: 'Passw0rd'
active_directory: true
allow_username_or_email_login: false
base: 'dc=secondcompany,dc=local'
user_filter: '(&(objectClass=person)(objectClass=user)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))'
## EE only
group_base: ''
admin_group: ''
sync_ssh_keys: false
EOS
Thank you very much!
Multiple LDAP servers is an EE feature so setting the config in CE won't do anything. You can see the feature in GitLab documentation.
With GitLab 14.7 (January 2022, seven years later), this is now possible! (for hosted instances)
LDAP failover support
You can now specify multiple hosts (using hosts) in your GitLab LDAP configuration.
GitLab will use the first reachable host. This ensures continuity of access to GitLab should one of your LDAP hosts become unresponsive.
Thanks to Mathieu Parent for the contribution!
See Documentation and Issue.
I’m trying to connect Jenkins (1.482) with TestLink (1.9.4) thru Jenkins configuration in order to retrieve tests, but while running the job in Jenkins I get the below error in the console log.
Please note that Jenkins is hosted on tomcat (linux) on network“gnb” and Testlink is hosted on php (linux) on another network “<company network name>”. It works well when both are on my localhost (in windows)
but this integration does not work when both Jenkins and TestLink are on separate networks/hosts.
I get the below error on the console while running the job:
Preparing TestLink client API.
Using TestLink URL: http://<hostname>/mr61_php5/testlink/lib/api/xmlrpc.php
FATAL: Error verifying developer key: HTTP server returned unexpected status: Found
br.eti.kinoshita.testlinkjavaapi.util.TestLinkAPIException: Error verifying developer key: HTTP server returned unexpected status: Found
at br.eti.kinoshita.testlinkjavaapi.MiscService.checkDevKey(MiscService.java:66)
at br.eti.kinoshita.testlinkjavaapi.TestLinkAPI.(TestLinkAPI.java:162)
at hudson.plugins.testlink.TestLinkBuilder.getTestLinkSite(TestLinkBuilder.java:244)
at hudson.plugins.testlink.TestLinkBuilder.perform(TestLinkBuilder.java:134)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:160)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1502)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:236)
Caused by: org.apache.xmlrpc.client.XmlRpcHttpTransportException: HTTP server returned unexpected status: Found
at org.apache.xmlrpc.client.XmlRpcSunHttpTransport.getInputStream(XmlRpcSunHttpTransport.java:94)
at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:152)
at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143)
at org.apache.xmlrpc.client.XmlRpcSunHttpTransport.sendRequest(XmlRpcSunHttpTransport.java:69)
at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56)
at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167)
at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:158)
at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:147)
at br.eti.kinoshita.testlinkjavaapi.BaseService.executeXmlRpcCall(BaseService.java:90)
at br.eti.kinoshita.testlinkjavaapi.MiscService.checkDevKey(MiscService.java:62)
... 12 more
ERROR: Error communicating with TestLink. Check your TestLink configuration.
I have below settings in my Jenkins’s global configuration for Testlink installation
Name: testlink
URL: http://<host name>/mr61_php5/testlink/lib/api/xmlrpc.php
Developer key: generated from Testlink (Settings->Generate a new key)
Can you please point me if I miss something?
Usually in the Testlink folder structure, the path that you have mentioned, does not contain the xmlrpc.php file
Probabaly worng URL: URL: http:///mr61_php5/testlink/lib/api/
The correct URL is usually of this format
.../testlink/lib/api/xmlrpc//xmlrpc.php
Kindly check the correct URL, or try opening the xmlrpc.php page, so that you can get the correct path of the file. As per my assumption it should be somewhat like this:
http:///mr61_php5/testlink/lib/api/xmlrpc/xmlrpc.php
Good answer In my case it is as below...
http://IP:PORT/testlink/lib/api/xmlrpc/v1/xmlrpc.php in 1.9.11 version of testlink