I have a .pem cert that I'm reasonably sure I generated correctly, and it is not being accepted by OpenSSL when I paste it into a Rails 3.0.2 model. What I do is this:
open up the .pem file in Textmate
select all and copy
user.cert = <paste into model>; user.save
OpenSSL::PKey::RSA.new(user.cert)
This gives me the error:
Neither PUB key nor PRIV key::
To test, I just loaded in the file instead, no errors:
OpenSSL::PKey::RSA.new(File.read("/path/to/cert.pem"))
I thought maybe it would be some encoding error or newline issue, I had tried gsub'ing out the newlines to no avail.
It was a weird copy and paste artifact indeed. I File.read'd it into the model instead of copy paste and it worked fine...
Perhaps late, but this is the answer:
You can put a public key inline in Ruby with copy/paste, but keep in mind that what looks like formatting to you is white space in the line - you need to make sure that the resulting pem string has no spaces. I just copied and pasted from a PEM file into Ruby code, and it did not work until I removed the extra spaces that text mate or whatever added to the lines.
Hard to show here:
SQS_PUBLIC_KEY = "-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs3VeTxEgLQLL11UN2G6c
oQsc0LbpoEs4VTmu0S4XU82N4h/25XX5k4t5oTJ0JGGSBP4/gzTwz15vS5mrlnsG
MISSINGLINES
rMV5ZCXToG0VCNPEHpZQnUHMCg/nF9jnk9i1ZZHv2dpYYG7GHMUPG3rtcTWJvZxI
3wIDAQAB
-----END PUBLIC KEY-----".force_encoding("us-ascii")
SQS_PUBLIC_KEY = "-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs3VeTxEgLQLL11UN2G6c
oQsc0LbpoEs4VTmu0S4XU82N4h/25XX5k4t5oTJ0JGGSBP4/gzTwz15vS5mrlnsG
MISSINGLINES
rMV5ZCXToG0VCNPEHpZQnUHMCg/nF9jnk9i1ZZHv2dpYYG7GHMUPG3rtcTWJvZxI
3wIDAQAB
-----END PUBLIC KEY-----".force_encoding("us-ascii")
ie - NOT the second one - ruby adds spaces to the start of each line, and the RSA tools do not ignore spaces - they only seem to ignore line feeds.
I use the copy/pasted key as a fallback - in other words if an ENV is set I use that, otherwise use the pasted in public key.
--Tom
Related
I'm using dotenv.
A Ruby gem to load environment variables from .env.
Is it possible to have multiline variables in my .env file?
e.g.
SOMETHING_CERTIFICATE="-----BEGIN CERTIFICATE-----
JSDFALDAFSSKLABVCXZLV2314IH4IHDFG9AYDF9DSSDF82QWEIWFHDSSD8SADF0=
-----END CERTIFICATE-----"
^ having the above just throws an error on that middle line, as if it's not part of the string and I'm trying to create an improperly formatted variable.
According to the documentation
Multi-line values
If you need multiline variables, for example private keys, you can double quote strings and use the \n character for newlines:
PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----\nHkVN9…\n-----END DSA PRIVATE KEY-----\n"
From the documentation Brian posted above:
Alternatively, multi-line values with line breaks are now supported for quoted values.
So the solution you sketched in your question is legit now!
If you are using node, you could use fixedKey = key.replaceAll('\\n', '\n')
And in the .env
KEY=-----BEGIN PRIVATE KEY----- xY=\n.....3ZaWjyKJqy+xY=\n-----END PRIVATE KEY-----\n
Fix the error replaceAll is not a function changing the lib option under compilerOptions in the tsconfig.json for "es2021" if you are using typeScript.
I am deploying a Ruby on Rails application to AWS using Elastic Beanstalk and have to set a private key as an environment variable
E.g
-----BEGIN RSA PRIVATE KEY-----
SpvpksXQIBA65ICOgQxV2TvMIICAiMeV9prhdJSKjjsk2
tYdz8lhn/ibROQW71utuHLAyHGMBxz3kIaaIq1kjdkkk
tYdz8lhn/ibROQW71utuHLAyHGMBxz3kIaaIq1kjdkkk
tYdz8lhn/ibROQW71utuHLAyHGMBxz3kIaaIq1kjdkkk
tYdz8lhn/ibROQW71utuHLAyHGMBxz3kIaaIq1kjdkkk
-----END RSA PRIVATE KEY-----
However this doesn't seem to work when deploying the app as it always fails with a
OpenSSL::PKey::RSAError: Neither PUB key nor PRIV key: nested asn1
error
I think it's because the RSA Key is malformed.
However unlike in Heroku, AWS EB does not accept multiline input (see below) so I have to use \n to create new lines.
I tried with few different styles but none of them seem to interpolate the \n properly and I always keep getting the same error.
I've tried with \n and the end of each line, then \\n and also tried tried double quotes \" to wrap the key but I still keep getting the same error.
How do I properly set a multiline environment variable in AWS Elastic Beanstalk ?
You can transform your private key in a base64, then you store that base64 as environment variable. When needed you decode this variable.
in unix:
$ base64 path/to/your/private_key_file
in your application:
def private_key
Base64.decode64(ENV['PRIVATE_KEY'])
end
You could set it in EB using \n and then convert the '\n' to newlines before you pass it to config.key - something like this (note the single and double quotes in the call to gsub):
single_line_key = ENV.fetch('CLOUDFRONT_KEY')
multi_line_key = single_line_key.gsub('\n', "\n")
config.key = multi_line_key
In I had the same problem with Golang and the elastic beanstalk,
I did this
went to AWS console and set the value like this:
-----BEGIN RSA PRIVATE KEY-----\nSpvpksXQIBA65ICOgQxV2TvMIICAiMeV9prhdJSKjjsk2\ntYdz8lhn/ibROQW71utuHLAyHGMBxz3kIaaIq1kjdkkk\ntYdz8lhn/ibROQW71utuHLAyHGMBxz3kIaaIq1kjdkkk\ntYdz8lhn/ibROQW71utuHLAyHGMBxz3kIaaIq1kjdkkk\ntYdz8lhn/ibROQW71utuHLAyHGMBxz3kIaaIq1kjdkkk\n-----END RSA PRIVATE KEY-----
inside my code
key := os.Getenv("PUSH_AUTH_KEY")
key = strings.Replace(key, `\n`, "\n", 5)
You need to 'export' your multiline string, e.g., your private or public key into the environment correctly.
Enclose in your shell export statement $'.....' where ...... is your multiline string, e.g., your private or public key.
Example:
export KEY = $'-----BEGIN RSA PRIVATE KEY-----\nSpvpksXQIBA65ICOgQxV2TvMIICAiMeV9prhdJSKjjsk2tYdz8lhn/ibROQW71utuHLAyHGMBxz3kIaaIq1kjdkkktYdz8lhn/ibROQW71utuHLAyHGMBxz3kIaaIq1kjdkkktYdz8lhn/ibROQW71utuHLAyHGMBxz3kIaaIq1kjdkkktYdz8lhn/ibROQW71utuHLAyHGMBxz3kIaaIq1kjdkk\n-----END RSA PRIVATE KEY-----'
I've got some certificate files, namely a .key file which says:
-----BEGIN RSA PRIVATE KEY-----
IEpAIBAAKCAQEAwAwxt4edIh3UuK8r5
....blablabla..................
QSNoquaasdsaKDybrezemVqCxsQjg==
-----END RSA PRIVATE KEY-----
So it's a RSA Private Key.
I used to load them from files like so:
#private_key = OpenSSL::PKey::RSA.new(File.read(private_key_file))
But since I am using Heroku, I intend to have my certificates saved as their values in environment variables.
So I've pasted them in my .env file
COMPANY_KEY="-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKC.....\n-----END RSA PRIVATE KEY-----"
Yeah, I switched the \n for explicits \\n based on Multi-line config variables in Heroku. So now my code looks like this:
#private_key = OpenSSL::PKey::RSA.new(ENV['COMPANY_KEY'])
And if I run it from the console I get the object built. But if I try to run it from the web server (Puma 3.4.0 over Rails 4.2.6, Ruby 2.2.3) it fails miserably saying: Neither PUB key nor PRIV key:: nested asn1 error when trying to run that same line.
If I use the debug console I get that the read file looks like
"Line 1\\nLine3\\nLinea3" and so on...
I'm pretty sure that it has something to do with the file format, but I'm all out of ideas and maybe you could help if you had a problem like mine.
I finally found a way to do it... mixing it all up!
So the file, for example company.key looks like
-----BEGIN PRIVATE RSA KEY ----
Mumbojumbomummbojumbo
-----END RSA PRIVATE KEY----
So I switched it to a one liner, making explicit \n in the string (so its a real \n)
COMPANY_KEY=""-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEA+ztKEj\n-----END RSA PRIVATE KEY-----\n"
Don't forget the last \n in the file.
Now, the last part, in the place where I used to do
#private_key = OpenSSL::PKey::RSA.new(File.read(private_key_file))
Now I do
#private_key = OpenSSL::PKey::RSA.new(ENV['COMPANY_KEY'].gsub("\\n", "\n"))
And now works like a charm! No public certificates, every piece of info in environment variables.
Save yourself some trouble and store only the certificate or key body in the environment variable. No need to put in newline characters.
SECRET = <<-SECRET
-----BEGIN PRIVATE KEY-----
#{ENV['SECRET_KEY']}
-----END PRIVATE KEY-----
SECRET
CERTIFICATE = <<-CERT
-----BEGIN CERTIFICATE-----
#{ENV['CERT']}
-----END CERTIFICATE-----
CERT
I switched the \n for explicits \\n based on Multi-line config variables in Heroku.
...
If I use the debug console I get that the read file looks like "Line 1\\nLine3\\nLinea3" ...
You problem should be here. The post you are linking is not suggesting to double escape your new lines, it is suggesting to wrap your multi-line text into "double quotes". In bash, it would allow to enter multi-line text at the terminal. The post also suggests to do it an in much easier way:
heroku config:add MASISA_KEY ="$(cat your_private_key.pem)"
I am not sure if I am doing something wrong here but I've been stuck on this issue for quite some time. I am using the Ruby-saml gem (https://github.com/onelogin/ruby-saml) and I am not sure if my settings.certificate is valid. I used OpenSSL to generate the public/private key pair. Here is my public key:
$ cat cert.pem
-----BEGIN CERTIFICATE-----
MIIE3zCCA8egAwIBAgIJANtTrhsq7mkmMA0GCSqGSIb3DQEBBQUAMIGlMQswCQYD
VQQGEwJVUzERMA8GA1UECBMITmV3IFlvcmsxDzANBgNVBAcTBkl0aGFjYTEbMBkG
A1UEChMSQ29ybmVsbCBVbml2ZXJzaXR5MQ4wDAYDVQQLEwVEeXNvbjEjMCEGA1UE
AxMaY3VtaW5vcnMuZHlzb24uY29ybmVsbC5lZHUxIDAeBgkqhkiG9w0BCQEWEW5t
YzUyQGNvcm5lbGwuZWR1MB4XDTE2MDQxMjE4MTUzOVoXDTI2MDQxMDE4MTUzOVow
gaUxCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcgWW9yazEPMA0GA1UEBxMGSXRo
YWNhMRswGQYDVQQKExJDb3JuZWxsIFVuaXZlcnNpdHkxDjAMBgNVBAsTBUR5c29u
MSMwIQYDVQQDExpjdW1pbm9ycy5keXNvbi5jb3JuZWxsLmVkdTEgMB4GCSqGSIb3
DQEJARYRbm1jNTJAY29ybmVsbC5lZHUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQCnVjE8GIJe19Ba+361+c7ATDhBrzpGQoe+IDrDWw8B68HayaAvC8Pq
WdNQNQ3SfHOdb+Vv0eywxHG7wRVVrJ+f8fLqmHBHfthzRG1JnGhReUXb/+wfkUEw
DFZPEnEcj6rBcSbX5nsLVvupMXw43XB7ev/NX1SLsRU4trS25YMOozxjL+SfcKsW
IQPgqD3usIArwS6b3UQ+ftuVfmWqKEqoUq25tUXoAporFkJyVqXZqe4g/Q+WqbX4
cD9e1u7q8OlbSeVXUyPwRsNXzn1n+8tUbCc2k8+glEW5UJk7DY0AP95ry0ZcpfLr
kgaOTqvbkUWCaZH1FP04SYG5Csw/8IDtAgMBAAGjggEOMIIBCjAdBgNVHQ4EFgQU
q3ybbMNZOEXWgJ7/K0mSMx3VeTMwgdoGA1UdIwSB0jCBz4AUq3ybbMNZOEXWgJ7/
K0mSMx3VeTOhgaukgagwgaUxCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcgWW9y
azEPMA0GA1UEBxMGSXRoYWNhMRswGQYDVQQKExJDb3JuZWxsIFVuaXZlcnNpdHkx
DjAMBgNVBAsTBUR5c29uMSMwIQYDVQQDExpjdW1pbm9ycy5keXNvbi5jb3JuZWxs
LmVkdTEgMB4GCSqGSIb3DQEJARYRbm1jNTJAY29ybmVsbC5lZHWCCQDbU64bKu5p
JjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUAA4IBAQBA8QEvcxLnjZURGb5/
a4JUVwU6noFdZpmI9FgBi7d3nKs+BNxL/1Het6Kk19T1kPlyDdG96asG6fbRH24G
cJOoDvVpx6lxMu85gFpJVv/vtDmnlpiBoDH+v2I7O4ENhve76B7Z5XtT5FsjEdy4
RAn1iczxq391vFNQJl0kCz2Khdv5CS3t6qNS42sPcRk9mjbnN0wz6jHxG5BfCVdk
dXxoLuJVLzT7/sbBkT2SLkwQkPiYitb3LFoNFu+Sk8y+L4cVaeoA5XoEjmIbtkgD
oLCrILf6t18C/R2AD0/huq2pFtxd/rng/yGMniTBc6aGDsv06RXo/5r7DsO0feXV
cRzc
-----END CERTIFICATE-----
In Rails I tried multiple different way to get this to work:
settings.certificate = "-----BEGIN CERTIFICATE-----
MIIE3zCCA8egAwIBAgIJANtTrhsq7mkmMA0GCSqGSIb3DQEBBQUAMIGlMQswCQYD
VQQGEwJVUzERMA8GA1UECBMITmV3IFlvcmsxDzANBgNVBAcTBkl0aGFjYTEbMBkG
A1UEChMSQ29ybmVsbCBVbml2ZXJzaXR5MQ4wDAYDVQQLEwVEeXNvbjEjMCEGA1UE
AxMaY3VtaW5vcnMuZHlzb24uY29ybmVsbC5lZHUxIDAeBgkqhkiG9w0BCQEWEW5t
YzUyQGNvcm5lbGwuZWR1MB4XDTE2MDQxMjE4MTUzOVoXDTI2MDQxMDE4MTUzOVow
gaUxCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcgWW9yazEPMA0GA1UEBxMGSXRo
YWNhMRswGQYDVQQKExJDb3JuZWxsIFVuaXZlcnNpdHkxDjAMBgNVBAsTBUR5c29u
MSMwIQYDVQQDExpjdW1pbm9ycy5keXNvbi5jb3JuZWxsLmVkdTEgMB4GCSqGSIb3
DQEJARYRbm1jNTJAY29ybmVsbC5lZHUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQCnVjE8GIJe19Ba+361+c7ATDhBrzpGQoe+IDrDWw8B68HayaAvC8Pq
WdNQNQ3SfHOdb+Vv0eywxHG7wRVVrJ+f8fLqmHBHfthzRG1JnGhReUXb/+wfkUEw
DFZPEnEcj6rBcSbX5nsLVvupMXw43XB7ev/NX1SLsRU4trS25YMOozxjL+SfcKsW
IQPgqD3usIArwS6b3UQ+ftuVfmWqKEqoUq25tUXoAporFkJyVqXZqe4g/Q+WqbX4
cD9e1u7q8OlbSeVXUyPwRsNXzn1n+8tUbCc2k8+glEW5UJk7DY0AP95ry0ZcpfLr
kgaOTqvbkUWCaZH1FP04SYG5Csw/8IDtAgMBAAGjggEOMIIBCjAdBgNVHQ4EFgQU
q3ybbMNZOEXWgJ7/K0mSMx3VeTMwgdoGA1UdIwSB0jCBz4AUq3ybbMNZOEXWgJ7/
K0mSMx3VeTOhgaukgagwgaUxCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcgWW9y
azEPMA0GA1UEBxMGSXRoYWNhMRswGQYDVQQKExJDb3JuZWxsIFVuaXZlcnNpdHkx
DjAMBgNVBAsTBUR5c29uMSMwIQYDVQQDExpjdW1pbm9ycy5keXNvbi5jb3JuZWxs
LmVkdTEgMB4GCSqGSIb3DQEJARYRbm1jNTJAY29ybmVsbC5lZHWCCQDbU64bKu5p
JjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUAA4IBAQBA8QEvcxLnjZURGb5/
a4JUVwU6noFdZpmI9FgBi7d3nKs+BNxL/1Het6Kk19T1kPlyDdG96asG6fbRH24G
cJOoDvVpx6lxMu85gFpJVv/vtDmnlpiBoDH+v2I7O4ENhve76B7Z5XtT5FsjEdy4
RAn1iczxq391vFNQJl0kCz2Khdv5CS3t6qNS42sPcRk9mjbnN0wz6jHxG5BfCVdk
dXxoLuJVLzT7/sbBkT2SLkwQkPiYitb3LFoNFu+Sk8y+L4cVaeoA5XoEjmIbtkgD
oLCrILf6t18C/R2AD0/huq2pFtxd/rng/yGMniTBc6aGDsv06RXo/5r7DsO0feXV
cRzc
-----END CERTIFICATE-----"
I've also tried to just have Rails read the cert.pem file directly:
settings.certificate = OpenSSL::X509::Certificate.new(File.read("#{Rails.root}/cert.pem")).to_s
The issue is (which I am not sure is an issue), my key is a long inline string in the XML file (metadata for the SP)
<md:KeyDescriptor use="encryption">
<ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
<ds:X509Data>
<ds:X509Certificate>
MIIE3zCCA8egAwIBAgIJANtTrhsq7mkmMA0GCSqGSIb3DQEBBQUAMIGlMQswCQYDVQQGEwJVUzERMA8GA1UECBMITmV3IFlvcmsxDzANBgNVBAcTBkl0aGFjYTEbMBkGA1UEChMSQ29ybmVsbCBVbml2ZXJzaXR5MQ4wDAYDVQQLEwVEeXNvbjEjMCEGA1UEAxMaY3VtaW5vcnMuZHlzb24uY29ybmVsbC5lZHUxIDAeBgkqhkiG9w0BCQEWEW5tYzUyQGNvcm5lbGwuZWR1MB4XDTE2MDQxMjE4MTUzOVoXDTI2MDQxMDE4MTUzOVowgaUxCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcgWW9yazEPMA0GA1UEBxMGSXRoYWNhMRswGQYDVQQKExJDb3JuZWxsIFVuaXZlcnNpdHkxDjAMBgNVBAsTBUR5c29uMSMwIQYDVQQDExpjdW1pbm9ycy5keXNvbi5jb3JuZWxsLmVkdTEgMB4GCSqGSIb3DQEJARYRbm1jNTJAY29ybmVsbC5lZHUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCnVjE8GIJe19Ba+361+c7ATDhBrzpGQoe+IDrDWw8B68HayaAvC8PqWdNQNQ3SfHOdb+Vv0eywxHG7wRVVrJ+f8fLqmHBHfthzRG1JnGhReUXb/+wfkUEwDFZPEnEcj6rBcSbX5nsLVvupMXw43XB7ev/NX1SLsRU4trS25YMOozxjL+SfcKsWIQPgqD3usIArwS6b3UQ+ftuVfmWqKEqoUq25tUXoAporFkJyVqXZqe4g/Q+WqbX4cD9e1u7q8OlbSeVXUyPwRsNXzn1n+8tUbCc2k8+glEW5UJk7DY0AP95ry0ZcpfLrkgaOTqvbkUWCaZH1FP04SYG5Csw/8IDtAgMBAAGjggEOMIIBCjAdBgNVHQ4EFgQUq3ybbMNZOEXWgJ7/K0mSMx3VeTMwgdoGA1UdIwSB0jCBz4AUq3ybbMNZOEXWgJ7/K0mSMx3VeTOhgaukgagwgaUxCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcgWW9yazEPMA0GA1UEBxMGSXRoYWNhMRswGQYDVQQKExJDb3JuZWxsIFVuaXZlcnNpdHkxDjAMBgNVBAsTBUR5c29uMSMwIQYDVQQDExpjdW1pbm9ycy5keXNvbi5jb3JuZWxsLmVkdTEgMB4GCSqGSIb3DQEJARYRbm1jNTJAY29ybmVsbC5lZHWCCQDbU64bKu5pJjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUAA4IBAQBA8QEvcxLnjZURGb5/a4JUVwU6noFdZpmI9FgBi7d3nKs+BNxL/1Het6Kk19T1kPlyDdG96asG6fbRH24GcJOoDvVpx6lxMu85gFpJVv/vtDmnlpiBoDH+v2I7O4ENhve76B7Z5XtT5FsjEdy4RAn1iczxq391vFNQJl0kCz2Khdv5CS3t6qNS42sPcRk9mjbnN0wz6jHxG5BfCVdkdXxoLuJVLzT7/sbBkT2SLkwQkPiYitb3LFoNFu+Sk8y+L4cVaeoA5XoEjmIbtkgDoLCrILf6t18C/R2AD0/huq2pFtxd/rng/yGMniTBc6aGDsv06RXo/5r7DsO0feXVcRzc
</ds:X509Certificate>
</ds:X509Data>
</ds:KeyInfo>
</md:KeyDescriptor>
The IdP people said it was not valid when they tried to use it, he gave me an example of theirs and theirs was split across many lines unlike mine which is just a long string with no space:
https://shibidp.cit.cornell.edu/idp/shibboleth
Am I doing something wrong here? All I did was take the output from cat and pasted it to my SAML Settings.
looks like your X509 doesn't have any line breaks. That might be your problem.
I want to edit project.pbxproj straight up using command line (for CI server script)
what tools can allow me to do this?
I used to use PlistBuddy to edit the output Info.plist; however, what i really want to do is to edit this user defined field, which is used in multiple places, and i really don't want to have to hunt that down in every plist location
project.pbxproj is an old-style ASCII property list file, too. So you can use /usr/libexec/PlistBuddy to edit it.
Print some User-Defined key's value like this,
# Get the key A83311AA20DA4A80004B8C0E in your project.pbxproj
# LZD_NOTIFICATION_SERVICE_BUNDLE_ID is defined by me,
# Replace key paths with your own.
/usr/libexec/PlistBuddy -c 'print :objects:A83311AA20DA4A80004B8C0E:buildSettings:LZD_NOTIFICATION_SERVICE_BUNDLE_ID' LAAppAdapter.xcodeproj/project.pbxproj
Set its value like this,
/usr/libexec/PlistBuddy -c 'set :objects:A83311AA20DA4A80004B8C0E:buildSettings:LZD_NOTIFICATION_SERVICE_BUNDLE_ID com.dawnsong.notification-service' LAAppAdapter.xcodeproj/project.pbxproj
UPDATE
PlistBuddy will automatically convert project.pbxproj into a xml-format plist file since macOS Catalina (or some earlier version). It's better to move the setting item into xcconfig file instead since xcconfig is much smaller and simpler than project.pbxproj and not easy to make mistakes when editing with perl script.
I know this has been answered for a while, but since the original question is about tools supporting the manipulation of .pbxproj files, and many other people may be looking for the same information, here's how I do it. It took me quite a while to figure this out because I was very unfamiliar with Xcode when I started attempting this, so I hope this saves others the hours of grief I had to put in.
You can use the plutil command to transform the .pbxproj file from the legacy .plist format into an XML or JSON format you will be able to manipulate more easily. I'm using JSON. To do so, just run:
plutil -convert json project.pbxproj
This will convert the format of project.pbxproj, but be aware that -contrary to common sense- the output won't be another file with a JSON extention such as project.json. What will happen is that project.pbxproj will be converted to JSON format, but retain it's cryptic .pbxproj extension. So even though the file's format has been changed, Xcode will still pick it up and use it in its new JSON format.
Then you can change project.pbxproj with ease using any JSON manipulation tool of your choosing. I'm using Groovy's JsonSlurper class in a Groovy script.
Note I also explored the XML option, but I found the project.pbxproj file in XML format to be cumbersome to parse. The elements are not properly nested to allow for traversing the tree with ease. It's plagued with:
<key>someKey</key>
<dict>
<!--More elements which provide configuration for the key above-->
</dict>
So it's positional in nature. You have to look for the key element corresponding to the setting you want to manipulate and then jump to the dict element just after it. Which means you have to mount the children of each XML element into an array, in order to index them.
Here are 3 open-source tools which implement .pbxproj file editing:
https://github.com/CocoaPods/Xcodeproj (Ruby based)
https://github.com/apache/cordova-node-xcode (NodeJS based)
https://github.com/kronenthaler/mod-pbxproj (Python based)
Personally, I made the best experience with the NodeJS based tool. So far it has covered all our needs reliably.
In the following is listed an example javascript file update-project.js which sets the developer team ID, app entitlements, adds a GoogleService-Info.plist file to the project and checks it as part of the build target. Take it as an inspiration and adapt the scripts and its paths to your needs:
const fs = require('fs')
const xcode = require('xcode')
if (process.argv.length !== 3) {
console.error("Please pass the development team ID as the first argument")
process.exit(1)
}
const developmentTeamId = process.argv[2]
const path = 'ios/App/App.xcodeproj/project.pbxproj'
const project = xcode.project(path)
project.parse(error => {
const targetKey = project.findTargetKey('App')
const appGroupKey = project.findPBXGroupKey({path: 'App'})
project.addBuildProperty('CODE_SIGN_ENTITLEMENTS', 'App/App.entitlements')
project.addBuildProperty('DEVELOPMENT_TEAM', developmentTeamId)
project.addFile('App.entitlements', appGroupKey)
project.removeFile('GoogleService-Info.plist', appGroupKey)
const f = project.addFile('GoogleService-Info.plist', appGroupKey, {target: targetKey})
f.uuid = project.generateUuid()
project.addToPbxBuildFileSection(f)
project.addToPbxResourcesBuildPhase(f)
fs.writeFileSync(path, project.writeSync())
})
Above script can be executed with
yarn run update-project <arguments...>
given that update-project is registered in package.json:
{
...,
"scripts": {
...
"update-project": "node update-project.js"
},
...
}