I have the same code with the same p12 file (checked md5 sums), same account_email, and same scope working on a number of computers, but not working in Docker containers on any of the working computers. My code snippet is as follows:
with open(self.pkcs12_file_path, 'rb') as f:
key = f.read()
scope = ['https://www.googleapis.com/auth/bigquery',
'https://www.googleapis.com/auth/cloud-platform']
credentials = SignedJwtAssertionCredentials(self.account_email,
key, scope)
http = httplib2.Http()
self.http = credentials.authorize(self.http)
service = discovery.build('bigquery', 'v2', http=self.http)
Whenever I try this inside of a Docker container, I get oauth2client.client.AccessTokenRefreshError: invalid_grant at the discovery.build line. I'm thinking it might have something to do with needing to expose ports, but have no idea which I'd need to expose, or if that's the actual problem. Anyone have any ideas?
This sounds like a clock issue. Google's OAuth access tokens are valid for one hour (Google refresh tokens are valid forever and you can use them to retrieve a new access token). Can you verify that the Docker container's clock is synced with the host machine (or set to your expected time zone?).
See: Will docker container auto sync time with the host machine?
Related
right now I'm deploying to cloud run and run
gcloud run deploy myapp --tag pr123 --no-traffic
I can then access the app via
https://pr123---myapp-jo5dg6hkf-ez.a.run.app
Now I would like to have a custom domain mapping going to this tag. I know how to point a custom domain to the service but I don't know how to point it to the tagged version of my service.
Can I add labels to the DomainMapping that would cause the mapping to got this version of my cloud run service? Or is there a routeName, eg. myapp#pr123 that would do the trick there?
In the end I would like to have
https://pr123.dev.mydomain.com
being the endpoint for this service.
With a custom domain, you configure a DNS to point to a service, not a revision/tag of the service. So, you can't by this way.
The solution is to use a load balancer with a serverless NEG. The most important is to define the URL mask that you want to map the tag and service from the URL which is received by the Load Balancer.
I ended up building the loadbalancer with a network endpoint group (as suggested). For further reference, here is my terraform snippet to create it. The part is then the traffic tag you assign to your revision.
resource "google_compute_region_network_endpoint_group" "api_neg" {
name = "api-neg"
network_endpoint_type = "SERVERLESS"
region = "europe-west3"
cloud_run {
service = data.google_cloud_run_service.api_dev.name
url_mask = "<tag>.preview.mydomain.com"
}
}
I'm working on a simple REST client for Docker Registry. For private registries, name resolution is pretty simple; if the image name is myregistry.io/myimage:latest, I look for https://myregistry.io/v2 and query the API there.
However, I notice that for docker hub, it doesn't quite work that way. If I'm looking for ubuntu, I can expand that to docker.io/ubuntu:latest, but https://docker.io/v2 returns a 307 redirect to https://www.docker.com/v2, which just returns HTML. The actual registry endpoint is at https://registry-1.docker.io/v2.
Is this just a hardcoded special case in the docker client, or is there some extra logic to looking up registry endpoints that I'm unaware of? If it is just a special case, is there more to it than always going to registry-1.docker.io instead of docker.io?
The central Docker registry is a well-known special case, similar to Maven central. You can see the defaults e.g. at https://github.com/docker/docker-ce/blob/ea449e9b10cebb259e1a43325587cd9a0e98d0ff/components/engine/registry/config.go#L42:
var (
// DefaultNamespace is the default namespace
DefaultNamespace = "docker.io"
// DefaultRegistryVersionHeader is the name of the default HTTP header
// that carries Registry version info
DefaultRegistryVersionHeader = "Docker-Distribution-Api-Version"
// IndexHostname is the index hostname
IndexHostname = "index.docker.io"
// IndexServer is used for user auth and image search
IndexServer = "https://" + IndexHostname + "/v1/"
// IndexName is the name of the index
IndexName = "docker.io"
// DefaultV2Registry is the URI of the default v2 registry
DefaultV2Registry = &url.URL{
Scheme: "https",
Host: "registry-1.docker.io",
}
)
I've setup a perfectly functioning application (in VB) that allows user to access his Google Sheets.
The application follows Google's OAuth documentation for displaying a sign-in dialog in a web browser, obtains user's permission and access codes, uses access codes to obtain access token, and then uses the Google Sheet's Query service to get hold of Google Sheets. Very simple. Works fine.
Problem occurs on computers that have internet proxy defined on them. In the rest of my application and most of Google Sheets API, I can define a manual internet proxy. GData's RequestFactory allows manually configuring proxy server. The only line of code that doesn't support (to my current knowledge) is the OAuthUtil library used for obtaining access token. It doesn't allow defining internet proxy server, hence it is unable to resolve host on computers behind proxy environment. Following is my pseudo code:
Dim parameters As New OAuth2Parameters
parameters.ClientId = CLIENT_ID
parameters.ClientSecret = CLIENT_SECRET
parameters.RedirectUri = REDIRECT_URI
parameters.Scope = SCOPE
>>Show browser window and obtain access code
parameters.AccessCode = login.Token
OAuthUtil.GetAccessToken(parameters) '<< Point of failure
Dim requestFactory As GOAuth2RequestFactory = New GOAuth2RequestFactory(Nothing, My.Application.Info.ProductName, parameters)
requestFactory.Proxy = GetProxySettings() '<< my code for defining proxy
myService = New SpreadsheetsService("Application")
myService.RequestFactory = requestFactory
Another important aspect is that my application works on Mac OSX as well using Wine (for web browser I use GeckoFX). If internet proxy is globally defined on the environment then the OAuthUtil works fine, but this doesn't work for Wine. I have tried setting internet proxy in the command-line environment, or in the registry and refreshed system settings, still the applications running in Wine do not understand that proxy is defined. Hence proxy has to be manually defined.
I need help to figure out a solution by any of the following:
* A way to forcefully/manually define proxy for OAuthUtil for obtaining access token
* Any other way to obtain OAuth access token if proxy cannot be defined as above (maybe WebClient can be used?)
* Some way to define global internet proxy in Wine so applications like GData API read and understand the proxy setting. Though I'd rather prefer manually defined proxy at application level.
Any ideas folks?
Regards
F.A.
I've figured it out. Turns out that the 'OAuthUtil.GetAccessToken' only uses system-defined proxy. There is no way to manually define internet proxy, like RequestFactory supports. So there is a work-around using WebClient:
Try
'// Get access token from code
Using WC As New WebClient
' Define proxy
WC.Proxy = GetProxySettings()
' Set parameters
WC.Headers(HttpRequestHeader.ContentType) = "application/x-www-form-urlencoded"
' Get response
Dim postURL = "https://www.googleapis.com/oauth2/v4/token"
Dim postParams = "code=" & parameters.AccessCode &
"&client_id=" & Uri.EscapeDataString(CLIENT_ID) &
"&client_secret=" & Uri.EscapeDataString(CLIENT_SECRET) &
"&redirect_uri=" & Uri.EscapeDataString(REDIRECT_URI) &
"&grant_type=authorization_code"
Dim responsebody As String = WC.UploadString(postURL, postParams)
' Read response
Dim jObj As JObject = JsonConvert.DeserializeObject(responsebody)
' Store token
parameters.AccessToken = jObj("access_token").ToString
parameters.RefreshToken = jObj("refresh_token").ToString
parameters.TokenType = jObj("token_type").ToString
parameters.TokenExpiry = Now().AddSeconds(CDbl(jObj("expires_in").ToString))
End Using
Catch ex As Exception
MsgBox("Error obtaining access token: " & ex.Message, MsgBoxStyle.Critical)
Return Nothing
End Try
My JMS client connects to WMQ through JNDI. The initial context factory used is com.ibm.mq.jms.context.WMQInitialContextFactory.
Currently, at WMQ side, there's a queue manager called TestMgr. Under this queue manager I created two channels. One is PLAIN.CHL which does not specify an SSL Cipher Spec, the other one is SSL.CHL which configured SSL Cipher Spec with RC4_MD5_US and SSL Authentication with Optional.
I have created a key store for the queue manager using IBM Key Management tool. The path of key db is [wmq_home]\qmgrs\TestMgr\ssl\key.
For channel PLAIN.CHL, I defined a queue connection factory like:
DEF QCF(PlainQCF) QMANAGER(TestMgr) CHANNEL(PLAIN.CHL) HOST(192.168.66.23) PORT(1414) TRANSPORT(client)
And under the SSL channel SSL.CHL, I defined a queue connection factory like:
DEF QCF(SSLQCF) QMANAGER(TestMgr) CHANNEL(SSL.CHL) HOST(192.168.66.23) PORT(1414) TRANSPORT(client) SSLCIPHERSUITE(SSL_RSA_WITH_RC4_128_MD5)
Now I only can create connection using the PlainQCF. But failed to look up the SSL queue connection factory. My code looks like:
Hashtable environment = new Hashtable();
environment.put(Context.INITIAL_CONTEXT_FACTORY, "com.ibm.mq.jms.context.WMQInitialContextFactory");
environment.put(Context.PROVIDER_URL, "192.168.66.23:1414/SSL.CHL");
Context ctx = new InitialContext( environment );
QueueConnectionFactory qcf = (QueueConnectionFactory) ctx.lookup("SSLQCF");
qcf.createConnection();
....
Am I missing some context properties when looking up the SSL factory? connection And then I found the code is hanging on the line new InitialContext( environment ) for a long time, almost 5 minutes, and I got CC=2;RC=2009;AMQ9208... error.
Any suggestion would be appreciated. Is it true that SSL channel can't be connected by JNDI?
#T.Rob, thanks for your reply very much. But we still want to use WMQInitialContextFactory, so I'm afraid I still need to find solution for this.
I just defined the connection factory one time. The displayed info for the SSL queue connection factory like:
InitCtx> DISPLAY QCF(SSLQCF)
ASYNCEXCEPTION(ALL)
CCSID(819)
CHANNEL(SSL.CHL)
CLIENTRECONNECTOPTIONS(ASDEF)
CLIENTRECONNECTTIMEOUT(1800)
COMPHDR(NONE )
COMPMSG(NONE )
CONNECTIONNAMELIST(192.168.66.23(1414))
CONNOPT(STANDARD)
FAILIFQUIESCE(YES)
HOSTNAME(192.168.66.23)
LOCALADDRESS()
MAPNAMESTYLE(STANDARD)
MSGBATCHSZ(10)
MSGRETENTION(YES)
POLLINGINT(5000)
PORT(1414)
PROVIDERVERSION(UNSPECIFIED)
QMANAGER(TestMgr)
RESCANINT(5000)
SENDCHECKCOUNT(0)
SHARECONVALLOWED(YES)
SSLCIPHERSUITE(SSL_RSA_WITH_RC4_128_MD5)
SSLFIPSREQUIRED(NO)
SSLRESETCOUNT(0)
SYNCPOINTALLGETS(NO)
TARGCLIENTMATCHING(YES)
TEMPMODEL(SYSTEM.DEFAULT.MODEL.QUEUE)
TEMPQPREFIX()
TRANSPORT(CLIENT)
USECONNPOOLING(YES)
VERSION(7)
WILDCARDFORMAT(TOPIC_ONLY)
The JNDI Provider should be fine because I can look up the plain connection factory successfully. Also, for my client app, I extracted the cert from the key store which created for MQ server and imported it to the trust store(cacerts) of my JRE with alias name ibmwebspheremqtestmgr.
You are correct, with 2009 error there are some log entries:
=================================================================
4/20/2012 20:24:27 - Process(13768.3) User(MUSR_MQADMIN) Program(amqzmur0.exe)
Host(xxxx_host of my MQ) Installation(mqenv)
VRMF(7.1.0.0) QMgr(TestMgr)
AMQ6287: WebSphere MQ V7.1.0.0 (p000-L111019).
EXPLANATION:
WebSphere MQ system information:
Host Info :- Windows Server 2003, Build 3790: SP2 (MQ Windows 32-bit)
Installation :- C:\IBM\WebSphereMQ (mqenv)
Version :- 7.1.0.0 (p000-L111019)
ACTION:
None.
-------------------------------------------------------------------------------
4/20/2012 20:24:27 - Process(7348.116) User(MUSR_MQADMIN) Program(amqrmppa.exe)
Host(xxxx_host of my MQ) Installation(mqenv)
VRMF(7.1.0.0) QMgr(TestMgr)
AMQ9639: Remote channel 'SSL.CHL' did not specify a CipherSpec.
EXPLANATION:
Remote channel 'SSL.CHL' did not specify a CipherSpec when the local channel
expected one to be specified.
The remote host is 'xxx_host of my app (192.168.66.25)'.
The channel did not start.
ACTION:
Change the remote channel 'SSL.CHL' on host 'xxx_host of my app (192.168.66.25)' to
specify a CipherSpec so that both ends of the channel have matching
CipherSpecs.
----- amqcccxa.c : 3817 -------------------------------------------------------
4/20/2012 20:24:27 - Process(7348.116) User(MUSR_MQADMIN) Program(amqrmppa.exe)
Host(my app host) Installation(mqenv)
VRMF(7.1.0.0) QMgr(TestMgr)
AMQ9999: Channel 'SSL.CHL' to host 'xxx_host of my app (192.168.66.25)' ended
abnormally.
====================================================================
I also got some confusion with the error log. My app staged at at a machine which is different from my MQ. But the log says the Change the remote channel 'SSL.CHL' on host 'xxx_host of my app (192.168.66.25)' to
specify a CipherSpec so that both ends of the channel have matching
CipherSpecs. How can I change the channel cipher spec on my app host?
updates on MQEnvironment...
reply the comments.
The value of MQEnvironment.sslCipherSuite is null, so it throws out NullPointerExcetpion when i put it the the env hashtable. But i tried another one environment.put(MQC.SSL_CIPHER_SUITE_PROPERTY, "SSL_RSA_WITH_RC4_128_MD5") and it still failed with 2009 error.
For JMSAdmin tool, i had changed the config to use WMQInitialContextFactory. The configuration like(JMSAdmin.config):
INITIAL_CONTEXT_FACTORY=com.ibm.mq.jms.context.WMQInitialContextFactory
PROVIDER_URL=192.168.66.23:1414/SYSTEM.DEF.SVRCONN
The rest configuration leaves as default.
Kindly note, here i use the default channel SYSTEM.DEF.SVRCONN so that i can logon to admin console. If I change the channel to the SSL oneSSL.CHL, I also can't logon to admin console. The error happened here is just like the one in my client app.
Another clarification, in my client, i use follow code can connect to connect qmgr(TestMgr) successfully through channel SSL.CHL.
MQConnectionFactory factory = new MQConnectionFactory();
factory.setTransportType(JMSC.MQJMS_TP_CLIENT_MQ_TCPIP);
factory.setQueueManager("TestMgr");
factory.setSSLCipherSuite("SSL_RSA_WITH_RC4_128_MD5");
factory.setPort(1414);
factory.setHostName("192.168.66.23");
factory.setChannel("SSL.CHL");
MQConnection connection = (MQConnection) factory.createConnection();
And now the problem is just like you said, that's the initial context failed connect to qmgr through SSL channel. The option(use plain channel for initial context and ssl channel for connection factory) you provided works too. But I still want to know how to get initial context with ssl channel work. Thanks for you patience very much. Your updates will be appreciated.
thanks
I never really liked com.ibm.mq.jms.context.WMQInitialContextFactory very much. It stores the managed objects on a queue. So in order to lookup the connectionFactory, which tells JMS how to connect to the QMgr, it is first necessary to connect to the QMgr to make the JNDI call. Therefore, before you can debug the SSL connection, you need to know whether the underlying JNDI provider is working.
If you want to skip the MQ-based JNDI provider and just use the filesystem, see the updated version of Bobby Woolf's article here. If you want to continue with com.ibm.mq.jms.context.WMQInitialContextFactory, read on but be prepared to provide more configuration info.
When you run the JMSAdmin tool, do you display the objects after creating them? For example, here is one of my JMSAdmin.bat scripts:
# Connection Factory for Client mode
# Delete the Connection Factory if it exists
DELETE CF(JMSDEMOCF)
# Define the Connection Factory
DEFINE CF(JMSDEMOCF) +
SYNCPOINTALLGETS(YES) +
SSLCIPHERSUITE(NULL_SHA) +
TRAN(client) +
HOST(127.0.0.1) CHAN(SSL.SVRCONN) PORT(1414) +
QMGR( )
# Display the resulting definition
DISPLAY CF(JMSDEMOCF)
This deletes the object (because JMSAdmin doesn't have a define with replace option) then defines the object, then displays it. Do you in fact see both objects defined? Can you connect and interactively display them both? Can you update your question with the contents displayed?
If so, then what does the JNDI provider configuration look like with each sample program? The 2009 indicates that there is at least a connection to the QMgr being made, so it is important to determine whether the thing that suffering the broken connection is your app or the JNDI provider. To diagnose that requires the config info you are using for the JNDI provider and whether it is the same in the working and failing cases. If not, how do they differ?
Once you know whether it's the app or the JNDI provider that is causing the problem (or switch to another JNDI provider that doesn't require an MQ connection such as the filesystem initial context) then it will be possible to determine the next steps.
The article linked above has samples of code and managed object scripts that use a filesystem JNDI provider. You may notice my scripts pasted in above use the same QMgr name. That's because I wrote that part of the article. When I want to switch to SSL using those same samples, I just update the connectionFactory to point to the SSL channel and it works.
Here are the other bits from the sample that I've modified:
java -Djavax.net.debug=ssl ^
-Djavax.net.ssl.trustStore=key2.jks ^
-Djavax.net.ssl.keyStore=key2.jks ^
-Djavax.net.ssl.keyStorePassword=???????? ^
-Djavax.net.ssl.trustStorePassword=???????? ^
-cp "%CLASSPATH%" ^
com.ibm.examples.JMSDemo -pub -topic JMSDEMOPubTopic %*
Note: The ^ is Windows version of line continuation.
Then if there are problems, I follow the debugging scenario I described in this SO answer. Note that the app will require a truststore, even if you have SSLCAUTH(OPTIONAL) on your channel. This is because the app must always validate the QMgr's certificate, even if the app does not present its own certificate. In my case I was using SSLCAUTH(REQUIRED) so my app needed both a keystore and a truststore. Your question mentions that the QMgr has a keystore but does not say what you did for the application.
Finally, a 2009 will usually generate an entry in the QMgr error logs. If you continue to get the problem, please update your question with those log entries.
UPDATE:
Responding to the comments, the JMSAdmin tool is part of the WMQ package. However, WMQ it comes with jars for filesystem context and LDAP context. The WMQInitialContextFactory is optional and is delivered as SupportPac ME01. When using WMQInitialContextFactory with the JMSAdmin tool (or the JMSAdmin GUI or with WMQ Explorer) it is necessary to configure the PROVIDER_URL with the host, port and channel. For example:
PROVIDER_URL: <Hostname>:<port>/<SVRCONN Channel Name>
192.168.66.23:1414/SSL.SVRCONN
So after reviewing your post again, I realized that you did provide the config info for WMQInitialContextFactory. I was looking for a JMSADmin.config file but you have it in the environment hash table. And that is where the problem is. You are attempting to use the SSL channel for both the WMQInitialContextFactory and the connection factory. This is what is causing the lookup to fail. The WMQInitialContextFactory first makes a Java connection to the QMgre in order to look in the queue to obtain the administered objects such as QCF. In order to do that, it needs to know the ciphersuite that the channel is set up for in order to negotiate the handshake. Right now, the *only * place that ciphersuite is recorded is in the QCF definition.
Try adding the following line:
environment.put(MQEnvironment.sslCipherSuite, "SSL_RSA_WITH_RC4_128_MD5");
As per this Infocenter page, that should tell the context factory classes what ciphersuite to use. Of course, they also need to know where the trust store is (and possibly keystore if the channel has SSLCAUTH(RQUIRED) set) so you still need to get those values in the environment. You can use the command-line variables or try loading them into the environment using code. You'll need both -Djavax.net.ssl.trustStore=key2.jks and -Djavax.net.ssl.trustStorePassword=????????.
The other option is to continue to use the plaintext channel for the WMQInitialContextFactory and the SSL channel for the application. If the plaintext channel has an MCAUSER for a non-privileged user ID, it can be restricted to only connect to the QMgr and access the queue that contains the administered objects. With those restrictions, anyone will be able to read the administered objects using that channel but not the application queues or administrative queues.
I have a client application which runs as a Java applet from a user's browser and connects to a server via a given port. The server is running on a publicly accessible cloud. Based on my previous experience of writing socket code , I can decide upon a random port number (say 5999) and use it for client server communication. However in this case the client can be any user machine and there can be many users accessing the server.
So the question is how to ensure that I use a port number which is least likely to be used by any other service on the client's computer.
I have also explored webservices based protocols for this purpose but I didnt use it for the reason that my requirement is really simple and it can be fulfilled with a simple socket communication and a custom protocol. I feel webservices tools and stuff like SOAP , CORBA are too heavy weight.
Choose one that is not in the Service Name and Transport Protocol Port Number Registry and hope for the best.
Also, a client can connect to many servers on the same port. When the clients connect, they will use a random port on there end.
Only the server needs to worry about using a free port, and the clients need to know what this port is else they will not be able to connect to your server.
You don't have to choose a portnumber on the users PC. Just the server port has to be one specific number.
When the client connects to the servers IP and port number, the operating system chooses a random free port for the client itself.
There are plenty of them as #thecoshman mentioned, and I compiled some of them for quick reference :)
258, 285, 325-332, 334-343, 703, 708, 717-728, 732-740, 743, 745-746, 755-757, 766, 768, 778-779, 781-785, 787, 788-799, 803-809, 811-827, 834-846, 849-852, 855-859, 863-872, 874-885, 889-899, 904-909, 914-952, 954-988, 1002-1007, 1009, 1491, 2194-2196, 2259, 2369, 2378, 2794, 2873, 3092, 3126, 3301, 3546, 3694, 3994, 4048, 4144, 4194-4196, 4198, 4315, 4317-4319, 4332, 4337-4339, 4363-4365, 4367, 4380-4388, 4397-4399, 4424, 4434-4440, 4459-4483, 4489-99, 4501, 4503-4533, 4539-4544, 4560-4562, 4564-4565, 4571-4572, 4574-4589, 4606-4620, 4622-4657, 4693-4699, 4705-4710, 4712-4724, 4734-4736, 4748, 4757-4773, 4775-4783, 4792-4799, 4805-4826, 4828-4836, 4852-4866, 4872-4875, 4886-4893, 4895-4898, 4903-4911, 4916-4935, 4938-4939, 4943-4948, 4954-4968, 4972-4979, 4981-4983, 4992-4998