I recently upgraded DSE to 5.X and OpsCenter to 6.0. Everything works great except the OpsCenter fails to start if I enable HTTPS using our own wildcard certificates. I have .pem file with cert chain and .key file with password removed. Same setup works perfectly with the OpsCenter 5.2.4 but with 6.0 I get the error below:
[opscenterd] ERROR: Traceback (most recent call last):
File "/usr/share/opscenter/jython/Lib/site-packages/opscenterd/OpsCenterdService.py", line 111, in setupWebServer
File "/usr/share/opscenter/jython/Lib/site-packages/opscenterd/WebServer.py", line 108, in makeWebServer
File "/usr/share/opscenter/jython/Lib/site-packages/opscenterd/SslUtils.py", line 24, in make_ssl_context_factory
File "/usr/share/opscenter/lib/py/twisted/internet/legacy_ssl.py", line 1133, in __init__ self.cacheContext()
File "/usr/share/opscenter/lib/py/twisted/internet/legacy_ssl.py", line 1142, in cacheContext ctx.load_cert_chain(self.certificateFileName, keyfile=self.privateKeyFileName) # Automatically checks against private key
File "/usr/share/opscenter/lib/jvm/jython-standalone-2.7.0.3.jar/Lib/ssl.py", line 1035, in load_cert_chain self._key_managers = _get_openssl_key_manager(certfile, keyfile, password, _key_store=self._key_store)
File "/usr/share/opscenter/lib/jvm/jython-standalone-2.7.0.3.jar/Lib/ssl.py", line 1035, in load_cert_chain self._key_managers = _get_openssl_key_manager(certfile, keyfile, password, _key_store=self._key_store)
File "/usr/share/opscenter/lib/jvm/jython-standalone-2.7.0.3.jar/Lib/_sslcerts.py", line 113, in _get_openssl_key_manager _certs, _private_key = _extract_certs_for_paths([cert_file], password)
File "/usr/share/opscenter/lib/jvm/jython-standalone-2.7.0.3.jar/Lib/_sslcerts.py", line 211, in _extract_certs_for_paths _certs, _private_key = _extract_cert_from_data(f, password, key_converter, cert_converter)
File "/usr/share/opscenter/lib/jvm/jython-standalone-2.7.0.3.jar/Lib/_sslcerts.py", line 230, in _extract_cert_from_data certs, private_key = _read_pem_cert_from_data(f, password, key_converter, cert_converter)
File "/usr/share/opscenter/lib/jvm/jython-standalone-2.7.0.3.jar/Lib/_sslcerts.py", line 246, in _read_pem_cert_from_data for br in _extract_readers(f):
File "/usr/share/opscenter/lib/jvm/jython-standalone-2.7.0.3.jar/Lib/_sslcerts.py", line 94, in _extract_readers raise SSLError(SSL_ERROR_SSL, "PEM lib (no start line or not enough data)")
SSLError: [Errno 1] PEM lib (no start line or not enough data)
(MainThread)
[opscenterd] ERROR: There was an error starting the OpsCenterd process: Traceback (most recent call last):
File "/usr/share/opscenter/jython/Lib/site-packages/opscenterd/OpsCenterdService.py", line 49, in startService
File "/usr/share/opscenter/jython/Lib/site-packages/opscenterd/OpsCenterdService.py", line 123, in setupWebServer
NameError: global name 'System' is not defined
(MainThread)
Note that OpsCenter starts if I use default certificate and key included in the installation.
Thanks for help!
Michael
Related
Not able to connect cosmos db using gramex mongo adapter.
From python, We can connect to local cosmosDB instance using:
# --------------------------------------
import pymongo
uri = r"mongodb://localhost:Tm%2BOzpb8BrV7DrZHSrGm3GMKyx9r%2Frl5ue9letmD1XRUUiafHFUyIQNenAQDla85nqVDrb8tr%2FtB0LR4azi1FQ%3D%3D#localhost:10255/admin?ssl=true"
client = pymongo.MongoClient(uri,
tls=True,
tlsCAFile='./documentdbemulatorcert.cer')
db = client.admin
print(db.command("serverStatus"))
# --------------------------------------
Please mind “tlsCAFile” parameter
However in gramex when I connect using:
# --------------------------------------
url:
envvartest-app-data:
pattern: /$YAMLURL/appdata
handler: FormHandler
kwargs:
url: "mongodb://localhost:Tm%2BOzpb8BrV7DrZHSrGm3GMKyx9r%2Frl5ue9letmD1XRUUiafHFUyIQNenAQDla85nqVDrb8tr%2FtB0LR4azi1FQ%3D%3D#localhost:10255/admin?ssl=true"
database: galaxy-dev
collection: threats
id: record_number
connect_args:
tls: True
tlsCAFile: $YAMLPATH/documentdbemulatorcert.cer
# ssl:
# ssl_ca: $YAMLPATH/documentdbemulatorcert.cer
# --------------------------------------
the connection fails reading:
ERROR 17-Feb 14:05:06 formhandler 9988 envvartest-app-data: filter failed
Traceback (most recent call last):
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\handlers\formhandler.py", line 158, in get
result[key] = yield val
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\tornado\gen.py", line 1133, in run
value = future.result()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\_base.py", line 428, in result
return self.__get_result()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\_base.py", line 384, in __get_result
raise self._exception
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\data.py", line 235, in filter
data = method(url=url, controls=controls, args=args, query=query, **kwargs)
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\data.py", line 1474, in _filter_mongodb
meta_cols = pd.DataFrame(list(table.find().limit(100)))
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\cursor.py", line 1238, in next
if len(self.__data) or self._refresh():
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\cursor.py", line 1130, in _refresh
self.__session = self.__collection.database.client._ensure_session()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\mongo_client.py", line 1935, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\mongo_client.py", line 1883, in __start_session
server_session = self._get_server_session()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\mongo_client.py", line 1921, in _get_server_session
return self._topology.get_server_session()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\topology.py", line 520, in get_server_session
session_timeout = self._check_session_support()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\topology.py", line 502, in _check_session_support
None)
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\topology.py", line 220, in _select_servers_loop
(self._error_message(selector), timeout, self.description))
pymongo.errors.ServerSelectionTimeoutError: localhost:10255: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1091), Timeout: 30s, Topology Description: <TopologyDescription id: 620e089c395cf9273486aa57, topology_type: Single, servers: [<ServerDescription ('localhost', 10255) server_type: Unknown, rtt: None, error=AutoReconnect('localhost:10255: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1091)')>]>
ERROR 17-Feb 14:05:06 __init__ 9988 500 GET /appdata (127.0.0.1) 30795.58ms envvartest-app-data
Not able to connect cosmos db using gramex mongo adapter.
Steps to reproduce
Install CosmosDB emulator
Generate access key:
.\Microsoft.Azure.Cosmos.Emulator.exe /GenKeyFile=D:\99exps\cosmosdb\key
Start cosmosDB emulator with mongoDB support:
.\Microsoft.Azure.Cosmos.Emulator.exe /FailOnSslCertificateNameMismatch /EnableMongoDbEndpoint=3.6 /EnableMongoDbEndpoint=3.2 /KeyFile=D:\99exps\cosmosdb\key
Create Gramex Application with above mentioned configuration
Observe the error
Edit 1:
The error is changed now:
ERROR 19-Feb 00:10:33 formhandler 9988 galaxy-app-data: filter failed
Traceback (most recent call last):
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\handlers\formhandler.py", line 158, in get
result[key] = yield val
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\tornado\gen.py", line 1133, in run
value = future.result()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\_base.py", line 428, in result
return self.__get_result()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\_base.py", line 384, in __get_result
raise self._exception
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\data.py", line 239, in filter
columns=columns, **kwargs)
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\data.py", line 1521, in _filter_mongodb
cols = [k for k in table.find().limit(1)[0].keys()]
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\cursor.py", line 694, in __getitem__
raise IndexError("no such item for Cursor instance")
IndexError: no such item for Cursor instance
WARNING 19-Feb 00:10:33 web 500 GET /appdata (::1): IndexError('no such item for Cursor instance')
ERROR 19-Feb 00:10:33 __init__ 9988 500 GET /appdata (::1) 846.11ms galaxy-app-data
As of Gramex 1.76.0 (Feb 2022), FormHandler does not connect to empty MongoDB collections, since there is no schema defined.
There is a plan to support empty MongoDB collections by specifying the schema explicitly.
But for now, you should be able to access the collection once you've added at least one row to it.
I am trying to config OAUTH_PROVIDERS using ambari
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SUPERSET/package/scripts/superset.py", line 184, in <module>
Superset().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 971, in restart
self.stop(env, upgrade_type=upgrade_type)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SUPERSET/package/scripts/superset.py", line 133, in stop
self.configure(env, upgrade_type=upgrade_type)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SUPERSET/package/scripts/superset.py", line 90, in configure
user=params.superset_user)
File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
self.env.run()
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run
returns=self.resource.returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'source /usr/hdp/current/superset/conf/superset-env.sh ; /usr/hdp/current/superset/bin/superset db upgrade' returned 1. Loaded your LOCAL configuration at [/usr/hdp/current/superset/conf/superset_config.py]
Traceback (most recent call last):
File "/usr/hdp/current/superset/bin/superset", line 12, in <module>
from superset.cli import manager
File "/usr/hdp/3.1.4.0-315/superset/lib/python3.6/site-packages/superset/__init__.py", line 180, in <module>
update_perms=utils.get_update_perms_flag(),
File "/usr/hdp/3.1.4.0-315/superset/lib/python3.6/site-packages/flask_appbuilder/base.py", line 135, in __init__
self.init_app(app, session)
File "/usr/hdp/3.1.4.0-315/superset/lib/python3.6/site-packages/flask_appbuilder/base.py", line 156, in init_app
self.sm = self.security_manager_class(self)
File "/usr/hdp/3.1.4.0-315/superset/lib/python3.6/site-packages/flask_appbuilder/security/sqla/manager.py", line 39, in __init__
super(SecurityManager, self).__init__(appbuilder)
File "/usr/hdp/3.1.4.0-315/superset/lib/python3.6/site-packages/flask_appbuilder/security/manager.py", line 199, in __init__
provider_name = _provider['name']
TypeError: string indices must be integers
I am able to setup superset Oauth without Ambari, but struggling to make the config in Ambari because even if make change in superset_cofig.py Ambari is overwriting the superset_cofig.py when we restart the service.
I never used Superset with Ambari, but I am currently struggling with it for standalone usage due to the lack of proper documentation and practical use cases.
To my understanding, in order to read the superset_config.py you need to export the PYTHONPATH and point down to the folder where the config is placed.
For example: export PYTHONPATH=/<folder where the config is placed>/:$PYTHONPATH
If you get this right, you should see in Superset's logs something like this
Loaded your LOCAL configuration at [/<folder where the config is>/superset_config.py]
I'm running a Dataflow job (Apache Beam 2.12.0) using python on the Google Cloud Platform. The pipeline is not terminating and keeps running.
The issue is the same to
https://issues.apache.org/jira/browse/BEAM-7266
It wasn't solved and says "open when met again". It also says that file writer is not closing.
There's only one error log:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py", line 178, in execute
op.finish()
File "dataflow_worker/native_operations.py", line 93, in dataflow_worker.native_operations.NativeWriteOperation.finish
def finish(self):
File "dataflow_worker/native_operations.py", line 94, in dataflow_worker.native_operations.NativeWriteOperation.finish
with self.scoped_finish_state:
File "dataflow_worker/native_operations.py", line 95, in dataflow_worker.native_operations.NativeWriteOperation.finish
self.writer.__exit__(None, None, None)
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/nativefileio.py", line 465, in __exit__
self.file.close()
File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/filesystemio.py", line 202, in close
self._uploader.finish()
File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/gcp/gcsio.py", line 606, in finish
raise self._upload_thread.last_error # pylint: disable=raising-bad-type
NotImplementedError: offset: 0, whence: 0, position: 51518, last: 0
I'm following this tutorial on using boto to access google cloud storage. I created a service account, downloaded the p12 file, and generated the .boto config file using "gsutil config -e" command. However, when I tried to list the buckets in my google cloud storage, I get following error.
import boto
import gcs_oauth2_boto_plugin
uri = boto.storage_uri('', 'gs')
for b in uri.get_all_buckets():
... print b.name
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/deployer/ENV/local/lib/python2.7/site-packages/boto/storage_uri.py", line 571, in get_all_buckets
return conn.get_all_buckets(headers)
File "/home/deployer/ENV/local/lib/python2.7/site-packages/boto/s3/connection.py", line 436, in get_all_buckets
response = self.make_request('GET', headers=headers)
File "/home/deployer/ENV/local/lib/python2.7/site-packages/boto/s3/connection.py", line 664, in make_request
retry_handler=retry_handler
File "/home/deployer/ENV/local/lib/python2.7/site-packages/boto/connection.py", line 1053, in make_request
retry_handler=retry_handler)
File "/home/deployer/ENV/local/lib/python2.7/site-packages/boto/connection.py", line 911, in _mexe
request.authorize(connection=self)
File "/home/deployer/ENV/local/lib/python2.7/site-packages/boto/connection.py", line 375, in authorize
connection._auth_handler.add_auth(self, **kwargs)
File "/home/deployer/ENV/local/lib/python2.7/site-packages/gcs_oauth2_boto_plugin/oauth2_plugin.py", line 70, in add_auth
self.oauth2_client.GetAuthorizationHeader()
File "/home/deployer/ENV/local/lib/python2.7/site-packages/gcs_oauth2_boto_plugin/oauth2_client.py", line 347, in GetAuthorizationHeader
return 'Bearer %s' % self.GetAccessToken().token
File "/home/deployer/ENV/local/lib/python2.7/site-packages/gcs_oauth2_boto_plugin/oauth2_client.py", line 318, in GetAccessToken
access_token = self.FetchAccessToken()
File "/home/deployer/ENV/local/lib/python2.7/site-packages/gcs_oauth2_boto_plugin/oauth2_client.py", line 395, in FetchAccessToken
credentials.refresh(http)
File "/home/deployer/ENV/local/lib/python2.7/site-packages/oauth2client/client.py", line 516, in refresh
self._refresh(http.request)
File "/home/deployer/ENV/local/lib/python2.7/site-packages/oauth2client/client.py", line 653, in _refresh
self._do_refresh_request(http_request)
File "/home/deployer/ENV/local/lib/python2.7/site-packages/oauth2client/client.py", line 710, in _do_refresh_request
raise AccessTokenRefreshError(error_msg)
oauth2client.client.AccessTokenRefreshError: invalid_grant
I have an issue whereby all clients cannot connect to openERP server 6.0.4 using port 8070.
It happened sometimes in a while (4-6 months). I wonder whats the problem, I checked the network traffic, processor, memory of the server, nothing wrong at all But it just happened few times.
When I checked on server logs, the error are same each time I met this issue, as below :
[2013-04-23 12:33:53,258][Server] ERROR:web-services:netrpc: cannot
deliver exception message to client Traceback (most recent call last):
File "/opt/openerp/server/bin/service/netrpc_server.py", line 89, in
run
ts.mysend(e, exception=True, traceback=tb_s) File "/opt/openerp/server/bin/tiny_socket.py", line 64, in mysend
self.sock.sendall('%8d%s%s' % (len(msg), exception and "1" or "0", msg)) File "/usr/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args) error: [Errno 32] Broken pipe
[2013-04-23 13:45:56,273][Server] ERROR:http:Could not run do_POST
Traceback (most recent call last): File
"/opt/openerp/server/bin/service/websrv_lib.py", line 299, in
_handle_one_foreign
method() File "/usr/lib/python2.7/SimpleXMLRPCServer.py", line 519, in do_POST
self.send_response(200) File "/usr/lib/python2.7/BaseHTTPServer.py", line 396, in send_response
(self.protocol_version, code, message)) File "/usr/lib/python2.7/socket.py", line 324, in write
self.flush() File "/usr/lib/python2.7/socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size]) error: [Errno 104] Connection reset by peer [2013-04-23
13:45:56,647][Server] ERROR:http:code 500, message Internal error
[2013-04-23 13:45:56,650][Server] ERROR:init:Server error in request
from ('192.168.0.132', 1880): Traceback (most recent call last):
File "/opt/openerp/server/bin/service/websrv_lib.py", line 528, in
_handle_request2
self.process_request(request, client_address) File "/usr/lib/python2.7/SocketServer.py", line 310, in process_request
self.finish_request(request, client_address) File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self) File "/opt/openerp/server/bin/service/websrv_lib.py", line 246, in init
SocketServer.StreamRequestHandler.init(self,request,client_address,server)
File "/usr/lib/python2.7/SocketServer.py", line 641, in init
self.finish() File "/usr/lib/python2.7/SocketServer.py", line 694, in finish
self.wfile.flush() File "/usr/lib/python2.7/socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size]) error: [Errno 32] Broken pipe
can anyone help me on this?
Broken-pipe error is a typical socket related error. It maybe if connect to slow from internet to server.
I suggest to use apache proxy to make available local server to internet. Mapping local server LOCALHOST:8069 to www.wxample.net:9000 using VirtualHost setting in apache. It may work for you.
For more information, Have a look at this link:
https://bugs.launchpad.net/openerp-web/+bug/927793
It may be helpful for you.