Local CosmosDB connectivity with pymongo for gramex - gramex

Question

Not able to connect cosmos db using gramex mongo adapter.
From python, We can connect to local cosmosDB instance using:
# --------------------------------------
import pymongo
uri = r"mongodb://localhost:Tm%2BOzpb8BrV7DrZHSrGm3GMKyx9r%2Frl5ue9letmD1XRUUiafHFUyIQNenAQDla85nqVDrb8tr%2FtB0LR4azi1FQ%3D%3D#localhost:10255/admin?ssl=true"
client = pymongo.MongoClient(uri,
tls=True,
tlsCAFile='./documentdbemulatorcert.cer')
db = client.admin
print(db.command("serverStatus"))
# --------------------------------------
Please mind “tlsCAFile” parameter
However in gramex when I connect using:
# --------------------------------------
url:
envvartest-app-data:
pattern: /$YAMLURL/appdata
handler: FormHandler
kwargs:
url: "mongodb://localhost:Tm%2BOzpb8BrV7DrZHSrGm3GMKyx9r%2Frl5ue9letmD1XRUUiafHFUyIQNenAQDla85nqVDrb8tr%2FtB0LR4azi1FQ%3D%3D#localhost:10255/admin?ssl=true"
database: galaxy-dev
collection: threats
id: record_number
connect_args:
tls: True
tlsCAFile: $YAMLPATH/documentdbemulatorcert.cer
# ssl:
# ssl_ca: $YAMLPATH/documentdbemulatorcert.cer
# --------------------------------------
the connection fails reading:
ERROR 17-Feb 14:05:06 formhandler 9988 envvartest-app-data: filter failed
Traceback (most recent call last):
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\handlers\formhandler.py", line 158, in get
result[key] = yield val
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\tornado\gen.py", line 1133, in run
value = future.result()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\_base.py", line 428, in result
return self.__get_result()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\_base.py", line 384, in __get_result
raise self._exception
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\data.py", line 235, in filter
data = method(url=url, controls=controls, args=args, query=query, **kwargs)
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\data.py", line 1474, in _filter_mongodb
meta_cols = pd.DataFrame(list(table.find().limit(100)))
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\cursor.py", line 1238, in next
if len(self.__data) or self._refresh():
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\cursor.py", line 1130, in _refresh
self.__session = self.__collection.database.client._ensure_session()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\mongo_client.py", line 1935, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\mongo_client.py", line 1883, in __start_session
server_session = self._get_server_session()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\mongo_client.py", line 1921, in _get_server_session
return self._topology.get_server_session()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\topology.py", line 520, in get_server_session
session_timeout = self._check_session_support()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\topology.py", line 502, in _check_session_support
None)
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\topology.py", line 220, in _select_servers_loop
(self._error_message(selector), timeout, self.description))
pymongo.errors.ServerSelectionTimeoutError: localhost:10255: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1091), Timeout: 30s, Topology Description: <TopologyDescription id: 620e089c395cf9273486aa57, topology_type: Single, servers: [<ServerDescription ('localhost', 10255) server_type: Unknown, rtt: None, error=AutoReconnect('localhost:10255: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1091)')>]>
ERROR 17-Feb 14:05:06 __init__ 9988 500 GET /appdata (127.0.0.1) 30795.58ms envvartest-app-data
Not able to connect cosmos db using gramex mongo adapter.
Steps to reproduce
Install CosmosDB emulator
Generate access key:
.\Microsoft.Azure.Cosmos.Emulator.exe /GenKeyFile=D:\99exps\cosmosdb\key
Start cosmosDB emulator with mongoDB support:
.\Microsoft.Azure.Cosmos.Emulator.exe /FailOnSslCertificateNameMismatch /EnableMongoDbEndpoint=3.6 /EnableMongoDbEndpoint=3.2 /KeyFile=D:\99exps\cosmosdb\key
Create Gramex Application with above mentioned configuration
Observe the error
Edit 1:
The error is changed now:
ERROR 19-Feb 00:10:33 formhandler 9988 galaxy-app-data: filter failed
Traceback (most recent call last):
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\handlers\formhandler.py", line 158, in get
result[key] = yield val
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\tornado\gen.py", line 1133, in run
value = future.result()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\_base.py", line 428, in result
return self.__get_result()
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\_base.py", line 384, in __get_result
raise self._exception
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\data.py", line 239, in filter
columns=columns, **kwargs)
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\data.py", line 1521, in _filter_mongodb
cols = [k for k in table.find().limit(1)[0].keys()]
File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\cursor.py", line 694, in __getitem__
raise IndexError("no such item for Cursor instance")
IndexError: no such item for Cursor instance
WARNING 19-Feb 00:10:33 web 500 GET /appdata (::1): IndexError('no such item for Cursor instance')
ERROR 19-Feb 00:10:33 __init__ 9988 500 GET /appdata (::1) 846.11ms galaxy-app-data

Answer 1

As of Gramex 1.76.0 (Feb 2022), FormHandler does not connect to empty MongoDB collections, since there is no schema defined.
There is a plan to support empty MongoDB collections by specifying the schema explicitly.
But for now, you should be able to access the collection once you've added at least one row to it.