I am trying to restart the airflow scheduler using the following command
airflow scheduler
I am using docker. I went inside my docker image for airflow and opened the CLI for my airflow image. That is where I used this command.
It throws an exception
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 25, in <module>
from airflow.configuration import conf
File "/usr/local/lib/python3.6/site-packages/airflow/__init__.py", line 31, in <module>
from airflow.utils.log.logging_mixin import LoggingMixin
File "/usr/local/lib/python3.6/site-packages/airflow/utils/__init__.py", line 24, in <module>
from .decorators import apply_defaults as _apply_defaults
File "/usr/local/lib/python3.6/site-packages/airflow/utils/decorators.py", line 36, in <module>
from airflow import settings
File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 37, in <module>
from airflow.configuration import conf, AIRFLOW_HOME, WEBSERVER_CONFIG # NOQA F401
File "/usr/local/lib/python3.6/site-packages/airflow/configuration.py", line 731, in <module>
conf.read(AIRFLOW_CONFIG)
File "/usr/local/lib/python3.6/site-packages/airflow/configuration.py", line 421, in read
self._validate()
File "/usr/local/lib/python3.6/site-packages/airflow/configuration.py", line 213, in _validate
self._validate_config_dependencies()
File "/usr/local/lib/python3.6/site-packages/airflow/configuration.py", line 247, in _validate_config_dependencies
self.get('core', 'executor')))
airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the LocalExecutor
I am looking for any way to restart the airflow scheduler.
This is expected.
Since sqlite doesn’t support multiple connections it can only be used with SequentialExecutor. This is also explained in the docs.
If you want to use LocalExecutor please set MySQL or PostgreSQL as backend.
Related
Describe the bug
Timeout occurred on objection explore
To Reproduce
Steps to reproduce the behavior:
Run command objection --gadget "com.apple.AppStore" explore
Evidence / Logs / Screenshots
Using USB device `iPhone`
Traceback (most recent call last):
File "/usr/local/bin/objection", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/objection/console/cli.py", line 114, in explore
agent.inject()
File "/usr/local/lib/python3.9/site-packages/objection/utils/agent.py", line 202, in inject
session = self.get_session()
File "/usr/local/lib/python3.9/site-packages/objection/utils/agent.py", line 169, in get_session
self.session = self.device.attach(self.spawned_pid)
File "/usr/local/lib/python3.9/site-packages/frida/core.py", line 76, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/frida/core.py", line 800, in attach
return Session(self._impl.attach(self._pid_of(target), **kwargs)) # type: ignore
frida.TimedOutError: unexpectedly timed out while initializing suspended process
Environment (please complete the following information):
Device: iPhone 7
OS: 15.3.1
Frida Version: 16.0.2
Objection Version: 1.11.0
Thanks to #Robert solved with pid choice. There are two important things here:
Don't use Apple-Slicon(m1/m2) MacOS
Use process-id instead of process-name. objection -g pid explore
Please, so trying to deploy a custom Pytorch based web-app on streamlit, everything works locally, however when deployed, I find the following Error in the Logs:
Downloading: "https://github.com/ultralytics/yolov5/archive/master.zip" to /home/appuser/.cache/torch/hub/master.zip
2022-08-14 12:23:17.584 Uncaught app exception
Traceback (most recent call last):
File "/home/appuser/venv/lib/python3.9/site-packages/streamlit/scriptrunner/script_runner.py", line 557, in _run_script
exec(code, module.__dict__)
File "app.py", line 28, in <module>
model = torch.hub.load('ultralytics/yolov5', 'custom', path=run_model_path)
File "/home/appuser/venv/lib/python3.9/site-packages/torch/hub.py", line 339, in load
model = _load_local(repo_or_dir, model, *args, **kwargs)
File "/home/appuser/venv/lib/python3.9/site-packages/torch/hub.py", line 368, in _load_local
model = entry(*args, **kwargs)
File "/home/appuser/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 74, in custom
return _create(path, autoshape=autoshape, verbose=_verbose, device=device)
File "/home/appuser/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 31, in _create
from models.common import AutoShape, DetectMultiBackend
File "/home/appuser/.cache/torch/hub/ultralytics_yolov5_master/models/common.py", line 14, in <module>
import cv2
File "/home/appuser/venv/lib/python3.9/site-packages/cv2/__init__.py", line 5, in <module>
from .cv2 import *
ImportError: libGL.so.1: cannot open shared object file: No such file or directory
Here's what the requrements.txt contains:
pillow<=9.2.0
numpy<=1.21.0
streamlit==1.11.0
torch<=1.8.2+cu111
opencv<=4.5.1
Would love your assistance.
If you're deploying the app, opencv-python-headless would be the appropriate package rather than opencv
I am trying to config OAUTH_PROVIDERS using ambari
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SUPERSET/package/scripts/superset.py", line 184, in <module>
Superset().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 971, in restart
self.stop(env, upgrade_type=upgrade_type)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SUPERSET/package/scripts/superset.py", line 133, in stop
self.configure(env, upgrade_type=upgrade_type)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SUPERSET/package/scripts/superset.py", line 90, in configure
user=params.superset_user)
File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
self.env.run()
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run
returns=self.resource.returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'source /usr/hdp/current/superset/conf/superset-env.sh ; /usr/hdp/current/superset/bin/superset db upgrade' returned 1. Loaded your LOCAL configuration at [/usr/hdp/current/superset/conf/superset_config.py]
Traceback (most recent call last):
File "/usr/hdp/current/superset/bin/superset", line 12, in <module>
from superset.cli import manager
File "/usr/hdp/3.1.4.0-315/superset/lib/python3.6/site-packages/superset/__init__.py", line 180, in <module>
update_perms=utils.get_update_perms_flag(),
File "/usr/hdp/3.1.4.0-315/superset/lib/python3.6/site-packages/flask_appbuilder/base.py", line 135, in __init__
self.init_app(app, session)
File "/usr/hdp/3.1.4.0-315/superset/lib/python3.6/site-packages/flask_appbuilder/base.py", line 156, in init_app
self.sm = self.security_manager_class(self)
File "/usr/hdp/3.1.4.0-315/superset/lib/python3.6/site-packages/flask_appbuilder/security/sqla/manager.py", line 39, in __init__
super(SecurityManager, self).__init__(appbuilder)
File "/usr/hdp/3.1.4.0-315/superset/lib/python3.6/site-packages/flask_appbuilder/security/manager.py", line 199, in __init__
provider_name = _provider['name']
TypeError: string indices must be integers
I am able to setup superset Oauth without Ambari, but struggling to make the config in Ambari because even if make change in superset_cofig.py Ambari is overwriting the superset_cofig.py when we restart the service.
I never used Superset with Ambari, but I am currently struggling with it for standalone usage due to the lack of proper documentation and practical use cases.
To my understanding, in order to read the superset_config.py you need to export the PYTHONPATH and point down to the folder where the config is placed.
For example: export PYTHONPATH=/<folder where the config is placed>/:$PYTHONPATH
If you get this right, you should see in Superset's logs something like this
Loaded your LOCAL configuration at [/<folder where the config is>/superset_config.py]
I'm running a Dataflow job (Apache Beam 2.12.0) using python on the Google Cloud Platform. The pipeline is not terminating and keeps running.
The issue is the same to
https://issues.apache.org/jira/browse/BEAM-7266
It wasn't solved and says "open when met again". It also says that file writer is not closing.
There's only one error log:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py", line 178, in execute
op.finish()
File "dataflow_worker/native_operations.py", line 93, in dataflow_worker.native_operations.NativeWriteOperation.finish
def finish(self):
File "dataflow_worker/native_operations.py", line 94, in dataflow_worker.native_operations.NativeWriteOperation.finish
with self.scoped_finish_state:
File "dataflow_worker/native_operations.py", line 95, in dataflow_worker.native_operations.NativeWriteOperation.finish
self.writer.__exit__(None, None, None)
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/nativefileio.py", line 465, in __exit__
self.file.close()
File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/filesystemio.py", line 202, in close
self._uploader.finish()
File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/gcp/gcsio.py", line 606, in finish
raise self._upload_thread.last_error # pylint: disable=raising-bad-type
NotImplementedError: offset: 0, whence: 0, position: 51518, last: 0
ubuntu#ubuntu-02:/reddit/r2$ paster serve --reload example.ini http_port=8080Starting subprocess with file monitor
/usr/local/lib/python2.6/dist-packages/Pylons-0.9.6.2-py2.6.egg/pylons/middleware.py:11: DeprecationWarning: The webhelpers.rails package is deprecated.
- Please begin migrating to the new helpers in webhelpers.html,
webhelpers.text, webhelpers.number, etc.
- Import url_for() directly from routes, and redirect_to() from
pylons.controllers.util (if using Pylons) or from routes.
- All Javascript support has been deprecated. You can write link_to_remote()
yourself or use one of the third-party Javascript libraries.
from webhelpers.rails.asset_tag import javascript_path
/reddit/r2/r2/lib/manager/tp_manager.py:22: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
import pylons, sha
Traceback (most recent call last):
File "/usr/local/bin/paster", line 8, in <module>
load_entry_point('PasteScript==1.7.3', 'console_scripts', 'paster')()
File "/usr/local/lib/python2.6/dist-packages/PasteScript-1.7.3-py2.6.egg/paste/script/command.py", line 84, in run
invoke(command, command_name, options, args[1:])
File "/usr/local/lib/python2.6/dist-packages/PasteScript-1.7.3-py2.6.egg/paste/script/command.py", line 123, in invoke
exit_code = runner.run(args)
File "/usr/local/lib/python2.6/dist-packages/PasteScript-1.7.3-py2.6.egg/paste/script/command.py", line 218, in run
result = self.command()
File "/usr/local/lib/python2.6/dist-packages/PasteScript-1.7.3-py2.6.egg/paste/script/serve.py", line 276, in command
relative_to=base, global_conf=vars)
File "/usr/local/lib/python2.6/dist-packages/PasteScript-1.7.3-py2.6.egg/paste/script/serve.py", line 313, in loadapp
**kw)
File "/usr/local/lib/python2.6/dist-packages/PasteDeploy-1.3.4-py2.6.egg/paste/deploy/loadwsgi.py", line 203, in loadapp
return loadobj(APP, uri, name=name, **kw)
File "/usr/local/lib/python2.6/dist-packages/PasteDeploy-1.3.4-py2.6.egg/paste/deploy/loadwsgi.py", line 224, in loadobj
return context.create()
File "/usr/local/lib/python2.6/dist-packages/PasteDeploy-1.3.4-py2.6.egg/paste/deploy/loadwsgi.py", line 617, in create
return self.object_type.invoke(self)
File "/usr/local/lib/python2.6/dist-packages/PasteDeploy-1.3.4-py2.6.egg/paste/deploy/loadwsgi.py", line 109, in invoke
return fix_call(context.object, context.global_conf, **context.local_conf)
File "/usr/local/lib/python2.6/dist-packages/PasteDeploy-1.3.4-py2.6.egg/paste/deploy/util/fixtypeerror.py", line 57, in fix_call
val = callable(*args, **kw)
File "/reddit/r2/r2/config/middleware.py", line 558, in make_app
load_environment(global_conf, app_conf)
File "/reddit/r2/r2/config/environment.py", line 54, in load_environment
config['pylons.g'] = app_globals.Globals(global_conf, app_conf, paths)
File "/reddit/r2/r2/lib/app_globals.py", line 173, in __init__
self.memcache = CMemcache(self.memcaches, num_clients = num_mc_clients)
File "/reddit/r2/r2/lib/cache.py", line 108, in __init__
client.behaviors.update(behaviors)
File "build/bdist.linux-x86_64/egg/pylibmc.py", line 105, in update
File "build/bdist.linux-x86_64/egg/pylibmc.py", line 172, in set_behaviors
_pylibmc.MemcachedError: memcached_behavior_set returned 45
I'm an absolute noob when it comes to running web services, just started learning Linux, and only know T-SQL and ActionScript 2. So, suffice to say that I'm a bit out of my depth here.
I know there are issues with various versions of python-webhelpers, and at least libmemcached, and at this point, I'm pretty stuck. I'm not great at Linux, so I'm never sure what version of a program I've got installed, and neither am I sure which versions are the working versions for what's in the git repository at the moment. What I'd like to do would be to uninstall libmemcached and webhelpers, and reinstall to the correct version. I get the feeling that doing this would require me to re-do much of the process, which is fine, provided it works.
Any help on how to resolve this error would be MUCH appreciated. I've gotten a lot of help previously from answered questions on this site, and I'm hoping someone much smarter than me has the answer to this one!
I encountered this exact same problem, and downgrading to libmemcached-0.48 fixed it.
Make sure you rebuild pylibmc after you do this.