I am running Airflow locally inside the Docker containers.
It works well and displays log files when I use standard operators, such as PythonOperator, BigQueryExecuteQueryOperator. The logs are created and stored correctly, as well as visible from Airflow UI.
However, whenever I try to use some of the (perfectly working in Cloud Composer) custom operators, no matter what services they connect to, how they work etc., all tasks display the same issue:
*** Log file does not exist: /home/airflow/airflow/logs/linkedin_avaus_daily_load/get_campaign/2022-09-19T09:57:50.869266+00:00/2.log
*** Fetching from: http://:8793/log/linkedin_avaus_daily_load/get_campaign/2022-09-19T09:57:50.869266+00:00/2.log
*** Failed to fetch log file from worker. Request URL missing either an 'http://' or 'https://' protocol.
It is right in saying that "Log file does not exist", as manually checking in the container confirms that. I presume that the task fails somewhere else and Airflow just doesn't manage to store logs. Has anyone else had this issue?
Thank you a lot for all the help.
Related
So I work with RubyMine, and I configured my docker-compose integration like in this tutorial, but I have an error when I simply hit the 'run' button:
ERROR: Duplicate mount points: [/home/kyrela/railsproject:/railsproject:rw, /home/kyrela/railsproject:/railsproject:rw]
A simple docker-compose up from the terminal works.
I founded when trying to launch the same command as RubyMine, but removing some arguments, that it was caused by the docker-compose.override.[number].yml file generated automatically by RubyMine, based on my configuration. Without it, it works perfectly.
But my configuration is extremely basic :
I only set the IP adress and port, the same as the ones it currently use with a simple docker-compose up from the terminal, I set the remote SDK (the one from my container), and the docker-compose method to up. That's all, I leaved the rest blank or with the default value.
After some research on this error, it's apparently a bug that can be fixed with a simple docker-compose restart. It didn't worked for me.
Does someone know how to get rid of this error ?
If some informations are missing, just leave a comment and I will edit my message with the specified informations.
I have just installed on my Ubuntu 16.04 the OpenRouteService into the docker as described at https://github.com/GIScience/openrouteservice/wiki/Running-with-Docker
It works, but when I try to get route between two coordinates, I am get an "unable to get an appropriate route profile for RoutePreference=driving-car" message. I have checked the config file of ors and there is a correct "driving-car" profile in.
Could you suggest something else to check or help?
I don't know what was the problem at the first runs, but the cache files cause the error. When I delete the content of "elevation cache", the app runs correctly in docker.
I performed a manual clean by deleting job folders directly into the filesystem, and now I find a stucked running job that I cannot abort.
I've tried the answers here to force it to be stopped, but it doesn't work as it is not able to find the existing job in the system.
Additionally, when I click over the running job I get a 404 error:
"Problem accessing <route_to_job_that_doesnt_exist_anymore>"
Reason: Not found
Is there something I can do to abort this running job without restarting the server?
A way to stop a build (Like actually aborting it) is by adding a /stop at the end of the job url, behind the Build Number.
Example: http://myjenkins/project/123/stop
If this doesn't work, there is also the "Hard Kill". Instead of adding /stop you add /kill. I guess you need Admin Access for that POST action.
Don't know though if it works for jobs that don't exist on the Jenkins Host anymore due to missing filesystems
I have restarted Jenkins using the following:
service jenkins stop
service jenkins start
Followed to that I can see some jobs are missing from the GUI.
I have also tried to go the job URL using http://<jenkins_url>/job/<JOBNAME>/
Unfortunately it is also giving:
HTTP ERROR 404
Problem accessing /job/<JOBNAME>/. Reason:
Not Found
Powered by Jetty://
Also performed Doing a Reload Configuration from Disk with no luck.
I checked the config.xml file and I can see it is corrupted. The size of config.xml file is around 110 MB. Why this file got corrupted? How to trace it.
Can anyone give me any pointer how to troubleshoot this problem?
I had the same symptoms, but I'm using a homebrew installation of Jenkins.
The Jenkins machine was shut down improperly, likely from a power outage, so when it came back up it was basically a clean instance. No jobs and no system configurations.
The following solution isn't for your exact use case, but it does solve the problem for some users who return to Jenkins to find it without any jobs.
The solution basically involves you checking to see if you have started the Jenkins service incorrectly or from the wrong place.
...
On to the specific homebrew issue:
For whatever reason, the homebrew.mxcl.jenkins.plist file was found in ~/Library/LaunchDaemons/
It belongs in ~/Library/LaunchAgents/ only.
If this happens, it can be solved as follows
Stop the service:
sudo launchctl unload ~/Library/LaunchDaemons/homebrew.mxcl.jenkins.plist
Reload the correct file, located in ~/Library/LaunchAgents/ by trying the following line in case it's running:
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.jenkins.plist
Note: the above line may yell at you if it's not running, which is ok.
Start it up again:
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.jenkins.plist
If all looks good when Jenkins loads again, you can and should
delete homebrew.mxcl.jenkins.plist in ~/Library/LaunchDaemons/:
sudo rm ~/Library/LaunchDaemons/homebrew.mxcl.jenkins.plist
i am getting this
Windows could not start the SphinxSearch service on Local Computer.
Error 1067: The process terminated unexpectedly.
i got installation instruction from this .
http://blog.robbsnet.com/2011/07/how-to-install-and-implement-sphinx.html
build process is complete but when i start the sphinx search service i got errors .
Try running searchd manually from Command Prompt. Maybe it will give you a useful error message.
Try also looking in searchd.log
For anyone who is still having problems with the Windows service:
Make sure your configuration is correct for both database and paths and the needed folders & files are created.
Make sure that the service "command" matches the correct paths and files, to do so you have to:
Run administrative tools, click on local services, then find SphinxSearch. Click on its properties and read the line the service is trying to execute. If the configuration path doesn't match the service start command, it will fail to start.
FYI, this can also happen because your search data files are corrupt. Easy fix for that is to nuke the files and refill them.