Erlang/Yaws: Cannot start web server within application using .conf file - erlang

I'm kind of at a loss. YAWS works fine starting as a service during boot in LXQt 19.04. But I intended on using a rate limiter; setting it in arg_rewrite_mod. Having one VM run YAWS and another my application, setting code paths appropriately, I believe would yield subpar performance because the rate limiting calls would use OS-based IPC and not Erlang IPC. Hence there should be OS IPC overhead and not EVM overhead, correct?
I basically just wanted everything under one hood to eliminate that. This is one thing were there are probably a few ways to approach the problem (i.e. split up my project and duplicate pieces where needed) but I like the "simplicity" of everything in one place.
I'm getting an error, {badmatch, {error, enoent}}, in the shell after:
code:add_patha("/usr/lib/yaws/ebin").
application:start(yaws).
It is occurring on line 548 in yaws_server:setup_dirs/1 (Github)
setup_dirs(GC) ->
Dir = yaws:id_dir(GC#gconf.id),
Ctl = yaws:ctl_file(GC#gconf.id),
ok = filelib:ensure_dir(Ctl),
case file:list_dir(Dir) of
{ok, LL} ->
lists:foreach(
fun(F) ->
file:delete(filename:join([Dir, F]))
end, LL -- ["CTL"]); %%% <---- LINE 548
{error, RSN} ->
error_logger:format("Failed to list ~p probably "
"due to permission errs: ~p",
[Dir, RSN]),
erlang:error(RSN)
end.
I made a UNIX group appname consisting of myself and the user yaws. I've gone around to various directories I found with sudo find / -group yaws -type d and set group permissions to the same as owner as well as reassigning the group from yaws to appname.... I believe since I did not set the id it is "default". My logs (/var/log/yaws/report.log) I was hoping would indicate the problem. They are empty.
I'm essentially using the default /etc/yaws/yaws.conf file. The server section has been removed and placed in /etc/yaws/conf.avail/ with a symlink in /etc/yaws/conf.d/.
UPDATE: The crash report ----
2020-03-27T08:30:04.131073-05:00 notice: Yaws: Using config file /etc/yaws/yaws.conf
2020-03-27T08:30:04.136142-05:00 error: use_old_ssl in yaws.conf is no longer supported - ignoring
2020-03-27T08:30:04.137441-05:00 notice: Yaws: Using global subconfig file /etc/yaws/conf.d/localhost.conf
2020-03-27T08:30:04.140979-05:00 error:
crasher:
initial call: yaws_server:init/1,
pid: <0.114.0>,
registered_name: [],
error: {{badmatch,{error,enoent}},
[{yaws_server,setup_dirs,1,[{file,"yaws_server.erl"},{line,548}]},
{yaws_server,init2,5,[{file,"yaws_server.erl"},{line,224}]},
{gen_server,init_it,2,[{file,"gen_server.erl"},{line,374}]},
{gen_server,init_it,6,[{file,"gen_server.erl"},{line,342}]},
{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]},
ancestors: [yaws_sup,<0.108.0>],
message_queue_len: 0,
messages: [],
links: [<0.109.0>,#Port<0.6>],
dictionary: [{gc,{gconf,"/usr/lib/yaws",false,612,"/var/log/yaws",
["/usr/local/lib/yaws-appmods/ebin","/usr/lib/yaws/examples/ebin"],
[],[],30000,nolimit,400,1000000,8000,nolimit,[],10240,[],0,30,
["/usr/local/lib/yaws-appmods/include","/usr/lib/yaws/examples/include"],
"/usr/bin/php-cgi","Yaws 2.0.6","default",false,[],8,undefined,
[inet],yaws_session_server,undefined,120000,3600000,disable}},
{start_time,{{2020,3,27},{8,30,4}}}],
trap_exit: true,
status: running,
heap_size: 1598,
stack_size: 27,
reductions: 32410;
neighbours:
2020-03-27T08:30:04.141195-05:00 error:
supervisor: {local,yaws_sup},
errorContext: start_error,
reason: {{badmatch,{error,enoent}},
[{yaws_server,setup_dirs,1,[{file,"yaws_server.erl"},{line,548}]},
{yaws_server,init2,5,[{file,"yaws_server.erl"},{line,224}]},
{gen_server,init_it,2,[{file,"gen_server.erl"},{line,374}]},
{gen_server,init_it,6,[{file,"gen_server.erl"},{line,342}]},
{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]},
offender: [{pid,undefined},
{id,yaws_server},
{mfargs,
{yaws_server,start_link,
[{env,false,false,false,false,false,false,"default",latin1}]}},
{restart_type,permanent},
{shutdown,120000},
{child_type,worker}]
2020-03-27T08:30:04.145621-05:00 error:
crasher:
initial call: application_master:init/4,
pid: <0.107.0>,
registered_name: [],
exit: {{{shutdown,{failed_to_start_child,yaws_server,{{badmatch,{error,enoent}},
[{yaws_server,setup_dirs,1,[{file,"yaws_server.erl"},{line,548}]},
{yaws_server,init2,5,[{file,"yaws_server.erl"},{line,224}]},
{gen_server,init_it,2,[{file,"gen_server.erl"},{line,374}]},
{gen_server,init_it,6,[{file,"gen_server.erl"},{line,342}]},
{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}}},
{yaws_app,start,[normal,[]]}},
[{application_master,init,4,[{file,"application_master.erl"},{line,138}]},
{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]},
ancestors: [<0.106.0>],
message_queue_len: 1,
messages: [{'EXIT',<0.108.0>,normal}],
links: [<0.106.0>,<0.43.0>],
dictionary: [],
trap_exit: true,
status: running,
heap_size: 987,
stack_size: 27,
reductions: 225;
neighbours:
2020-03-27T08:30:04.147171-05:00 notice:
application: yaws,
exited: {{shutdown,{failed_to_start_child,yaws_server,{{badmatch,{error,enoent}},
[{yaws_server,setup_dirs,1,[{file,"yaws_server.erl"},{line,548}]},
{yaws_server,init2,5,[{file,"yaws_server.erl"},{line,224}]},
{gen_server,init_it,2,[{file,"gen_server.erl"},{line,374}]},
{gen_server,init_it,6,[{file,"gen_server.erl"},{line,342}]},
{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},
{line,249}]}]}}},{yaws_app,start,[normal,[]]}},
type: temporary
SOLUTION:
From the help from the answer below and from having explored the various YAWS paths using find above I found the "default" home directory for me was `/var/cache/yaws/'. The following series of commands in the shell worked for me:
os:putenv("YAWSHOME","/var/cache/yaws/").
code:add_patha("/usr/lib/yaws/ebin").
application:start(yaws).

The source code line 548 shown in your question can't return {error, enoent}, but line 548 of yaws_server.erl for version 2.0.6 is
ok = filelib:ensure_dir(Ctl),
and this is what's returning the error tuple, which fails to match the expected atom ok and causes the failure.
The function filelib:ensure_dir/1 verifies that the parent directories of its argument exist, attempting to create them if they don't. Solving this problem therefore requires determining the Ctl argument pathname in the code shown above.
Since your server id is "default", Ctl in this case is defined as the path
<HOME>/.yaws/yaws/default/CTL
where <HOME> comes either from the setting for the YAWSHOME environment variable if it exists, otherwise from the setting for the HOME environment variable. Make sure one of these environment variables is set to a suitable path and that the .yaws/yaws/default subdirectory under it, if already present, has the appropriate permissions for your yaws user id and appname group id.

Related

Erlang: starting gen_server on another node fails after init

I am stuck in a bit of a fix trying to run gen_server on another node. So I have a common gen_server class which looks like this
start(FileName) ->
start_link(node(), FileName).
start_link(ThisNode, FileName) ->
gen_server:start_link({local, ?MODULE}, ?MODULE, [ThisNode, FileName], []).
init([ThisNode, FileName]) ->
process_flag(trap_exit, true),
{ok, Terms} = file:consult(FileName),
{A1, B1, C1} = lists:nth(1,Terms),
place_objects(A1, B1, C1).
Now I want to start multiple nodes that will run the same gen_server and somehow communicate with each other, and use a another node to orchestrate that. (All these nodes are started on my local terminal).
So I start a new node in one terminal using erl -sname bar where I intend to run the gen_server, and compile the gen_server module on this node. Then I start another node called 'sup' which I intend to use as a coordinator for all the other nodes. If I run the command my_gen_server:start("config_bar.txt"). on bar, it successfully returns but when I run the command rpc:call('bar#My-MacBook-Pro', my_gen_server, start, ["config_bar.txt"]). on sup, it successfully returns from the init method (I checked this by putting in the logs) but immediately after that, I get this error:
{ok,<9098.166.0>}
(sup#My-MacBook-Pro)2> =ERROR REPORT==== 21-Feb-2022::11:12:30.443051 ===
** Generic server my_gen_server terminating
** Last message in was {'EXIT',<9098.165.0>,
{#Ref<0.3564861827.2990800899.137513>,return,
{ok,<9098.166.0>}}}
** When Server state == {10,10,#Ref<9098.1313723616.3973185546.82660>,
'bar#My-MacBook-Pro'}
** Reason for termination ==
** {#Ref<0.3564861827.2990800899.137513>,return,{ok,<9098.166.0>}}
=CRASH REPORT==== 21-Feb-2022::11:12:30.443074 ===
crasher:
initial call: my_gen_server:init/1
pid: <9098.166.0>
registered_name: my_gen_server
exception exit: {#Ref<0.3564861827.2990800899.137513>,return,
{ok,<9098.166.0>}}
in function gen_server:decode_msg/9 (gen_server.erl, line 481)
ancestors: [<9098.165.0>]
message_queue_len: 0
messages: []
links: []
dictionary: []
trap_exit: true
status: running
heap_size: 1598
stack_size: 29
reductions: 3483
neighbours:
I can't seem to figure out what causes the error and if there's anything I need to add to my gen_server code to fix it. Would really appreciate some help on this one!
The gen_server in the remote node is linked to an ephemeral process created for the rpc call.
As this ephemeral process exits with a term that's different from normal (the actual result of the rpc call), the exit signal propagates to the gen_server, killing it.
You can use gen_server:start instead of gen_server:start_link or, if you want the gen_server to be part of the supervission tree, instruct its supervisor to spawn it:
rpc:call('bar#My-MacBook-Pro', my_gen_sup, start_child, ["config_bar.txt"]).

How to debug headless pdf printing problems in chrome?

Note: this is not (directly) a question about how to print PDF in chrome, instead this is a question about how to get more information when printing fails.
In short: I cannot solve a printing PDF problem, which occurs only for certain (presumably large) pages and could use some assistance in debugging the actual issue.
Background: I am using the chromedriver (v83) and chromium-browser (v83) to print PDF files from webpages by utilizing python selenium. I am building a docker image to contain the required dependencies for this. I have tried to use Debian (buster and stretch) as well as Alpine base images, but all eventually result in the same error, when trying to print some pages. The odd thing is that for other (smaller) pages the printing works, but when many assets and pages are to be printed, the printing fails. I might add that this docker images is eventually being run inside of a Kubernetes cluster, where I assigned up to 4GB of RAM.
What code am I running?
This project was written for python3, so here are some relevant code fragments. Please note that I removed all error handling and waiting for the page loads to complete here.
from selenium import webdriver
appState = {
"recentDestinations": [
{
"id": "Save as PDF",
"origin": "local"
}
],
"selectedDestinationId": "Save as PDF",
"version": 2
}
def get_chrome_options(headless: bool, enable_logging: bool) -> Options:
chrome_options = webdriver.ChromeOptions()
profile = {'printing.print_preview_sticky_settings.appState': json.dumps(appState)}
chrome_options.add_experimental_option('prefs', profile)
if headless:
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--window-size=1920,1080')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--disable-web-security')
chrome_options.add_argument('-–allow-file-access-from-files')
chrome_options.add_argument('--run-all-compositor-stages-before-draw')
chrome_options.add_argument('--kiosk-printing')
if enable_logging:
chrome_options.add_argument('--enable-logging')
return chrome_options
def print_the_page(url):
driver = webdriver.Chrome(chrome_options=get_chrome_options(headless, enable_logging))
driver.execute(driver_command=Command.GET, params={'url': url})
command_url = f"{driver.command_executor._url}/session/{driver.session_id}/chromium/send_command_and_get_result"
response = driver.command_executor._request('POST', command_url, json.dumps({'cmd': 'Page.printToPDF', 'params': {}}))
Then what happens?
For some pages this fails - meaning - there is this message in the response:
{'status': 500, 'value': '{"value":{"error":"unknown error","message":"unknown error: unhandled inspector error: {\\"code\\":-32000,\\"message\\":\\"Printing failed\\"}\\n (Session info: headless chrome=83.0.4103.116)","stacktrace":""}}'}
[UPDATE]
I have managed to produce some more error output when using the --print-to-pdf option directly, which seems to hint at an "out-of-memory" issue here:
[0923/135406.102857:WARNING:discardable_shared_memory_manager.cc(194)] Less than 64MB of free space in temporary directory for shared memory files: 23
[0923/135406.110108:WARNING:dns_config_service_posix.cc(341)] Failed to read DnsConfig.
[0923/135406.180892:WARNING:dns_config_service_posix.cc(341)] Failed to read DnsConfig.
[0923/135406.613221:FATAL:memory.cc(38)] Out of memory. size=796176
Received signal 6
r8: 00007fa6f39dadc4 r9: 0000000000000000 r10: 0000000000000008 r11: 0000000000000246
r12: 0000557efd1b0660 r13: 0000000000000000 r14: 00007fa6f39db240 r15: 0000000000000043
di: 0000000000000002 si: 00007fa6f39dac90 bp: 00007fa6f39dac90 bx: 0000000000000000
dx: 0000000000000000 ax: 0000000000000000 cx: 00007fa6fd347a71 sp: 00007fa6f39dac88
ip: 00007fa6fd347a71 efl: 0000000000000246 cgf: 002b000000000033 erf: 0000000000000000
trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.
[0923/135406.626313:ERROR:headless_shell.cc(399)] Abnormal renderer termination.
I will note here that I have been running this docker container locally on my machine (which has more than enough RAM) as well as on a Kubernetes cluster where this image has requested 4GB RAM. I also monitored the RAM usage and it didn't seem to be an issue - although - that could be illusive if the RAM usage is so radically high that chrome just fails and you never really see that in the overall RAM usage.
[UPDATE 2]
I have tried to use the --print-to-pdf option again, but I am seeing issues with that as well. The resources are loading, but the printing still fails.
│ [0923/144355.169080:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
...
│ [0923/141758.393923:WARNING:dns_config_service_posix.cc(341)] Failed to read DnsConfig. │
│ [0923/141758.401925:ERROR:zygote_host_impl_linux.cc(262)] Failed to adjust OOM score of renderer with pid 32: Permission denied (13) │
│ [0923/141758.413475:ERROR:zygote_host_impl_linux.cc(262)] Failed to adjust OOM score of renderer with pid 36: Permission denied (13)
... loading all the resources ...
│ [0923/141824.611661:ERROR:print_render_frame_helper.cc(1889)] Printing failed. │
│ [0923/141824.612439:ERROR:headless_shell.cc(562)] Print to PDF failed
What's the question(s)?
How can I get more information about why the "Printing failed" - unfortunately the "unknown error: unhandled inspector error" hasn't given me any ideas about how to proceed.
Are there potentially any additional flags to get more debug output from chrome or is there a log somewhere that I should be able to find?
What else have I tried?
I have initially been running this under Debian buster with the latest google-chrome and chromium binaries (v85). I have switched to an Alpine base image and chromium - hoping that this might change something, but it didn't.
I have experimented with setting up Xvfb ${DISPLAY} -screen ${SCREEN} ${RESOLUTION} & in Docker, but it didn't seem to have any effect either.
I have tried to switch to using the direct cli google-chrome --print-to-pdf= option, but since it's a page that requires to pass through login authentication, I could only get the login page to print and it also seems to have some not so nice formatting issues.
I have been running this on my machine, outside of Docker, and was able to print as expected, but as soon as I put the same code inside a Docker container, it fails.
Unfortunately, I cannot share the page where this fails with you.
The relevant warning from your logs seems to be this:
[0923/135406.102857:WARNING:discardable_shared_memory_manager.cc(194)] Less than 64MB of free space in temporary directory for shared memory files: 23
The problem appears to stem from Docker's mounted /dev/shm being too small for Chromium to do things like you're trying to do.
I found a closed bug report against chromium referencing this issue in certain limited environments such as AWS Lambda and Docker, it was fixed in chromium v65 behind a command line flag --disable-dev-shm-usage.
The last few comments reference another bug report (now closed) about this issue in chromium v83 where the command line flag was not working properly. It has been fixed in version 84 - per comment 28:
You can find the fix in current stable release of Chrome (version 84.0.4147.89 and above).
You've indicated you're using chromium v83, so you'll need to update at least version 84.0.4147.89, then use the command line flag --disable-dev-shm-usage.

Jenkins is spawning a lot of daemon processes and server crashes

I've recently installed Jenkins on a cheap VM on Azure. The specs are very low, since I use this server for testing the setup: 1vCPU & 1GB RAM. There will usually only be 1 build at the same time, with a max. of 3, in very rare occassions.
During the build process from Jenkins quite frequently my server would crash completely and stay so for +- 10 - 15 minutes until being able to be used again.
I checked the processes on the server and this is the result:
The full line is like this:
/etc/alternatives/java -Dcom.sun.akuma.Daemon=daemonized -Djava.awt.headless=true -DJENKINS_HOME=/var/lib/jenkins -jar /usr/lib/jenkins/jenkins.war --logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war --daemon --httpPort=8080 --debug=5 --handlerCountMax=100 --handlerCountMaxIdle=20
It is the same for every single one of those daemons, not a single parameter is different.
Is this normal behavior, and is this the reason why my server is crashing? Or are my specs just too low for Jenkins to run on to?
Thanks in advance!
EDIT:
My jenkins.log file looks pretty normal except for one NullPointerException that keeps coming back up:
2020-01-08 12:43:17.702+0000 [id=148] WARNING h.ExpressionFactory2$JexlExpression#evaluate: Caught exception evaluating: h.filterDescriptors(it,attrs.descriptors) in /configure. Reason: java.lang.NullPointerException: Descriptor list is null for context 'class hudson.model.Hudson' in thread 'Handling GET /configure from 85.154.65.124 : qtp2085857771-148 Jenkins/configure.jelly GlobalLibraries/config.jelly LibraryConfiguration/config.jelly SCMRetriever/DescriptorImpl/config.jelly MultiSCM/DescriptorImpl/config.jelly'
java.lang.NullPointerException: Descriptor list is null for context 'class hudson.model.Hudson' in thread 'Handling GET /configure from 85.154.65.124 : qtp2085857771-148 Jenkins/configure.jelly GlobalLibraries/config.jelly LibraryConfiguration/config.jelly SCMRetriever/DescriptorImpl/config.jelly MultiSCM/DescriptorImpl/config.jelly'
at hudson.model.DescriptorVisibilityFilter.apply(DescriptorVisibilityFilter.java:73)
...

Can't start supervisor from yaws runmod

I have a yaws runmod defined in yaws.conf as:
runmod = sg_app
the module contains an exported function:
start()->
io:format("~p start~n", [ sg_sup:start_link() ]).
When I start yaws I see a call to the runmod:
=INFO REPORT==== 29-Oct-2015::16:46:51 === sync call sg_app:start
{ok,<0.61.0>} start
But the supervisor is nonexistent:
1> whereis(sg_sup).
undefined
If I call the runmod:start manually, the supervisor hangs around.
2> sg_app:start().
{ok,<0.73.0>} start
ok
3> whereis(sg_sup).
<0.73.0>
What have I done wrong?
Your runmod's start/0 function is starting the supervisor with start_link/0, which means it's getting linked to the parent process. When that process dies, it takes your runmod process down with it, due to the link. The runmod feature isn't designed for starting a supervision tree.
You might instead consider using a yapp, which allows your code to run as a regular Erlang application in the same Erlang node as Yaws and be registered to have Yaws dispatch requests into it.
Another option is to launch your application using a separately spawned, infinite process:
start()->
spawn(fun () ->
application:start(my_app, permanent),
receive after infinity -> ok end
end).

How to always log/show the error reason when a supervisor child returns error from start_link?

When starting gen_server's from a supervisor (which itself is started by a application) I have the problem that when the start_link of the gen_server doesn't return {ok, ...} but {error, Reason} the only error message I see is:
=INFO REPORT==== 20-Jan-2011::13:14:43 ===
application: foo
exited: {shutdown,{foo_app,start,[normal,[]]}}
type: temporary
The Reason that for terminating is not shown/logged.
Is there a way to see/log these error returns to the supervisor?
The childspec I'm using is e.g.:
{ok, {{one_for_one, 3, 10}, ...
{usb_mux_1,
{usb_mux, start_link,
[Some_Params]},
permanent,
10000,
worker,
[usb_mux]}, ...
Edit: Clarification
I know about error_logger and using it already. The question is not how to get something logged but how to get supervisor to log the reason for it terminating, e.g. log who terminated with an error return and what did it return.
And just to get this also out of the way, yes I start erlang with sasl on:
-boot start_sasl
Just discovered the answer myself:
The supervisor is really logging the error exit as crash report.
The problem is the shell doesn't show these crash reports. Just to confuse me it shows info/warning and error reports but no progess reports and crash reports from the supervisor.
If I look in the on disk log there is a detailed crash report there:
10> rb:show(4).
CRASH REPORT <0.53.0> 2011-01-20 17:33:52
===============================================================================
Crashing process
initial_call {usb_mux,init,['Argument__1']}
pid <0.53.0>
registered_name []
error_info
{exit,{undef,[{usb_port,get_gw_hw_spec,[<0.59.0>]},
...
The reason the SASL events were not shown on the screen was a ommission in the -config file, which looked like this:
[{sasl, [
{sasl_error_logger, false}, %% no SASL error logger installed
{error_logger_mf_dir,"./log"},
{error_logger_mf_maxbytes,10485760}, % 10 MB
{error_logger_mf_maxfiles, 10}
]}].
Meaning there was a multi-file erroro logger installen (all the error_logger_mf_* entries) but no on screen logger for SASL events.
Changing the entry like this fixed it:
{sasl_error_logger, tty}, %% SASL reports to tty
From the sasl manpage:
sasl_report_tty_h:
Formats and writes supervisor reports, crash reports and progress reports to stdio .

Resources