inets httpd: server example in User's Guide not working - erlang

Question

I copied the code straight from the inets User's Guide:
$ erl
Erlang/OTP 19 [erts-8.2] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V8.2 (abort with ^G)
1> inets:start().
ok
2> {ok, Pid} = inets:start(httpd, [{port, 0}, {server_name,"httpd_test"}, {server_root,"/tmp"}, {document_root,"/tmp/htdocs"}, {bind_address, "localhost"}]).
=ERROR REPORT==== 25-Feb-2018::03:08:14 ===
Failed initiating web server:
undefined
{invalid_option,{non_existing,{document_root,"/tmp/htdocs"}}}
** exception error: no match of right hand side value
{error,
{{shutdown,
{failed_to_start_child,
{httpd_manager,{127,0,0,1},60152,default},
{error,
{invalid_option,
{non_existing,{document_root,"/tmp/htdocs"}}}}}},
{child,undefined,
{httpd_instance_sup,{127,0,0,1},60152,default},
{httpd_instance_sup,start_link,
[[{port,60152},
{bind_address,{127,0,0,1}},
{server_name,"httpd_test"},
{server_root,"/tmp"},
{document_root,"/tmp/htdocs"}],
15000,
{<0.73.0>,#Port<0.904>},
[]]},
permanent,infinity,supervisor,
[httpd_instance_sup]}}}
3>
document_root is an invalid option? Okay, I'll check the list of valid options and correct the mistake in the example....hmmmm, there doesn't seem to be one.
Okay, I needed to do this:
$ cd /tmp
$ mkdir htdocs
Now, I'm trying to bind to an ipv6 version of localhost, but I'm having no luck. The httpd docs say:
{bind_address, ip_address() | hostname() | any}
and ip_address() is defined as:
ip_address() = {N1,N2,N3,N4} % IPv4 | {K1,K2,K3,K4,K5,K6,K7,K8} % IPv6
but N and K are not defined. If N is an integer, what is a K? I tried:
{bind_address, {0,0,0,0,0,0,0,1}}
but I got an error:
$ erl
Erlang/OTP 19 [erts-8.2] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V8.2 (abort with ^G)
1> inets:start().
ok
2> inets:start(httpd, [{port, 0}, {server_name, "httpd_test"}, {server_root, "."}, {document_root, "./htdocs"}, {bind_address,{0,0,0,0,0,0,0,1}}]).
{error,{listen,{exit,badarg}}}
Yet, with an ipv4 address everything works as expected:
3> inets:start(httpd, [{port, 0}, {server_name, "httpd_test"}, {server_root, "."}, {document_root, "./htdocs"}, {bind_address,{127,0,0,1}}]).
{ok,<0.74.0>}
4> httpd:info(pid(0,74,0)).
[{mime_types,[{"htm","text/html"},{"html","text/html"}]},
{server_name,"httpd_test"},
{bind_address,{127,0,0,1}},
{server_root,"."},
{port,63069},
{document_root,"./htdocs"}]
5> httpc:request("http://localhost:63069/file1.txt").
{ok,{{"HTTP/1.1",200,"OK"},
[{"date","Mon, 26 Feb 2018 03:02:33 GMT"},
{"etag","nCZT0114"},
{"server","inets/6.3.4"},
{"content-length","14"},
{"content-type","text/plain"},
{"last-modified","Mon, 26 Feb 2018 02:51:52 GMT"}],
"Hello, world!\n"}}
/ets/hosts:
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting. Do not change this entry.
##
127.0.0.1 localhost
255.255.255.255 broadcasthost
::1 localhost
Next, I tried {bind_address, any}:
$ erl
Erlang/OTP 19 [erts-8.2] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V8.2 (abort with ^G)
1> inets:start().
ok
2> {ok, Server} = inets:start(httpd, [{port, 0}, {server_name, "httpd_test"}, {server_root, "."}, {document_root, "./htdocs"}, {bind_address, any}]).
{ok,<0.72.0>}
3> httpd:info(Server).
[{mime_types,[{"htm","text/html"},{"html","text/html"}]},
{server_name,"httpd_test"},
{bind_address,any},
{server_root,"."},
{port,63679},
{document_root,"./htdocs"}]
But, I can't perform a get request with an ipv6 address:
4> httpc:request("http://[::1]:63679/file1.txt").
{error,{failed_connect,[{to_address,{"::1",63679}},
{inet,[inet],nxdomain}]}}
5> httpc:request("http://127.0.0.1:63679/file1.txt").
{ok,{{"HTTP/1.1",200,"OK"},
[{"date","Mon, 26 Feb 2018 03:13:35 GMT"},
{"etag","nCZT0114"},
{"server","inets/6.3.4"},
{"content-length","14"},
{"content-type","text/plain"},
{"last-modified","Mon, 26 Feb 2018 02:51:52 GMT"}],
"Hello, world!\n"}}
Okay, I solved the error I got when I tried to bind the server to an ipv6 address: I needed to specify the option {ipfamily, inet6}:
inets:start(httpd, [{port, 0},
{server_name, "httpd_test"},
{server_root, "."},
{document_root, "./htdocs"},
{ipfamily, inet6},
{bind_address,{0,0,0,0,0,0,0,1}}]).
However, my httpc:request() still fails:
4> httpd:info(Server).
[{mime_types,[{"htm","text/html"},{"html","text/html"}]},
{ipfamily,inet6},
{server_name,"httpd_test"},
{bind_address,{0,0,0,0,0,0,0,1}},
{server_root,"."},
{port,51284},
{document_root,"./htdocs"}]
5> httpc:request("http://[::1]:51284/file1.txt").
{error,{failed_connect,[{to_address,{"::1",52489}},
{inet,[inet],nxdomain}]}
I can use curl to make a get request with an ipv6 address:
~$ curl -v "http://[::1]:52489/file1.txt"
* Trying ::1...
* TCP_NODELAY set
* Connected to ::1 (::1) port 52489 (#0)
> GET /file1.txt HTTP/1.1
> Host: [::1]:52489
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Mon, 26 Feb 2018 05:07:07 GMT
< Server: inets/6.3.4
< Content-Type: text/plain
< Etag: nCZT0114
< Content-Length: 14
< Last-Modified: Mon, 26 Feb 2018 02:51:52 GMT
<
Hello, world!
* Connection #0 to host ::1 left intact
That leads me to believe that httpc:request() is having trouble with the ipv6 address.
Okay, I tried to configure the client for ipv6:
1> inets:start().
ok
2> {ok, Client} = inets:start(httpc, [{profile, client1_config}] ).
{ok,<0.72.0>}
3> Client.
<0.72.0>
7> httpc:set_options([{ipfamily, inet6}], client1_config).
ok
Fingers crossed....
8> httpc:request("http://[::1]:52489/file1.txt", client1_config).
{error,
{failed_connect,
[{to_address,{"::1",52489}},
{inet6,[inet6],nxdomain}]}}
Then I tried (spacing added by me for legibility):
9> httpc:request(
get,
"http://[::1]:52489/file1.txt",
[],
[{ipv6_host_with_brackets, true}],
client1_config
).
** exception error: no function clause matching httpc:request(get,"http://[::1]:52489/file1.txt",[],
[{ipv6_host_with_brackets,true}],
client1_config) (httpc.erl, line 149)
The error makes no sense to me. There is a five arg version of httpc:request(), and I've carefully checked the types of all the args, and my types are correct:
httpc:request(atom, string, list_of_tuples, list_of_tuples, atom)
Okay, the second argument is actually a tuple: {string, []}. Here's where I'm at now with the server:
7> httpd:info(Server).
[{mime_types,[{"htm","text/html"},{"html","text/html"}]},
{ipfamily,inet6},
{server_name,"httpd_test"},
{bind_address,{0,0,0,0,0,0,0,1}},
{server_root,"."},
{port,53686},
{document_root,"./htdocs"}]
And the client:
32> httpc:get_options(all, client1_config).
{ok,[{proxy,{undefined,[]}},
{https_proxy,{undefined,[]}},
{pipeline_timeout,0},
{max_pipeline_length,2},
{max_keep_alive_length,5},
{keep_alive_timeout,120000},
{max_sessions,2},
{cookies,disabled},
{verbose,verbose},
{ipfamily,inet6},
{ip,default},
{port,default},
{socket_opts,[]}]}
But my client still can't connect with an ipv6 address. I don't know if I'm supposed to be using the httpc:request() Option {ipv6_host_with_brackets, true} or not, so I've been trying it both ways:
34> httpc:request(get, {"http://[::1]:52489/file1.txt", []}, [], [{ipv6_host_with_brackets, true}], client1_config).
(<0.124.0>) << {dbg,{ok,[{matched,nonode#nohost,1}]}}
(<0.124.0>) << {#Ref<0.0.3.431>,
{ok,<<16,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0>>}}
(<0.124.0>) << {inet_async,#Port<0.981>,5,{error,econnrefused}}
(<0.124.0>) << {'EXIT',#Port<0.981>,normal}
(<0.124.0>) << {init_error,error_connecting,
{#Ref<0.0.3.426>,
{error,
{failed_connect,
[{to_address,{"::1",52489}},
{inet6,[inet6],econnrefused}]}}}}
{error,{failed_connect,[{to_address,{"::1",52489}},
{inet6,[inet6],econnrefused}]}}
35> httpc:request(get, {"http://[::1]:52489/file1.txt", []}, [], [], client1_config).
(<0.126.0>) << {dbg,{ok,[{matched,nonode#nohost,1}]}}
(<0.126.0>) << {#Ref<0.0.3.447>,
{ok,<<16,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0>>}}
(<0.126.0>) << {inet_async,#Port<0.982>,6,{error,econnrefused}}
(<0.126.0>) << {'EXIT',#Port<0.982>,normal}
(<0.126.0>) << {init_error,error_connecting,
{#Ref<0.0.3.442>,
{error,
{failed_connect,
[{to_address,{"::1",52489}},
{inet6,[inet6],econnrefused}]}}}}
{error,{failed_connect,[{to_address,{"::1",52489}},
{inet6,[inet6],econnrefused}]}}
36> httpc:request(get, {"http://[0:0:0:0:0:0:0:1]:52489/file1.txt", []}, [], [{ipv6_host_with_brackets, true}], client1_config).
(<0.128.0>) << {dbg,{ok,[{matched,nonode#nohost,1}]}}
(<0.128.0>) << {#Ref<0.0.3.463>,
{ok,<<16,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0>>}}
(<0.128.0>) << {inet_async,#Port<0.983>,7,{error,econnrefused}}
(<0.128.0>) << {'EXIT',#Port<0.983>,normal}
(<0.128.0>) << {init_error,error_connecting,
{#Ref<0.0.3.458>,
{error,
{failed_connect,
[{to_address,{"0:0:0:0:0:0:0:1",52489}},
{inet6,[inet6],econnrefused}]}}}}
{error,{failed_connect,[{to_address,{"0:0:0:0:0:0:0:1",
52489}},
{inet6,[inet6],econnrefused}]}}
37> httpc:request(get, {"http://[0:0:0:0:0:0:0:1]:52489/file1.txt", []}, [], [], client1_config).
(<0.130.0>) << {dbg,{ok,[{matched,nonode#nohost,1}]}}
(<0.130.0>) << {#Ref<0.0.3.479>,
{ok,<<16,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0>>}}
(<0.130.0>) << {inet_async,#Port<0.984>,8,{error,econnrefused}}
(<0.130.0>) << {'EXIT',#Port<0.984>,normal}
(<0.130.0>) << {init_error,error_connecting,
{#Ref<0.0.3.474>,
{error,
{failed_connect,
[{to_address,{"0:0:0:0:0:0:0:1",52489}},
{inet6,[inet6],econnrefused}]}}}}
{error,{failed_connect,[{to_address,{"0:0:0:0:0:0:0:1",
52489}},
{inet6,[inet6],econnrefused}]}}

Answer 1

Okay! I got a client to successfully make a request with an ipv6 address. For all the requests at the bottom of my question, I specified the wrong port. Once I got the client port to match the server port, then all those requests succeeded. Here's my setup:
Server config:
7> httpd:info(Server).
[{mime_types,[{"htm","text/html"},{"html","text/html"}]},
{ipfamily,inet6},
{server_name,"httpd_test"},
{bind_address,{0,0,0,0,0,0,0,1}},
{server_root,"."},
{port,53686},
{document_root,"./htdocs"}]
8>
Client config:
52> httpc:get_options(all, client1_config).
{ok,[{proxy,{undefined,[]}},
{https_proxy,{undefined,[]}},
{pipeline_timeout,0},
{max_pipeline_length,2},
{max_keep_alive_length,5},
{keep_alive_timeout,120000},
{max_sessions,2},
{cookies,disabled},
{verbose,false},
{ipfamily,inet6},
{ip,default},
{port,default},
{socket_opts,[]}]}
53>
And here's the shortest syntax for a request:
53> httpc:request("http://[::1]:53686/file1.txt", client1_config).
{ok,{{"HTTP/1.1",200,"OK"},
[{"date","Mon, 26 Feb 2018 10:21:39 GMT"},
{"etag","nCZT0114"},
{"server","inets/6.3.4"},
{"content-length","14"},
{"content-type","text/plain"},
{"last-modified","Mon, 26 Feb 2018 02:51:52 GMT"}],
"Hello, world!\n"}}
So the httpc:request() Option {ipv6_host_with_brackets, true} isn't required.
When I read the httpc docs, the profile part didn't really register with me. But a profile consists of the configuration options for the client, and it's also where cookies are stored, so you need to include the profile with subsequent requests.
There's a default profile, which I guess automatically gets sent with every request when you don't specify your own profile, and I think the default profile will handle cookies for you. (Nope, not by default. I found the following below the definition for httpc:set_options():
CookieMode = enabled | disabled | verify
If cookies are enabled, all
valid cookies are automatically saved in the cookie database of the
client manager. If option verify is used, function store_cookies/2 has
to be called for the cookies to be saved. Default is disabled.
You can also add configuration options to the default profile.
But if you need to use a configuration like ipv6 for some requests and not others, then you can create a named profile and use the named profile when needed in your httpc:request()'s and the default profile for other requests (by not specifying a named profile). See httpc:set_options() for the client configuration options. Somewhat confusingly, httpc:request() has an Options argument, which allows you to specify other options (look in the docs below the definition of httpc:request/5 for a list of those options). Some of the request Options, like sync and stream, seem to be more appropriate for a profile:
A profile keeps track of proxy options, cookies, and other options
that can be applied to more than one request.
http://erlang.org/doc/man/httpc.html
If you create a profile by calling inets:start(httpc, profile_name), the return value is the Pid of the client, which gets spun off in a separate process and handles requests when using that profile. You can kill the client with either of the following:
inets:stop(httpc, name_of_profile)
inets:stop(httpc, ClientPid)