Rails 5.2, Capybara launched Puma server does not set session cookie - ruby-on-rails

Question

My rails app makes use of the session, or in other words, makes use of the session id that is set in the cookie header automatically by controller responses.
This works fine except for when the app is launched via Capybara, and then for some reason the response headers never include a Set-Cookie.
It doesn't matter if the request is coming from the selenium launched browser or not. I can curl to the Capybara launched Puma server and also will never get the cookie set.
I took a look at the middleware launched by the Capybara Puma server, and it seemed to include sessions and cookies components like normal. I can't find anything about this online though, and it seems like it would be a big problem to never have sessions for doing system tests in Rails 5.2. So I suppose I'm doing something wrong somehow, but have no idea what it could be.
Here is an example normal request:
Host: localhost:3000
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1.2 Safari/605.1.15
Accept-Language: en-us
DNT: 1
Accept-Encoding: gzip, deflate
HTTP/1.1 200 OK
X-Frame-Options: ALLOWALL
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
Content-Type: text/html; charset=utf-8
ETag: W/"bb9f6e144809fdf8359cec8ca0cbfc09"
Cache-Control: max-age=0, private, must-revalidate
Set-Cookie: _rails_app_session=......................; path=/; HttpOnly
X-Meta-Request-Version: 0.6.0
X-Request-Id: a2f2591b-7523-4c32-98a0-1d1fdc0fd231
X-Runtime: 0.867249
Vary: Origin
Transfer-Encoding: chunked
Note the Set-Cookie: _rails_app_session=......................; is present.
and here is an example when hitting capybara's server:
GET /calculator HTTP/1.1
Host: localhost:3030
Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
HTTP/1.1 200 OK
X-Frame-Options: ALLOWALL
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
Content-Type: text/html; charset=utf-8
ETag: W/"040559a7884c3251acd2082604a43cf3"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 2bc191c6-66e9-4fe6-a8d2-7980f6cc66ed
X-Runtime: 0.146430
Vary: Origin
Transfer-Encoding: chunked