Elixir: how can I leverage release_handler? - erlang

Question

I know there are projects such as exrm which do that for you, and it seem to do it extremely well. However,　I would like to manually play with release_handler for educational purposes, and I find little documentation or articles on the subject.

Answer 1

Have you read the Learn You Some Erlang article about release handling?
Doing upgrades with release handler is both easy and difficult. Easy because once you know all the nitty gritty details it's pretty much automatic. But difficult, because getting all the details right in the first place is quite a task.
I spent quite a lot of time trying to understand how to get the process right and the effect is builderl, a build system, something like rebar, but doing everything from start to finish the OTP way.
It's probably not of much use if you are using Elixir, and I don't know much about how the process differs for Elixir. But I can give you a few points that were relevant for Erlang, which you can try to adapt to your case (in the context of builderl, as that will allow me to share some code examples).
The bird's view
In general the process is as follows:
Create the initial release. This is a compiled version of your project processed with systools and reltool that contains all the folders and files as release handler expects them to be. This is what builderl creates in tmp/rel for gmake rel. It's a self-contained folder also called an embedded Erlang installation. More about it later.
Copy the release to the production system, install and start the node.
In your development environment make some updates to one or more of the applications running in the release.
Create a new version of each application that has changed. Create appropriate appup files for those applications. Increase the release version.
Create the relup file with systools. The easiest way is to have two release versions unpacked alongside each other and the newer version installed and running. Then in the Erlang shell of the new version call make_relup referencing the old version of applications from the old version of the release (see snippets at the bottom of the answer).
Create the upgrade archive. This is simply a tarball of all applications that have been upgraded, the releases folder, and the release definition file (again snippets at the bottom).
Copy the upgrade archive to the production system and unpack with release_handler:unpack_release/1. This will unpack the new versions of your applications alongside the old versions in the lib folder of the release folder.
Then finally install the new version with release_handler:install_release/1.
A release
In order to use release handler the code must be structured in a specific way. The Releases document specifies how a release folder should look like. The humbundee application is an example and here are its folders once builderl compiles it and creates the release using systools (in tmp/rel):
> tmp/rel/
bin/
erts-7.2.1/
etc/
lib/
releases/
The folders are as follows:
lib
Main folder with applications. Each application may be either without the version suffix (-Vsn) or all versions of the same applications have to have the suffix. You can't mix application folders where some of them contain the version suffix and some don't.
But when doing upgrades with release handler all folders must have a proper -Vsn suffix, otherwise upgrades don't make sense - upgrades are between versions of applications and Erlang distinguishes the versions by the suffix in the application's folder name.
> l tmp/rel/lib/
builderl-0.2.7/
compiler-6.0.2/
deploy-0.0.1/
goldrush-0.1.7/
humbundee-0.0.1/
kernel-4.1.1/
lager-3.0.1/
mnesia-4.13.2/
sasl-2.6.1/
stdlib-2.7/
syntax_tools-1.7/
yajler-0.0.1/
yolf-0.1.1/
erts-Vsn
Usually this is a copy of the erts folder from where Erlang is installed (/usr/local/lib/erlang/erts-7.2.1 in my case).
releases
Contains files that define the release. Each file is created slightly differently.
> l tmp/rel/releases/
RELEASES
hbd-0.0.1/
start_erl.data
humbundee.rel
> cat release/RELEASES
[{release,"humbundee","hbd-0.0.1","7.2.1",
[{kernel,"4.1.1","/usr/home/g/work/humbundee/lib/kernel-4.1.1"},
{stdlib,"2.7","/usr/home/g/work/humbundee/lib/stdlib-2.7"},
{sasl,"2.6.1","/usr/home/g/work/humbundee/lib/sasl-2.6.1"},
{yolf,"0.1.1","/usr/home/g/work/humbundee/lib/yolf-0.1.1"},
{yajler,"0.0.1","/usr/home/g/work/humbundee/lib/yajler-0.0.1"},
{mnesia,"4.13.2","/usr/home/g/work/humbundee/lib/mnesia-4.13.2"},
{syntax_tools,"1.7",
"/usr/home/g/work/humbundee/lib/syntax_tools-1.7"},
{compiler,"6.0.2","/usr/home/g/work/humbundee/lib/compiler-6.0.2"},
{goldrush,"0.1.7","/usr/home/g/work/humbundee/lib/goldrush-0.1.7"},
{lager,"3.0.1","/usr/home/g/work/humbundee/lib/lager-3.0.1"},
{humbundee,"0.0.1",
"/usr/home/g/work/humbundee/lib/humbundee-0.0.1"}],
permanent}].
> cat releases/start_erl.data
7.2.1 hbd-0.0.1
> cat tmp/rel/releases/humbundee.rel
%% rel generated at {2016,3,24} {11,9,39}
{release,{"humbundee","hbd-0.0.1"},
{erts,"7.2.1"},
[{kernel,"4.1.1"},
{stdlib,"2.7"},
{sasl,"2.6.1"},
{yolf,"0.1.1"},
{yajler,"0.0.1"},
{mnesia,"4.13.2"},
{syntax_tools,"1.7"},
{compiler,"6.0.2"},
{goldrush,"0.1.7"},
{lager,"3.0.1"},
{humbundee,"0.0.1"}]}.
> l tmp/rel/releases/hbd-0.0.1/
builderl.config
cmd.boot
cmd.data
cmd.rel
cmd.script
hbd.config
humbundee.boot
humbundee.data
humbundee.rel
humbundee.script
start.boot -> humbundee.boot
sys.config -> hbd.config
sys.config.src -> ../../etc/sys.config.src
Those files are created automatically by builderl, which in turn generates them with the help of Release Handler:
RELEASES
Contains the definitions of current and past releases currently available on the system. Can be created with release_handler:create_RELEASES/4. builderl creates it by starting a node with this command (which runs the command and shuts down the node afterwards):
run_erl -daemon ../hbd/shell/ ../hbd/log "exec erl ../hbd releases releases/start_erl.data -config releases/$APP_VSN/hbd.config -args_file ../hbd/etc/vm.args -boot releases/$APP_VSN/humbundee -noshell -noinput -eval \"{ok, Cwd} = file:get_cwd(), release_handler:create_RELEASES(Cwd, \\\"releases\\\", \\\"releases/$APP_VSN/humbundee.rel\\\", []), init:stop()\""
start_erl.data
Can be created manually.
humbundee.rel
Contains a definition of a release - which applications should be included in the release in which versions. It can be created manually but it's difficult to keep track of the application versions. So, very often it's generated from a more generic file. In case of builderl it generates the .rel file from a reltool.config file based on these reltool examples.
bin
Contains links to Erlang executables, script and boot files for releases. In case of builderl they are simply copied from the Erlang installation.
> l tmp/rel/bin/
builderl -> ../lib/builderl-0.2.7
config.esh -> builderl.esh
configure.esh -> builderl.esh
deps.esh -> builderl.esh
init.esh -> builderl.esh
migresia.esh -> builderl.esh
mk_dev.esh -> builderl.esh
mk_rel.esh -> builderl.esh
start.esh -> builderl.esh
stop.esh -> builderl.esh
update_root_dir.esh -> builderl.esh
builderl.esh
ct_run
epmd
erl
escript
run_erl
to_erl
start
cmd.boot
humbundee.boot
start_clean.boot
start_sasl.boot
start.boot
start.script
The .esh files belong to builderl so you can skip them. The executables are copied from /usr/local/lib/erlang/bin/ (which is system dependent - I am using FreeBSD). The start* boot and script files are also copied from /usr/local/lib/erlang/bin/ and cmd.boot and humbundee.boot are simply copied from tmp/rel/releases/hbd-0.0.1/.
etc
Used by builderl to store configuration files, not required by OTP release handler.
Snippets
Finally some code snippets to help you work with the release tools:
Create a relup file that upgrades release version 1.1.1 to 9.9.9 (run in the version having the old version of the release alongside, see The bird's view at the top of the answer):
systools:make_relup("releases/rel-9.9.9/my_release", ["../1.1.1/releases/rel-1.1.1/my_release"], ["../1.1.1/releases/rel-1.1.1/my_release"], [{path, ["../1.1.1/lib/lager-2.6/ebin", "../1.1.1/lib/backup_app-1.1.2/ebin", "../1.1.1/lib/goldrush-2.5/ebin"]}, {outdir, "releases/rel-9.9.9"}]).
Create a relup file that upgrades two release versions, 1.1.1 and 2.2.2 to 9.9.9
systools:make_relup("releases/rel-9.9.9/my_release", ["../1.1.1/releases/rel-1.1.1/my_release", "../2.2.2/releases/rel-2.2.2/my_release"], ["../1.1.1/releases/rel-1.1.1/my_release", "../2.2.2/releases/rel-2.2.2/my_release"], [{path, ["../1.1.1/lib/lager-2.6/ebin", "../2.2.2/lib/lager-2.4/ebin", "../1.1.1/lib/backup_app-1.1.2/ebin", "../2.2.2/lib/other_app-1.1/ebin"]}, {outdir, "releases/rel-9.9.9"}]).
Manually create the release upgrade package
tar -czf update_to_rel-9.9.9.tar.gz releases/my_release.rel releases/rel-9.9.9 lib/lager-2.6.1 lib/builderl-0.2.6 lib/deploy-0.4.2 lib/backup_app-2.0.0 lib/other_app-2.5.3
Unpack the release upgrade package (after it has been copied to the releases folder in the release on the production node):
release_handler:unpack_release("rel-9.9.9").
Verify the release has been unpacked - the new version should be marked as unpacked:
release_handler:which_releases().
Install the new release:
release_handler:install_release("rel-9.9.9").
Make it permanent once it's working as expected:
release_handler:make_permanent("rel-9.9.9").
Rollback the release:
release_handler:install_release("rel-1.1.1").
release_handler:remove_release("rel-1.1.1").