I have two Erlang applications: single app mylogger and umbrella app myapp.
I want to include a current version of mylogger as a dependency to myapp.
Rebar3 only suggests to add deps from git like
{mylogger, {git, "https://github.com/someuser/mylogger", {branch, "master"}}}
or from hex.pm.
How to add my custom app from the file system with rebar3 structure?
Use checkout dependencies. If you add your application in the _checkout folder it will do the following.
Any application/plugin in _checkouts will take precedence over the
same application if it is additionally listed in the rebar.config's
deps, plugins or project_plugins
In the past there was an issue where the directory in _checkout was used for the artifacts too, if I read the last release notes correctly, this was corrected already.
Related
Let's say, I have a completely new VPS server which I've just rolled out, which I haven't installed anything on yet.
And I've compiled and build a production release of Phoenix application on my local machine which is identical to a VPS server Linux distributive- and version-wise.
In the directory _build/prod/rel/my_app123 there have been generated 4 subdirectories:
bin
erts-12.3
lib
releases
Will copying the content of rel/my_app123/, that is, these 4 subdirectories, over to a VPS will be absolutely enough in order to run an application?
Or will I have install something extra as well? Elixir and Erlang?
How about production dependencies from mix.exs? Or are these have been included and compiled into into a release?
P.S. Assume that my web application has no "js", "css" and the like files, and doesn't use a database.
When you run mix release, it bundles all of your Elixir/Erlang dependencies for the MIX_ENV in question into the release directory, the erlang BEAM runtime/VM that you were using in your build, and any files that you specify in your mix project in mix.exs.
Because the BEAM runtime and code that bootstraps loading your code are included in the release, you won't need to install Elixir or Erlang on the target machine.
Things that are not included include:
any non-Elixir dependencies. For example, if you rely on openssl, you'll need to make sure you have a binary-compatible version of that installed on the machine you plan to run on (typically, the equivalent major verson release).
Portable bytecode. BEAM isn't like the Java VM. The compiled BEAM code needs to run on a substantially similar architecture. Build on an Arm64 machine for deployment on an Arm64 virtual machine, or x86 for Intel-compatible hardware, for instance. And it's probably best to use the same major OS distribution. There may be cases where "Any Linux * Same CPU architecture" is fine, but for example, building on a Windows or MacOS install of Elixir/OTP and deploying on Linux is a non-starter; you'd need to use a sufficiently similar OS.
As an example, one of my projects has its releases built on Alpine using Docker, so we only really have to worry about CPU compatibility. In our case we do need to make sure some external non-Elixir dependencies our app binds to are included on the docker image.
RUN apk add --no-cache libstdc++ openssl ncurses-libs wkhtmltopdf xvfb \
fontconfig \
freetype \
ttf-dejavu
(ignore the fact that wkhtmltopdf is kind of deprecated, we're working on it. But for now it's a non-elixir dependency we rely on).
If you're building for a, say, an EC2 instance and not using Docker, you'd just need to make sure your release is built on a similar OS to what you're using for production, and make sure the production AMI (image) has those non-Elixir dependencies on it, or will at the time of deployment, perhaps using apt or another package manager. For a VPS, the solution for non-elixir dependencies will depend on whether they have the option for customizing the base machine image (maybe with Packer or Ansible)
Since you may seem to have been a bit confused about it in the comments, yes, MIX_ENV=prod mix release will build all of your production Elixir/Erlang dependencies and include them in the /_build/prod folder.
I include the whole ./prod folder in our release, but it looks like protocol consolidation binaries and the lib folder .Beam files are all in the rel folder so that's a bit unnecessary.
If you do a default build, the target will be inside your _build directory, with sub-directories for the config environment and your application, e.g. _build/dev/rel/your_app/. That directory should contain everything you need to run your app -- the prompt after running mix release provides some clues for this when it says something like:
Release created at _build/dev/rel/your_app!
I find it more useful, however, to zip up the app into a single portable file (and yes, I agree that the details about how to do this are not necessarily the first things you see when reading about Elixir releases). The trick is to customize your mix.exs by fleshing out the releases option -- this is usually done via a dedicated private function but the organization of how you supply the options is up to you.
What I find is often useful is the generation of a single zipped .tar.gz file. This can be accomplished by specifying the include_executables_for option along with steps. It looks something like this:
# mix.exs
defmodule YourApp.MixProject do
use Mix.Project
def project do
[
# ...
releases: releases()
# ...
]
end
defp releases do
[
my_app: [
include_executables_for: [:unix],
steps: [:assemble, :tar]
]
]
end
When you configure your application this way, running mix release will generate a nice portable file containing your app with everything it needs. Unzipping this file is education for understanding everything your app needs. By default this file will be created at a location like _build/dev/yourapp-1.0.0.tar.gz. You can configure the build path by specifying a path for your app. See Mix.Release for more options.
I'm trying to compile my app (app_1) with a dep from another project (proj2) which has some apps (e.g. app_a, app_b, app_c...).
proj2 is has an apps folder in which there are the said apps (app_a etc.).
I'm trying to get app_a, but it also depends on app_b.
Whatever I do, I cannot seem to get it to work.
Using this in rebar.config of app_1:
{deps, [{proj2, {git, "git://github.com/scm/pr/proj2", {branch, "master"}}}]}.
I get
Dependency failure: source for proj2 does not contain a recognizable project and can not be built
Similar attempts with app_a instead of proj2, adding the /apps/app_a path etc. do not help.
Running rebar3 3.12.0.
Any help would be appreciated in how is it possible. Thanks
With rebar3 3.12.0, I don't think this is possible. You might be able to get it to work by using a rebar3.config.script file that downloads the dependency manually before putting it somewhere rebar3 can find it.
rebar3 3.14.0 introduced a git_subdir dependency type, which would look something like this:
{deps, [{proj2, {git_subdir, "git://github.com/scm/pr", {branch, "master"}, "proj2"}}]}.
This is mentioned in an example in the documentation on dependencies, but there aren't many details about it.
So if you're able to upgrade to a newer rebar3 version, it might be worth trying.
When looking at bower.json, I assume putting things in devDependencies is the recommended way to deal with things such unit testing dependencies. I'd like to use qunit as my testing framework, but I guess it should apply to other frameworks as well.
The problem I have is that when I put qunit in devDependencies, it doesn't get picked up when building vendor.js. That's expected, but then how do I go with asking brunch to include it? Obviously, I'd prefer to have test-vendor.js where I'd have dependencies + devDependencies, while keeping devDependencies out of vendor.js.
I have this in config.coffee:
exports.config =
# See http://brunch.readthedocs.org/en/latest/config.html for documentation.
files:
javascripts:
joinTo:
'javascripts/app.js': /^app/
'javascripts/vendor.js': /^(bower_components|vendor)/
'test/javascripts/test.js': /^test\/(?!vendor)/
'test/javascripts/test-vendor.js': /^test\/(?=vendor)/
Obviously, the last line wouldn't pick up any bower_component items as-is. Any suggestions here?
I saw this ticket:
https://github.com/paulmillr/read-components/pull/7
but it looks it's still up in the air.
This SOq:
Is it possible to split production and development Bower components?
suggests there's no way to do it from bower side at the moment.
Until this is implemented natively in Brunch, you can hack around it like this, so long as you aren't using Windows:
Add quint and other test dependencies to devDependencies in bower.json. They will now be managed under bower_components.
Add a symlink for each test dependency from your test/ directory to the relevant file under bower_components, eg:
ln -s ../bower_components/qunit/qunit.js test/
The dependencies will then be included in your unit-tests.js.
When brunch supports devDependencies, you can delete the symlinks.
See Ignore directories in brunch production build
You should be able to do something like:
overrides:
production:
conventions:
ignored: /[\/\\]_|bower_components[\/\\]qunit/
And then generate your production build with brunch b -P (brunch build --production)
I'm developing a series of related applications that will eventually be integrated into a single release. Several of the applications share identical record structures.
Should I:
a) Duplicate the *.hrl files that define the record structure in the include directory of each of the applications?
b) Put a single file somewhere else in my application tree--- and, if so where?
Here's my current tree:
zpt$
apps
app1
ebin
include
myrecords.hrl
priv
src
app2
ebin
include
myrecords.hrl
priv
src
etc
Many thanks,
LRP
One approach I tried was to create an application which doesn't do anything, but contains the record definitions common to multiple projects. Then I used rebar to include it as a dependency. When including the hrl files, I use the include_lib syntax. That syntax allows you to include hrl files from another application.
app
ebin
include
priv
src
some_src.erl
deps
common_hrl_app
include
common_records.hrl
src
ebin
other_dep_app
src
other_src.erl
.
.
.
include_lib example which could appear in either some_src.erl or other_src.erl:
-include_lib("common_hrl_app/include/common_records.hrl").
I like this approach because:
It plays nicely with the rebar dependency system
It allows me to track the hrls in one place in version control
I can version this application, which allows me to pull specific versions if I want a new application to be compatible with another using the same records.
Answering questions from the comments:
I have a skeleton app file in the ebin directory which specifies the name and version of the application so rebar can verify the version. Here's an example
{application,common_hrl_app,
[{description,[]},
{vsn,"1"},
{registered,[]},
{applications,[kernel,stdlib]},
{env,[]},
{modules,[]}]}.
With rebar, you have the top level application, which can have multiple applications as dependencies. When rebar fetches these dependencies, it places them in the deps directory. If any of those applications has their own dependencies, those are also fetched to the deps directory, and so on. There isn't an infinitely nested hierarchy of deps directories.
I have an erlang application I have been writing which uses the erldis library for communicating with redis.
Being a bit of a newbie with actually deploying erlang applications to production, I wanted to know if there was anyway to 'bundle' these external libraries with the application rather than installing into my system wide /usr/lib/erlang/lib/ folder.
Currently my directory structure looks like...
\
--\conf
--\ebin
--\src
I have a basic Makefile that I stole from a friend's project, but I am unsure how to write them properly.
I suspect this answer could involve telling me how to write my Makefile properly rather than just which directory to plonk some external library code into.
You should really try to avoid project nesting whenever possible. It can lead to all sorts of problems because of how module/application version is structured within Erlang.
In my development environment, I do a few things to simplify dependancies and multiple developed projects. Specifically, I keep most of my projects sourced in a dev directory and create symlinks into an elibs dir that is set in the ERL_LIBS environmental variables.
~/dev/ngerakines-etap
~/dev/jacobvorreuter-log_roller
~/dev/elib/etap -> ~/dev/ngerakines-etap
~/dev/elib/log_roller -> ~/dev/jacobvorreuter-log_roller
For projects that are deployed, I've either had package-rpm or package-apt make targets that create individual packages per project. Applications get boot scripts and init.d scripts for easy start/stop controls but libraries and dependancy projects just get listed as package dependencies.
I use mochiweb-inspired style. To see example of this get your copy of mochiweb:
svn checkout http://mochiweb.googlecode.com/svn/trunk/ mochiweb
and use
path/to/mochiweb/scripts/new_mochiweb.erl new_project_name
to create sample project of the structure (feel free to delete everything inside src afterwards and use it for your project).
It looks like this:
/
/ebin/
/deps/
/src/
/include/
/support/
/support/include.mk
Makefile
start.sh
ebin contains *.beam files
src contains ***.erl files and local *.hrl files
include contains global *.hrl files
deps contains symlinks to root directories of dependencies
Makefile and include.mk takes care of including appropriate paths when project is built.
start.sh takes care of including appropriate paths when project is run.
So using symlinks in deps directory you are able to fine tune the versions of libraries you use for every project. It is advised to use relative paths, so afterwards it is enough to rsync this structure to the production server and run it.
On more global scale I use the following structure:
~/code/erlang/libs/*/
~/code/category/project/*/
~/code/category/project/*/deps/*/
Where every symlink in deps points to the library in ~/code/erlang/libs/ or to another project in the same category.
The simplest way to do this would be to just create a folder named erldir and put the beams you need into it and then in your start script just use the -pa flag to the erlang runtime to point out where it should fetch the beams.
The correct way (at least if you buy into the OTP distribution model) would be to create a release using reltool (http://www.erlang.org/doc/man/reltool.html) or systools (http://www.erlang.org/doc/man/systools.html) which includes both your application and erldis.
Add the external libraries that you need, anywhere you want them, and add them to your ERL_LIBS environment variable. Separate the paths with colon in unix or semicolon in dos.
Erlang will add the "ebin"-named subdirs to its code loading path.
Have your *.app file point out the other applications it depends on.
This is a good halfway-there approach for setting up larger applications.
Another way is put your lib path in ~/.erlang.
code:add_pathz("/Users/brucexin/sources/mochiweb/ebin").
code:add_pathz("/Users/brucexin/sources/webnesia/ebin").
code:add_pathz("./ebin").
code:add_pathz("/Users/brucexin/sources/erlang-history/ebin/2.15.2").