Rails 7: Compiling assets with different folder structure - ruby-on-rails

Question

I am having a problem with my rails app. Today I upgraded from Rails 6 to Rails 7. In Rails 7 webpacker is kinda removed, so I am now using ESBuild. In my project I have TypeScript and SASS files. To compile these assets I am running the following script:
import esbuild from "esbuild";
import { sassPlugin } from "esbuild-sass-plugin";
import postcss from 'postcss';
import autoprefixer from 'autoprefixer';
// Generate CSS/JS Builds
esbuild
.build({
entryPoints: ['app/assets/scss/app.scss', 'app/assets/ts/app.ts'],
outdir: "dist",
bundle: true,
metafile: true,
plugins: [
sassPlugin({
async transform(source) {
const { css } = await postcss([autoprefixer]).process(source);
return css;
},
}),
],
})
.then(() => console.log("⚡ Build complete! ⚡"))
.catch(() => process.exit(1));
In the app/assets/config/manifest.js I have the following content:
// = link_tree ../images
// = link_tree ../scss .css
// = link_tree ../ts .js
// = link_tree ../builds
But I am using a different folder structure that is probably causing the issues. This is my folder structure for .scss and .ts files:
app/
└── assets/
├── scss/
│ ├── math/
│ │ └── calculate.scss
│ └── app.scss
└── ts/
├── math/
│ └── calculate.ts
└── app.ts
As you can see, under assets I have created the folders scss and ts, which makes more sense to me than putting the typescript in the javascript folder first. The problem is that when I use the include tag:
<%= javascript_include_tag 'math/calculate', 'data-turbolinks-track': 'reload' %>
The asset cannot be found, I think it is caused by the fact that it is still looking in the assets/javascript folder.
I can see in the public/assets that all my ts files are now .js files and all the .scss files are .css files, so ESBuild does his job, but the problem is in the including part I think. I get this error:
The asset "math/calculate.js" is not present in the asset pipeline.
Can someone help me fix this, I hope I can keep my folder structure like it is now!?

Answer 1

First, I have to mention that rails comes with tools to build javascript and css, we'll be doing our own set up, but these are still useful as a reference:
https://github.com/rails/jsbundling-rails
https://github.com/rails/cssbundling-rails
In Rails 7 js, css, and any other local assets are served by sprockets. For sprockets to find your assets they have to be in Rails.application.config.assets.paths; any directory under app/assets/ will be automatically be part of asset paths:
>> Rails.application.config.assets.paths
=> ["/home/alex/code/SO/ts/app/assets/builds",
"/home/alex/code/SO/ts/app/assets/config",
"/home/alex/code/SO/ts/app/assets/scss",
"/home/alex/code/SO/ts/app/assets/ts",
...
Most of the time an app would serve application.js and application.css. Any assets that go into javascript_include_tag, stylesheet_link_tag or anything else that needs to be served in the browser, also have to be declared for precompilation. For js and css these are the entrypoints, where you #import other files to be part of a bundle:
# app/assets/config/manifest.js
//= link_tree ../builds
app/assets/builds directory is in asset paths and every file in that directory will be precompiled in production. Because sprockets works with js and css, there has to be an additional build step to compile ts => js and scss => css.
The asset source files can be anywhere and organized however you like, rails doesn't care about them, as long as compiled assets end up in app/assets/builds.
CSS:
// esbuild.css.js
import esbuild from "esbuild";
import { sassPlugin } from "esbuild-sass-plugin";
import postcss from "postcss";
import autoprefixer from "autoprefixer";
const watch = process.argv.includes("--watch");
esbuild
.build({
// declare you entrypoint, where, I hope, you're importing all other styles.
entryPoints: ["app/assets/scss/app.scss"],
// spit out plain css into `builds`, so that sprockets can serve them.
outdir: "app/assets/builds",
// you'll want this running with --watch flag
watch: watch,
publicPath: "assets",
bundle: true,
metafile: true,
plugins: [
sassPlugin({
async transform(source) {
const { css } = await postcss([autoprefixer]).process(source);
return css;
},
}),
],
})
.then(() => console.log("⚡ CSS build complete! ⚡"))
.catch(() => process.exit(1));
JS:
// esbuild.js
import esbuild from "esbuild";
import glob from "glob";
const watch = process.argv.includes("--watch");
esbuild
.build({
// you say you want every file compiled separately, best I could come up:
entryPoints: glob.sync("app/assets/ts/**/*.ts"),
// spit out plain js into `builds`
outdir: "app/assets/builds",
watch: watch,
publicPath: "assets",
bundle: true,
metafile: true,
})
.then(() => console.log("⚡ JS build complete! ⚡"))
.catch(() => process.exit(1));
Rails compiles js and css separately, and so can we. This also avoids nested build/ts/ and build/scss/ directories.
In case you don't have this:
# bin/dev
#!/usr/bin/env sh
if ! gem list foreman -i --silent; then
echo "Installing foreman..."
gem install foreman
fi
exec foreman start -f Procfile.dev "$#"
# Procfile.dev
web: bin/rails server
_js: yarn build --watch
css: yarn build:css --watch
Add build scripts:
// package.json
{
...
"scripts": {
"build": "node esbuild.js",
"build:css": "node esbuild.css.js"
}
}
I think this should be everything:
app/assets
├── builds # <= this is the only directory "servable" by sprockets
├── config
│ └── manifest.js # //= link_tree ../builds
├── scss
│ ├── app.scss # #import "./math/calculate.scss"
│ └── math
│ └── calculate.scss
└── ts
├── app.ts # no imports here? ¯\_(ツ)_/¯
└── math
└── calculate.ts # console.log("do the calc")
You can run bin/dev to start the server and start compiling scss and ts:
$ bin/dev
or run build scripts manually:
$ yarn build && yarn build:css
⚡ JS build complete! ⚡
⚡ CSS build complete! ⚡
And it compiles into builds directory:
app/assets/builds
├── app.css
├── app.js
└── math
└── calculate.js
These ^ are the assets you can use in your layout:
<%= stylesheet_link_tag "app", "data-turbo-track": "reload" %>
<%= javascript_include_tag "app", "math/calculate", "data-turbo-track": "reload", defer: true %>
Now, when you ask for math/calculate, sprockets will find it in Rails.application.config.assets.paths and it will check if it is declared for precompilation:
# NOTE: if it is, you get a digested url to that file
>> helper.asset_path("math/calculate.js")
=> "/assets/math/calculate-11273ac5ce5f76704d22644f4b03b94908a318451578f2d10a85847c0f7f2998.js"
# everything as expected here
>> puts URI.open(helper.asset_url("math/calculate.js", host: "http://localhost:5555")).read
(() => {
// app/assets/ts/math/calculate.ts
console.log("do the calc");
})();
Hook your custom build scripts into a few rails tasks, like assets:precompile so that everything gets built automatically when deploying:
https://github.com/rails/jsbundling-rails/blob/v1.1.1/lib/tasks/jsbundling/build.rake
# lib/tasks/build.rake
namespace :ts_scss do
desc "Build your TS & SCSS bundle"
task :build do
# put your build commands here VVVVVVVVVVVVVVVVVVVVVVVVVVVV
unless system "yarn install && yarn build && yarn build:css"
raise "Command build failed, ensure yarn is installed"
end
end
end
if Rake::Task.task_defined?("assets:precompile")
Rake::Task["assets:precompile"].enhance(["ts_scss:build"])
end
if Rake::Task.task_defined?("test:prepare")
Rake::Task["test:prepare"].enhance(["ts_scss:build"])
elsif Rake::Task.task_defined?("spec:prepare")
Rake::Task["spec:prepare"].enhance(["ts_scss:build"])
elsif Rake::Task.task_defined?("db:test:prepare")
Rake::Task["db:test:prepare"].enhance(["ts_scss:build"])
end
OMG! Do not precompile assets in development! I don't know where people got that idea. You'll be serving assets from public/assets and they will not update automatically. If you did, just undo it:
bin/rails assets:clobber
Long story short:
app/assets/ts/app.ts # compile to `js` with esbuild
V # output into builds
app/assets/builds # is in `Rails.application.config.assets.paths`
V # is in `manifest.js` (//=link_tree ../builds)
javascript_include_tag("app")
V
asset_path("app.js") # served by `sprockets`
V # or by something else in production (thats why we precompile)
Browser!