On these websites https://staging.blockbutler.io and https://blockbutler.io (RoR on heroku both)
Google and Yandex search bots can't index with reason: 'robots.txt blocks'
but robots.txt is fine. And if you will remove
javascript_include_tag and stylesheet_link_tag it perfectly indexed the page. I spent 3 days of trying different tests - nothing helps:
if there are only javascript_include_tag with empty
application.js or only stylesheet_link_tag with empty
application.scss - no indexing
add gem 'rails_12factor' - no
indexing
rake assets:precompile RAILS_ENV=production and push static files from public/assets to server - no indexing
put assets in footer - no indexing
wrap assets in <noindex> and rel: "nofollow" - no indexing
I really don't know what da magic is going on and be really happy for any ideas
Logs when run google search console live test:
production.rb:
config.assets.js_compressor = :uglifier
config.assets.enabled = true
config.assets.version = '1.0'
config.assets.compile = true
robots.txt:
User-agent: *
Allow: /
User-agent: Yandex
Allow: /
User-agent: Google
Allow: /
Sitemap: https://blockbutler.io/sitemap.xml
p.s. sorry for bad english - will appreciate editing my language (:
Some freaking magic over here.
Change <%= javascript_include_tag 'application', rel: "nofollow" %> to <script src="/assets/application.js" rel="nofollow"></script>
And now google indexer works just fine. GooGLe InDExER DoesN'T LiKE BIg fiLe NamES. Will think about how to prevent cache assets file.
p.s. ok. Now every time Im updating assets - Im changing assets name, like applicationv0.js and etc. Still have no idea, why google wasn't ok with default application-hash.js filename
Related
According to rails/webpacker documentation, extract_css is default to true in production environment and false in development. From what I observed:
With extract_css true, webpacker will emit a css file from each stylesheet_pack_tag in application.html.erb.
And, with extract_css false, stylesheet_pack_tag return nil & stylesheet that gets imported in js files will get extracted and bundle into blobs and send to browser. Hence, link tags to blob url exist.
So, I assume that using extract_css true yield the same result as using inline styles in header since styles get downloaded to browser with the website document file. If what I understand is true then setting extract_css to true on production should be ok.
Is what I understand about extract_css option correct?
You mostly correct, you can read more about extract_css in css.md or v4-upgrade.md
With extract_css: true, webpacker will emit a single css <link rel="stylesheet"... from each stylesheet_pack_tag.
With extract_css: false, stylesheet_pack_tag return nil & stylesheet that gets imported in js files will get extracted and bundle into blobs and injected into as an inline .
In the end extract_css: false is the one that yields the same result as using inline styles.
I don't have anything to add other than "extract_css" in webpacker.yml has been a source of confusion for me as well. When it is "extract_css: false" in development and production, a stylesheet IS included in the document head (shouldn't this be "extract_css: true"?). And when I use "extract_css: true", styles are not included in the document.
I access some static images (favicon, etc) through /assets/image.jpg.
It works as expected on localhost but when I push in production, I can't access fonts and images.
Css and js are compiled and working fine.
I've added this to my production.rb file but it still doesn't work :
config.serve_static_files = true
config.serve_static_assets = true
config.assets.compile = true
What do I have to write to access them and where ?
Ok, I figured, it was dumb :
In production we have to use rails link helpers to provide assets, for example, you can't access favicon with :
<link rel="shortcut icon" type="image/gif" href="/assets/images/favicon.gif"/>
we have to use
<%= favicon_link_tag 'favicon.gif' %>
because a sha is generated and produces the following link for example
/assets/favicon-02168c53f101e2059920863c64a71d6abc53b4fbec334f2e0b002f7866e63b69.gif
I have in my .html file:
<%= javascript_include_tag "processing-1.4.1.min" %>
<canvas data-processing-sources="/assets/pjs/my.pde"></canvas>
and the asset lies exactly there: app/assets/pjs/my.pde.
I get this error in the server:
Served asset /pjs/my.pde - 404 Not Found (10ms)
and this error in the javascript:
Uncaught Processing.js: Unable to load pjs sketch files: /assets/pjs/my.pde ==> Invalid XHR status 404
My application.rb says:
config.assets.enabled = true
Might be a really stupid mistake but i just don't get it. I'd really appreciate if anyone can tell me how to solve this.
use the erb extension for your view file, then use asset_path 'my.pde'. When using the asset pipeline you can't link directly to a path because files will have fingerprints added to them.
I'm struggling with the asset pipeline. I'm loading dojo from Google CDN putting this in my template:
= javascript_include_tag 'http://ajax.googleapis.com/ajax/libs/dojo/1.6.1/dojo/dojo.xd.js', :'data-dojo-config' => %Q(dojoBlankHtmlUrl:'/blank.html', baseUrl: 'assets/', modulePaths: {custom: 'javascripts/modules'})
I just want a fallback to a local version if running locally or if the CDN is down. I thought of doing this:
script typeof(dojo) === "undefined" && document.write(unescape('%3Cscript src="js/libs/dojo-1.6.1.min.js"%3E%3C/script%3E'));
But I don't like it as it works out of the asset pipeline. I want to keep dojo in vendors/assets/javascripts/dojo. How can I get the fallback to be served by the asset pipeline.
Is there a way do declare conditional require in the asset pipeline. What I want is to run some javascript tests, and depending on the result serve a file.
Thanks
I suggest you use yepnope, a lightweight library for loading libraries like this in parallel (for speed) and it gives you the option to run some other code to test if the library is loaded. For example:
yepnope([{
load: 'http://ajax.googleapis.com/ajax/libs/dojo/1.6.1/dojo/dojo.xd.js',
complete: function () {
if (!window.jQuery) {
yepnope('asset_path('you_local_copy_of_dojo') ');
}
}
}])
(Note: You will need erb tags around the asset_path helper)
The local dojo file would be in the assets/javascript folder, but not included in the application manifest. You need to add the dojo file to the precompile array:
config.assets.precompile += 'your_local_file.js'
And this will make it available to the asset_path helper.
Thanks Richard!
I don't want to have yepnope to load one library. It would be overkill imo. Here is the solution I came up with, based on your help (written in slim):
1/ In vendors/assets/javascripts/, I have my dojo.js.
2/ In config/application.rb:
# Precompile these assets files
config.assets.precompile += ['dojo.js']
3/ In the template:
= javascript_include_tag "http://ajax.googleapis.com/ajax/libs/dojo/#{Settings.dojoVersion}/dojo/dojo.xd.js", :'data-dojo-config' => %Q(dojoBlankHtmlUrl:'/blank.html', baseUrl: 'assets/', modulePaths: {custom: 'javascripts/modules'})
script = "typeof(dojo) === \"undefined\" && document.write(unescape('%3Cscript src=\"#{asset_path('dojo')}\"%3E%3C/script%3E'));".html_safe
I also posted on the Rails Google Group to request the addition of two options to the javascript_include_tag, :test and :local that would take care of all the work. We'll see.
We're using Rails asset caching for JS and CSS like this:
<%= stylesheet_link_tag 'reset','global','admins','autocomplete', 'date_input', 'tablesorter', 'partners', 'jqmodal', :media => 'screen', :cache => set_asset_cache(:admins) %>
<%= javascript_include_tag :defaults, 'autocomplete', 'searchbox', 'jqmodal', :cache => set_asset_cache(:admins) %>
In our deploy we call rake tmp:assets:clear each time. The problem is that the first few page loads after a deploy come up with no css or js on the page. I guess until the cached all.js and all.css have been regenerated.
We deploy many times per day and this is scary for any users who happen to come across a busted page.
Have people found any way to make this smoother so the new cached assets are guaranteed to be there on the first new page load?
The AssetHat gem addresses this exact problem. Instead of concatenating assets the first time a page is loaded (which increases that page's load time), it concatenates assets on deploy instead. As a bonus, the gem also minifies your CSS and JS, which saves precious bytes.
After setup, usage is pretty simple:
Use include_css :bundle => 'admins' and include_js :bundle => 'admins' in your layout. (The bundle contents are set in a config file to keep your layout lightweight.)
Add rake asset_hat:minify to your deploy script. My company has been using it in production with Capistrano for about a year now.
There's more info in the readme and docs, and I'd be happy to hear any questions/ideas!
You could try warming the cache during deployment using wget, as an example (shamelessly reposted):
wget -r -nd --delete-after http://whatever.com/~popular/page/
However, this would have to be executed after you switch your symlink to your new deployment. A possibly more elegant solution might be to call the asset caching methods manually in your deploy, though I'm not sure how feasible that is. Here's where the caching is performed in Rails:
# File vendor/rails/actionpack/lib/action_view/helpers/asset_tag_helper.rb, line 273
273: def javascript_include_tag(*sources)
274: options = sources.extract_options!.stringify_keys
275: concat = options.delete("concat")
276: cache = concat || options.delete("cache")
277: recursive = options.delete("recursive")
278:
279: if concat || (ActionController::Base.perform_caching && cache)
280: joined_javascript_name = (cache == true ? "all" : cache) + ".js"
281: joined_javascript_path = File.join(joined_javascript_name[/^#{File::SEPARATOR}/] ? ASSETS_DIR : JAVASCRIPTS_DIR, joined_javascript_name)
282:
283: unless ActionController::Base.perform_caching && File.exists?(joined_javascript_path)
284: write_asset_file_contents(joined_javascript_path, compute_javascript_paths(sources, recursive))
285: end
286: javascript_src_tag(joined_javascript_name, options)
287: else
288: expand_javascript_sources(sources, recursive).collect { |source| javascript_src_tag(source, options) }.join("\n")
289: end
290: end
You might be able to modify the caching code and run it manually on deploy.