I'm setting up varnish-devicedetect VCL in Varnish 4.0.2:
https://github.com/varnish/varnish-devicedetect/blob/master/INSTALL.rst
I'm following the directions for method #1: "Send HTTP header to backend"
I've read through this readme and have Googled for quite some time now and still quite a few concepts are escaping me.
Here's my code (excerpts):
default.vcl
include "devicedetect.vcl";
sub vcl_recv {
call devicedetect;
# ... snip ...
}
sub vcl_backend_response {
# device detect
if (bereq.http.X-UA-Device) {
if (!beresp.http.Vary) { # no Vary at all
set beresp.http.Vary = "X-UA-Device";
} elseif (beresp.http.Vary !~ "X-UA-Device") { # add to existing Vary
set beresp.http.Vary = beresp.http.Vary + ", X-UA-Device";
}
}
# ... snip ...
}
sub vcl_deliver {
# device detect
if ((req.http.X-UA-Device) && (resp.http.Vary)) {
set resp.http.Vary = regsub(resp.http.Vary, "X-UA-Device", "User-Agent");
}
# ... snip ...
}
Here's my questions.
When I inspect the response in Chrome Dev Tools, why is the Vary header set to User-Agent. Isn't the whole approach of method #1 NOT to use user agent, and instead use X-UA-Device?
Based on other guides I read... it seems this will hit the origin for EACH type of mobile (if you look in device detect, its split up into... mobile-iphone, mobile-android, mobile-smartphone, etc). Is this true in my code above? I definitely DONT want to hit the origin server more than twice for any given URL (desktop, and mobile ... I don't want all the mobile-* cached separately).
Can someone describe what the 3 code blocks above actually do? In somewhat laymen's terms. About the only one I truly understand is the first code block. call devicedetect just looks at the User-Agent and then sets X-UA-Device header with the appropriate grouping on the request to the backend. I'm a bit confused what the other 2 code blocks do though.
Can I delete the bit with X-UA-Device-force if I don't intend to allow the user to 'use desktop site'?
The guide mentions that I should be setting something in the backend in my app code. Right now this is all I have (rails). I'm not changing headers or changing anything about the response. I'm only changing the way the HTML looks (for the mobile version of the site). Should I be changing a header or something? This is what I have so far:
Rails:
def detect_device
if request.headers['X-UA-Device'] =~ /^mobile/
#device = 'mobile'
prepend_view_path Rails.root + 'app' + 'views_mobile'
else
#device = 'desktop'
end
end
As to point 1, your X-UA-Device is a custom header for internal consumption, ie by default not exposed to the external world. To ensure the external caches/proxies understand you are considering the device/user-agent in the response, you have to update the Vary with a header which reflect this. this is where the user-agent comes in, as thats where you have derived the X-UA-Device from.
note the comment within the link you indicate
to keep any caches in the wild from serving wrong content to client #2 behind them, we need to transform the Vary on the way out.
Related
I have following problem:
We are overriding the tt_content TCA with a custom column which has an itemsProcFunc in it's config. In the function we try to retrieve the TypoScript-Settings, so we can display the items dynamically. The problem is: In the function we don't receive all the TypoScript-Settings, which are included but only some.
'itemsProcFunc' => 'Vendor\Ext\Backend\Hooks\TcaHook->addFields',
class TcaHook
{
public function addFields($config){
$objectManager = \TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance('TYPO3\\CMS\\Extbase\\Object\\ObjectManager');
$configurationManager = $objectManager->get('TYPO3\\CMS\\Extbase\\Configuration\\ConfigurationManagerInterface');
$setup = $configurationManager->getConfiguration(
\TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface::CONFIGURATION_TYPE_FULL_TYPOSCRIPT
);
}
$setup is now incomplete and doesn't contain the full TypoScript, for example some of the static-included TypoScript is missing.
Used TYPO3 7 LTS (7.6.18), PHP 7.0.* in composer-mode.
Does anybody know where the problem is? Is there some alternative?
You maybe misunderstood the purpose of TypoScipt. It is a way of configuration for the Frontend. The Hook you mentioned is used in the TCA, whích is a Backend part of TYPO3. TypoScript usually isn't used for backend related stuff at all, because it is bound to a specific page template record. Instead in the backend, there is the TSConfig, that can be bound to a page, but also can be added globally. Another thing you are doing wrong is the use of the ObjectManager and the ConfigurationManager, which are classes of extbase, which isn't initialized in the backend. I would recommend to not use extbase in TCA, because the TCA is cached and loaded for every page request. Instead use TSConfig or give your configuration settings directly to the TCA. Do not initialize extbase and do not use extbase classes in these hooks.
Depending on what you want to configure via TypoScript, you may want to do something like this:
'config' => [
'type' => 'select',
'renderType' => 'singleSelect',
'items' => [
['EXT:my_ext/Resources/Private/Language/locallang_db.xlf:myfield.I.0', '']
],
'itemsProcFunc' => \VENDOR\MyExt\UserFunctions\FormEngine\TypeSelectProcFunc::class . '->fillSelect',
'customSetting' => 'somesetting'
]
and then access it in your class:
class TypeSelectProcFunc{
public function fillSelect(&$params){
if( $params['customSetting'] === 'somesetting' ){
$params['items'][] = ['New item',1];
}
}
}
I had a similar problem (also with itemsProcFunc and retrieving TypoScript). In my case, the current page ID of the selected backend page was not known to the ConfigurationManager. Because of this it used the page id of the root page (e.g. 1) and some TypoScript templates were not loaded.
However, before we look at the solution, Euli made some good points in his answer:
Do not use extbase configuration manager in TCA functions
Use TSconfig instead of TypoScript for backend configuration
You may like to ask another question what you are trying to do specifically and why you need TypoScript in BE context.
For completeness sake, I tested this workaround, but I wouldn't recommend it because of the mentioned reasons and because I am not sure if this is best practice. (I only used it because I was patching an extension which was already using TypoScript in the TCA and I wanted to find out why it wasn't working. I will probably rework this part entirely.)
I am posting this in the hope that it may be helpful for similar problems.
public function populateItemsProcFunc(array &$config): array
{
// workaround to set current page id for BackendConfigurationManager
$_GET['id'] = $this->getPageId((int)($config['flexParentDatabaseRow']['pid'] ?? 0));
$objectManager = GeneralUtility::makeInstance(ObjectManager::class);
$configurationManager = $objectManager->get(BackendConfigurationManager::class);
$setting = $configurationManager->getTypoScriptSetup();
$templates = $setting['plugin.']['tx_rssdisplay.']['settings.']['templates.'] ?? [];
// ... some code removed
}
protected function getPageId(int $pid): int
{
if ($pid > 0) {
return $pid;
}
$row = BackendUtility::getRecord('tt_content', abs($pid), 'uid,pid');
return $row['pid'];
}
The function getPageId() was derived from ext:news which also uses this in an itemsProcFunc but it then retrieves configuration from TSconfig. You may want to also look at that for an example: ext:news GeorgRinger\News\Hooks\ItemsProcFunc::user_templateLayout
If you look at the code in the TYPO3 core, it will try to get the current page id from
(int)GeneralUtility::_GP('id');
https://github.com/TYPO3/TYPO3.CMS/blob/90fa470e37d013769648a17a266eb3072dea4f56/typo3/sysext/extbase/Classes/Configuration/BackendConfigurationManager.php#L132
This will usually be set, but in an itemsProcFunc it may not (which was the case for me in TYPO3 10.4.14).
I have setup rollbar in my rails application. It keeps reporting recordnotfound which is as a result of SEO scrawlers (i.e Google bot, Baidu, findxbot etc..) searching for deleted post.
How to prevent rollbar from reporting SEO scrawler activities.
TL;DR:
# ./initializers/rollbar.rb
#
# https://stackoverflow.com/questions/36588449/how-to-prevent-rollbar-from-reporting-seo-crawlers-activities
#
# frozen_string_literal: true
crawlers = %w[Facebot Twitterbot YandexBot bingbot AhrefsBot crawler MJ12bot Yahoo GoogleBot Mail.RU_Bot SemrushBot YandexMobileBot DotBot AppleMail SeznamBot Baiduspider]
regexp = Regexp.new(Regexp.union(*crawlers).source, Regexp::IGNORECASE)
Rollbar.configure do |config|
ignore_bots = lambda do |options|
agent = options.fetch(:scope).fetch(:request).call.fetch(:headers)['User-Agent']
raise Rollbar::Ignore if agent.match?(regexp)
end
config.before_process << ignore_bots
...
end
======================
Be careful with magic comment frozen_string_literal and use =~ instead of match? if you have Ruby version less than 2.3.
Here I use an array that will be transformed into regexp. I did this because I wanted to prevent syntax and escaping related errors of developers in future and add ignorecase thing for same reason.
So in regexp you will see a Mail\.RU_Bot, instead of anything wrong.
Also in your case you can use simply word bot instead of many crawlers, but be careful with unusual user-agents. In my case, I want to know all crawlers on my site, so I came up with this solution. Yet another example of working part: there are crawler and crawler4j on my production site. I use just crawler in array to prevent notifing for both of them.
Last thing I want to say — my solution is not very optimal, but it just works. I hope someone will share an optimized version of my code. That's also the main reason I recommend to send data asynchronously, i.e. use sidekiq, delayed_job or whatever you want, don't forget to check related wikis.
My answer is based on #AndrewSouthpaw's solution (?), that wasn't working for me. Hoping that approved wiki-copy-pasted #Jesse Gibbs will be moderated some way.
=======
EDIT1: it's nice idea to check the https://github.com/ZLevine/rollbar-ignore-crawler-errors repo if you need to prevent rollbar to notify on js.
Looks like you are using rollbar-gem, so you'd want to use Rollbar::Ignore to tell Rollbar to ignore errors that were caused by a spider
handler = proc do |options|
raise Rollbar::Ignore if is_crawler_error(options)
end
Rollbar.configure do |config|
config.before_process << handler
end
where is_crawler_error detects if the request that led to the error was from a crawler.
If you are using rollbar.js to detect errors in client-side Javascript, then you can use the checkIgnore option to filter out client-side errors caused by bots:
_rollbarConfig = {
// current config...
checkIgnore: function(isUncaught, args, payload) {
if (window.navigator.userAgent && window.navigator.userAgent.indexOf('Baiduspider') !== -1) {
// ignore baidu spider
return true;
}
// no other ignores
return false;
}
}
Here's what I did:
is_crawler_error = Proc.new do |options|
return true if options[:scope][:request]['From'] == 'bingbot(at)microsoft.com'
return true if options[:scope][:request]['From'] == 'googlebot(at)googlebot.com'
return true if options[:scope][:request]['User-Agent'] =~ /Facebot Twitterbot/
end
handler = proc do |options|
raise Rollbar::Ignore if is_crawler_error.call(options)
end
config.before_process << handler
Based on these docs.
I have a collection called customer_devices and I can't change the name. Can I expose it via deployd as /devices ? How?
There are a few ways that I can think of. If you really just want to rename the collection, you can do so from the dashboard, as #thomasb mentioned in his answer.
Alternatively, you can create a "proxy" event resource devices and forward all queries to customer_devices. For example, in devices/get.js you would say
dpd.customer_devices.get(query, function(res, err) {
if (err) cancel(err);
setResult(res);
});
Update
Finally, here is a "hack" to redirect all requests from one resource path to a different path. This is poorly tested so use at your own risk. This requires that you set up your own server as explained here. Once you have that, you can modify the routing behaviour using this snippet:
server.on('listening', function() {
var customer_devices = server.router.resources.filter(function (res) {
return res.path === '/customer_devices';
})[0];
// Make a copy of the Object's prototype
var devices = Object.create(customer_devices);
// Shallow copy the properties
devices = extend(devices, customer_devices);
// Change the routing path
devices.path = "/devices";
// Add back to routing cache
server.router.resources.push(devices);
});
This will take your customer_devices resource, copy it, change the path, and re-insert it into the cached routing table. I tested it and it works, but I won't guarantee that it's safe or a good idea...
Can't you change the name via dashboard?
Mouseover your collection customer_devices
Click the down arrow
Select 'Rename'
Enter the new name and click 'Rename'
My application has a couple of services that make external calls via httpClient (GET and POST) that are unlikely to change in months, but they are slow; making my application even slower.
Clarification: this is NOT about caching GORM/hibernate/queries to my db.
How can I cache these methods (persistence on disk gets bonus points...) in grails 2.1.0?
I have installed grails-cache-plugin but it doesn't seem to be working, or i configured it wrong (very hard to do since there are 2-5 lines to add only, but i've managed to do it in the past)
I also tried setting up an nginx proxy cache in front of my app, but when i submit one of my forms with slight changes, I get the first submission as result.
Any suggestions/ideas will be greatly appreciated.
EDIT: Current solution (based on Marcin's answer)
My config.groovy: (the caching part only)
//caching
grails.cache.enabled = true
grails.cache.clearAtStartup = false
grails.cache.config = {
defaults {
timeToIdleSeconds 3600
timeToLiveSeconds 2629740
maxElementsInMemory 1
eternal false
overflowToDisk true
memoryStoreEvictionPolicy 'LRU'
}
diskStore {
path 'cache'
}
cache {
name 'scoring'
}
cache {
name 'query'
}
}
The important parts are:
do not clear at startup (grails.cache.clearAtStartup = false)
overflowToDisk=true persists all results over maxElementsInMemory
maxElementsInMemory=1 reduced number of elements in memory
'diskStore' should be writable by the user running the app.
Grails Cache Plugin works quite well for me under Grails 2.3.11. Documentation is pretty neat, but just to show you a draft...
I use the following settings in Config.groovy:
grails.cache.enabled = true
grails.cache.clearAtStartup = true
grails.cache.config = {
defaults {
maxElementsInMemory 10000
overflowToDisk false
maxElementsOnDisk 0
eternal true
timeToLiveSeconds 0
}
cache {
name 'somecache'
}
}
Then, in the service I use something like:
#Cacheable(value = 'somecache', key = '#p0.id.toString().concat(#p1)')
def serviceMethod(Domain d, String s) {
// ...
}
Notice the somecache part is reused. Also, it was important to use String as key in my case. That's why I used toString() on id.
The plugin can be also set up to use disk storage, but I don't use it.
If it doesn't help, please provide more details on your issue.
This may not help, but if you upgrade the application to Grails 2.4.x you can use the #Memoize annotation. This will automagically cache the results of each method call based upon the arguments passed into it.
In order to store this "almost static" information you could use Memcached or Redis as a cache system. (There are many others)
This two cache systems allows you to store key-value data (in your case something like this "key_GET": JSON,XML,MAP,String ).
Here is a related post: Memcached vs. Redis?
Regards.
Just new in Prestashop (1.6.0.6), I've a problem with my product page in admin. All translatable-field are to display:none (I inspect the code with chrome).
So when I want to create a new product I can't because the name field is required.
I thought that it was simple to find the .js whose do that but it isn't.
If somebody could help me, I would be happy.
Thank you for your help
Hi,
I make some searches and see that the function hideOtherLanguage(id) hide and show translatable-field element.
function hideOtherLanguage(id)
{
console.log(id_language);
$('.translatable-field').hide();
$('.lang-' + id).show();
var id_old_language = id_language;
id_language = id;
if (id_old_language != id)
changeEmployeeLanguage();
updateCurrentText();
}
When I set the Id to 1 (default language), it works. It seems that when I load the page, the function is called twice and the last calling, the id value is undefined. So the show() function will not work.
If somebody could help me. Thank you.
In my console, I see only one error
undefined is not a function.
under index.php / Line 1002
...
$("#product_form").validate({
...
But I find the form.tpl template and set this lines in comment but nothing change.
EDIT: According to comment on this link http://forge.prestashop.com/browse/PSCFV-2928 this can possibly be caused by corrupted installation file(s) - so when on clean install - try to re-download and reinstall...
...otherwise:
I got into a similar problem - in module admin page, when creating configuration form using PrestaShop's HelperForm. I will provide most probable cases and their possible solutions.
The solution for HelperForm was tested on PS 1.6.0.14
Generally there are 2 cases when this will happen.
First, you have to check what html you recieve.
=> Display source code - NOT in developer tools/firebug/etc...!
=> I really mean the pure recieved (JavaScript untouched) html.
Check if your translatable-fields have already the inline style "display: none":
Case 1 - fields already have inline style(s) for "display: none"
This means the template/html was already prepared this way - most probably in some TPL file I saw codes similar to these:
<div class="translatable-field lang-{$language.id_lang}"
{if $language.id_lang != $id_lang_default}style="display:none"{/if}>
Or particularly in HelperForm template:
<div class="translatable-field lang-{$language.id_lang}"
{if $language.id_lang != $defaultFormLanguage}style="display:none"{/if}>
Case 1 is the most easy to solve, you just have to find, where to set this default language.
Solutions
HelperForm
Look where you've (or someone else) prepared the HelperForm object - something like:
$formHelper = new HelperForm();
...
Somewhere there will be something like $formHelper->default_form_language = ...;
My wrong first solution was to get default form language from context - which might not be set:
$this->context->controller->default_form_language; //THIS IS WRONG!
The correct way is to get the default language from configuration - something like:
$default_lang = new Language((int)Configuration::get('PS_LANG_DEFAULT'));
$formHelper->default_form_language = $default_lang->id;
...this particularly solved my problem...
Other form-creations
If there is something else than HelperForm used for form creations, the problem is still very similar.
You have to find where in files(probably tpls) is a condition for printing display:none for your case - then find where is the check-against-variable set and set it correctly yourself.
Case 2 - fields don't have inline style(s) for "display: none"
This means it is done after loading HTML by JavaScript. There are two options:
There is a call for hideOtherLanguage(), but there is wrongly set input language - that means no language will be displayed and all hidden.Solution for this one can be often solved by solving Case 1 (see above). In addition there can be programming error in not setting the after-used language id variable at all... then you would have to set it yourself (assign in JavaScript).
Some script calls some sort of .hide() on .translatable-field - you will have to search for it the hard way and remove/comment it out.
PS: Of course you can set the language to whatever you want, it is just common to set it to default language, because it is the most easier and the most clear way how to set it.