Websites detecting and blocking Selenium Chromedriver - url

I got a task to update a URL in a test (using Selenium, Java and Chromedriver) to a new URL. The page is the same, just at a new location. This test worked fine with the old URL.
I brought up an instance of Chrome and verified I could access the URL.
Then I changed the URL, committed to git, and ran on our Jenkins machine and it failed. The screenshot said "This site can’t be reached". I told a co-worker that I think the URL must be wrong (though it worked on my desktop without Selenium). He tried it and it loaded fine. I tried it again from a regular Chrome window and it loaded fine. I tried from Eclipse using Selenium and I got the message about the site could not be accessed.
So I ran Eclipse, stopped it right when Chrome came up but hadn't loaded a URL yet. I manually entered the URL (from copy/paste) and got "This site cannot be accessed). Then I brought up a regular Chrome window and pasted the URL in and it loaded fine.
I decided to look for differences. So in my regular Chrome window it was "Version 77.0.3865.120 (Official Build) (64-bit)". I did the same on the version running under Chromedriver and got "Version 77.0.3865.120 (Official Build) (64-bit)". So the same.
I did a search for the latest Chromedriver. It said for version 77 to please use 77.0.3865.40. This was not what I was using. So I downloaded the correct version and thought "surely this will fix it". I verified it was 77.0.3865.40 coming up. But, as you guessed it, I got the same problem after updating to the most compatible Chromedriver.
Some searches said that basically nothing would be able to tell, but there was one property that Chromedriver inserts. I don't know whether this would be enough to block a session using Chromedriver?
The only thing I can think of is that we have iBoss and possibly iBoss is blocking it? Either that or somehow the site is blocking it.
I would like to share the URL with you but I do not know whether that would be violating company policy. If I get permission, I will share the URLs.
by the way, the error is "ERR TUNNEL CONNECTION FAILED"

Got some internal help to help fix some settings

Related

Atlassian JavaScript load error We tried to load scripts but something went wrong

We tried to load scripts but something went wrong.
Please make sure that your network settings allow you to download scripts from the following domain:
https://aid-frontend.prod.atl-paas.net/atlassian-id/front-end/4.5.0
Tried Google Chrome, Firefox, Internet explorer also tried with different PC.
Not able to login into bitbucket
I'm using Chrome Version 77.0.3865.120 (Official Build) (64-bit). I got the same error trying to log in to my group's Atlassian site, and I was able to fix it by specifically allowing JavaScript from Atlassian:
Open Chrome settings
Click on Advanced
Click on Privacy and Security
Click on Site Settings
Click on JavaScript
Make sure JavaScript is set to allowed
Next to Allow, click Add
Paste the URL from the error message in and click Add
I've faced a similar problem recently while traveling and trying to work from a new location.
First I've tried the steps from the previous answer which unfortunately didn't help me (but they definitely may be useful in a different case scenario).
I'm renting a house with an existing pre-configured WIFI router which I could get an access to manage it over the web UI.
I'm not sure which of the following helped me. I did the following changes to the router configuration:
Added the 8.8.8.8 as a primary DNS Server
Enabled IPV6
Disabled the built-in firewall (this one is the more likely one which fixed the issue)

Service Worker: files are updated on the server but old version showing in browser

I am building a static app with PouchDB on Google AppEngine.
When I open the site in a browser window, it is showing a version I uploaded several hours ago.
If I open the site in an incognito window, the updated version is displayed (therefore I don't think it is actually an error in the console).
I put a new version number in app.yaml
I have migrated all traffic to the new version.
I have cleared my cache, deleted cookies, checked my application data, everything. I even reinstalled Chrome and Firefox.
I updated my Python version and my Google AppEngine Launcher yesterday; the problem pre-dated that update.
Also: just discovered that if I go to the URL of the updated version
http://4.[app-id].appspot.com, it displays the correct, updated version.
This is happening in Chrome, Firefox, and Edge.
Edit: probably should have mentioned that my site uses Service Workers and IndexedDB. I assume my service workers are caching the previous version, but I would have thought that Ctrl + F5 would clear the cache and show the new version.
I think it must be the Service Workers caching the pages (which is, after all, what they are supposed to do). This is actually really annoying when you are developing though.
A guy called Rich Harris has documented this behaviour and some workarounds on a Github Gist.
Reloading the page doesn't behave as you'd expect
If you make a change to your service worker, then reloading the page
won't kill the old one and activate the new one (it detects the change
by requesting the service worker file each time and comparing the old
and the new byte-for-byte), leaving you somewhat confused as to why
your changes haven't taken effect. This is because the old window is
never actually closed, meaning there's never a time to swap them out –
you need to kill the tab completely then reopen it.
Or you can have the browser do that for you by going to the
Application tab in devtools (in Canary, not stable Chrome yet), going
to the Service Workers section, and checking 'Update on reload'.
UPDATE (13 Nov 2017): This functionality is now available in Chrome, so you don't even need to download Canary.
And here's which bits of the application cache to clear:

Intern test framework hangs/does nothing when used with Appium 1.4.13 on an iOS device for mobile web

I just started looking into using the intern test framework to write functional tests for our mobile web apps. I'm looking to run these tests on actual iOS devices.
I setup Appium as the selenium server talking to the intern client. It works as expected on the simulator (although I get 'method not implemented' for many of the touch related methods).
I then decided to configure everything to work on my iPhone 6 plus, running iOS 9.0.2
After much wrangling with instructions from here, I got all the various pieces working in tandem (ios-webkit-debug-proxy, appium-server from the repo, etc.)
Finally, I decided to run my tests. When I run the test runner, Safari on my device starts up (via SafariLauncher App), and ios-webkit-debug-proxy and appium-server seem to be talking to each other. However, intern simply posts a 'created session on...' message, and then does nothing.
To ensure that the server was running properly, I decided to use a rest client to send it POST and GET requests using the JSONWire Protocol methods, and it seems to be loading the web-page url and responding to JSONWire requests. So clearly, something is going wrong with Appium talking to intern, or vice-versa.
Any help/insight on this would be greatly appreciated. Thanks!
I'm also in the unique position of having to run these locally, so using SauceLabs, BrowserStack, etc. is out of the question.
Got it working!!!
So the trick was to specify the proxyUrl config option.
My original conjecture was that there might have been some issues with basePath, etc., but I noticed my console.log statements that were outside of test cases were running properly.
That's when I noticed that the Safari page tried to navigate to localhost right after the test had started. This is my first time ever working with Intern or Appium or any Selenium stuff for that matter. Because you cannot access localhost (which in this case was meant for accessing the Intern server on my computer, since you're running the tests on your computer, and not on an external device), from the iPhone, I specified proxyUrl and proxyPort in the config, and specified my computer's local IP (192.168.x.x) and the Intern server port (9000). It still didn't work. I looked at the URL in Safari and noticed that it wasn't using the proxyPort field - so that the URL looked like: 192.168.x.x/__intern/clie... So I just tried to enter the port in the proxyUrl option - as that option's a string - 192.168.x.x:9000. And voila - it worked!!!
So hopefully this helps some other soul looking to have a native mobile test working with this setup!

Grails withForm and AWS interactions

The following issue is using the following:
AWS
PostgreSQL
Grails 2.3.3
Redis
On our account creation page, we are having some REALLY obscure functionality with the Grails withForm{}.invalidToken{} closure.
Upon hitting the page for the first time, everything works fine. You can post back to the server fine as long as you do not leave this page.
Upon leaving this page, either through navigation links or logging off, returning to the page (Again through navigation links or logging on and heading there), we can no longer submit, it hits the invalidToken closure every time.
I know AWS is involved as we took the project and deployed it to local machines both with IntelliJ and Tomcat by itself and both work fine. This issue only occurs upon deploying the WAR to AWS. (This occurs both with local builds and automated builds. They work locally but not on AWS)
We have spent almost a week on this issue trying to figure out why this is occuring, and all we have to show for it is we know AWS is somehow involved, but that's as far as we have gotten.
Does anyone have any insight into what would be causing our session to act like this?
After a LOT of searching about this issue, me and my team finally figured it out. Taken directly from our JIRA:
"This issue is caused by the implementation of tomcat-redis-session-manager used on AWS. As per their documentation (https://github.com/jcoleman/tomcat-redis-session-manager#session-change-tracking), there are "unintended consequence of hiding writes if you implicitly change a key in the session or if the object's equality does not change even though the key is updated." Specifically, the "useToken" implementation is Grails 2.3.8 is: "String generateToken(String url) { final UUID uuid = UUID.randomUUID() getTokens(url).add(uuid) return uuid }" The combination of these native implementations are there for incompatible.
The tomcat-redis-session-manager does support a manual dirty tracking mode by setting: RedisSession.setManualDirtyTrackingSupportEnabled(true); but this would require a forked build of SynchronizerTokensHolder in grails-core."

Chrome browser doesn't like *.loc domain without http://?

For web development on localhost I'm using domains with .loc extension at the end of a domain name.
For example: if I work on a site roses.com, the local development domain would be roses.loc
(defined in hosts file and IIS as a host-header in binding)
My preferred browser (the main browser I use for development is Chrome) but unfortunately
it does not recognize a domain name with .loc extension as http://rose.loc - it throws me onto a google search page each time I would type in rose.loc (without http://) in the beginning..
Have you experienced this in a similar way ? Is there some solution to that ?
Because during intensive development testing with clearing browser cache and restarting the browser for various reasons, it's getting pretty annoying to be thrown at a google search page instead of the development page where I expect to notice some changes, each time I forget to type in http:// before the url (and Chrome is the one who hides it by default, anyway..)
Google Chrome is pretty "smart" with this. It uses a list of known-good TLDs and assumes everything else is just a search term that happens to end in a dot followed with some characters.
99% of the time that's perfectly fine. It's "only" us developers and a few people with strange network setups that have to suffer for the good of the majority ;-)
You can try using .local as your TLD, as that's a defined domain for referencing local domain names (at least it's used in some mDNS systems).
The issue has been entered as #30636 in the Chromium bug tracker. One workaround that often (but not always) seems to work is to append / to your hostname. So try roses.loc/.
I've created a search engine with a keyword of 'l' (my local TLD is .l). The URL for the search engine is http://%s.l. Then, I simply type "l mysite" in the address bar and it takes me to mysite.l.
Here is a workaround I came up with for this bug: http://code.google.com/p/chromium/issues/detail?id=30636#c38
I have Chromium installed on Linux Mint, and have a few localhost websites here. (I use Firefox for all of my work, so I have just discovered something here with Chromium.) My local sites are called morse and a.z. I had to enter http://morse/ and a.z/ respectively to get these sites to load the first time. They produced quick links on the new tab's otherwise blank page.
After closing Chromium, I reopened it and I could enter just morse/ and a.z to visit these sites. Since I've never seriously used this browser, I have not tailored any settings in it. (I did not use the quick link icons, but instead typed in the address bar.)
My findings confirm the localhost example.TLD/ entry does work when entered for the first time.
About Chromium: I am using Version 106.0.5249.119 (Official Build) for Linux Mint (64-bit).

Resources