Sometimes, when running automated tests in PhantomJS using Cucumber for Rails 4, it would be really, really useful to actually sit in front of my screen, look in a window, and see exactly what the browser is doing.
There are times when your code is right and your test is right, but testing fails intermittently nonetheless. It's often because of a script, or an animation, or some CSS that gets in the way. But seeing a screenshot, drilling into a DOM inspector, or using the debugger is not enough to catch those edge cases.
Is there any way to have a window looking at what PhantomJS is doing in the background? It could be something in X Window, or running in a VNC Server, etc. Anything visual would greatly help with debugging, especially in with those finicky details.
I found a little program called PhantomVNC, but it's not telling me much on how it works. It looks like something just feeding a series of screenshots through VNC.
I tried PhantomJS and Capybara-WebKit, but neither of those headless browsers other a "head" option. Selenium-WebDriver seems complicated to set up and only seems to work with a full browser like Firefox, and that may cause more problems than it solves.
If you have any ideas, please let me know. Thank you in advance.
Not sure specifically about Cuc/Rails - but I specifically have a second chrome/webdriver Virtual Machine for this. I use PhantomJS specifically for the headless automatic acceptance testing, but when there is an issue with the tests and I'm trying to diagnose it can be helpful to view the browser live.
In other words, spin up a Ubuntu desktop VM, install chrome, install selenium server and the chrome webdriver. Log in to the VM, open chrome, make sure selenium is running, then configure your test suite to connect to the selenium webdriver service (usually port :4444) of this instance. Run your tests and you should be able to watch the chrome instance on the VM.
Related
Background
I have a makefile which fetches and spins up a Dockerized Mediawiki instance on my local machine, from scratch, with a single make command:
https://gitlab.wikimedia.org/mhurd/mediawiki-docker-make
This works fairly well. Please try it!
Goal
I'd also like to be able to run Mediawiki's Selenium tests (as-is) in Docker containers and watch them execute "live", with the makefile, again, making it easy to kick this process off.
Approach
This work-in-progress branch adds a container for running Mediawiki Selenium tests, and a browserless-chrome container so you can watch and debug the tests "live", as they run:
https://gitlab.wikimedia.org/mhurd/mediawiki-docker-make/-/tree/selenium
As you can see from the top of this branch's readme, after running make to spin everything up, running make runseleniumtests kicks the tests off. When doing so a browser window is automatically opened which lets you see the Selenium tests which are running in the browserless-chrome container. This works... to a point.
Problem
Unfortunately, after a few tests are seen to be running, I'm seeing this error:
Protocol error (Input.dispatchMouseEvent): Target closed.
I suspect the way the browserless-chrome container manages its Chrome sessions may be to blame, as it appears the error is happening when a test causes a new page to be loaded, but I'm not sure.
Any ideas appreciated, and please try it yourself - the whole point of the makefile is to make spinning this up from scratch super simple. It only takes a couple commands and you should be able to see the tests running and the resulting error. Thanks!
Misc
Running curl -s http://127.0.0.1:3000/sessions | python3 -m json.tool on the host machine after attempting to run the tests shows there are multiple browserless-chrome sessions, but I'm unsure how to make it behave more like a non-dockerized setup - which has no problem with tests which cause page loads.
In email communication with Browserless support, they mention seeing similar issues but haven't been able to track down why:
It seems to be that the root cause may be due to "click" events aren't
working properly when your viewing the live debugger. Can you confirm
that if you don't open the live viewer, it allows the test to progress
and eventually throws new errors regarding selectors not being found? (Edit: I did)
I've also run into this live debugger behavior before and in my
experience I stopped watching the live debugger, and since I couldn't
see which selector it wasn't finding I exported a screenshot and in
that particular case, it ended up being an issue with my viewport,
since the viewport in the remote session was smaller, the styles
rendered differently then locally on my machine, when I set the
viewport size, the selectors were found - but then again, that was my
particular issue, might not be yours.
But yes, for some odd reason when you view live sessions, you can run
into this odd issue which we haven't been able to understand.
I'm testing my Rails project and there are some problems when the test is running.
cucumber features/test.feature
I have installed chromedriver for WSL2 and the test can run fine. However, the browser showing up is not rendered at all, it only has a border and transparent body like this.
I can still open google-chrome and it is displayed correctly. And the project used to work on WSL1. I can also take screenshots when testing, it's just that the browser's UI is not rendered.
I have installed the driver for GPU as stated in the WSL guide but the problem still persisted.
Has anyone encountered this problem and how can I fix it? Thank you very much.
I have a file upload control in my UI marked with webkitdirectory. When I upload a folder in the following way, running in non-headless mode, it works:
find('.directory-upload', visible: false).set(Rails.root + "test/data/directory_to_upload")
However, when I run headless (with the :headless_chromium driver) it fails silently, i.e. the code runs but the directory isn't actually selected in the control (so the ensuing form validation fails).
Are there differences in supported HTML between headless and non-headless that could explain this?
You've run into a known bug in headless Chrome - https://bugs.chromium.org/p/chromedriver/issues/detail?id=2521&q=directory%20upload&colspec=ID%20Status%20Pri%20Owner%20Summary - nothing your can really do about it.
I'm running a fairly large suite of python-based tests with a much larger number of steps on an Ubuntu Linux VM. When I run them via any number of methods manually (via the console) they all run and pass just fine.
After I ported them to a Jenkins server, four out of the thirty fail. I tried the usually recommended fix - increasing the wait time for keywords to work to 1s before every single click - so I'm fairly certain it isn't a timing issue. The site loads a lot faster than that on Windows, which I know is slower than Jenkins on Linux.
After Googling around a little for an answer, I found that apparently no one has come up with an accepted answer, either on this site or other Q/A sites.
Here's the error messages I'm getting from Jenkins.
ElementNotVisibleException: Message: element not visible
(Session info: chrome=61.0.3163.79
(Driver info: chromedriver=2.26.436382 (70eb799289ce4c2208441fc057053a5b07ceabac),platform=Linux 4.10.0-33-generic x86_64)
WebDriverException: Message: unknown error: Cannot read property 'innerHTML' of undefined
(Session info: chrome=61.0.3163.79
(Driver info: chromedriver=2.26.436382 (70eb799289ce4c2208441fc057053a5b07ceabac),platform=Linux 4.10.0-33-generic x86_64)
The other two are both element not visible exceptions identical to the first, both of which happen on a Click Button keyword that is not the first Click Button keyword in the test suite. The first one happens on a Click Element keyword that has worked perfectly since I wrote it, and the last one happens on tried-and-true JavaScript call to get the text of an element.
Why would something work locally on two different operating systems and then fail on Jenkins?
Why would something work locally on two different operating systems and then fail on Jenkins?
The most common might be that the jenkins system is running slower, and your tests aren't being hyper-vigilant about waiting for pages to finish loading before trying to interact with it. Jenkins boxes often can be under a heavy load, and if both the client and server are running on the same box, either or both could be contributing to the problem.
Another reason could be that you're running different versions of the browsers and/or selenium drivers on the jenkins box.
Another reason could be that the resolution of the (virtual?) displays is different, causing elements to be shifted to a different position.
The browsers on the jenkins box could have different profiles, resulting in a different set of plugins or antivirus software running. These can contribute to the speed at which a page renders, or could cause unwanted popups that cover portions of the screen.
I just started looking into using the intern test framework to write functional tests for our mobile web apps. I'm looking to run these tests on actual iOS devices.
I setup Appium as the selenium server talking to the intern client. It works as expected on the simulator (although I get 'method not implemented' for many of the touch related methods).
I then decided to configure everything to work on my iPhone 6 plus, running iOS 9.0.2
After much wrangling with instructions from here, I got all the various pieces working in tandem (ios-webkit-debug-proxy, appium-server from the repo, etc.)
Finally, I decided to run my tests. When I run the test runner, Safari on my device starts up (via SafariLauncher App), and ios-webkit-debug-proxy and appium-server seem to be talking to each other. However, intern simply posts a 'created session on...' message, and then does nothing.
To ensure that the server was running properly, I decided to use a rest client to send it POST and GET requests using the JSONWire Protocol methods, and it seems to be loading the web-page url and responding to JSONWire requests. So clearly, something is going wrong with Appium talking to intern, or vice-versa.
Any help/insight on this would be greatly appreciated. Thanks!
I'm also in the unique position of having to run these locally, so using SauceLabs, BrowserStack, etc. is out of the question.
Got it working!!!
So the trick was to specify the proxyUrl config option.
My original conjecture was that there might have been some issues with basePath, etc., but I noticed my console.log statements that were outside of test cases were running properly.
That's when I noticed that the Safari page tried to navigate to localhost right after the test had started. This is my first time ever working with Intern or Appium or any Selenium stuff for that matter. Because you cannot access localhost (which in this case was meant for accessing the Intern server on my computer, since you're running the tests on your computer, and not on an external device), from the iPhone, I specified proxyUrl and proxyPort in the config, and specified my computer's local IP (192.168.x.x) and the Intern server port (9000). It still didn't work. I looked at the URL in Safari and noticed that it wasn't using the proxyPort field - so that the URL looked like: 192.168.x.x/__intern/clie... So I just tried to enter the port in the proxyUrl option - as that option's a string - 192.168.x.x:9000. And voila - it worked!!!
So hopefully this helps some other soul looking to have a native mobile test working with this setup!