Resolve: Test setup with Selenium Chrome Driver on CentOS

These are rough notes and I need to separate my resolution of Firefox from my resolution of Chrome. I also need to clean up these notes on ChromeDriver solution. I wanted to publish, however, just for future reference since I did a lot of research on the Internet and no single solution helped (but thankful for all the many solutions leading up to this solution).

=================
Who would think that setting up Selenium to run with Google Chrome on a CentOS would be so challenging? Maybe I'm just stupid?

Here's my problem, or actually my set of problems. I'm trying to run Selenium 3.13.0 with Google Chrome 61 on CentOS 7.

Problem #1: Error regarding the "webdriver.chrome.driver" system property not set.
There are so many useless articles including on StackOverflow answering this problem. Some point to Google's ChromeDriver website. Here's an example:
https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver
let me to
https://sites.google.com/a/chromium.org/chromedriver/help/chrome-doesn-t-start
but I still didn't understand clearly what this site it saying (in troubleshooting the "doesn't start" issue).
Why hasn't anyone on the Google ChromeDriver just create a dumbdown version (without all these security features)? Just a very basic browser for testing an application displays decently in a Chrome browser?
Anyhow, I found this one StackOverflow answer which actually helped.
https://stackoverflow.com/questions/18674092/how-to-implement-chromedriver-in-selenium-in-linux-platform
I was simply pointing to the wrong driver. I thought the issue was with running the selenium-chrome-driver and not actually google-chrome.

Someone should update this page to be more clear in stating the default locations of Chrome installations (as mentioned in the SO article above).
https://sites.google.com/a/chromium.org/chromedriver/getting-started

Answer (#1):
Find the location of my ChromeDriver in CentOS 7.
$ whereis google-chrome
Output: 
/usr/bin/google-chrome

Set this location as the system property expected by Selenium Chrome Driver.
NOTE: If this location is already set in your $PATH, then you shouldn't have to do this.
String pathToChrome = "/usr/bin/google-chrome";
System.setProperty("webdriver.chrome.driver", pathToChrome);

Result (#1):
The Google Chrome browser launches. But, it might not actually finish loading. Sigh.

Problem #2: Error regarding the "driver.version" system property not set.
Really! Now I need to set another system property and I need to know the version? Why is my software program so version dependent? Either I can run what I need, or I can't. Backwards compatibility, anyone?
Enough complaining, here's how I resolve this issue.

Answer (#2):
Find the version of my ChromeDriver.
$ /usr/bin/google-chrome -version
Output:
Google Chrome 61.0.3163.100

Set this version as the system property expected by Selenium Chrome Driver.
String driverVersion = "Google Chrome 61.0.3163.100";
System.setProperty("driver.version", pathToChrome);
Output:
Driver info: driver.version: ChromeDriver
    at org.openqa.selenium.remote.service.DriverService.waitUntilAvailable(DriverService.java:##);

Since "ChromeDriver" is stated as the driver.version, we know that the driver.version is not getting recognized.
https://stackoverflow.com/questions/49282494/java-net-connectexception-failed-to-connect-to-localhost-error-with-selenium-3

Result (#2):
This didn't actually solve the issue. Sigh, again. However, there's another error message. Let's move forward.

Problem #3: Error regarding "[...ERROR:sandbox_linux.cc (344)] InitializeSandbox() called with multiple threads in process gpu-process."
What is this "sandbox"?
Google Search "chrome sandbox" for answers.
A couple results:
https://blog.chromium.org/2008/10/new-approach-to-browser-security-google.html
https://chromium.googlesource.com/chromium/src/+/master/docs/design/sandbox.md

I guess I didn't know that I need to know more about browser security in order to implement a simple test case using Selenium.

Answer?
https://stackoverflow.com/questions/39041146/how-do-i-pass-arguments-to-google-chrome-when-running-selenium

First step, see if running this argument in command line will work without error.
$ /usr/bin/google-chrome --no-sandbox
Output:
[...ERROR:sandbox_linux.cc (344)] InitializeSandbox() called with multiple threads in process gpu-process.
Note:
The browser opens with a banner stating "You are using an unsupported command-line flag: --no-sandbox. Stability and security will suffer."

Okay, let's see the options available for my google-chrome program.
$ /usr/bin/google-chrome --help

Let's try the the option: --app=URL
$ /usr/bin/google-chrome --app=http://www.mywebsite.com

Step Result:
The page displays, yes! I still get the the "sandbox" error, but I'm not going to investigate further especially considering the warning message under the OPTIONS when reading the --help.
It says "Google Chrome has hundreds of undocumented command-line flags that are added and removed at the whim of the developers. Here, we document relatively stable flags." In other words, if you really need help, call Google's Customer Support. Smh. I must be missing something. Like, there must be a Chrome for Developers and a Chrome for People (and I'm using the Chrome for Developers). I see now why certain developers like Mozilla Firefox more.

Okay, now let's try adding this argument into the driver we're creating...


Answer? Nope, this didn't matter in my scenario.
Set the different attributes manually in DesiredCapabilities for ChromeDriver.
@Before
public void createDriver() {
  DesiredCapabilities chromeCapabilities = DesiredCapabilities.chrome();
  chromeCapabilities.setPlatform(Platform.LINUX);
  chromeCapabilities.setJavascriptEnabled(true);
  chromeCapabilities.setAcceptInsecureCerts(true);
  driver = new RemoteWebDriver(service.getUrl(), chromeCapabilities);
}

PROBLEM (ChromeDriver): Error WebDriverException: java.net.ConnectException: Failed to connect to localhost/127.0.0.1:####
https://stackoverflow.com/questions/49227055/webdriverexception-java-net-connectexception-failed-to-connect-to-localhost-er?rq=1
Answer: 
Update and ensure all programs are compatible among Selenium, Chrome, and Java. See details in the link highlighted in the Problem.
Currently what I see listed is...
java 8v201
selenium 3.14x
chromedriver 2.46
google-chrome v72.x  (chrome://settings/help)

Okay, since I see that I had a "driver" issue with Firefox, I'm led back to this article here.
https://stackoverflow.com/questions/18674092/how-to-implement-chromedriver-in-selenium-in-linux-platform
When I run the following command, I don't have a chromedriver install.
$ chromedriver --version
Output:
chromedriver: command not found

Download and install driver from here:
https://sites.google.com/a/chromium.org/chromedriver/downloads
https://chromedriver.storage.googleapis.com/index.html?path=2.46/

<backfill by saying we installed and got browser to display, but got other error messages now>
downloading and installing chromedriver...

Resolving error message "Chrome controlled by automated test software"
Now, to remove the banner in browser stating that Chrome is controlled by a automated test software, see this:
http://www.automationtestinghub.com/selenium-chromedriver/
https://help.applitools.com/hc/en-us/articles/360007189411--Chrome-is-being-controlled-by-automated-test-software-notification

Resolving error message "Only local connections are allowed."
If the url expected is not displaying in the Chrome browser, but instead "data;", then it's related to the error regarding "Only local connections are allowed." Try these steps to resolve.
https://stackoverflow.com/questions/25080500/when-running-webdriver-with-chrome-browser-getting-message-only-local-connect

None of these search results resolved. Instead, was led back to find the correct chromedriver version compatible with my Chrome browser.
On this Download page, the Current Release section suggests going to the Version page.
http://chromedriver.chromium.org/downloads
This Version page doesn't have the drivers I'm looking for.
http://chromedriver.chromium.org/downloads/version-selection
With more research effort, I finally found the URL I need to find the correct driver for my browser version.
https://chromedriver.storage.googleapis.com/index.html

From the bottom of this page:
http://chromedriver.chromium.org/downloads
we see the last message regarding the chromedriver version going back is version 2.35 for browser version 62-64:

ChromeDriver 2.35

Supports Chrome v62-64

Changes include:

• Supports persistent connections between client application and ChromeDriver.
• Adds more devices types for mobile emulation.
• Fixes a bug in get local storage command.
• Fixes a compatibility bug that causes JavaScript code execution to fail on some versions of Chrome.
• Uses absolute time in log file.

I have browser version 61. Hahahahaaa.
Okay, well, hopefully the driver version is 2.34 according to this chromedriver build pattern.
https://chromedriver.storage.googleapis.com/index.html?path=2.34/

Before we try this version, let's quickly recap what is installed at this point.
Currently here's what is installed:
CentOS version: 7.5.1804
Google Chrome version: 61.0.3163.100
ChromeDriver version: 2.34.522913
Selenium version: 3.13.0
Note: Selenium version is found in my POM.xml and specifies 3.13.0.

Let's double check the chromedriver version installed.
$ chromedriver --version
Output:
ChromeDriver 2.34.522913 (36222509aa6e819815938cbf2709b4849735537c)

Now, let's see what happens. Yes, I got it to work!!

Result:
Finally, got my Chrome test to work.

My Code Example:
@Test
public void testChrome() throws IOException {

    WebDriver driver;

    String pathToChrome = "/usr/bin/chromedriver"; // links to actual chromedriver dir (not google-chrome)
    System.setProperty("webdriver.chrome.driver", pathToChrome);

    // System Property addition suggested by -
        // https://wiki.saucelabs.com/display/DOCS/Test+Configuration+Options#TestConfigurationOptions-ChromeDriverVersion
    // System.setProperty("chromedriverVersion", "2.34.522913");

    // == Useful Options: Use only what is required ==
    ChromeOptions options = new ChromeOptions();
//  options.setBinary(new File(pathToChrome));
//  options.addArguments("--no-sandbox");
//  options.addArguments("--app");
//  options.addArguments("--force-app-mode");
//  options.addArguments("--kiosk");
//  options.addArguments("--start-maximized");
//  options.addArguments("--disable-web-security");

    // For info, see https://www.automationtestinghub.com/selenium-chromedriver/
    options.addArguments("disable-infobars");

    driver = new ChromeDriver(options);
    driver.get(APP_URL);
    String title = driver.getTitle();

    assertTrue("Incorrect login page title!", LOGIN_PAGE_TITLE.equalsIgnoreCase(title));
    LOG.info("Title: "+title);

    // driver.quit();
}

Cheers!


PROBLEM (FirefoxDriver): Error WebDriverException: java.net.ConnectException: Failed to connect to localhost/127.0.0.1:####
Here's an insightful SO response.
https://stackoverflow.com/questions/53107233/error-org-openqa-selenium-webdriverexception-java-net-connectexception-failed

Here are the specific versions after updating my CentOS7 box.
CentOS version: 7.5.1804
Firefox version: 60.5.0
GeckoDriver version:  ???
Selenium version: 3.13.0
Note: Selenium version is found in my POM.xml and specifies 3.13.0.

Here are my Linux commands to find the versions:
$ cat /etc/redhat-release
$ firefox --version
$ geckodriver --version

Here's some background information as I investigate how to find the GeckoDriver version:
https://github.com/mozilla/geckodriver/issues/510
https://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/firefox/FirefoxOptions.html
https://stackoverflow.com/questions/43272919/difference-between-webdriver-firefox-marionette-webdriver-gecko-driver/43920453

Here's some information regarding the compatibility issues with Firefox and Selenium:
https://www.guru99.com/first-webdriver-script.html
https://stackoverflow.com/questions/43757984/how-to-start-firefoxdriver-using-selenium-3-4-0-using-maven
http://www.automationtestinghub.com/selenium-3/
http://www.automationtestinghub.com/selenium-3-0-launch-firefox-with-geckodriver/

Answer:
So, the root issue is that I don't have the "geckodriver" installed and I need this installed. Thus, I do the following:

1. Download latest version @ https://github.com/mozilla/geckodriver/releases/tag/v0.24.0
2. Install in my /usr/bin directory.
3. Run geckodriver version command to ensure correct version appears.
4. Update my test script by removing the property for "webdriver.firefox.driver" and adding the property "webdriver.gecko.driver".

Run Command Example:
$ geckodriver --version
Output:
geckodriver 0.24.0 ( 2019-01-28)

Code Sample (using FirefoxDriver)
@Test
public void testFirefox() throws IOException {

    WebDriver driver;
    
    String pathToGecko = "/usr/bin/geckodriver";
    System.setProperty("webdriver.gecko.driver", pathToGecko);
    
    driver = new FirefoxDriver();
    driver.get(myUrl);

    assertTrue("Incorrect homepage title!", driver.getTitle().equalsIgnoreCase(myUrlTitle));

    driver.quit();
}

Run Command Sample
$ mvn -Dtest=FirefoxTest test

Result:
Selenium opens Firefox browser to the expected page, retrieves the title, compares title, and build passes.


Using Example from Selenium's ChromeDriver class api page
Now that we did dug so deep to solve, what should have been, a simple issue - let's try using this code from ChromeDriver class. Specifically, only using the argument "DesiredCapabilities.chrome()" into creating the RemoteWebDriver.

Did it work?
Nope, got the same error regarding the sandbox initialization.

Why doesn't this [code example] work?
I'm not sure yet, but I think this might be the reason. Seems to me, Selenium uses this constructor as the default for instantiating the DesiredCapabilities object where it simply needs parameters including the browser, version, and platform.
https://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/remote/DesiredCapabilities.html#DesiredCapabilities-java.lang.String-java.lang.String-org.openqa.selenium.Platform-
However, on Linux platform, there are more security measures to take and Chrome understands this by enforcing user to provide further parameters.

TRY THIS:
https://www.seleniumhq.org/docs/03_webdriver.jsp#htmlunit-driver

Notes:
Selenium API (Java)
https://seleniumhq.github.io/selenium/docs/api/java/
Code Example (ChromeDriver):
https://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/chrome/ChromeDriver.html


https://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/remote/DesiredCapabilities.html

http://chromedriver.chromium.org/capabilities
https://stackoverflow.com/questions/38335671/where-can-i-find-a-list-of-all-available-chromeoption-arguments
https://chromium.googlesource.com/chromium/src/+/master/chrome/common/chrome_switches.cc
https://chromium.googlesource.com/chromium/src/+/master/chrome/common/pref_names.cc

https://peter.sh/experiments/chromium-command-line-switches/

https://www.blazemeter.com/blog/how-to-set-up-your-automated-functional-gui-tests-with-selenium-webdriver

https://medium.com/@sahajamit/selenium-chrome-dev-tools-makes-a-perfect-browser-automation-recipe-c35c7f6a2360

https://wiki.saucelabs.com/display/public/DOCS/The+Sauce+Labs+Cookbook+Home