How to check URL for 404 using Selenium WebDriver?

We can check the URL for 404 using Selenium webdriver. A 404 check is actually done to verify if there are broken links in a page. On clicking such a link, we shall not be directed to the correct page.

A broken link can occur due to the following reasons −

• The landing page is no longer available.

• Some parts of the URL have been modified.

• An incorrect URL has been specified on the page.

• Firewall or geolocation limitations.

A URL can have the following status code −

• 5XX − Represents issue in server.

• 4XX − Represents resource cannot be determined.

• 3XX − Represents redirection.

• 2XX − Represents correct condition.

Thus we see that with only 2XX status code we can have a correct URL. We shall send a HTTP request and analyse its response code for all the links on the page.

Example

import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebElement; import org.openqa.selenium.firefox.FirefoxDriver; import java.util.concurrent.TimeUnit; import java.net.HttpURLConnection; import java.net.URL; import java.net.URLConnection; import java.util.List; public class BrokenURL{    public static void main(String[] args) throws    InterruptedException{       System.setProperty("webdriver.gecko.driver",          "C:\Users\ghs6kor\Desktop\Java\geckodriver.exe");       WebDriver driver = new FirefoxDriver();       // wait of 5 seconds       driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS);       driver.get("https://www.google.com/");       //get list of elements with anchor tag       List<WebElement> l = driver.findElements(By.tagName("a"));       //iterate links       for(int j=0; j<l.size(); j++) {          WebElement e = l.get(i);          //get URL of links with getAttribute()          String u = e.getAttribute("href");          // to catch MalFormedURLException          try{             //object of URL class             URL link = new URL(u);             // establish connection URL object             HttpURLConnection c = (HttpURLConnection)link.openConnection();             //have timeout             c.setConnectTimeout(1000);             // connection began             c.connect();             //getResponseCode() to obtain response code             if(c.getResponseCode()== 200) {                System.out.println(u+" − "+ c.getResponseMessage());             }             if(c.getResponseCode()== 404) {                System.out.println(u+" − "+c.getResponseMessage());             }          }          catch (Exception ex) {          }       }    } }

Output