Embed Shiny

Please wait...

Web Scraping Tutorial 4- Getting the busy information data from Popular time page from Google

Popular Times

In this blog we will try to scrape the busy text from popular times section in google maps.

Step 0: Importing the libraries

Step 1: Start a headless Firefox browser

driver <- rsDriver( 
    browser = c("firefox"), 
    chromever = NULL, 
    verbose = F, 
    extraCapabilities = list("firefoxOptions" = list(args = list("--headless"))) 
) 
web_driver <- driver[["client"]] 

# This link contains Restaurant links for Cedele
nm<-"cedele restaurant "
ad_url<-str_c("https://www.google.co.id/maps/search/ ",nm)

web_driver$navigate(ad_url)

The page looks like the below image

Step 2: Get the url(links) of one of these restuarants to start with

In order to gt the link, we have to right click on the first store and click on inspect

If you right click on the first restaurant, then the link to the restaurant is at a tag

Just right click on this element and get the xml path

# the xml path of the link 
nm1<-"/html/body/div[2]/div[3]/div[8]/div[9]/div/div/div[1]/div[2]/div/div[1]/div/div/div[1]/div[1]/div[3]/div/a"
nm1

## [1] "/html/body/div[2]/div[3]/div[8]/div[9]/div/div/div[1]/div[2]/div/div[1]/div/div/div[1]/div[1]/div[3]/div/a"

<>br

Using the xml component to access the link

link_restuarants <- web_driver$findElements(using = "xpath", value = nm1)
rest_url<-link_restuarants[[1]]$getElementAttribute("href")[[1]]
rest_url

## [1] "https://www.google.co.id/maps/place/Cedele+Bakery+Kitchen+-+The+Woodleigh+Mall/data=!4m7!3m6!1s0x31da1793c89df043:0xf72df23d7aafbfac!8m2!3d1.3379161!4d103.8723492!16s%2Fg%2F11v05s7v9f!19sChIJQ_CdyJMX2jERrL-vej3yLfc?authuser=0&hl=en&rclk=1"

Navigating to the URL

web_driver$navigate(rest_url)

Step 3: Scrolling Down to the popular times section(*****MOST IMPORTANT)

Google maps, reviews and popular times work very different as compared to other websites when you have to scroll up or down.In most websites, you can just do a scroll down command and the page will scroll down.But in google reviews or google maps for example, there are essentially two pages and you have to scroll down/up in the left section. This is shown below.