This content originally appeared on DEV Community 👩‍💻👨‍💻 and was authored by Anuoluwapo Balogun
Purpose Of Project :
To analyze how each fin tech apps is being used and widely accepted by Nigerians, as well as understanding the impact fin tech companies has made on Nigerians finances and Nigeri's Tech Sector.
Project Process:
- Web scraping data from playstore
- Performing Exporatory Data Analysis On data
- Visualizing data on Power BI and Tableau
Project Tools:
- Notebook
- Power BI
- Tableau
Importing Necessary Libraries
import time
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
Accessing PlayStore Using Selenium
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get('https://play.google.com/store/search?q=fintech%20in%20nigeria&c=apps')
time.sleep(10)
Getting All The Apps In The Required Page
SCROLL_PAUSE_TIME = 5
# Get scroll height
last_height = driver.execute_script("return document.body.scrollHeight")
time.sleep(SCROLL_PAUSE_TIME)
while True:
# Scroll down to bottom
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
# Wait to load page
time.sleep(SCROLL_PAUSE_TIME)
# Calculate new scroll height and compare with last scroll height
new_height = driver.execute_script("return document.body.scrollHeight")
if new_height == last_height:
break
last_height = new_height
Scraping App Links
links_fintech = []
elems = driver.find_elements_by_xpath("//a[@href]")
for elem in elems:
if "details?id" in elem.get_attribute("href"):
links_fintech.append((elem.get_attribute("href")))
links_fintech = list(dict.fromkeys(links_fintech))
Scraping The Necessary Informations From Each App
list_all_elements = []
for iteration in links_fintech:
try:
driver.get(iteration)
print(iteration)
time.sleep(3)
header1 = driver.find_element_by_tag_name("h1")
downloads = driver.find_elements_by_class_name("htlgb")
list_downloads = []
for x in range (len(downloads)):
if x % 2 == 0:
list_downloads.append(downloads[x].text)
titles = driver.find_elements_by_class_name("AHFaub")
comments = driver.find_element_by_class_name("EymY4b")
list_elements = [iteration,header1.text, list_downloads.append(downloads[x].text), comments.text.split()[0]]
for x in range (len(titles)):
if titles[x].text == "Download":
list_elements.append(list_others[x])
if titles[x].text == "Developer":
for y in list_others[x].split("\n"):
if "@" in y:
list_elements.append(y)
break
list_all_elements.append(list_elements)
except Exception as e:
print(e)
https://play.google.com/store/apps/details?id=ng.gov.cbn.speed.wallet.temp
https://play.google.com/store/apps/details?id=com.oxloan.loan.money.credit.nigeria
https://play.google.com/store/apps/details?id=com.cowrywise.android
https://play.google.com/store/apps/details?id=com.fastmoney.loan.credit.cash.nigeria
https://play.google.com/store/apps/details?id=com.wemabank.alat.prod
https://play.google.com/store/apps/details?id=com.ikonik.lamp_pay......
Creating A CSV File For Scraped DataFrame
import pandas as pd
df = pd.DataFrame(list_all_elements,columns=['URL', 'Name', 'downloads', 'install'])
df_1 = df.to_csv('fintech_playstore.csv', header = True, index=False, encoding="utf-8")
df_1
https://www.danielherediamejias.com/scraping-google-play-store-with-python-and-selenium/
https://play.google.com/store/search?q=fintechs%20in%20nigeria&c=apps
This content originally appeared on DEV Community 👩‍💻👨‍💻 and was authored by Anuoluwapo Balogun
Anuoluwapo Balogun | Sciencx (2022-09-27T11:34:40+00:00) Scrapping Google Playstore. Retrieved from https://www.scien.cx/2022/09/27/scrapping-google-playstore/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.