Python web scrapping on GCP

Content

build container images.
tag images.
upload images to GCP.
create services.

build container images.

selenium.
web scrapping.

tag images.

image with GCP pattern.

docker pull selenium/standalone-chrome
# tag wi…


This content originally appeared on DEV Community and was authored by goffity

Content

  1. build container images.
  2. tag images.
  3. upload images to GCP.
  4. create services.

build container images.

  • selenium.
  • web scrapping.

tag images.

  • image with GCP pattern. 
docker pull selenium/standalone-chrome
# tag with pattern $GCP_REGISTRY/$PROJECT_NAME/$IMAGE_NAME
docker tag selenium_standalone-chrome asia.gcr.io/$PROJECT_NAME/selenium-standalone-chrome
# push image
docker push asia.gcr.io/$PROJECT_NAME/selenium-standalone-chrome

upload images to GCP.

  • authentication on browser. 
gcloud auth login
  • setup project. 
gcloud config set project $PROJECT_NAME

create services.

  • Create from container images. 
  # create selenium service.
gcloud run deploy selenium-chrome --image asia.gcr.io/$PROJECT_NAME/selenium_standalone-chrome --port 4444 --memory 2G --region asia-southeast1 --platform managed
  • Create from cloud build. 
# gcloud builds submit --tag asia.gcr.io/$PROJECT_NAME/$IMAGE_NAME

gcloud builds submit --tag asia.gcr.io/$PROJECT_NAME/web-scrpping
# deploy service
gcloud run deploy selenium-chrome --image asia.gcr.io/$PROJECT_NAME/selenium_standalone-chrome

Reference:
https://www.roelpeters.be/how-to-deploy-a-scraping-script-and-selenium-in-google-cloud-run/
https://dev.to/googlecloud/using-headless-chrome-with-cloud-run-3fdp


This content originally appeared on DEV Community and was authored by goffity


Print Share Comment Cite Upload Translate Updates
APA

goffity | Sciencx (2021-07-05T16:42:28+00:00) Python web scrapping on GCP. Retrieved from https://www.scien.cx/2021/07/05/python-web-scrapping-on-gcp/

MLA
" » Python web scrapping on GCP." goffity | Sciencx - Monday July 5, 2021, https://www.scien.cx/2021/07/05/python-web-scrapping-on-gcp/
HARVARD
goffity | Sciencx Monday July 5, 2021 » Python web scrapping on GCP., viewed ,<https://www.scien.cx/2021/07/05/python-web-scrapping-on-gcp/>
VANCOUVER
goffity | Sciencx - » Python web scrapping on GCP. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2021/07/05/python-web-scrapping-on-gcp/
CHICAGO
" » Python web scrapping on GCP." goffity | Sciencx - Accessed . https://www.scien.cx/2021/07/05/python-web-scrapping-on-gcp/
IEEE
" » Python web scrapping on GCP." goffity | Sciencx [Online]. Available: https://www.scien.cx/2021/07/05/python-web-scrapping-on-gcp/. [Accessed: ]
rf:citation
» Python web scrapping on GCP | goffity | Sciencx | https://www.scien.cx/2021/07/05/python-web-scrapping-on-gcp/ |

Please log in to upload a file.



There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.

Related Posts