This content originally appeared on DEV Community and was authored by peter279k
Instructions
In Taiwan, it's COVID-19 outbreak on May 13 and there're about 100 local cases every day. And it's zero local case since October.
At that moment, I'm really concerned about daily local cases so I develop this project and publish it to the GitHub. I use the cron job feature to fetch the latest COVID-19 news and publish my new web page during GitHub action running.
My Workflow
My completed workflow files are available here and it includes following YAML files:
-
fetcher.yml
, this YAML file is for fetching latest COVID-19 news and store them to be JSON file. This web crawler is written in Python. And It will be run every hour. -
web_build.yml
, this YAML file is for publishing new website and loading new JSON file. This website is written JavaScript with Vue.js framework. And it will be run every seventy minutes.
Submission Category:
This category is for DIY-Deployments and I hope developers love this and can be referenced :-).
YAML files and link to repository
Here are repository preview and completed codes about above two YAML files:
peter279k / covid19-tw-news
The Taiwan COVID-19 News
COVID-19 Taiwan News
References
Data Sources (資料來源)
- https://heho.com.tw/page/page_number/?s=%E5%85%A8%E5%8F%B0%E6%9F%93%E7%96%AB%E8%B6%B3%E8%B7%A1%E4%B8%80%E6%AC%A1%E7%9C%8B+
- http://od.cdc.gov.tw/icb/%E6%8C%87%E5%AE%9A%E6%8E%A1%E6%AA%A2%E9%86%AB%E9%99%A2%E6%B8%85%E5%96%AE.csv
- https://www.mohw.gov.tw/rss-16-1.html
- https://od.cdc.gov.tw/eic/Day_Confirmation_Age_County_Gender_19CoV.csv
- https://od.cdc.gov.tw/eic/covid19/covid19_global_cases_and_deaths.csv
-
fetcher.yml
name: Update CSV Dataset Records from Taiwan CDC website
on:
schedule:
- cron: '0 * * * *'
workflow_dispatch:
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- name: Get working copy
uses: actions/checkout@master
with:
fetch-depth: 1
- name: Set up Python 3.8
uses: actions/setup-python@v2
with:
python-version: '3.8'
- name: Install Requirements
run: pip install -r requirements.txt
- name: Update COVD-19 Labs
run: python Fetcher/LabFetcher.py
- name: Update MOHW RSS News
run: python Fetcher/MohwRssFetcher.py
- name: Update COVID-19 global cases and deaths
run: python Fetcher/GlobalCasesFetcher.py
- name: Update COVID-19 TW datasets
run: python Fetcher/DailyCovid19TW.py
- name: Copy JSON files to assets folder
run: cp datasets/*.json web/src/assets/
- name: Commit and push if it changed
run: |
git config user.name "peter279k"
git config user.email "peter279k@gmail.com"
git add -A
timestamp=$(date -u)
git commit -m "Last Commit: ${timestamp}(TW)" || exit 0
git push origin master
-
web_build.yml
name: 'Web Build and Deploy'
on:
schedule:
- cron: '10 * * * *'
workflow_dispatch:
push:
branches:
- main
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout 🛎️
uses: actions/checkout@v2.3.1
with:
persist-credentials: false
- name: Install and Build 🔧
working-directory: web
run: |
cp .env.example .env
export VUE_APP_MAPKEY=${{ secrets.VUE_APP_MAPKEY }}
sed -i -e "s/MAP_KEY/$VUE_APP_MAPKEY/g" ./.env
npm install
npm run build
cd dist
cp index.html 404.html
- name: Deploy 🚀
uses: JamesIves/github-pages-deploy-action@3.6.2
with:
GIT_CONFIG_NAME: "peter279k"
GIT_CONFIG_EMAIL: "peter279k@gmail.com"
SINGLE_COMMIT: true
GITHUB_TOKEN: ${{ secrets.TOKEN }}
BRANCH: gh-pages
FOLDER: web//dist
Additional Resources / Info
Here are some useful references about GitHub action workflows:
This content originally appeared on DEV Community and was authored by peter279k
peter279k | Sciencx (2021-12-07T15:27:28+00:00) Introduce to my Taiwan COVID-19 news website and crawler. Retrieved from https://www.scien.cx/2021/12/07/introduce-to-my-taiwan-covid-19-news-website-and-crawler/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.