This content originally appeared on DEV Community 👩💻👨💻 and was authored by SeongKuk Han
Python3 Flask: Interacts with Docker Containers
Last Friday, I was talking with my co-worker at my work. Just about work, you know. He told me he had to implement a server for serving his features. Although I didn't understand what he was going to do exactly, it sounded really interesting and I wanted to try. I got really busy at my company a couple of weeks ago, I've had to write a bunch of HTML. It wasn't technically interesting(I love my work but the work was almost the same in this release). I kind of got tired of that. I want to do something new. While I was feeling like this, I had a small talk with the coworker.
He was going to do
- Make an API server that gets an image file
- Execute a docker container and pass the image file to the container and the container will create the text file that is generated from the image file
- Execute another docker container and pass the text
- The API Server knows that the container has done with its work and notify to another server(The backend that servers the data to users).
It sounds interesting, ha?, I decided to implement the system on the weekend. I thought it's going to be fun. But it's been really a long time since I used Python and Docker. So, I first had to learn these. During the weekends, I couldn't learn all the things, so that, I would use the minimum skills for implementing the system. If you're a python developer or working with the docker, it might be not in your mind at some points. Please, comment your opinion comment below. I'd love that.
Anyways, Let's get started!
Prerequisite
- pipenv
Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world. Windows is a first-class citizen, in our world.
When I first use python a few years ago, I used conda or venv for managing packages. When I searched for package management, I saw this. It seems likenpm
, I didn't know there is like this tool. I thought it must be worth it to try.
- docker
Docker is a platform designed to help developers build, share, and run modern applications. We handle the tedious setup, so you can focus on the code.
As I introduced at first, this is the main in this post.
- Flask
Flask is a web application framework written in Python. It was developed by Armin Ronacher, who led a team of international Python enthusiasts called Poocco.
To implement a simple API web server, I usedFlask
. there were other libraries like Fast API. Since I used to Flask a little, I went with Flask For reducing my hours.
This is the process that I'm going to implement.
The API server receives a file from the user
Container A processes A file then saves it to the B File
Container B processes the B File then saves it to the C File
Users can see the result through API
Python Apps and Docker Images
I made two images and uploaded to my them repository in docker-hub.
A tokenizer and word-counting app.
[Tokenizer]
import sys, json, os, requests
from textblob import TextBlob
def extract_nouns(text):
blob = TextBlob(text)
filtered_tags = list(filter(lambda tag: tag[1] == "NN", blob.tags))
nouns = list(map(lambda tag: tag[0], filtered_tags))
return nouns
def read_file(path):
with open(path) as f:
contents = f.read()
return contents
def save_data(path, data):
with open(path, "w") as f:
json.dump(data, f)
def get_filename_from_path(path):
return os.path.splitext(os.path.basename(path))[0]
def notify_done(url, file_name):
requests.get(f"{url}/docker/tokenizer_done?file_name={file_name}")
if __name__ == "__main__":
if len(sys.argv) < 4:
print("You must pass file path as an argument")
print("python3 main.py [file path to read] [dir to save] [notification api]")
print("Example) python3 main.py ./test.txt ./ http://host.docker.internal:20000")
sys.exit()
api_url = sys.argv[3]
file_path = sys.argv[1]
file_name = get_filename_from_path(file_path)
target_path = os.path.join(sys.argv[2], file_name + ".json")
text = read_file(file_path)
nouns = extract_nouns(text)
save_data(target_path, {"nouns": nouns})
notify_done(api_url, file_name)
print("Done")
[word-counting]
import sys, json, os, requests
def count_word(nouns_list):
count_dict = dict()
for noun in nouns_list:
if noun in count_dict:
count_dict[noun] += 1
else:
count_dict[noun] = 1
return count_dict
def load_data(path):
with open(path) as f:
json_data = json.load(f)
return json_data
def save_data(path, data):
with open(path, "w") as f:
json.dump(data, f)
def get_filename_from_path(path):
return os.path.splitext(os.path.basename(path))[0]
def notify_done(url, file_name):
requests.get(f"{url}/docker/word_count_done?file_name={file_name}")
if __name__ == "__main__":
if len(sys.argv) < 4:
print("You must pass file path as an argument")
print("python3 main.py [file path to read] [dir to save] [notification api]")
print("Example) python3 main.py ./test.txt ./ http://host.docker.internal:20000")
sys.exit()
api_url = sys.argv[3]
file_path = sys.argv[1]
file_name = get_filename_from_path(file_path)
target_path = os.path.join(sys.argv[2], file_name + ".json")
json_data = load_data(file_path)
count_dict = count_word(json_data["nouns"])
save_data(target_path, {"result": count_dict})
notify_done(api_url, file_name)
print("Done")
For running the apps from the API server, I built both python files with below Dockerfiles.
[Tokenizer]
FROM python:3.9
WORKDIR /app
COPY . .
RUN pip install pipenv
RUN pipenv install
RUN pipenv run python3 -m textblob.download_corpora
ENTRYPOINT ["pipenv", "run", "python3", "./main.py"]
[word-counting]
FROM python:3.9
WORKDIR /app
COPY . .
RUN pip install pipenv
RUN pipenv install
ENTRYPOINT ["pipenv", "run", "python3", "./main.py"]
API Server
This is the main code and it's kind of simple.
from flask import Flask
from dotenv import load_dotenv
load_dotenv()
app = Flask(__name__)
import routes
Routes
[routes/docker.py]
import os
from flask import jsonify, request
from server import app
from lib import docker, json
result = []
@app.route('/docker/tokenizer_done')
def get_tokenizer_done():
file_name = request.args.get("file_name")
docker.run_word_count_container(file_name)
return "run a word_count container"
@app.route('/docker/word_count_done')
def get_word_count_done():
file_name = request.args.get("file_name")
json_data = json.load_data(
os.path.join(os.getenv("SHARED_VOLUME_PATH"),
"word_count_output",
f"{file_name}.json"
))
result.append(json_data)
return "all works done"
@app.route('/docker/result')
def get_result():
file_name = request.args.get("file_name")
return jsonify({
"result": result
})
[routes/upload.py]
import os
from flask import jsonify, request
from werkzeug.utils import secure_filename
from server import app
from lib import docker
@app.route("/upload", methods=["POST"])
def upload_file():
f = request.files["file"]
file_name = secure_filename(f.filename)
f.save(os.path.join(os.getenv("SHARED_VOLUME_PATH"), "input", file_name))
docker.run_tokenizer_container(file_name)
return "succeed to upload"
[routes/__init__.py]
from routes import docker, upload
[lib/docker.py]
import os
API_URL = os.getenv("API_URL")
VOLUME_ROOT_PATH = os.getenv("SHARED_VOLUME_PATH")
RUN_TOKENIZER_CONTAINER = 'docker run -it --add-host=host.docker.internal:host-gateway -v "' + VOLUME_ROOT_PATH + ':/shared_volume" hskcoder/tokenizer:0.2 /shared_volume/input/{FILE_NAME_WITH_EXTENSION} /shared_volume/tokenizer_output ' + API_URL
RUN_WORD_COUNT_CONTAINER = 'docker run -it --add-host=host.docker.internal:host-gateway -v "' + VOLUME_ROOT_PATH + ':/shared_volume" hskcoder/word_count:0.2 /shared_volume/tokenizer_output/{FILE_NAME_WITHOUT_EXTENSION}.json /shared_volume/word_count_output ' + API_URL
def run_tokenizer_container(file_name):
print(RUN_TOKENIZER_CONTAINER.format(
FILE_NAME_WITH_EXTENSION = file_name
))
os.popen(RUN_TOKENIZER_CONTAINER.format(
FILE_NAME_WITH_EXTENSION = file_name
))
def run_word_count_container(file_name):
os.popen(RUN_WORD_COUNT_CONTAINER.format(
FILE_NAME_WITHOUT_EXTENSION = file_name
))
[iib/json.py]
import json
def load_data(path):
with open(path) as f:
json_data = json.load(f)
return json_data
This app read environment variables from .env
file, so you need to set up like this.
The below variables just fit my system.
API_URL=http://host.docker.internal:20000
ROOT_PATH=C:\Users\hskco\OneDrive\바탕 화면\stuff\docker\api
SHARED_VOLUME_PATH=C:\Users\hskco\OneDrive\바탕 화면\stuff\docker\api\shared_volume
You can run the server with this script
python3 -m pipenv run flask run -h 0.0.0.0 --port 20000
Before running this command, you need to set an environment variable FLASK_APP
.
Since I was developing in Windows, I ran this command in api
dir.
$env:FLASK_APP = './server.py'
If you enter http://127.0.0.1/docker/result
you will see this page.
Let's send a file to the API server and see the result.
Conclusion
It was really fun. I've learned a lot of things.
Regardless of your position, I think It would be really good to try anything you're interested in.
This example is a really beginner level though, I mean.
It should've considered like
- Authorization and Security (In this example, the API server exposes all routes to the public)
- Communication between containers (I adopted RestfulAPI for interacting between containers, however there must be better ways)
- Managing containers (When containers that have done, they need to be deleted. There are also tools for this like Kubernetes)
- Deployment for production
there are many things that I didn't mention and you need to consider. (I respect for backend developers) I was just focusing on implementing the system, If I tried to make it perfect, I couldn't write this article. Haha,, I have to go back to work tomorrow...
Anyways, it was worth it, it's definitely true :)
I hope it'll be helpful for someone.
References
Github Source Code
Python
- https://realpython.com/python-modules-packages/
- https://www.geeksforgeeks.org/reading-and-writing-json-to-a-file-in-python/
Pipenv
Flask
Docker
- https://www.atlassian.com/microservices/microservices-architecture/kubernetes-vs-docker
- https://blog.payara.fish/do-you-need-kubernetes
- https://docs.docker.com/build/building/multi-stage/
- https://textblob.readthedocs.io/en/dev/quickstart.html#noun-phrase-extraction
- https://www.baeldung.com/ops/dockerfile-run-cmd-entrypoint
- https://www.howtogeek.com/devops/how-to-connect-to-localhost-within-a-docker-container/
This content originally appeared on DEV Community 👩💻👨💻 and was authored by SeongKuk Han
![](https://www.radiofree.org/wp-content/plugins/print-app/icon.jpg)
SeongKuk Han | Sciencx (2022-10-30T18:16:45+00:00) Python Flask: Interacts with Docker Containers. Retrieved from https://www.scien.cx/2022/10/30/python-flask-interacts-with-docker-containers/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.