This content originally appeared on DEV Community and was authored by Ramki Pitchala
Crunch Pokemon Data with Python and Deta Base
Setup and work on a datastore faster than Pikachu
 on [Unsplash](https://unsplash.com/photos/DypO_XgAE4Y)](https://res.cloudinary.com/practicaldev/image/fetch/s--B6RhuBDl--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://cdn-images-1.medium.com/max/2000/0%2Ac2l5Bfc5dK9J3Tox)
This article was originally posted on Medium. If you prefer reading it from there, please do check it out.
Introduction
Performance is a must when it comes to building software. However, in certain situations, the speed to set up and integrate new services for proof of concept evaluation or infrastructure is overlooked. Specifically, in the realm of databases, there are many options, but I find Deta to be most seamless to set up and use. Deta offers Deta Base (I will refer to it as Base), a NoSQL data store optimized for developer simplicity. In this article, I aim to show how to set up a Deta project and interact with your Base to store and manipulate Pokemon data.
Let’s get started!
Agenda
Setup
CRUD
Querying
Setup
To begin, navigate to Sign Up, create a new account, and verify your account. Once you sign in, you should be on the following page:
Click the arrow in the top left, we will create a new project from scratch.
Enter a name of your choice and hit create! A popup with your project key and project id will appear. Make sure you save the key!
With the project key saved, create a new directory, walk into it and run the following command.
pip install flask deta
Flask is a web framework that we will use to create endpoints to listen to incoming requests. Based on the requests, we will interact with our remote database using the deta package.
Here is how the folder structure will look like:
app.py
config.py
In config.py, we will store our project key
DETA_KEY= "YOUR_COPIED_KEY"
In our project, we will create a Base called pokemon and utilize it to store, access, and manipulate Pokemon data. Before that, let’s go over the schema for a record in pokemon :
Each record in a Base must correspond to a unique identifier called key. When we insert a Pokemon record into our Base, we will provide name as our key. As a result, if we needed to get information on Charizard, we just have to ask pokemon to find the associated record with Charizard as its key.
In app.py, let’s set up the Flask app and our connection to our Base, pokemon.
from config import DETA_KEY
from flask import Flask, request
from deta import Deta
from json.decoder import JSONDecoder
app = Flask(__name__)
deta_project = Deta(DETA_KEY)
db = deta_project.Base("pokemon")
decoder = JSONDecoder()
if __name__ == "__main__":
app.run()
All the data access and manipulations will occur through db.
Our setup is now complete! Onto the CRUD!
CRUD
CRUD is an acronym for creating, reading, updating, and deleting data in a database. We will explore how to perform each of the above operations in our pokemon Base. All the work will be done in app.py.
Insertion
Deta provides 2 ways of inserting data. The first is through the put method. put is the faster method of insertion. If you call put on a record that already exists in the base, put will overwrite the record. In contrast, insert is 2x slower than put. In the case that you try to insert into the Base with an already existing key, it will throw an error.
Let’s create an endpoint to insert a new Pokemon with put.
Here is what the endpoint would look like using insert :
Deta also supports inserting multiple records at the same time with put_many. According to the documentation, it is possible to insert at most 25 items into the Base in a single call (Deta Base SDK).
Let’s test out what we have so far. Let’s insert the data for Pichu into the Base via an HTTP POST through /pokemon on Postman.
Once you hit send, navigate back to Deta.
Click on your Base under the Bases section. You will now be able to view the data in your Base.
As you can see, Pichu is in our pokemon Base!
Access
We can use the get method to access the record of a given key.
Since our Pokemon name is the key, we can directly access its record by providing its name. We can test it with the following GET request.
Updation
As stated previously in Insertion, the put function can be used for overwriting records. However, put completely overwrites the record and can remove prior fields that are not part of the new updates. As result, if we want to partially update the record, we can use the update function. In fact, update also allows for fine-grained updations like incrementing values and appending, prepending, and removing elements in a list (Deta Base SDK).
Let’s make sure updating works with an example. First, I will insert the following data for Charizard.
{
"region": "Johto", # Charizard is from Kanto region
"name": "Charizard",
"height": 1.7,
"weight": 90.5,
"type": ["Fire"], # Charizard also Flying type
"evolution": ""
}
region should be Kanto, and Charizard is also a flying type. Let’s update Charizard with a POST request to /pokemon/update/Charizard.
After a GET request to /pokemon/Charizard, it is clear that region and type are updated.
Deletion
Deta provides delete, a function that takes in a key and deletes the record associated with the key.
If I wanted to delete Charizard, I would make an HTTP DELETE request to /pokemon/delete/Charizard.
We have now explored how to insert, access, update and delete from our Base. Let’s learn how to query our Base.
Querying
Prior to querying, make sure to fill up your Base with some more Pokemon.
Querying is done through the fetch method. To elaborate, fetch takes in a query or a list of queries and accumulates a list of records whose fields match the query or queries.
A query is nothing more than a dictionary where the mapping between the keys and values represents the query condition. For instance, suppose I wanted to get Blastoise’s record with a query instead of get, here is how it would work:
query = {"name": "Blastoise"}
results = next(db.fetch(query))
blastoise = results[0]
We accumulate all the records with name equal to Blastoise.
It is also possible to query based on inequalities. For instance, we can query for all Pokemon that weigh greater than 100kg and are less than 1 meter tall.
query = {"weight?gt": 100, "height?lt": 1}
pokemon = next(db.fetch(query))
We can append “?gt” and “?lt” at the end of numerical fields to query for records with respective values greater than or less than a threshold. There are a lot more suffixes that can be added to the end of a query field, so I recommend reading the documentation for your specific use case (Deta Base SDK).
Let’s create an endpoint that will return Pokemon which are of a parameter type.
“?contains” checks if a provided query element exists in the list associated with the field. To provide an example, if I wanted all the fire-type Pokemon, I would send a GET request to /pokemon/type/Fire.
There are 13 Pokemon returned from the above request.
This is a little overwhelming since this query will return all the fire-type Pokemon in our Base. Deta also provides us with the ability to limit the number of query results with buffer. Let’s set the buffer arg in fetch to 2.
Now, we only get two records back.
Finally, by changing the pages argument in fetch, it is possible to spread the result data over multiple pages.
Conclusion
My primary goal with this writing was to shed greater light on a database that you can set up and work on in the blink of an eye. It took hardly much time to create the project and based on the above examples, CRUD and Querying are as simple as they can get. For these reasons, Deta Base is perfect for proof of concepts, serverless applications, hackathons, and many more situations and projects that require simplicity and speed.
Resources
This content originally appeared on DEV Community and was authored by Ramki Pitchala
Ramki Pitchala | Sciencx (2021-04-18T21:09:50+00:00) Crunch Pokemon Data with Python and Deta Base. Retrieved from https://www.scien.cx/2021/04/18/crunch-pokemon-data-with-python-and-deta-base/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.