This content originally appeared on Level Up Coding - Medium and was authored by Skolo Online Learning
Build an application to classify Tweets, retrieved from the Twitter V2 API using the OpenAI classification API.
In this tutorial, we will build an application that will classify Tweets, retrieved from the Twitter V2 API using the OpenAI classification API. We will combine both:
- OpenAI API for AI functions to perform the classification.
- The Twitter API V2 to search for the tweets we are going to classify.
This tutorial has both a YouTube video that fully covers all the steps and the GitHub repo with the code.
GitHub Starting Code Repo:
GitHub - skolo-online/twitter-v2-api
GitHub Finishing Code Repo:
GitHub - skolo-online/Twitter-Search-API-V2-Tweet-Classification-OpenAI
Youtube Video:
Steps we are going to cover to complete Tweet Classification with OpenAI API:
- Download the starting code — the Flask Application
- Create AI training document
- Upload training document to OpenAI
- Classify tweets
- Update the Flask application with classification functionality
Download the Starting Twitter API V2 Code
Clone the repo:
git clone https://github.com/skolo-online/twitter-v2-api.git
Enter your API keys as required in the config file, install the pip libraries required and run the application. Make sure you can see the app from the Twitter tutorial, everything else builds on it and you need it to proceed with the rest of the tutorial.
Create an AI Classification Document
All AI models need to be trained to perform the task that they are built for. OpenAI provides the models, and some training but certain tasks are too specific and require targeted training.
Tweet classification is just one of those tasks that required specific training because every use case is different. In our example we will be classifying Tweets into three categories:
- Positive
- Negative
- Neutral
Therefore we need to create a file with example tweets that we classify into these groups above so we can teach/train the AI how to classify the tweets. So we will tell it, what a positive tweet looks like, same with negative and neutral.
In training data, the more variety we give it — the better the model will be at classifying the results, but with tweets specifically, the training data must be similar to the actual tweets that will be classified.
Therefore, we will be pulling the Tweets from Twitter using our API V2 Tweepy code and manually classifying them.
If you run this function:
return50Tweets(query)
At the bottom of the file, and provide a query — you will get a list of tweets saved in a JSONL file in the root of your app. Each line of the file will look like this:
{"text": "@Tbose01 I\u2019m attending a funeral in Polokwane \ud83d\ude01"}What you need to do is add a label to every line on this file like so:
{"text": "@Tbose01 I\u2019m attending a funeral in Polokwane \ud83d\ude01", "label": "Negative"}Upload training document to OpenAI for Classification Tasks
Once you have created the document, you need to upload it to OpenAI. The documentation that explains this process can be found here:
The code for doing that will look like this:
import os
import openai
import config
openai.api_key = config.OPENAI_API_KEY
def uploadClassificationDocument():
filename = 'tweet-training.jsonl'
response = openai.File.create(file=open(filename), purpose="classifications")
return response
Run the code:
def uploadClassificationDocument()
Save the response, it should look like this:
# {
# "bytes": 9401,
# "created_at": 111111111111,
# "filename": "tweet-training.jsonl",
# "id": "file-1234567890123456",
# "object": "file",
# "purpose": "classifications",
# "status": "uploaded",
# "status_details": null
# }Classify Tweets
Take note of the “id” — you will need it to run the second part of the code:
class_file = "file-1234567890123456"
def classifyTweet(query):
response = openai.Classification.create(
file=class_file,
query=query,
search_model="ada",
model="curie",
max_examples=3)
return response['label']
Update the Flask Application with Tweet Classification
The final Flask application should look like this:
The last step is to integrate the above code into our existing Flask application so that a user can enter the search query from the front-end of the application and view the results of that search — classified into the specific buckets pre-defined in the training file.
The full explanation is available on the YouTube video at the top of this tutorial.
The Flask App.py Update
Inside the app.py file, we need to create a new route for the second page like so:
This route will work as the first one on the home page, when you first access the route the default query will be run:
query = 'Polokwane lang:en -is:retweet'
The maximum number of results that will be fetched is 20, you can adjust this number between 10 and 100.
The tweets will be classified into these three python lists that will be made available to the frontend.
positiveTweets = []
negativeTweets = []
neutralTweets = []
The Python Flask HTML Template for Classification
The front-end template (class.html) will look like so:
Tutorial Comments
Feel free to leave comments, notes, or suggestions on this approach for Tweet classification.
More content at plainenglish.io. Sign up for our free weekly newsletter. Get exclusive access to writing opportunities and advice in our community Discord.
Tweet Classification with Twitter Search API V2, OpenAI API, and Python was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Skolo Online Learning
Skolo Online Learning | Sciencx (2022-01-12T22:25:35+00:00) Tweet Classification with Twitter Search API V2, OpenAI API, and Python. Retrieved from https://www.scien.cx/2022/01/12/tweet-classification-with-twitter-search-api-v2-openai-api-and-python/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.