This content originally appeared on Level Up Coding - Medium and was authored by Javier Martín
So, as a FullStack intern, I was asked to integrate an AI model into a web app to start getting in touch with the technologies we were using (React.js and FastAPI mainly).
At that moment, I was shocked. I had never tried to do that on my own. But you only learn when trying so I gave it a try.
Implementing the API route for the client was very easy. I had been working with cloud services for a while, so I was not scared.
I read the docs.
I coded.
Here you have the first result:
router = APIRouter()
class Prompt(BaseModel):
"""Class for sending or receiving messages"""
prompt: str
@router.post("/ai/chat")
async def chat(prompt: Prompt):
"""In charge of executing and obtaining the connection with the model"""
api_key = os.getenv("YOUR_API_KEY")
api_url = os.getenv("OPENAI_API_URL")
if not api_key or not api_url:
raise HTTPException(status_code=500, detail='API KEY/URL not good')
client = AzureOpenAI(api_key=api_key, azure_endpoint=api_url)
# response from the model
response = client.chat.completions.create(
model="gpt-35-turbo-4k-0613",
messages=context
)
# Extract the model's response
model_response = response.choices[0].message.content
return {"response": model_response}
It worked. Not on the first try, but in the end it worked.
The main problem was that the AI integrated does not have any memory. That sucks, especially when you are trying to maintain a fluid conversation with it.
And to be honest, I didn’t know how to implement this. I was lost. But after researching for a while, I came up with an idea: what if I just send again the entire conversation plus the new message?
I discovered that that was the way. An AI just by itself can’t remember what was told to it (at least not OpenAI). But if I casually remembered what we were talking about, we wouldn’t have any trouble.
Well, that was the idea. And it was a good idea. What did I do? I coded it.
I just added a dictionary that was in charge of saving the user and the AI’s messages depending on the local time hour when the message was sent (may God bless JavaScript).
Then, I would only need to pass that entire prompt to the model, crossing my fingers and praying for not receiving a weird output.
router = APIRouter()
class Message(BaseModel):
"""Class representing the message of a conversation"""
role: str # either 'user' or 'assistant'
content: str
class Prompt(BaseModel):
"""Class for sending or receiving messages"""
prompt: str
conversation_id: str
conversations: Dict[str, List[Message]] = {}
@router.post("/ai/chat")
async def chat(prompt: Prompt):
"""In charge of executing and obtaining the connection with the model"""
api_key = os.getenv("YOUR_API_KEY")
api_url = os.getenv("OPENAI_API_URL")
if not api_key or not api_url:
raise HTTPException(status_code=500, detail='API KEY/URL not good')
client = AzureOpenAI(api_key=api_key, azure_endpoint=api_url)
# Initialize conversation if not present
if prompt.conversation_id not in conversations:
conversations[prompt.conversation_id] = []
# Add the new user message to the conversation history
conversations[prompt.conversation_id].append(Message(role='user', content=prompt.prompt))
# Create the context for the API request
context = [{"role": msg.role, "content": msg.content} for msg in conversations[prompt.conversation_id]]
# Request completion from the model
response = client.chat.completions.create(
model="gpt-35-turbo-4k-0613",
messages=context
)
model_response = response.choices[0].message.content
conversations[prompt.conversation_id].append(Message(role='assistant', content=model_response))
return {"response": model_response}
Basically, this works because the AI is not stupid. It understands Natural Language and I would even say that has a little bit of common sense. Because the prompt was well structured, the model was able to identify that it was receiving a history of the previous messages.
But I must be critical of my code. If not, this manuscript wouldn’t be useful for you. And believe me, this code has a lot of problems to be fixed:
- As it saves the conversation by the local time of the machine, we have several keys corresponding to the same conversation. This means that we have to iterate even more through the dictionary, making it efficient (time complexity of O(n)).
- It works only for one user. As I am just me and the local machine; I don’t care especially about this one, but it is important to take into account. This could be easily resolved by adding another parameter for the user who is chatting with the AI.
- It uses a dictionary, whereas a database would be more efficient and “correct”.
So, this has been the work of an entire morning. I know, it is not that impressive. But just remember that I have 19 years and no more than 2 years of experience in programming. Give me some time, guys.
Keep coding,
Javier Martín
Mastering AI Context: The Code was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Javier Martín
Javier Martín | Sciencx (2024-08-04T17:28:38+00:00) Mastering AI Context: The Code. Retrieved from https://www.scien.cx/2024/08/04/mastering-ai-context-the-code/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.