Welcome to my blog! I’m Jan Wilhelm T. Sy, a Computer Science student from Bicol University.
Over the past few weeks, I’ve had the incredible opportunity to undertake an On-the-Job Training (OJT) at Iraya Energies.
This experience will be a pivotal part of my academic journey, allowing me to apply and expand my knowledge in real-world settings.
Over the past few weeks, I’ve had the incredible opportunity to undertake an On-the-Job Training (OJT) at Iraya Energies.
This experience will be a pivotal part of my academic journey, allowing me to apply and expand my knowledge in real-world settings.
Research
Report
Report
Welcome to my blog! This week marks the second week of my On-the-Job Training (OJT) program, focusing on the development of a chatbot. From June 17, 2024, to June 21, 2024. I engaged in the continuation of my research and trial and error approach in creating a chatbot.
This week marked the continuation of my On-the-Job Training (OJT) program, focusing on chatbot development. My main goal was to create a comprehensive report on my research and progress in developing a chatbot, as outlined in the mentorship plan. Additionally, I worked on developing a prototype to ensure our ideas were feasible at our current skill level.
This week marked the continuation of my On-the-Job Training (OJT) program, focusing on chatbot development. My main goal was to create a comprehensive report on my research and progress in developing a chatbot, as outlined in the mentorship plan. Additionally, I worked on developing a prototype to ensure our ideas were feasible at our current skill level.
ACTIVITY PERFORMED
In coordination with the mentorship plan, I had conducted and documented my research about chatbot development. I had also started other experiments in testing the functionality of the chatbot.
For more info about our research report: Click Me
For more info about our research report: Click Me
Experimenting with Retrieval Augmented Generation (RAG)
I had researched and learned about RAG as a means to retrieve possible similar context from the documents. The process is straightforward. I utilized the sampled data / documents as input, splitted it into chunks, embed it, and stored it into a Vector Database where we can retrieve chunks with similar semantics to the prompt.
For this experimentation, I researched and learned more about LangChain to implement Retrieval Augmented Generation. In addition, I experimented with different embedding models such as the sentences-transformers/All-Mini-LM-L6-V2 and the hkunlp/instructor-xl with Phi-3 Mini as the Large Language Model.
Image from LangChain
By utilizing RAG, we are essentially feeding real information from the documents to the language model in order for it to provide an inference that is related to the documents. It also helps in reducing hallucination of AI Models
Image from LangChain
For this experimentation, I researched and learned more about LangChain to implement Retrieval Augmented Generation. In addition, I experimented with different embedding models such as the sentences-transformers/All-Mini-LM-L6-V2 and the hkunlp/instructor-xl with Phi-3 Mini as the Large Language Model.
Image from LangChainBy utilizing RAG, we are essentially feeding real information from the documents to the language model in order for it to provide an inference that is related to the documents. It also helps in reducing hallucination of AI Models
Image from LangChain
Implemented Retrieval Augmented Generation in the Chatbot Prototype
As part of the experiment I had integrated the chatbot implementation in the Chatbot. It helped enhance the chatbot’s response because the language model has more information to work with, especially questions that are focused in the documents.
I had also updated the StreamLit UI to better help visualize the chunks of related text being retrieved by the RAG System. I had created 2 sub versions of the prototype, one that can save previous conversation for continuous interaction, albeit with high memory usage, and another version that does not save previous conversation to save memory usage.
I had also updated the StreamLit UI to better help visualize the chunks of related text being retrieved by the RAG System. I had created 2 sub versions of the prototype, one that can save previous conversation for continuous interaction, albeit with high memory usage, and another version that does not save previous conversation to save memory usage.
Researching Memory Optimizations
To help with the memory problem when saving the conversation for continuous interaction, I had explored methods to reduce memory usage when passing conversation history such as history summarization, Model Quantization, and Conversation Summary Memory from Langchain.
In conclusion, this week, in alignment with the mentorship plan, I made significant progress in chatbot development. I focused on Retrieval Augmented Generation (RAG) to improve response accuracy and documented my research findings in a comprehensive report. Additionally, I enhanced the Streamlit UI and explored memory optimization techniques. These efforts have resulted in a more robust and efficient chatbot prototype, setting a strong foundation for future advancements.