OpenAI Completion Models
Text vs. Chat Completion Models
When going through the documentation for the LangChain application programming interface (API), we come across text completion models and chat completion models as distinct ways to interact with OpenAP models. The LangChain API gives the LLM developer two Python objects to use:
Interestingly, the v0.2 API page for OpenAI comes with a caveat: You are currently on a page documenting the use of OpenAI text completion models. The lastest and most popular OpenAI models are chat completion models. Unless you are specifically using gpt-3.5-turbo-instruct, you are probably looking for [the chat completion model] instead.
OpenAIfor text completionChatOpenAIfor chat completion
As the LangChain documentation states, each one “offers a spectrum of models with different levels of power suitable for different task.” But what exactly is the difference between the two?
In her book Building LLM Powered Applications, Valentina Alto points out that:
A [text] completion model is a type of LLM that takes a text input and generates a text output, which is called a completion. The completion model tries to contiue the prompt in a coherent and relevant way, according to the task and the data it was trained on. For example a completion model can generate summaries, translations, code, lyrics and more, depending on the prompt.
A chat model is a special kind of completion model that is designed to generation conversational responses. A chat model takes a list of messages as input, where each message has a role (either system, user, or assistant) and content. The chat model tries to generate a new message for the assistant role, based on the previous messages and the system instruction.
The main difference between completion and chat models is that completion models expect a single text input as a prompt, while chat models expect a list of messages as input.
The blog site Dreamsavior gives a more nuanced comparison. Let’s summarize here by laying out the pro and con of each completion model.
It’s worth noting that in the same blog post, Dreamsavior also presents examples of English-to-French translation as further elaborations.
| Text Completion | Advantage: Full control over input and output. Customization of settings such as temperature, and top_p. Widest use, from summarization and paraphrasing to ranking, classification and sentiment analysis. |
Disadvantage: Requires careful prompt design to achieve fluent natural sounding results. Depending on the quality of the prompt, the output text may be inconsistent or incomplete. |
| Chat Completion | Advantage: Consistent persona and tone leads to natural and fluent output. Leverage conversation history to build context and continuity for the model to use. | Disadvantage: Requires more input and processing power. Works best with elaborate input messages. Token limit constraint. |
Python-Langchain: Comparing Prompt Outputs
LangChain is a popular framework for building LLM applications, belonging to a small group that includes Semantic Kernel and AutoGen, just to name a few. LangChain is too big a topic to cover here. Instead let’s take some highlights from an AWS article that explains what the framework is all about:
Jane Huang shares a great article comparing LangChain, Semantic Kernel and AutoGen frameworks.
- LangChain is an open source framework for building applications based on LLMs.
- It provides tools and abstractions to improve the customization, accuracy and relevancy of the information the models generate.
- Developers use LangChain components to build new prompt chains or customize existing templates.
- LangChain also has components that allow LLMs to access new data sets without retraining (cf. retrieval augmented generation).
Below are the output from two Python scripts that use LangChain APIs to interact with OpenAI models. In the first, we use OpenAI, the API for the text completion model. The section following this is the output when we use ChatOpenAI, the API for the chat completion model.
Here are the Python code for the langchain-openai-text-completion.py and langchain-openai-chat-completion.py respectively.
Comparing the two techniques, we see differences in the LLM parameters the two APIs expose. (The text completion API gives us more LLM parameters to customize.) We also note that the text completion model, which runs on the default gpt-3.5-turbo-instruct model, is limited to its September 2021 training set. Lastly, its clear that in terms of tone and nuance, the output of the two completion models vary widely, particularly in the case of the “Tech Entrepreneur” prompt.
=== Text Completion LLM Parameters ===
temperature 0.7
top_p 1
frequency_penalty 0
presence_penalty 0
n 1
logit_bias {}
max_tokens 512
=== Funny Lines ===
Why couldn't the bicycle stand up by itself? Because it was two-tired!
=== The Bard Sayeth ===
"To thine own self be true, and it must follow, as the night the day, thou canst not then be false to any man." - Hamlet
=== Yen:USD Exchange ===
As of September 24, 2021, the exchange rate for Japanese Yen to US Dollar is 0.0091 USD for 1 JPY.
=== Tech Entrepreneur ===
Ruby's typical working day starts early in the morning. After waking up, she takes a quick shower and has a cup of coffee while checking her emails and responding to urgent messages from her team or clients.
At 9 AM, Ruby heads to her office, which is located in a co-working space. She greets her team and starts the day with a team meeting to discuss the tasks and goals for the day. She makes sure that everyone is on the same page and motivated to work towards the company's success.
After the meeting, Ruby spends most of her time managing and overseeing different aspects of her start-up. This includes meeting with potential investors, attending networking events, and conducting market research to stay updated with the latest trends and competition.
She also spends a significant amount of time brainstorming and strategizing with her team to come up with innovative ideas and solutions to grow the business. Ruby is also involved in the hiring process, interviewing potential candidates to join her team.
During lunchtime, Ruby takes a break to recharge and socialize with her team or other entrepreneurs in the co-working space. She believes that networking and building relationships are crucial for the success of her start-up.
In the afternoon, Ruby spends time on administrative tasks such as reviewing contracts, managing finances, and responding to emails. She also makes time for creative tasks, such as designing marketing materials or updating the company's website and social media accounts.
As the day comes to an end, Ruby checks in with her team to ensure that everything is on track and addresses any issues or concerns that may have arisen. She also takes this time to plan for the next day and prioritize tasks for the following week.
Ruby usually leaves the office around 6 PM, but she often works late into the night, especially when there are urgent deadlines or important meetings. She believes in hard work and dedication and is willing to put in extra hours to make her start-up a success.
After work, Ruby either attends networking events or goes home to spend time with her family and unwind. She makes sure to get enough rest and recharge for the next day, ready to tackle any challenges that come her way.=== Chat Completion LLM Parameters ===
model gpt-3.5-turbo
stream False
n 1
temperature 0.7
=== Funny Lines ===
Sure thing! Here's one for you: Why did the math book look sad? Because it had too many problems.
=== The Bard Sayeth ===
"All the world's a stage, and all the men and women merely players." - As You Like It, William Shakespeare
=== Yen:USD Exchange ===
The exchange rate for Japanese Yen to US Dollar can fluctuate throughout the day due to market conditions. I recommend checking with a local bank or currency exchange service for the most up-to-date rate.
=== Tech Entrepreneur ===
Ruby's typical working day involves a mix of tasks related to running her startup. Here's an idea of what her day might look like:
1. Morning: Ruby starts her day early with a healthy breakfast and some exercise to get energized for the day ahead.
2. Check emails and messages: She spends some time going through her emails and messages to stay updated on any important communications or developments.
3. Team meetings: Ruby meets with her team to discuss progress on ongoing projects, address any challenges, and set goals for the day.
4. Business development: She spends time working on business development activities such as networking, building partnerships, and exploring new opportunities for growth.
5. Product development: Ruby is always looking for ways to improve her product or service, so she may spend time meeting with her product team to discuss new features or enhancements.
6. Marketing and sales: She also dedicates time to marketing and sales activities, such as planning campaigns, analyzing data, and meeting with potential clients or partners.
7. Strategic planning: Ruby sets aside time to work on the long-term strategy for her startup, including setting goals, making decisions about the direction of the business, and planning for the future.
8. Networking events: In the evenings, Ruby may attend networking events, industry conferences, or other opportunities to meet potential investors, partners, or customers.
9. Reflect and unwind: At the end of the day, Ruby takes some time to reflect on her accomplishments, unwind, and prepare for the next day.
Overall, Ruby's typical working day is busy and varied, with a focus on driving her startup forward and making it a success.