Build own AI (LLMs) Chatbot
Nowadays, a plethora of AI tools based on large language models are emerging daily. Since OpenAI released API on March 1, 2023 (https://openai.com/blog/introducing-chatgpt-and-whisper-apis), ChatGPT-like chatbots are everywhere. Except for the official applications and ecosystem integrations, many tech enterprises, start-ups, and developers are bringing a variety of chatbots into daily life. For instance, I use Chrome + Monica instead of new Bing & Edge, and MacGPT instead of OpenAI Chat online, it’s convenient.
However, almost all of them are powered by a unified API with a public knowledge base.
How to build own AI(LLMs) chatbot ?
1. Model
There are numerous open-source models available, such as Google’s BERT and T5, OpenAI’s GPT, Meta’s LLaMA, and many more. However, utilizing these models can be quite challenging for regular users and even SMEs. Several prerequisite conditions limit their adoption:
- Research Team: AI talents, especially experienced researchers working on LLM teams, play a crucial role. For instance, former Google employees founded OpenAI, and former OpenAI members established Anthropic, highlighting the ongoing competition for AI talent(As I mentioned on my previous blog: ChatGPT Effect: Part 4, Competing for AI Talent). These researchers can customize and enhance AI models to suit specific objectives. However, for individuals and SMEs, effectively working with LLM models is nearly impossible. While some researchers may aspire to create their “own” AI chatbot, why not start up? By the way, collaborating with universities is a compromise, many scholars specialize in AI, DL, ML, and related fields. Even tech giants like Google also have close contact with universities in this domain.
- Costs: Computing power is a significant expense. Each major AI company has its own cloud service: Google has Google Cloud, OpenAI has Microsoft Azure, and Baidu has Baidu Cloud. According to CNBC’s report titled “ChatGPT and generative AI are booming, but the costs can be extraordinary” – The critical process of training a large language model such as GPT-3 could cost over $4 million (MAR 13 2023). As the model parameters continue to grow larger, the costs also increase. However, smaller fine-tuning models like Vicuna-13B have lower costs, with training expenses of around $300. Additionally, Google’s PLAM 2 has a lightweight model called Gecko that can run on mobile devices. However, this does not imply a reduction in costs, as the underlying large language model (LLM) still incurs substantial expenses. Furthermore, the costs of building, deploying, and providing services need to be considered as well.
2. Project
There are popular open-source repos available that enable users to efficiently build their own language models (such as chatbots) in a few steps. However, it’s important to note that these repositories primarily simplify the process of training large language models (LLMs) and allow for local deployment on reduced resources (CPU/GPU). Users are still required to possess relevant AI knowledge.
- How to install the package?
- How to prepare the dataset? How to format the dataset?
- How to perform fine-tuning?
- How to evaluate the model?
- …
Furthermore, most of these repos are under Apache License 2.0. And they are limited to study and research and prohibited from commercial use. To SMEs, this way is hardly feasible.
| Project | Github | Model |
|---|---|---|
| LMFlow | https://github.com/OptimalScale/LMFlow | GPT-2, GPT-Neo, Galactica, LLama… |
| FastChat | https://github.com/lm-sys/FastChat | FastChat-T5, Vicuna |
| Chinese-LLaMA-Alpaca | https://github.com/ymcui/Chinese-LLaMA-Alpaca | LLama |
| Auto-GPT | https://github.com/Significant-Gravitas/Auto-GPT | GPT-4 |
| DeepSpeed | https://github.com/microsoft/DeepSpeed | AlexaTM, BLOOM, GLM, GPT-NeoX, Jurassic-1, METRO-LM, Megatron-Turing NLG, Turing NLG, YaLM |
| Colossal-AI | https://github.com/hpcaitech/ColossalAI | BERT, BLOOM, GPT-2, GPT-3, OPT, PaLM, VIT |
| gpt4all | https://github.com/nomic-ai/gpt4all | Alpaca, Fastchat, GPT-J, GPT4All, Koala, LLama, Mosaic, Open Assistant Pythia, Pythia, StableLM, StableVicuña, Wizard |
| OpenAssistant | https://github.com/LAION-AI/Open-Assistant |
3. Platform
Cloud leaders have recently introduced new products for large language models (LLMs) in recent months. Microsoft was the first to launch the Azure OpenAI Service on January 17, 2023.
“Large language models are quickly becoming an essential platform for people to innovate, apply AI to solve big problems, and imagine what’s possible. Today, we are excited to announce the general availability of Azure OpenAI Service as part of Microsoft’s continued commitment to democratizing AI, and ongoing partnership with OpenAI. “- source
According to CNBC’s report titled “Cloud leaders Amazon, Google, and Microsoft show the once-booming market is cooling down”- As growth in traditional tech equipment and software slowed to a trickle in recent years, cloud computing gobbled up spending, reflecting a dramatic change in how companies were choosing to run applications and store data(Feb 4, 2023).
After ChatGPT introduced a new experience, LLMs quickly became popular, and enterprises faced the challenge of empowering their businesses with the power of AI models. As we know, the cost of AI (LLMs) is mainly in computing and processing user prompts. Cloud leaders have seized this opportunity and combined their own products to provide a “one-stop platform” solution for enterprises.
On these AI platforms, the Model Zoo, Deploy environment, Training, and Security… are all in one. Developers can use the AI toolkit to start quickly. But one thing to consider is whether your existing business is already on the platform and whether you need to change Cloud providers for this AI service.
It is also important to note that AI knowledge is necessary, as the platform simply simplifies the process. Without any IT experience, accessing the platform may be difficult at first.
4. API
APIs provide a direct and cost-effective way to incorporate AI (LLMs) into your products. Just Input > Output.
Companies like OpenAI, Google, Baidu, and Anthropic provide API services with a wide range of application scenarios. With these APIs, developers can quickly tap into the capabilities of AI for their businesses or specific goals.
However, it’s important to note that APIs are primarily designed for developers and coding knowledge is usually required to effectively utilize them.
- OpenAI API, https://platform.openai.com/overview
- Baidu wenxin API, https://wenxin.baidu.com/api
- Google PaLM API, https://developers.generativeai.google/
- Anthropic API, https://console.anthropic.com/docs/api
5. Service
“Build own Chatbot in x minutes” – these service platforms claim. They cater to regular users and SMEs by providing a no-code/zero-code solution to easily create an AI Chatbot through a simple interactive interface, functioning as a lightweight SaaS. (Most of these platforms are powered by OpenAI’s GPT, including GPT-4. It appears that OpenAI has dominated the market and built a vast ecosystem.)
In just three steps:
- Sign up
- create a knowledge base tailored to your specific goals.
…….Auto training your exclusive ChatGPT-like Chatbot……..
- Share or embed it in your websites/products
| Product | Powered by | Website | Knowledge base | note |
|---|---|---|---|---|
| Chatbase | OpenAI(GPT-3.5-turbo) | https://www.chatbase.co/ | doc, docx, pdf, txt, url | |
| SiteGPT | OpenAI(GPT) | https://sitegpt.ai/ | url, xml | for website |
| ChatMasters | OpenAI(GPT-3/4) | http://chatmasters.io/ | txt | |
| CustomGPT | OpenAI(GPT-4) | https://customgpt.ai/ | doc, pdf, url, zip, etc. | |
| YourGPT Chatbot | OpenAI(GPT) | https://yourgpt.ai/ | csv, doc, docx, pdf, ppt, txt, url | for website |
| Dante | OpenAI(GPT-3.5/4) | https://dante-ai.com/ | docx, etc., images, pdf, url, video | |
| AskAI | OpenAI(GPT) | https://myaskai.com/ | csv, doc, docx, pdf, txt, url | |
| Arsturn | OpenAI(GPT-3.5-turbo) | https://www.arsturn.com/ | url | for website |
| Wonderchat | OpenAI(GPT-3.5/4) | https://wonderchat.io/ | pdf, url | for website |
| Gptify.io | https://gptify.io/ | YouTube video link, csv, docx, pdf, txt, url | ||
| Quickchat | OpenAI(GPT) | https://www.quickchat.ai/ | Notion, pdf, txt, url | |
| Chat Thing | OpenAI(GPT-4) | https://chatthing.ai/ | Notion, YouTube video link, pdf, txt, url | |
| anywebsite.ai | OpenAI(GPT-3.5-turbo) | https://anywebsite.ai/ | url | for website |
However, many users have concerns about “Data Security.” Although these services claim to prioritize the security of personal data, storing content only on secure and encrypted Cloud servers without reading or accessing these documents.
According to a Bloomberg report, some Samsung employees uploaded sensitive code to ChatGPT, leading Samsung to ban the use of AI. They worry about after data be fed to the AI model on training and would be leaked.
Additionally, Jailbreak Prompts can be used to manipulate and abuse Chatbots easily, raising further issues.
Therefore, when dealing with these platforms that require no technical expertise, it becomes more likely that sensitive, confidential, and personal data may be inadvertently uploaded. It is crucial to establish clear usage boundaries when employing AI (LLMs).
Within these SaaS platforms, some only support building Chatbots for websites, allowing the use of public links to create a chatbot for enhancing customer experience (CX) or developing an AI assistant for public data sources like Notion, blogs, and papers to improve search and summarization capabilities.
Currently, when referring to a “Chatbot,” most people envision a ChatGPT-like chatbot. The chatbot used in CX closely resembles ChatGPT and can easily be integrated with AI (LLMs) to harness the power of AI.
Has the Chatbot/Conversational AI field undergone significant changes amid this AI revolution?
….