- AI Something: The no BS everyday AI Newsletter
- Posts
- ChatGPT vs Google Gemini Pro: Unveil the Superior AI with data - December 2023.
ChatGPT vs Google Gemini Pro: Unveil the Superior AI with data - December 2023.
Dive into an unbiased, data-driven comparison of Google Gemini and ChatGPT technology. Discover the king of Language Large Models (LLMs) and see how close-sourced and open-sourced LLM projects measure up.
In my previous post, I mentioned the assumed capabilities of Google Gemini from its launch and how it is compared to ChatGPT. Since AI Something’s mission is a no-BS AI newsletter (no offense, Google, I’m not implying any BS), I want to dive deeper into this comparison, backed by numbers and not speculation. Worry not; it will be a quick and interesting read, as usual.
In today’s rundown:
🥊 What is ChatBot arena?
🏇 Google Gemini vs ChatGPT: Which one is really better?
Read time: 1 minute
ChatBot Arena
LMSYS’s org homepage and projects.
Description: an innovative open-source project collaboratively developed by LMSYS and UC Berkeley SkyLab teams. They aim to establish a comprehensive, crowdsourced platform for collecting valuable human feedback and rigorously evaluating Language Large Models (LLMs) in diverse, real-world scenarios. 🌐🤖💡
Why we use ChatBot Arena leaderboard: ChatBot Arena uses 130K+ user votes to compute Elo ratings (read research Paper). It includes three reputed benchmarks. These data-driven methods give us confidence in how to judge the best LLM models accurately and unbiased:
Chatbot Arena is a crowdsourced, randomized battle platform. We use 100K+ user votes to compute Elo ratings.
MT-Bench is a set of challenging multi-turn questions. We use GPT-4 to grade the model responses.
MMLU (5-shot) is a test to measure a model’s multitask accuracy on 57 tasks.
Check out the project website here 👉️ link.
Google Gemini vs ChatGPT: Which one is really better?
ChatBot Arena ranking result - December 2023
Quick summary from the ranking:
👑 OpenAI is still currently the king of LLMs
Google Gemini's performance is not currently remarkable, but it is comparable to ChatGPT 3.5 at most.
🔒️ Close-sourced LLMs generally outperform open-sourced LLM projects (but the gap is getting smaller 🏃)
Mistral 8×7b is the best open-source project as of December 2023
✅ Verdict: ChatGPT4/ OpenAI is currently 👑 King of LLMs as of December 2023
Check out the AI Leaderboard here 👉️ link.
Give love to your favorite tools 💞
Cast your vote to win a $20 gift card and contribute toward your favorite tools. Your selection will guide others to discover the emerging useful tool, and the top pick will be honored as the AI overlord for the month.
Which one is the better AI? |
That’s a wrap! 🌯
Thank you for reading ❤️I hope you will find these insights useful. Please contact us at [email protected] if you have suggestions, feedback, or anything!