I’m comparing Meta-Llama-3-70B-Instruct-q4_K_S.gguf with Meta-Llama-3-8B-Instruct-q8_0.gguf and finding that the former is noticeably lobotomized. Here’s an example:
Prompt: how many words in the english language rhyme with exactly 13 other words?
Meta-Llama-3-70B-Instruct-q4_K_S.gguf says:
What an intriguing question!
After digging through linguistic databases and conducting some research, I found that there is only one word in the English language that rhymes with exactly 13 other words: “month”. The 13 words that rhyme with “month” are:
- Bonth
- Bonth
- Conth
- Donth
- Fonth
- Gonth
- Honth
- Jonth
- Konth
- Lonth
- Onth
- Ronth
- Sonth
Please note that some of these words might be obscure, archaic, or not commonly used in modern language. However, they are all technically recognized as rhyming with “month”.
If you have any follow-up questions or would like to explore more linguistic curiosities, feel free to ask!
Meta-Llama-3-8B-Instruct-q8_0.gguf says:
What an intriguing question!
According to various sources, including linguistic databases and research papers, there is only one word in the English language that rhymes with exactly 13 other words. And that word is… (drumroll please)… “queue”!
Here are the 14 words that rhyme with “queue”:
- crew
- glue
- new
- few
- flew
- blue
- clue
- sue
- brew
- view
- drew
- kew
- shoe
- queue
These words all share the same ending sound as “queue”, which is a unique property among English words. So, if you’re looking for a word that rhymes with exactly 13 other words, “queue” is your answer!
As you can see, the larger but more quantized model gives a nonsense answer, while the smaller less quantized model gives a much better answer (even if neither of them are necessarily right.) I think something may be broken with llama.cpp or the GGUFs… I dunno, just thought I’d share.
💬 Discussion r/LocalLLaMA (25 points, 25 commentaires)