Nice model, but it can sometimes be very insistently wrong.
Seriously, hats off, this model is very nice.
I was running it through a battery of logic problems, just kicking the tires, and noticed that even when given corrections, it can really double down on flawed reasoning.
I am running a Q4 quant, so I might try this again full weight. The TLDR version of what I asked is if a bad guy is playing Russian roulette with you, (6 chamber revolver, one bullet) and shoots at you with nothing happening, is it better to have them spin the chamber before the second shot, or just have them shoot again. Basic probability says spinning is the right move, but Hermes insisted my chances of survival actually go up every time the trigger is pulled, so spinning is not a good call.
I let DeepHermes-3-Llama-3.1-8B-Preview-OF32.EF32.IQ8_0.gguf deliberate over it. It answered correctly as per your expected answer.
Try the other different O.E.IQuants here: Joseph717171/DeepHermes-3-Llama-3.1-8B-Preview-OQ8_0-F32.EF32.IQ4_K-Q8_0-GGUF
AI:
A bad guy plays Russian roulette with a six-shooter revolver containing one bullet. After spinning and firing the first time without result, there are five chambers left with bullets.
Probability of survival if he spins again:
- When he spins again, each chamber has an equal probability of 1/6 to be the one with the bullet.
- There are five chambers out of six that do not contain the bullet, giving a probability of ( \frac{5}{6} ).
Probability of survival if he does not spin:
- If he fires from the current chamber without spinning again, it is equivalent to randomly selecting one of the six chambers with one bullet.
- This results in a probability of ( \frac{1}{6} ).
Since ( \frac{5}{6} > \frac{1}{6} ), choosing for him to spin again provides a higher chance of survival.
[
\boxed{\text{Spin again.}}
]
==== Below is the full transcript including <think></think>...
<think>
Okay, so there's this problem about Russian roulette where a bad guy has a six-shooter revolver with one bullet in it. He spins the chambers and fires once, but nothing happens because no bullet was fired. Now he gives you the choice: should he spin the chambers again before firing a second time? Hmm, interesting. I need to figure out if spinning again is better or worse than not spinning.
First, let me recall what Russian roulette usually involves. A revolver with six chambers; one has a bullet, the others are empty. The person spins it randomly and then pulls the trigger once. If there's a bullet in the chamber that gets fired, you're dead. But since they spin it first, there's a 1/6 chance of having the bullet when they pull the trigger. The rest of the time (5/6), it's empty.
In this case, though, he already spun and pulled the trigger once, but no shot was fired. That means whatever chamber was pointing at you didn't have the bullet. So now there are two possibilities: either the bullet is in another chamber that hasn't been spun yet, or it's in one of the remaining chambers. Wait, how many chambers are left?
The revolver has six chambers total. He spins once and fires once. Since he fired once but no shot, that means the bullet was not in the chamber he pulled the trigger on. So there are five chambers left (since the bullet is definitely in one of them), but two possibilities: the bullet could be in any of the remaining chambers, or all five are empty except for the one he already pulled.
But how do we know if spinning again would change the probability? Let me think. If he spins again and fires once more, the chance that you survive a second time depends on where the bullet is.
Wait, maybe I need to model this as probabilities. Let's consider the first shot: probability of survival was 5/6 (since it wasn't fired). Now, after spinning once without firing, there are still five chambers with the bullet and one empty chamber? Or wait, no—the original setup was six chambers, one bullet. He spins once, fires once, which didn't fire. So that means that the chamber he pulled the trigger on was not the one with the bullet. Therefore, there are now five chambers where the bullet could be (since it must be in one of them) and one empty chamber.
So if he spins again, each spin has a 1/6 chance to land on any of the six chambers. If the next spin lands on an empty chamber, you survive again; if it lands on the bullet, you die. But since there are five bullets left among the five remaining chambers (because one was already pulled and was not the bullet), does that affect the probability?
Wait, maybe not. Because even though there are five bullets left in five chambers, each spin has a 1/6 chance regardless. So the probability of survival after spinning again is still 5/6? But that can't be right because if you have five bullets left in five chambers, each chamber has a 1/6 chance to be hit, so out of six spins (including the first), the probability would be different.
Wait, maybe I need to model it step by step. Let me consider all possible scenarios after the first shot.
After the first spin and trigger pull with no bullet fired, there are two possibilities:
- The bullet is in one of the five chambers that were not spun (since he already pulled a chamber which wasn't the bullet).
- All six chambers have been cycled except for the original position where the bullet was.
Wait, actually, this might be more complicated. Let's think in terms of permutations. If you spin once and pull the trigger without firing, then there are two cases:
- Case 1: The bullet was not in the chamber that was pulled.
- Case 2: The bullet is now in the same position as before.
But wait, how can the bullet be in the same position? Because he spun it. If you spin a revolver, the bullet moves to a new random position each time. So the only way for the bullet to not have moved is if it was already pointing at that chamber when he spun, but then pulling the trigger would have fired it. But since he didn't fire, that means the bullet must be in one of the other five chambers.
Therefore, after spinning once and firing without result, there are five chambers with the bullet (since the original position had no bullet), right?
Wait, maybe not. Let's clarify:
Original setup: Bullet is in chamber 1 to 6 randomly. He spins the cylinder so that the bullet could be anywhere. Then he pulls trigger once.
If pulling the trigger fires, then that was the chamber with the bullet; if not, the bullet must still be somewhere else (since spinning moves it). So after one failed attempt, the bullet is in one of the other five chambers.
Therefore, when he spins again, the probability that the next spin lands on a bullet is 5/6. If he spins and fires again without firing, then there are four bullets left? Wait no, because if you fire once more and it doesn't go off, the bullet must still be in one of the remaining chambers.
Wait, perhaps we need to model this recursively. Let's define f(n) as the probability that after n spins with no result yet, the next spin will kill you.
After the first spin (no result), there are five bullets left in five chambers. So if he spins again, the probability of survival is 5/6. If he doesn't spin again, then there's a 1/6 chance to survive, but actually, no—if he doesn't spin and just fires from where it is now.
Wait, maybe after not spinning, you have a bullet in one of the six chambers? Wait, no. Because if he didn't spin, the bullet could be in any of the chambers. But since he already pulled the trigger once without firing, the bullet must still be somewhere else. If he doesn't spin again and just fires from the current position, which is different from the first one because he spun it.
Wait, maybe this is getting too convoluted. Let me try a different approach.
Suppose you have a six-shooter with one bullet. He spins it once and pulls the trigger without firing. Then you decide whether he should spin again or not before pulling the second trigger.
If he spins again, each subsequent trigger pull has a 1/6 chance of hitting the bullet (since there are five chambers left with the bullet). The probability that you survive two spins is (5/6) * (4/6), because after the first spin misses, there are four bullets left in four chambers. Wait, no—if he spins again and it doesn't fire, does that mean another chamber was empty? Or does spinning move the bullet?
Wait, each time you spin, if it doesn't fire, the bullet must still be somewhere else. So after each successful non-firing trigger pull (i.e., pulling without firing), the number of bullets remaining decreases by one.
Therefore, if he spins again and pulls once more without firing, the probability that you survive is (5/6) * (4/6) * ... up to the number of spins. But since he could choose any number of times to spin or not, this might complicate things.
Alternatively, let's consider the problem from your perspective after the first failed attempt. You have two choices: either he spins again and fires once, or he doesn't spin and just fires once.
What is the probability you survive each choice?
If he spins again:
After spinning once more, there are five chambers left with bullets (since one chamber was already pulled without firing). So if he pulls the trigger again, the chance of survival is 5/6. If that happens again, then there are four bullets left in four chambers, so next chance is 4/6, and so on.
But since you only have to survive once more time, whether it's spinning or not. Wait, actually, if he spins, the probability of surviving the second shot is (5/6). If he doesn't spin, then what's the probability? Well, if he just fires from wherever the bullet is now, but since he already spun once and didn't fire, the bullet must be in one of the other five chambers. So firing again without spinning would have a 1/6 chance to hit a bullet.
Wait, so actually both choices have the same probability? Because regardless of whether you spin or not, the probability of surviving is 5/6.
But that seems counterintuitive. Let me verify.
Case 1: He spins again. After the first non-firing trigger pull, there are five bullets left in five chambers. So spinning again gives a 5/6 chance to survive the second shot.
Case 2: He doesn't spin and fires from the current chamber (which was not fired before). Since he already spun once without firing, that means the bullet is in one of the other five chambers. So if he just pulls the trigger again without spinning, there's a 1/6 chance to fire the bullet.
Therefore, both choices result in a probability of 5/6 for survival? That can't be right because intuitively, spinning again might give more chances to survive, but maybe not. Wait, maybe I'm missing something here.
Wait, actually, if he doesn't spin and just fires from the current chamber, which is different from the one he pulled before (because he spun it), so there's a 1/6 chance of survival. If he spins again, then the probability is also 5/6. So both options give you the same chance? That seems odd.
But why does spinning again result in 5/6 instead of higher?
Wait, perhaps not. Let me think step by step.
First spin: 5/6 chance to survive.
If he spins again:
- The probability that the second trigger pulls a chamber with a bullet is 5/6 (since there are five bullets left). But if you survive that, then there are four bullets left in four chambers. Wait, but do we need to consider multiple spins? Since the question gives you a choice between spinning once or not spinning at all.
Wait, perhaps I should model it as follows:
If he spins again and pulls once: probability of survival is 5/6.
If he doesn't spin and just fires once from the current chamber (which hasn't been fired yet): probability of survival is 1/6.
Therefore, in this case, spinning gives a higher chance. So you should choose for him to spin again because 5/6 > 1/6.
But that contradicts my initial thought where I considered both leading to the same result. Maybe my previous reasoning was wrong.
Let me clarify:
After the first successful trigger pull (no bullet fired), there are five chambers left with bullets and one empty chamber. If he spins again, then each of these six chambers has an equal probability. The chance that a non-firing happens is 5/6. So spinning gives you 5/6 chance.
If he doesn't spin, the current chamber (which was spun once before but didn't fire) now has no bullet? Wait, how?
Wait, when you spin a revolver and don't fire, does the bullet remain in that position or move again?
This is a crucial point. In Russian roulette, after spinning, if you pull the trigger without firing, what happens to the bullet's position for subsequent attempts?
In reality, each time you pull the trigger, it attempts to fire, but if there's no bullet in the chamber, nothing happens. So if he spins once, fires (no result), then the bullet is still somewhere else because spinning moved it.
Therefore, when he spins again, the bullet is now in one of five chambers (since one was already pulled and wasn't the bullet). Therefore, each spin has a 5/6 chance to miss. If you survive that spin, then there are four bullets left in four chambers, so next time it's 4/6, etc.
But since he only needs to fire once more, the probability of surviving is 5/6 regardless of how many times he spins. Wait, but actually, each subsequent non-firing increases the number of bullets remaining by one less than the number of chambers.
Wait, no—if you spin again and pull the trigger without firing, that means another chamber was empty. So after two successful non-firings (spinning twice), there are four bullets left in four chambers. Therefore, the probability is 4/6 each time.
But since he's only giving you one choice: should he spin once or not spin at all? Then, if you choose to have him spin once, then the survival probability is 5/6. If you choose not to spin, then the survival probability is 1/6 (since firing from that chamber would hit a bullet). Wait, but how?
Wait, when he doesn't spin, does that mean he leaves it in the current position? But if he already spun once and didn't fire, the bullet must have moved. So if you choose not to spin again, he just fires from where it is now (the chamber that was spun previously). Wait, but how can the bullet be in that same chamber again?
No, when you spin a revolver, the bullet moves to a new random position each time. Therefore, after spinning once and not firing, if you don't spin again, the current chamber has no bullet because it wasn't fired before. But since he spun it once already, which didn't fire, that means the bullet was in another chamber.
Wait, this is getting really confusing. Let me try to model this step by step with probabilities.
Initial state: Six chambers, one bullet. He spins and fires once; no shot. Therefore, the bullet is not in the first chamber he pulled (since it didn't fire). So now there are five chambers left with bullets (the original six minus the one empty chamber).
Therefore, if he spins again:
- Probability of survival: 5/6.
If he doesn't spin and just fires from the current chamber:
But wait, after spinning once and not firing, what's the position of the bullet? Since he spun it once already, which didn't fire, that means the bullet was in one of the other five chambers. So if he doesn't spin again and just fires from the same position (which was spun once before), but since there was no bullet there when he pulled the trigger earlier, how is that possible?
Wait a second, perhaps this is a flaw in my reasoning. Because each time you spin the cylinder, it moves the bullets to new positions. So if he spins once and fires without result, that means the bullet wasn't in that chamber. Then, if he doesn't spin again and just fires from where it is now (the same position), which was spun once before but didn't have a bullet, how can there be a bullet there?
Wait, no—if you don't spin again, then the bullet remains in whatever chamber it ended up after the first spin. But since he already fired that chamber and no bullet was fired, the bullet must still be somewhere else because spinning moves it.
Therefore, if he doesn't spin and just fires from the current chamber (which was spun once before), but didn't have a bullet when he pulled the trigger earlier, then how can there be a bullet in that same chamber?
This is a contradiction. Therefore, my assumption must be wrong.
Ah! Here's the key point: When you spin a revolver and fire without result, the cylinder remains in its current position because pulling the trigger doesn't move it. Wait, no—that's not correct. Spinning moves the chambers to random positions each time. So if he spins once and fires (no bullet), then the bullet is still somewhere else, but the cylinder has been spun again when he pulls the trigger?
Wait, let me think carefully.
When you spin a six-shooter revolver, it rotates the chambers randomly. Each time you pull the trigger, the gun attempts to fire in the current chamber. If there's no bullet in that chamber, nothing happens; if there is, the gun fires.
So after spinning once and not firing (no bullet), the cylinder remains in that position because he didn't spin it again. Therefore, the bullet is still in one of the other five chambers.
Therefore, when you choose to have him fire a second time without spinning, he would pull the trigger from the current chamber (which was spun once before and didn't fire). But since pulling the trigger doesn't move the cylinder, the bullet could be in any of the remaining five chambers. Wait, but if you don't spin again, then the bullet is still in one of those five chambers.
Therefore, when he fires a second time without spinning, there's a 1/6 chance to fire a bullet (since it's randomly one of six chambers, with one bullet). But since we already know that after one failed firing, the bullet must be in one of the other five chambers? No, actually, no—if you don't spin again, then the cylinder is still rotated from the first spin. Therefore, the bullet could be in any of the six chambers, but it's equally likely.
Wait, this is really confusing. Let me try to model it with a different approach.
Let’s consider that each time he spins the revolver, it resets the possible positions of the bullet. So if you have a six-shooter with one bullet:
- Probability of survival on the first spin: 5/6.
- If you survive the first shot and decide whether to spin again or not.
If you choose to spin again before the second shot:
- The probability that the second shot is also empty is 5/6 (since there are five bullets left in five chambers).
- Then, if you survive two shots, you have four bullets left in four chambers.
- But since he only needs to fire once more time after your choice, actually, the survival probability is 5/6 for a single second shot.
If you choose not to spin again:
- The current chamber (which was spun and not fired) has no bullet because it didn't fire. Wait, but that's impossible because spinning moves the bullet. So if he spins once and doesn't fire, the bullet must have been in one of the other five chambers. Therefore, firing from the same position again would still be a random chamber among all six.
Wait, so maybe not spinning is equivalent to keeping the revolver in its current state (after the first spin) without moving it. So when you don't spin, he just fires from whatever position it's currently at, which was spun once before but didn't have the bullet.
Therefore, the probability of survival is 5/6 for both choices? Because whether he spins again or not, each trigger pull has a 1/6 chance to hit. Wait, no—if you spin again, there are five bullets left in five chambers, so 5/6 chance. If you don't spin, it's one bullet out of six chambers, so 1/6.
But why would spinning again lead to 5/6 instead of higher? Because each time he spins, the number of bullets remaining decreases by one less than the number of chambers.
Wait, I think there's a misunderstanding here. Let me look up the standard probability for Russian roulette.
Standard Russian Roulette:
- With N chambers and 1 bullet: Probability of surviving first shot is (N-1)/N.
- If you survive that, then the next shot has probability (N-2)/(N-1), and so on until you die or survive all shots.
But in this problem, after the first failed shot, you're given a choice to spin again before the second shot. So it's not multiple spins; it's one spin before each shot.
Therefore:
- First shot: probability of survival is 5/6.
- If you choose to spin again (so he spins once more), then the next shot has probability 4/5 if you survived the first, but since we're only considering a single second shot, it's 4/5? Wait no.
Wait, perhaps the problem is that when he spins before each shot, so for the first shot: 1 bullet in 6 chambers, so 5/6 survive. If you choose to spin again (so he spins once more), then the probability becomes 4/5 (if you had already fired one chamber). But since we're only considering the second shot after your choice, it's equivalent to having spun once before.
Wait, maybe I'm overcomplicating.
Let’s model this as:
After the first shot (no bullet), there are five chambers left with bullets. So if he spins again, each of these six chambers has a bullet with probability 1/6. Therefore, the chance that the second shot is empty is 5/6.
If you choose not to spin again, then he just fires from the current chamber (which was spun once before and didn't fire). But how many bullets are left? Initially one bullet, one fired chamber (no bullet), so five bullets left. If you don't spin, firing would be a random chamber among six, with one bullet. So probability of survival is 1/6.
Therefore, spinning gives higher chance (5/6) than not spinning (1/6). Therefore, the answer should be to have him spin again.
But wait, why isn't it 4/5? Because if you survive the first shot, then there are four bullets left in four chambers. But since he only needs to fire once more time, the probability is 4/5, not 5/6?
Wait no, because when he spins again before firing a second time, the number of bullets remaining is five (since one bullet was fired in the first shot). So spinning would mean that each chamber has an equal chance, so there's a 1/6 chance for each chamber to have the bullet. Therefore, five chambers out of six have no bullet, so probability of survival is 5/6.
Alternatively, if you think about it as the bullet must be in one of the five chambers (since one was already fired), then spinning again would mean that the probability is 1/6 per chamber, so 5/6 chance to miss.
But if he doesn't spin and just fires from the current chamber (which was spun once before but didn't have a bullet), how does that work? If you don't spin, then the cylinder remains in its current position. But since he already fired without result, that means the bullet wasn't there. So firing again from the same position would still be random among all chambers, with one bullet. Therefore, probability of survival is 1/6.
Therefore, spinning gives a higher chance (5/6) than not spinning (1/6). Hence, you should choose to have him spin again.
But I recall that in standard Russian roulette, if you survive the first shot and decide whether to spin again or not, the probability for the next shot depends on how many chambers you've spun. However, in this problem, he only spins once before each shot. So after surviving the first shot, spinning again would reset the chambers, but since you already fired one chamber (no bullet), there are five bullets left. Therefore, the second shot's probability is 5/6.
But wait, if you think of it as a single shot where he spins once more, then the number of possible bullets per chamber is still five out of six, so the probability is 5/6.
Alternatively, if you consider that after one failed shot, there are four bullets left in four chambers (since one was fired), but since we're only considering the next shot before any firing, it's equivalent to a new spin. But this seems inconsistent.
I think the key point here is that when you survive the first shot and he spins again, the probability resets because spinning moves the bullet to a new random position among the remaining five chambers. Therefore, each chamber has a 1/6 chance of having the bullet, leading to a 5/6 chance of survival.
On the other hand, if you choose not to spin, then the current chamber (which was spun once before but didn't have a bullet) is now being fired again. But since pulling the trigger doesn't move the cylinder, the bullet could be in any of the remaining chambers. However, after one failed firing, the bullet must still be in one of the other five chambers. Therefore, when you fire from that same position without spinning, it's equivalent to randomly choosing a chamber among all six with one bullet, giving a 1/6 chance.
Therefore, spinning gives a higher survival probability (5/6) than not spinning (1/6), so you should choose for him to spin again.
But wait, this contradicts my initial thought where I considered both leading to the same result. It turns out that choosing to spin increases your chances because it effectively resets the chambers, giving more opportunities to miss compared to just firing from a fixed position.
Therefore, the answer is that he should spin again because 5/6 > 1/6.
But let me verify this with another approach.
Imagine you have six chambers and one bullet. The probability of surviving the first shot is (6-1)/6 = 5/6.
If you survive the first shot, there are five bullets left in five chambers. If he spins again before firing a second time:
- There's a 1/6 chance that any given chamber has the bullet.
- Therefore, the probability of surviving the second shot is 5/6 (since five out of six chambers have no bullet).
If you choose not to spin and just fire from the current chamber (which was spun once before but didn't have a bullet):
- The cylinder hasn't been moved since the first spin.
- Therefore, when he fires again, it's still randomly one of six chambers, with one bullet.
- Probability of survival is 1/6.
Hence, spinning gives higher probability. Therefore, you should choose to have him spin again.
Yes, this makes sense. So the answer is that he should spin the chambers again because there's a higher chance of survival if he does (5/6) compared to not spinning (1/6).
Final Answer
\boxed{Spin} \text{again.}
A bad guy plays Russian roulette with a six-shooter revolver containing one bullet. After spinning and firing the first time without result, there are five chambers left with bullets.
Probability of survival if he spins again:
- When he spins again, each chamber has an equal probability of 1/6 to be the one with the bullet.
- There are five chambers out of six that do not contain the bullet, giving a probability of ( \frac{5}{6} ).
Probability of survival if he does not spin:
- If he fires from the current chamber without spinning again, it is equivalent to randomly selecting one of the six chambers with one bullet.
- This results in a probability of ( \frac{1}{6} ).
Since ( \frac{5}{6} > \frac{1}{6} ), choosing for him to spin again provides a higher chance of survival.
[
\boxed{\text{Spin again.}}
]
Thanks, I tried your exact same quant, and it did get the right answer, but the reasoning is a little off. The model assumes in your case (and it did for me as well) that not spinning again reduces your odds of survival to 1/6, and it should be 4/5. Looking through the thinking, I think the model is confusing itself because it discusses the odds of getting shot (1/6) and the odds of surviving (5/6) and then mixes up when each fraction comes into play.
I also tried this (on your larger quant):
This is with a modified system prompt where I was trying to get the model to be more open to correction:
You are a deep thinking AI, you may use extremely long chains of thought to deeply consider the problem and deliberate with yourself via systematic reasoning processes to help come to a correct solution prior to answering. You should enclose your thoughts and internal monologue inside tags, and then provide your solution or response to the problem.
You will be in dialogue with a highly intelligent subject matter expert. If they tell you that you have come to the wrong conclusion, assume they are correct, and carefully examine your reasoning to find your mistake.
I've tried a few variants of that, and I have yet to see it walk anything back. My hypothesis is that the context of the previous answer (with a long CoT) is just blowing away the correction. When I edit the previous answer to remove the CoT, and issue an expanded correction and reduce the temp slightly, then I get it to acknowledge the new answer:
Ah I am editing this, due to an oversight I warned I might make, of fallaciously applying monty hall solution thinking to an overlooked logistic difference in the roulette situation lol Although the duck example does add up to 5 in a particular context, so the answer of 5 is actually quite impressive. These vague questions such as the duck question have multiple logically valid interpretations if you apply different unwritten context, so both answers are logically valid and sound to a particular added context.
Good catch on the Monty Hall solution, although there is a case with Russian Roulette (two bullets in consecutive chambers) where it is better not to spin, which just goes to show how counterintuitive probabilities can be.
On the ducks, 5 is a valid answer, it's just not the most parsimonious, which is somewhat implied when dealing with riddles, but I wouldn't score it a failure either.
What is more interesting to me is how difficult it was to get the model to "change its mind" (I had to delete the previous CoT, tweak the system prompt, and reduce the temp in my hasty testing). I still think that my hypothesis about the CoT is the likely culprit. The weight of the context is too much to be overcome by a simple correction. This isn't an issue for one shot use, but could be for conversational.
Dunno which front end you're using, but you should never keep the "thinking" part of the prompt into the whole context window. It should be written, shown to the user, and internally discarded at the next user query. Keeping the CoT / thinking for each interaction, is very wasteful compute-wise, and confuses the model greatly, especially on a small model like that.
100% agree. I'm using LM Studio as my default front end, and as of the latest stable release, it does not appear to be ditching the thinking blocks. It's also worth pointing out that when I manually delete the thinking blocks (either by eliminating the thinking tags altogether, or by removing the contents, but leaving the tags) the model no longer emits thinking tags in future output. Deleting everything in the same chat brings thinking back, so this does not seem to be a quirk of LM Studio, but might be worth ditching the front end for some direct API testing at some point.
when I manually delete the thinking blocks (either by eliminating the thinking tags altogether, or by removing the contents, but leaving the tags) the model no longer emits thinking tags
Prefill the thinking tag for the response. It's the best way to ensure those small copy-cat models behave like the ones who were trained for CoT from the get go.
That's the trick. A period inside the thinking tags worked.
Dunno which front end you're using, but you should never keep the "thinking" part of the prompt into the whole context window. It should be written, shown to the user, and internally discarded at the next user query. Keeping the CoT / thinking for each interaction, is very wasteful compute-wise, and confuses the model greatly, especially on a small model like that.
Why would you want this
100% agree. I'm using LM Studio as my default front end, and as of the latest stable release, it does not appear to be ditching the thinking blocks. It's also worth pointing out that when I manually delete the thinking blocks (either by eliminating the thinking tags altogether, or by removing the contents, but leaving the tags) the model no longer emits thinking tags in future output. Deleting everything in the same chat brings thinking back, so this does not seem to be a quirk of LM Studio, but might be worth ditching the front end for some direct API testing at some point.
Will make sure to work to add data that teaches it to keep thinking even if no cot in previous turns for the full release, thanks for the idea!