This is a good time to promote running your own models. I have been running my own models locally and I would wager a local model will meet 85-95% of your needs if you really learn to use it. These models have gotten great. For anyone wanting to get into this, the smartest models to run recently that is consumer friendly was just released, checkout Qwen3.5 the 27B and 35B variants. They are small and I recommend running full Q8 quants. The easiest way to run these without dealing with complex GPU is to get a mac. For the example I gave, a 64gb mac will handle it well. If you are really cash strapped then you can manage with a 32gb but will have to run with less resolution quants. If you are not cashed strap, then get at least a 128gb and if possible a 256gb. The models are so good you will regret not getting a better system. You can join the r/LocalLlama community in reddit to learn some more. But this is pretty easy. Grab llama.cpp, grab a gguf quant from huggingface.co - the unsloth quants are great - https://huggingface.co/unsloth/models
And if you don't want to buy a Mac? A 80 GB NVidia GPU costs $10,000K (equivalent to 30 years of ChatGPT Plus subscription) and will probably be obsolete in 5-7 years anyway. What are my options if I want a decent coding agent at a reasonable price?
A laptop with an iGPU and loads of system RAM has the advantage of being able to use system ram in addition to VRAM to load models (assuming your gpu driver supports it, which most do afaik), so load up as much system RAM as you can. The downside is, the system RAM is less fast than dedicated GDDR5. These GPUs would be Radeon 890M and Intel Arc (previous generations are still decently good, if that's more affordable for you).
A laptop with a discrete GPU will not be able to load models as large directly to GPU, but with layer offloading and a quantized MoE model, you can still get quite fast performance with modern low-to-medium-sized models.
Do not get less than 32GB RAM for any machine, and max out the iGPU machine's RAM. Also try to get a bigass NVMe drive as you will likely be downloading a lot of big models, and should be using a VM with Docker containers, so all that adds up to steal away quite a bit of drive space.
Final thought: before you spend thousands on a machine, consider that there are at least a dozen companies that provide non-Anthropic/non-OpenAI models in the cloud, many of which are dirt cheap because of how fast and good open weights are now. Do the math before you purchase a machine; unless you are doing 24/7/365 inference, the cloud is fastly more cost effective.
I have a 24GB Macbook Pro. I will note, do get the 'Pro' models, the Mac Mini and the Macbook Air do not have internal fans. The Macbook Pro has an internal fan, and the Mac Studio (bigger Mac Mini) has a fan. If you get a Mini, you might want to get one of those docks that cools the Mini. Your hardware will get very hot very quickly.
Also, because Apple in their infinite wisdom despite giving you a fan, very lazily turn it on (I swear it has to hit 100c before it comes on) and they give you zero control over fan settings, you may want to snag something like TG Pro for the Mac. I wound up buying a license for it, this lets you define at which temperature you want to run your fans and even gives you manual control.
On my 24G RAM Macbook Pro I have about 16GB of Inference. I use Zed with LM Studio as the back-end. I primarily just use Claude Code, but as you note, I'm sure if I used a beefier Mac with more RAM I could probably handle way more.
There's a few models that are interesting on the Mac with LM Studio that let you call tooling, so it can read your local files and write and such:
mistralai/mistralai-3-3b this one's 4.49GB - So I can increase my context window for it, not sure if it auto-compacts or not, have only just started testing it
zai-org/glm-4.6v-flash - This one is 7.09GB, same thing, only just started testing it.
mistralai/mistral-3-14b-reasoning - This one is 15.2GB just shy of the max, so not a TON of wiggle room, but usable.
If you're Apple or a company that builds things for Macs or other devices, please build something to help with airflow / cooling for the MBP / Mac Mini, it feels ridiculous that it becomes a 100c device I'm not so sure its great for device health if you want to use inference for longer than the norm.
I will probably buy a new Mac whenever the inference speeds increase at a dramatic enough rate. I sure hope Apple is considering serious options for increasing inference speed.
An even easier way to get into this is simply by downloading a program called LM Studio. You can mount a model and chat to it within 10-15 mins with no experience whatsoever, and no configuration at all.
That said, last time I tried local LLMs (around when gpt-oss came out) it still seemed super gimmicky (or at least niche, I could imagine privacy concerns would be a big deal for some). Very few use cases where you want an LLM but can't benefit immensely from using SOTA models like Claude Opus.
The big AI labs are almost certainly selling inference below cost and burning mountains of money. With the insane increase in hardware prices, running models locally just doesn’t make any financial sense.
Nobody is saying it makes "financial sense", it's about control.
I have always taken plenty of care to try and avoid becoming dependent on big tech for my lifestyle. Succeeded in some areas failed in others.
But now AI is a part of so many things I do and I'm concerned about it. I'm dependent on Android but I know with a bit of focus I have a clear route to escape it. Ditto with GMail. But I don't actually know what I'd do tomorrow if Gemini stopped serving my needs.
I think for those of us that _can_ afford the hardware it is probably a good investment to start learning and exploring.
One particular thing I'm concerned about is that right now I use AI exclusively through the clients Google picked for me, coz it makes financial sense. (You don't seem to get free bubble money if you buy tokens via API billing only consumer accounts). This makes me a bit of a sheep and it feels bad. There's so much innovation happening and basically I only benefit from it in the ways Google chooses.
(Admittedly I don't need local models to fix that particular issue, maybe I should just start paying the actual cost for tokens).
I really hope at some point in the near future AI models shrink enough or laptops get strong enough to run AI models locally. I haven't tried in the past year, but when I did it was very slow token output + laptop was on fire to make that happen.
I've wanted to try some of the more recent 8B models for local tab completion or agentic, any experience with those kinds of smaller models?
> NTransformer
High-efficiency C++/CUDA LLM inference engine. Runs Llama 70B on a single RTX 3090 (24GB VRAM) by streaming model layers through GPU memory via PCIe, with optional NVMe direct I/O that bypasses the CPU entirely.
Isn't between Q4-Q6 the usual recommendation for quants? Can you explain the Q8 recommendation, as I was under the impression that if you can run a model at Q8, you should probably run a bigger model in Q4 instead
I just can't help but imagine ChatGPT's sycophancy mixed with military operations. "Sharp insight bombing that wedding! Next would you like tips on mosques to bomb, or I can suggest some new napalm recipes that are extra spicey. Your call!"
Department of Defense: You just bombed the wrong Georgia! The people of Atlanta are furious!
ChatGPT: You're absolutely right, and you're right to call that out. Upon examination it does appear that there might have been a mistake with the coordinates of the bomb. Let's try again, this time we will double check before we launch any missiles! :missile emoji:
The point is it will be autonomous, the prompt could just be 'keep me safe' which will be interpreted who knows how and presumably no further prompting.
Assuming it's not smart enough to write logs that make it less likely to be prosecuted/ disabled by coming up with fake reasons.
It can just say you were a terrorist because you were an adult male traveling with something in your hands. Humans already do this to justify strikes, likely the AI would do the same.
I actually cancelled my ChatGPT subscription in late 2024 and documented the process, kind of as a social media thing because it had gotten so bad and I realized nobody in my family was using it anymore. I asked my wife if she was getting any use out of it and she told me she had been using Gemini and Grok for months because "GPT is very lazy now".
After a while another charge came in for the subscription, but I had the receipts: we had cancelled before the next billing cycle. I decided to try and reach out to OpenAI to resolve this, but they only let you chat with GPT itself for this, which it failed at and told me they weren't in the wrong and none of the information matched what actually happened.
I took this and used it to submit a chargeback request with Privacy.com, which I use for all of my online purchases. Normally I don't have to worry about this because I set a limit or cancel the cards I issue manually, but I had an OpenAI API account using the same card and I had been a bit lazy in using the same card for technically two different services.
Well, Privacy.com won that dispute and I got that money back. It's worth mentioning this is actually different than most banks will do now days. For the most part when you try to get a bank to do a chargeback they just roll it into their insurance and refund you the customer as a cost of doing business, but the actual scammer or shady merchant got to keep their stolen money, whereas I can be certain OpenAI didn't keep my money.
Your last statement is false. A shady merchant never gets to keep the stolen money. The card issuer/bank refunds you immediately because of consumer protection laws. But that charge is immediately charged to the processor. The processor then gets the merchant involved in a dispute process. If the merchant loses the processor charges the merchant. One way they do it is to immediately deduct it from their current processed transactions. If the merchant is no longer processing, they will usually go try to claw it back from their bank account if they have no held reserves, and if they can't get it, they send the merchant to collection. At the end the merchant must eat the cost or the processor. So in your case, the bank didn't eat the cost. OpenAI certainly ate the cost and the chargeback fees.
> Your last statement is false. A shady merchant never gets to keep the stolen money.
Or any merchant for that matter. Chargebacks (from bad actors) are one of the most annoying things when you sell online when you’re a honest legit business. Stripe even charges you a penalty fee on top of that.
Chase uses a "provisional credit" system, but for small amounts, this credit often becomes permanent almost instantly.
Wells Fargo utilizes an automated system called the Wells Fargo Dispute Manager which is also similar.
Technically, it is Self-Insurance. Banks set aside a portion of their interchange revenue (the fees they charge merchants for every swipe) into a "Provision for Credit Losses." They use this pool of money to "buy" customer satisfaction for small errors rather than paying an employee $30/hour to investigate a $12 dispute.
Your credibility is shot when you claim that banks will just give you money. They absolutely do not. In fact, Discover has admitted to me in writing, that they always rule in favor of the Merchant if that Merchant responds to the dispute -- regardless of what their response says.
I've dealt with multiple chargebacks over the years and have only ever lost once -- when the Manager at Lowes' showed a check they wrote me [after I opened the dispute].
They absolutely do not just do anything and "write it off".
Please be human and don't just rattle of high-confidence, baseless claims, especially as a giant billboard to Privacy.com
> Discover has admitted to me in writing, that they always rule in favor of the Merchant if that Merchant responds to the dispute -- regardless of what their response says
What, always? Like, literally 100% of the time if the merchant responds at all, they automatically win?
That's very hard to believe. I don't know Discover but I do know Visa and that's not how their system works at all.
I use Amex as much as possible because it’s basically never a fight. If I dispute, I get my money back. Granted, I don’t abuse the power so maybe I’ve earned some trust over the decades.
> Well, Privacy.com won that dispute and I got that money back.
Well, it seems like ChatGPT’s automated litigation resolution with Privacy.com got lazy. I wonder how a company with an AI can lose in a dispute instead of smokescreening the opponent with legitimate arguments and legalese.
It helps when you have a video posted on social media the day you cancelled and a video of talking to a clueless AI customer retention system that seems to not agree or understand how time works.
Also, chargeback dispute is limited to 3 rounds of back and fourth by Visa and MasterCard both. They don't get to endlessly come back etc.
I had been considering ditching everyday ChatGPT use in favor of Claude anyway, but hadn’t gotten around to it mostly out of habit. Now I have a good reason to do it.
Same, I had put Claude in my metaphorical shopping cart about two weeks ago but I already had some inertia with ChatGPT + Codex and figured it wouldn't be better enough to justify changing.
That has changed, so I canceled my ChatGPT membership and signed up for Claude. I still have five bucks of credit I bought a year ago for the OpenAI API that I do not believe I can have refunded back, so some of my apps are going to have to stick to OpenAI until those credits run out since I'm not going to just donate five bucks to them.
Playing with it now, I honestly can't tell too much of a difference, which as far as I am concerned is a good thing.
Consider carefully the usage limits of both services before deleting your account (as you cannot create a new one later with the same email). Claude's €20/month sub offers very little and this has unfortunately kept me from switching when I tried earlier this month.
I had been using both, ChatGPT mostly for chats and Claude mostly for code. Now I cancelled the ChatGPT subscription and turned on extra usage in Claude instead.
I get a fair amount of use out of it. I'm not using it for professional software development, just hobby stuff that I don't to write the boring parts of. For 20 bucks a month that seems pretty reasonable.
What value does this give you? Part of why I deleted my account was I couldn't think of a single thing of value in my chats from the past couple years? Maybe some nostalgia looking at what bugs I was fixing?
For me this is very valuable. The results of personal "research projects" are in there. I use it for reference. Of course I could ask Claude to get me those answers but why waste the energy?
Thanks, but I guess I understand the sentiment. I probably should have not said that I couldn't think of "a single thing of value" when that is a bit of a judgement along with my question. Anyways, it is interesting hearing what people ask it, I think I've only ever used it like a search engine / bug fixing while it seems some people have much deeper conversations or discussions that are worth remembering.
I'm glad I upvoted. Your perspective and questions are valid, no matter the depth of conversation. You'd be surprised what fresh questions can do for a topic.
I for one might use these chats as an input for switching over to keep the learning process fast. For me it took a while for ChatGPT to get me. I know that other people delete memories because they want a clean slate experience with every chat. I use chatGPT mostly private (use claud code for work for instance) and I prefer that memories travel across chats.
I'm gonna have to see if I can get my company to switch off openAI. Hopefully we can make a small dent and if enough of us do it, a larger dent.
Sounds like it won't really be a pain for me though based off comments on HN indicating Claude is the better product and I doubt I personally would hit any sort of token limits with the amount I use agentic coding.
I've found the free tier to be extremely limited recently, but in a stochastic way. Some days I ask one friggin question and it tells me I only have two questions left and should upgrade. I just switch to a different model.
I re-registered an account last year to the same email that was used previously. That account was deleted in 2023. Although they used to request your phone number back in the day and do not do so anymore, so email retention policies may also have changed.
A few days ago I went to cancel mine and it just said they'd give me a free month instead so I said OK. I thought it was funny all the patterns to keep you on
I love that the tool in question is very calm and collected, in contrast with the emotional wreck that is the US regime. I got a very helpful response to this prompt and I will make it continue working on a python script to get my historical chats looking good in Obsidian.
> Ok. So I'm cancelling the subscription to ChatGPT and moving over to Claude because of the news of OpenAI striking a deal with us department of war. (https://www.techradar.com/pro/openai-just-signed-a-huge-deal...) Please line out a good exit strategy where I can keep the information in my chats and projects on my own hard drive.
How long does data export usually take for three years of medium usage? I started it eight hours ago, got a confirmation email that export had started but so far no email with a download link.
Virtue signaling is when you stop giving people your money because you don't like what they do.
It's hilarious how mad the hogs get when you suggest maybe not supporting their powerful daddies. It doesn't matter which daddy it is, inevitably taking your ball and going home is 'virtue signaling'
I canceled all services and deleted my account with OpenAI right after the announcement. They can get money from the current US regime but I will not contribute to their violations of the constitution.
It's frustrating. Sam Altman already has everything. He's a billionaire, he can buy literally anything he wants, he can live anywhere he wants, he can buy a brand new sports car every day just to blow it up, he can buy a new house every week just to demolish and replace it with a trampoline park. He can afford to do anything.
He can fucking afford to have some fucking principles. He's not going to end up on the street for not being a fucking coward.
Because of some bullshit minor PTSD from a few years ago, I sort of swore an oath to myself that I wouldn't let being a coward stop me from doing the right thing, regardless of the consequences, and by doing things that I think are right it has cost me opportunities and money. I'm not homeless, but it made the job hunt harder when I was unemployed. I can actually feel consequences from standing up for what I believe in. Sam Altman being a coward is not equivalent, he's choosing to do the wrong thing for no reason.
> He can fucking afford to have some fucking principles.
Who is to say he doesnt? Just because they dont align with yours doesnt mean he doesnt have his own principles.
> he's choosing to do the wrong thing
To many millions he is doing the right thing. I am on the fence personally, but I know many people who think that increasing defense capabilities at any cost is something that the governmetn should be doing. Any company that helps them do that is 'doing the right thing'.
> I wouldn't let being a coward stop me from doing the right thing
The 'right thing' is always subjective, and for you it is decided by you alone. Try to remember that and see things from both sides.
He posted like seven hours ago about these principles and changed them like twenty minutes after the president had a temper tantrum about Anthropic.
Whether or not he agrees with my principles isn’t the issue. He doesn’t even agree with his own stated principles. He posted his stipulations about AI models used by the department of defense to presumably get social credit, and then changed his mind over the course of a few hours.
He claims that the Department of Defense principles just happen to now align with these principles but as far as I can tell he seems to just be trusting their word. The word of a Fox News TV host and a convicted fraudster.
You can judge his actions all you like, but unless you know the man and sit down and discuss it with him everything you are saying is just speculation and opinion. That is fine, just realise that.
Umm, no shit? This isn’t actually saying anything. I am saying that his stated principles are inconsistent with his actions.
Yes, sure, maybe deep down on his heart of hearts he actually is the most kind caring person who ever existed, but I have no way of knowing what is in his heart of hearts so I can only judge him by what he has said and done, and I am arguing that his actions don’t match his words.
You really think that a CEOs public statements match their real thoughts and feelings?
Everything you hear from any CEO is constructed to form a desired image and narrative. Any public statements from a CEO that you are using to judge their character is completely false and misguided. You have no idea what the real man is behind the image.
> It's frustrating. Sam Altman already has everything. He's a billionaire, he can buy literally anything he wants, he can live anywhere he wants, he can buy a brand new sports car every day just to blow it up, he can buy a new house every week just to demolish and replace it with a trampoline park. He can afford to do anything.
No, he doesn't have everything. See, maybe he's worth $3 billion. Or maybe $30 billion. But he's not worth $300 billion. That's a lot more worth he could have! And even then, he could be worth $3 trillion instead!
But yes, $100 million is the maximum amount of assets one individual should ever be allowed to hold. Potentially less. Anything higher is enormously harmful to society. People would get used to it very quickly and would work just as hard to reach that $100 million as they do now to reach $100 billion.
“Yes, but I have something he will never have — ENOUGH” - Joseph Heller.
After a billion dollars, I doubt another billion will make you happier. In fact, I don’t think another trillion will make you happier. In fact, I don’t think another quadrillion dollars will make you happier, etc.
After a certain point you have effectively infinite money. Enough money to live dozens of extremely comfortable lifetimes. And importantly enough money to afford to actually have some principles. Oh no, he wouldn’t be able to afford to have his house re-covered in 24 karat gold again if he doesn’t fellate our lolcow president.
We live in a capitalist economy. What do you expect, a company to just say 'Thats fine we have enough money we dont need any more'?
How does a $100 billion dollar company grow? By taking on massive government and military contracts, they are the only clients big enough left in the world.
If a company does not show continual growth then it is classed as failing. That is the society we have built, and you cannot blame one man for following those principles. Every CEO in existence does the same.
Somehow Anthropic’s CEO managed to reach a different conclusion.
They don’t have to do business with every single entity who asks them to and they don’t have to bend over for every stipulation that that entity asks for.
Yep, in exactly the same way that for years Google was 'not evil'.
When anthropic can no longer grow through developer subscriptions and deals with ethical companies, Lets see how long it takes their shareholders to force them to remove such sweet statements from their company mission statement.
"Only when the tide goes out do you discover who's been swimming naked." -- Warren Buffet
The original context was very different, about financial markets, but I've been thinking about it a lot the past 12 months. There's a lot of cowards in high places in tech, surprisingly cowardly people. Or they have sold out their principles to be friends with terrible people, which is also a form of cowardice. Hard to say which.
The whole Epstein thing is a really really great marker of this too. Though I'm not sure if the tide has gone out all the way (we mostly know what's going on), or if there's a lot more tide to fall.
LBJ was a real son of a bitch, who, when he finally was thrust into power as president, did something pretty surprising by going all-in on the civil rights movement. Power reveals who people are, and times of trials reveal who people really are.
I think we merely have a system where the best people are selfless and poor and the worse people are rich and in charge. It makes sense; we have a system that rewards immoral behavior so we shouldn't be surprised that immoral people have made it to the top.
Such systems are nothing new, and are in fact the norm. The current system is perhaps even notable for how it has deviated from the past, and in particular Silicon Valley was a means for promoting some of the most selfless and poor into positions of great wealth and influence, especially going back to the Fairchild Semiconductor days. Always been greedy venal and immoral people here, but perhaps less than in other systems of power.
The stoics, people that Zuckerberg and others pretend to understand and follow, would have nothing but disdain for the lack of virtue that's apparent in those like Zuckerberg.
History is full of cowards who are arguably as guilty as the people who committed the atrocities. The people who are remembered positively in history are the people who overcame their fears and did what they thought was right, even if it carried a real risk of it blowing up in their faces.
Sam (and Greg Brockman) want something they do not have, very desperately. They want to win, to be Great Men, to be remembered by history with Jobs and Gates and the other tech luminaries. This is mentioned in Karen Hao's Empire of AI.
They are both a lesson to me that no matter how much you have, you will not necessarily be satisfied.
Anthropic must be in some way better in that they do have some red lines and do truly stand on them (and if they didn’t, like every other company doesn’t seem to, we’d never have even known)?
anthropic previously agreed to deploy their models in this context with nothing but a contract to enforce their red lines -- they even disabled their safety systems!
per announcement, openai can include safety systems of their own making, including ones to prevent their red lines from being crossed. that seems to be a more robust solution, including in the face of an untrustworthy government
Notice that Anthropic doesn't support spying on Americans specifically - spying on anyone else is fine. Can't spy on three specific 300m people, but spying on the other 7.7B totally ok.
I'm afraid that AI weapons will follow a similar dynamic to nuclear ones where, as much as we'd like to avoid them, someone will build them. Which means, everyone will need them. I'm worried that we're repeating the pre-Ukraine war mindset of US Tech keeping their distance from defense while other countries have a joined tech/military base.
I never really understood people's need to post these cynical doomer posts. "Things can't be perfect so don't bother doing anything ever I am so smart".
Will a few dozen people canceling their accounts change anything? Probably not, but at least we know that we're not actively giving our money to Sam Altman.
There's not a lot in the world that any of us have control over. Most of us aren't billionaires who can buy a government. Really the only variable we have any amount of freedom with is how we spend our money.
As a former soldier and as someone who married into a family from the now russian occupied parts of Ukraine i feel that this is a great mindset, but also somewhat of a luxury believe. I agree that ideally we'd stand up to aggression and weapon production and that all other citizens around the world would do the same, and we'd live in peaceful equilibrium.
But they don't- and so our best bet is to be so strong that no one wants to attack us. For that, we can't leave the cutting edge of military technology to others.
This mindset used to be anathema to the tech community, but then briefly changed after the Russian invasion of Ukraine as people briefly understood that there are in fact aggressive actors in the world and war might come to us. But it seems we went back on that.
This administration has relentlessly demonstrated they will let no law or human right get in the way of their accumulation of unchecked power.
He tried to take congress's power but not even his hand picked SCOTUS lackeys would permit that.
So instead he kidnapped presidents, had masked unaccountables shooting people in the streets, is threatening to seize elections all while covering up countless crimes by flushing enough evidence down toilets that they needed plumbers to come out.
Along with that, defunding science, shutting down research centers, tearing up international treaties, threatening to invade Canada and Denmark all while building 24 camps and shutting down pbs.
He runs MLMs and cryptocurrency pump and dumps from a demolished Whitehouse where he peddles cheap glitzy trinkets from his online store, sells pardons, and tried to orchestrate a coup.
This is a Whitehouse that uses the 14 words, makes references to 1488, puts out AI deep fakes and fraudulent photographs as press releases that read like North Korean propaganda. One that defunds the weather service because of conspiracy theories.
One that shuts down battery research, puts mercury in the air, pollutes water, and relentlessly defunds and dismantles powerful growth sectors.
There is no excuse whatsoever for empowering them.
If any replies accuse me of being a democrat (I'm not) or try to deflect, I will not engage.
It's an effortful, exhaustive, strategic sabotaging of the american economy and our fundamental civil government.
OpenAI step in to work with defence department on stuff so questionable that another company took a public stance to distance itself from?
Myself, I’ve always “followed the money” when the current administration has taken public positions on things from media company mergers to data centres etc. So a bit of me wonders how much of the “anthropic is a threat to national security” is genuine and how much is about getting another company into lucrative defence contracts instead?
Trump family has major investments in data centers etc and is heavy benefiting from OpenAI footprint but they recently declined an investment opportunity in anthropic citing it’s political leanings
What does domestic surveillance have to do with national security objectives or keeping denizens of the U.S. safe (that is, the ones who are not lucky enough to be in its cosseted ruling class)?
A laptop with an iGPU and loads of system RAM has the advantage of being able to use system ram in addition to VRAM to load models (assuming your gpu driver supports it, which most do afaik), so load up as much system RAM as you can. The downside is, the system RAM is less fast than dedicated GDDR5. These GPUs would be Radeon 890M and Intel Arc (previous generations are still decently good, if that's more affordable for you).
A laptop with a discrete GPU will not be able to load models as large directly to GPU, but with layer offloading and a quantized MoE model, you can still get quite fast performance with modern low-to-medium-sized models.
Do not get less than 32GB RAM for any machine, and max out the iGPU machine's RAM. Also try to get a bigass NVMe drive as you will likely be downloading a lot of big models, and should be using a VM with Docker containers, so all that adds up to steal away quite a bit of drive space.
Final thought: before you spend thousands on a machine, consider that there are at least a dozen companies that provide non-Anthropic/non-OpenAI models in the cloud, many of which are dirt cheap because of how fast and good open weights are now. Do the math before you purchase a machine; unless you are doing 24/7/365 inference, the cloud is fastly more cost effective.
Also, because Apple in their infinite wisdom despite giving you a fan, very lazily turn it on (I swear it has to hit 100c before it comes on) and they give you zero control over fan settings, you may want to snag something like TG Pro for the Mac. I wound up buying a license for it, this lets you define at which temperature you want to run your fans and even gives you manual control.
On my 24G RAM Macbook Pro I have about 16GB of Inference. I use Zed with LM Studio as the back-end. I primarily just use Claude Code, but as you note, I'm sure if I used a beefier Mac with more RAM I could probably handle way more.
There's a few models that are interesting on the Mac with LM Studio that let you call tooling, so it can read your local files and write and such:
mistralai/mistralai-3-3b this one's 4.49GB - So I can increase my context window for it, not sure if it auto-compacts or not, have only just started testing it
zai-org/glm-4.6v-flash - This one is 7.09GB, same thing, only just started testing it.
mistralai/mistral-3-14b-reasoning - This one is 15.2GB just shy of the max, so not a TON of wiggle room, but usable.
If you're Apple or a company that builds things for Macs or other devices, please build something to help with airflow / cooling for the MBP / Mac Mini, it feels ridiculous that it becomes a 100c device I'm not so sure its great for device health if you want to use inference for longer than the norm.
I will probably buy a new Mac whenever the inference speeds increase at a dramatic enough rate. I sure hope Apple is considering serious options for increasing inference speed.
That said, last time I tried local LLMs (around when gpt-oss came out) it still seemed super gimmicky (or at least niche, I could imagine privacy concerns would be a big deal for some). Very few use cases where you want an LLM but can't benefit immensely from using SOTA models like Claude Opus.
I have always taken plenty of care to try and avoid becoming dependent on big tech for my lifestyle. Succeeded in some areas failed in others.
But now AI is a part of so many things I do and I'm concerned about it. I'm dependent on Android but I know with a bit of focus I have a clear route to escape it. Ditto with GMail. But I don't actually know what I'd do tomorrow if Gemini stopped serving my needs.
I think for those of us that _can_ afford the hardware it is probably a good investment to start learning and exploring.
One particular thing I'm concerned about is that right now I use AI exclusively through the clients Google picked for me, coz it makes financial sense. (You don't seem to get free bubble money if you buy tokens via API billing only consumer accounts). This makes me a bit of a sheep and it feels bad. There's so much innovation happening and basically I only benefit from it in the ways Google chooses.
(Admittedly I don't need local models to fix that particular issue, maybe I should just start paying the actual cost for tokens).
I've wanted to try some of the more recent 8B models for local tab completion or agentic, any experience with those kinds of smaller models?
untested:
https://github.com/xaskasdf/ntransformer
ChatGPT: You're absolutely right, and you're right to call that out. Upon examination it does appear that there might have been a mistake with the coordinates of the bomb. Let's try again, this time we will double check before we launch any missiles! :missile emoji:
(Spoilers for the ending of the movie: https://youtu.be/h73PsFKtIck?si=tTm9TidmEMBHsXq1 )
It can just say you were a terrorist because you were an adult male traveling with something in your hands. Humans already do this to justify strikes, likely the AI would do the same.
Alternatively: Assuming it's smart enough not to consider logging to /dev/null a reasonable way to speed up execution times.
I actually cancelled my ChatGPT subscription in late 2024 and documented the process, kind of as a social media thing because it had gotten so bad and I realized nobody in my family was using it anymore. I asked my wife if she was getting any use out of it and she told me she had been using Gemini and Grok for months because "GPT is very lazy now".
After a while another charge came in for the subscription, but I had the receipts: we had cancelled before the next billing cycle. I decided to try and reach out to OpenAI to resolve this, but they only let you chat with GPT itself for this, which it failed at and told me they weren't in the wrong and none of the information matched what actually happened.
I took this and used it to submit a chargeback request with Privacy.com, which I use for all of my online purchases. Normally I don't have to worry about this because I set a limit or cancel the cards I issue manually, but I had an OpenAI API account using the same card and I had been a bit lazy in using the same card for technically two different services.
Well, Privacy.com won that dispute and I got that money back. It's worth mentioning this is actually different than most banks will do now days. For the most part when you try to get a bank to do a chargeback they just roll it into their insurance and refund you the customer as a cost of doing business, but the actual scammer or shady merchant got to keep their stolen money, whereas I can be certain OpenAI didn't keep my money.
Or any merchant for that matter. Chargebacks (from bad actors) are one of the most annoying things when you sell online when you’re a honest legit business. Stripe even charges you a penalty fee on top of that.
Chase uses a "provisional credit" system, but for small amounts, this credit often becomes permanent almost instantly.
Wells Fargo utilizes an automated system called the Wells Fargo Dispute Manager which is also similar.
Technically, it is Self-Insurance. Banks set aside a portion of their interchange revenue (the fees they charge merchants for every swipe) into a "Provision for Credit Losses." They use this pool of money to "buy" customer satisfaction for small errors rather than paying an employee $30/hour to investigate a $12 dispute.
I've dealt with multiple chargebacks over the years and have only ever lost once -- when the Manager at Lowes' showed a check they wrote me [after I opened the dispute].
They absolutely do not just do anything and "write it off". Please be human and don't just rattle of high-confidence, baseless claims, especially as a giant billboard to Privacy.com
What, always? Like, literally 100% of the time if the merchant responds at all, they automatically win?
That's very hard to believe. I don't know Discover but I do know Visa and that's not how their system works at all.
Go read your banks terms and you'll find the provision. Do you want me to read your banks terms for you and point them out?
Well, it seems like ChatGPT’s automated litigation resolution with Privacy.com got lazy. I wonder how a company with an AI can lose in a dispute instead of smokescreening the opponent with legitimate arguments and legalese.
Also, chargeback dispute is limited to 3 rounds of back and fourth by Visa and MasterCard both. They don't get to endlessly come back etc.
That has changed, so I canceled my ChatGPT membership and signed up for Claude. I still have five bucks of credit I bought a year ago for the OpenAI API that I do not believe I can have refunded back, so some of my apps are going to have to stick to OpenAI until those credits run out since I'm not going to just donate five bucks to them.
Playing with it now, I honestly can't tell too much of a difference, which as far as I am concerned is a good thing.
Sounds like it won't really be a pain for me though based off comments on HN indicating Claude is the better product and I doubt I personally would hit any sort of token limits with the amount I use agentic coding.
BTW, what's going to hurt their business more, deleting my account or using the free tier?
So if I no longer want their services for any period of time, they no longer want me as a customer for any reason?
What other business work like that?
> Ok. So I'm cancelling the subscription to ChatGPT and moving over to Claude because of the news of OpenAI striking a deal with us department of war. (https://www.techradar.com/pro/openai-just-signed-a-huge-deal...) Please line out a good exit strategy where I can keep the information in my chats and projects on my own hard drive.
https://github.com/openai/codex/issues/26#issuecomment-28116...
2. Click on your profile icon and select New Chat icon.
3. Formulate a polite prompt in the regard of subscription cancellation.
4. Wait for a reply from Mr. Altman.
It's hilarious how mad the hogs get when you suggest maybe not supporting their powerful daddies. It doesn't matter which daddy it is, inevitably taking your ball and going home is 'virtue signaling'
Anthropic usage credits purchased.
Message those that work forces.
mv: 'OpenAI': No such file or directory
bash> ls
ClosedAI
He can fucking afford to have some fucking principles. He's not going to end up on the street for not being a fucking coward.
Because of some bullshit minor PTSD from a few years ago, I sort of swore an oath to myself that I wouldn't let being a coward stop me from doing the right thing, regardless of the consequences, and by doing things that I think are right it has cost me opportunities and money. I'm not homeless, but it made the job hunt harder when I was unemployed. I can actually feel consequences from standing up for what I believe in. Sam Altman being a coward is not equivalent, he's choosing to do the wrong thing for no reason.
Who is to say he doesnt? Just because they dont align with yours doesnt mean he doesnt have his own principles.
> he's choosing to do the wrong thing
To many millions he is doing the right thing. I am on the fence personally, but I know many people who think that increasing defense capabilities at any cost is something that the governmetn should be doing. Any company that helps them do that is 'doing the right thing'.
> I wouldn't let being a coward stop me from doing the right thing
The 'right thing' is always subjective, and for you it is decided by you alone. Try to remember that and see things from both sides.
Whether or not he agrees with my principles isn’t the issue. He doesn’t even agree with his own stated principles. He posted his stipulations about AI models used by the department of defense to presumably get social credit, and then changed his mind over the course of a few hours.
He claims that the Department of Defense principles just happen to now align with these principles but as far as I can tell he seems to just be trusting their word. The word of a Fox News TV host and a convicted fraudster.
Yes, sure, maybe deep down on his heart of hearts he actually is the most kind caring person who ever existed, but I have no way of knowing what is in his heart of hearts so I can only judge him by what he has said and done, and I am arguing that his actions don’t match his words.
Everything you hear from any CEO is constructed to form a desired image and narrative. Any public statements from a CEO that you are using to judge their character is completely false and misguided. You have no idea what the real man is behind the image.
Until that line has been reached, we can safely assume there are no principles at play.
Ugh
No, he doesn't have everything. See, maybe he's worth $3 billion. Or maybe $30 billion. But he's not worth $300 billion. That's a lot more worth he could have! And even then, he could be worth $3 trillion instead!
But yes, $100 million is the maximum amount of assets one individual should ever be allowed to hold. Potentially less. Anything higher is enormously harmful to society. People would get used to it very quickly and would work just as hard to reach that $100 million as they do now to reach $100 billion.
After a billion dollars, I doubt another billion will make you happier. In fact, I don’t think another trillion will make you happier. In fact, I don’t think another quadrillion dollars will make you happier, etc.
After a certain point you have effectively infinite money. Enough money to live dozens of extremely comfortable lifetimes. And importantly enough money to afford to actually have some principles. Oh no, he wouldn’t be able to afford to have his house re-covered in 24 karat gold again if he doesn’t fellate our lolcow president.
How does a $100 billion dollar company grow? By taking on massive government and military contracts, they are the only clients big enough left in the world.
If a company does not show continual growth then it is classed as failing. That is the society we have built, and you cannot blame one man for following those principles. Every CEO in existence does the same.
We live in a multifaceted (are we allowed to use that word again? I think 2026 models have stopped using it) economy.
> That is the society we have built
Maybe you have, I sure haven't. Luckily "we" also haven't, as many - no, the overwhelming majority of people - aren't like that.
> and you cannot blame one man for following those principles.
You absolutely can, as much as you can blame the guards at Auschwitz.
> Every CEO in existence does the same.
A shocking, bald-faced lie. This is on the level of "the Mexicans are eating cats and dogs out there".
They don’t have to do business with every single entity who asks them to and they don’t have to bend over for every stipulation that that entity asks for.
When anthropic can no longer grow through developer subscriptions and deals with ethical companies, Lets see how long it takes their shareholders to force them to remove such sweet statements from their company mission statement.
Is this really the best backup?
Sam Altman has demonstrated that he's a piece of ** with this move.
We can now safely assume that all the pronouncements and grand statements before were simply posturing.
The original context was very different, about financial markets, but I've been thinking about it a lot the past 12 months. There's a lot of cowards in high places in tech, surprisingly cowardly people. Or they have sold out their principles to be friends with terrible people, which is also a form of cowardice. Hard to say which.
The whole Epstein thing is a really really great marker of this too. Though I'm not sure if the tide has gone out all the way (we mostly know what's going on), or if there's a lot more tide to fall.
LBJ was a real son of a bitch, who, when he finally was thrust into power as president, did something pretty surprising by going all-in on the civil rights movement. Power reveals who people are, and times of trials reveal who people really are.
The stoics, people that Zuckerberg and others pretend to understand and follow, would have nothing but disdain for the lack of virtue that's apparent in those like Zuckerberg.
I completely support the sentiment of what you wrote. But it doesn't directly seem relevant to the parent question.
Very few of the comments on this thread are actually about the act of canceling the subscription.
They are both a lesson to me that no matter how much you have, you will not necessarily be satisfied.
per announcement, openai can include safety systems of their own making, including ones to prevent their red lines from being crossed. that seems to be a more robust solution, including in the face of an untrustworthy government
But the desire and ability to control non-citizens en masse is nothing like the threat a government is to its own citizens.
In practice the two activities have very different natures and impact.
I'm afraid that AI weapons will follow a similar dynamic to nuclear ones where, as much as we'd like to avoid them, someone will build them. Which means, everyone will need them. I'm worried that we're repeating the pre-Ukraine war mindset of US Tech keeping their distance from defense while other countries have a joined tech/military base.
Will a few dozen people canceling their accounts change anything? Probably not, but at least we know that we're not actively giving our money to Sam Altman.
There's not a lot in the world that any of us have control over. Most of us aren't billionaires who can buy a government. Really the only variable we have any amount of freedom with is how we spend our money.
He tried to take congress's power but not even his hand picked SCOTUS lackeys would permit that.
So instead he kidnapped presidents, had masked unaccountables shooting people in the streets, is threatening to seize elections all while covering up countless crimes by flushing enough evidence down toilets that they needed plumbers to come out.
Along with that, defunding science, shutting down research centers, tearing up international treaties, threatening to invade Canada and Denmark all while building 24 camps and shutting down pbs.
He runs MLMs and cryptocurrency pump and dumps from a demolished Whitehouse where he peddles cheap glitzy trinkets from his online store, sells pardons, and tried to orchestrate a coup.
This is a Whitehouse that uses the 14 words, makes references to 1488, puts out AI deep fakes and fraudulent photographs as press releases that read like North Korean propaganda. One that defunds the weather service because of conspiracy theories.
One that shuts down battery research, puts mercury in the air, pollutes water, and relentlessly defunds and dismantles powerful growth sectors.
There is no excuse whatsoever for empowering them.
If any replies accuse me of being a democrat (I'm not) or try to deflect, I will not engage.
It's an effortful, exhaustive, strategic sabotaging of the american economy and our fundamental civil government.
Myself, I’ve always “followed the money” when the current administration has taken public positions on things from media company mergers to data centres etc. So a bit of me wonders how much of the “anthropic is a threat to national security” is genuine and how much is about getting another company into lucrative defence contracts instead?
Trump family has major investments in data centers etc and is heavy benefiting from OpenAI footprint but they recently declined an investment opportunity in anthropic citing it’s political leanings