So, as an unemployed person I'm trying to work my way into AI. I'm building an app that puts higgsfield like generation in your mobile phone.
As I'm doing that I'm using a lot of inference. I have adopted Hermes as a harness and bounce around mostly between Nous Research, Openrouter, opencodego, and Nvidia for inference.
I have run out of my monthly quota with basically all of it and I don't want to keep throwing money at it. So, recently I have purchased a RTX 3090 and it fits into my rig well enough. Alongside of it is a 5070. The 5070 has 16 speedy GB of Vram, which does lots of things well, but local AI is using somewhere between 16-24 depending on how you set it up.
I keep trying different toolsets, but I keep running into a CUDA problem between having a 3090 and a 5070. Apparently this hasn't been patched yet, and it's a known bug/problem.
So, I've been pulling my hair out trying to get local AI to work. The gold standard right now is QWEN 3.6-27B and my goal is to have MTP working since that's supposed to speed it up quite a bit (processes more than 1 token at a time).
Last night I bricked my comp by trying to down grade my nvidia driver. Had to disassemble and resassemble it a few times with help from google ai to try to get it all back to functional.
Fell asleep around 4am, and back at it this morning.
Nous Research sometimes has new models cycling on free inference for testing and marketing. I think I'm using Step 3.5 Flash, which is a nice little model. I haven't tried calling it for much app building, but it's running tests seemlessly to try to get local Qwen working and it needs practically no attention from me to do so.
I've been testing Grok a little. Grok is an amazing research and planning tool. I am a huge fan, but I can't stand the build experience. I've tried 4.2, 4.3, and build .1 and I hate them.
It takes so much cajoling, and even after it does something it can't debug anything it's built. it also can't operate on it's own for more than 30 seconds to a minute before it's asking me to check on it. It refuses to QA it's own work, or it does QA it and does a terrible job.
The XAI team is one of teh bests out there, so I'm not expecting this to be a major problem for too long, and they also now have cursor that's pretty good at programming in their toolkit as well.
Separately, I've filed a report with the FBI, local law enforcement, and am waiting to get connected through my attorney to a secret service team to try to get stolen funds back.
Need to wait a little longer to get some of the services back up and running after the hack. Chugging along.
I can't believe the number of terrible things happening in my life these days. Hoepfully it's a sign of breakthrough rather than further breakdown.
Oh, and as always I'm retardedly bullish on the long term for alts. My new favorite chart hit me recently and it's a 9 year wycof accumulation of alts vs btc. If it's accurate alts are going to go parabolic against BTC, which I also think is going to go parabolic.
I've thought it was going to do this years ago. So, I feel dumber by the day that it hasn't, but assuming I can hang in there long enough while getting squeezed by court, life, hacks, inference, and food then there's a glorious payday waiting. Until then it's just pain.
