- 17 Posts
- 296 Comments
6·12 hours agoThose are starting to become a thing
deleted by creator
What would you like to know about it?
I glossed over it the other day here -
https://aussie.zone/comment/24336560
From go to woah (including the compute), total project cost is about $1400 USD / ~$2K AUD. I realise I have the privilege of claiming that’s not a lot, but for a hobby spend that makes me happy / does useful stuff (see TV example above), it’s not a lot.
OK, here’s a real life use case, small time as it is, that has nothing to do with anything rarefied.
I have a TCL Android TV that, after some “unapproved measures”, runs without a Google account - FLauncher as the front end, telemetry blocked, home media streaming, YouTube ad-free. The yar har har works.
Two days ago the kids come in. TV’s not working.
Jellyfin: fine. Nova Player: fine. Stremio, YouTube, anything hitting the net: dead.
We’d had a brownout that day, so I figured the router was scrambled. Reset it, re-added the network… and Smarttube now launches, plays for 30 seconds, then hard-locks. Just weird JSON vomit and then dead.
At that point I’m thinking the TV is fucked. Options:
-
factory reset and reinstall the workarounds - 2-3hrs of pain-in-the-ass fuckery, but it wouldn’t be the first time.
-
crawl the net decoding cryptic error codes.
-
take it in for repair (good luck with that).
-
fall down a TV diagnostics rabbit hole.
Instead, I chose option 5)
I told my local LLM: "This TV is on this IP. This laptop has ADB access. I’ve given you MCP tools with ADB libraries. Here’s the issue, here’s what I tried. Check prior history of what I did for this project, figure out what’s wrong now and fix it, or tell me it’s broken.
If it looks like the brownout caused corruption, factory reset, stream-install the backup APKs directly, sign in my accounts, get it back up and running.
Failing that, if it’s looking like a hardware issue, push a diagnostic tool, test the TV and tell me which parts I might need, as well as likely costs using eBay or Amazon. I’m happy to spend $50 on this shitbox and not one cent more. If the fix looks like it’s more than $50, find me a TV of this size under $200. Prefer local shop X or Y.
Present everything as markdown in the usual folder, using house style. Ping me on my phone if you need me to authorise anything or when you’re done with the TL;DR."
The agent ADBs in, pokes around, tests, checks the WiFi, and figures out the brownout had corrupted the date and time.
Turns out this p.o.s doesn’t have a RTC back up battery.
Wrong clock = certs flagged as invalid = no net.
It syncs the date and time with my location. Boom, everything comes back.
Now, could I have twigged it myself?
Yes - eventually. But the agent already had the tools, knew the TV’s prior state, and had a record of what I’d already tried.
It could’ve reset the TV, sideloaded all my apps, re-signed into my accounts - the whole lot - if needed.
This is not the only time I’ve had real world use of my local, and it had very little to do with “writing functions”.
I had a project - jailbreak my tcl. Over time, I’d stored enough context of what I had done in the LLM and its tools, so that it could do what I’d do (and more), faster, based on the constraints I had put in place.
People that call these things “spicy autocorrect” are demonstrating either a skill issue or a lack of imagination.
-
1·2 days agoI’ve found Qwen 27b better adheres to rules and tool calls, but that depends on lots of things.
One that a bit of a dark horse is Arcee Trinity Mini (26B-A3B)
https://huggingface.co/arcee-ai/Trinity-Mini-GGUF
On paper it’s not in the same bookclub as the others (unless that bookclub involved eating books) but it is obedient with tool calls IMHE
Save us PrismML, you’re our only hope!
Just in defence of your point:
I prefer to self host, and am in the process of setting up my entire system to run off solar (tiny 2x250 watt solar array, 3 deep cycle 100Ah marine batteries and sundry equipment).
End to end, my AI rig - a BC-250 (a cut down PS5, basically) consumes 50w standby / 200w on inference and can run Qwen 3.6-27b (Ternary Bonsai specifically, about par with Sonnet 4.6 on the AAII) at good speeds (~30 tok/s)
With a smart power plug and Raspberry pi (uses about 5w) as the pilot fish, I can run my entire system “off grid”, to say nothing of the excess solar that accumulates (several days battery back up to run home router, pi server, hard drive array and 2-3hrs daily of inference).
For $1500 AUD, “Poor man’s Sonnet 4.6 at home” ain’t nothing to sneeze at.
Outside of folks such as myself, this blog post may be of interest -
https://blog.andymasley.com/p/individual-ai-use-is-not-bad-for
It’s too verbose, I find (system prompt and user setting be damned). Claude’s generally more to the point.
I still prefer to self host, and am in the process of setting up my entire system to run off solar (tiny 2x250 watt solar array, 3 deep cycle marine batteries and sundry equipment).
End to end, my AI rig - a BC-250 (a cut down PS5, basically) consumes 50w standby / 200w on inference and can run Qwen 3.6-27b (on par with Sonnet 4.5 on the AAII) at good speeds.
With a smart power plug and Raspberry pi (2w) as the pilot fish, I can run my entire system “off grid”, with about 7 days battery back up.
For $1500 AUD, “Poor man’s Sonnet 4.5 at home” ain’t nothing to sneeze at.
Still fine tuning this. I’m finding that Grug has a particular affinity for larger -b and -ub sizes. I’m now able to hit 90-110 tok/s prefil (upto 155 tok/s fresh). Specific details below
- 155.42 tok/s fresh 2,009-token prefill
- roughly 90–110 tok/s on larger incremental/agentic prefills
- roughly 11.3–12.9 tok/s sustained generation
- 64°C peak CPU temperature
- approximately 33 W peak CPU package power
- no thermal throttling
- ub and -b both 2048
All this on a Quadro P1000 4GB card.
In-sane.
-m "%MODEL_PATH%" ^ -t 8 ^ -tb 8 ^ -ngl 99 ^ --n-cpu-moe 38 ^ --flash-attn on ^ --no-mmap ^ --mlock ^ -c 16384 ^ -b 2048 ^ -ub 2048 ^ -np 1 ^ --host 0.0.0.0 ^ --port %PORT% ^ --ui-mcp-proxyPS: Interestingly, dropping cache precision away from FP16 reduced tok/s generation by 18%. I don’t know why. I’m just fine tuning MTP now to see if I can eke out a few more tok/s, as MTP enabled grug-v2 just dropped
https://huggingface.co/ProCreations/grug-35b-mtp-gguf
PPS: Sadly also discovered my Tesla card has intermittent electrical fault. Took some troubleshooting to figure it out, but the long and short of it is you probably shouldn’t buy second-hand server cards off Ebay. Oh well, $100. On the upside, tweaking throttlestop, re-pasting and re-seating greatly improved thermals.
22·7 days agoThis is my final reply in this thread. The developer has said their piece, and I have said mine. Now you’ve waded in - so let me set the record straight.
I am a developer. I had genuine interest in this project. I read the Hister documentation and inspected parts of the repository because the documentation did not clearly answer several basic questions I had:
-
How SQLite, Bleve, and stored HTML relate.
-
Whether TTL or storage quotas exist.
-
How browser-history deletion affects stored data.
-
How previews differ from a real web archive.
-
What multi-user isolation actually covers.
Yes, I used AI to assemble a plain-language summary and labelled it accordingly. Not everyone keeps the Hister codebase in their head, not everyone talks in code review and if I had these questions, I’m willing to bet others did too. The AI wrote for a lay audience because I didn’t ask it to do QA, I asked it to ELI-5.
The summary contained errors. Fine. That’s AI for you. However, if neither I nor the AI could find clear answers after cloning the repo, that supports my point about opacity.
At no point did I request a line-by-line audit. “Points 2 and 5 are wrong” would have answered the question.
Declining would also have been reasonable. Hell, side stepping it would have been fine too. Instead the dev decided to note the inaccuracies and rudely brush them off.
Both you and the dev seem to be under the impression !selfhosted is a one way distribution channel.
The developer came here, invited questions, then turned the raw prawn when questions arrived.
I didn’t go to their their Github. I didn’t abuse them. I genuinely wanted to know more about their project and share it, perhaps even work to help improve it.
They - and now you, ostensibly a happy clapper for Hister - came here.
Your claims about my effort and intent are assumptions followed by personal abuse.
Try and walk a mile in someone else’s shoes before calling them low effort and shitty next time.
-
That is not what happened.
I fed your GitHub repository to a clanker because the documentation did not answer my questions. I then shared its summary here.
You replied afterwards and said the summary was wrong. Fair enough. I then asked which specific points were wrong.
You could have answered, declined, or ignored the post.
Instead, you deigned only to dismiss the effort, then blamed me for objecting.
You also asked which parts were confusing, although my previous reply had already listed those issues.
You did not address them then, either.
A prospective user should not need ChatGPT, a cloned repository, and several follow-up questions to understand key functions.
You invited feedback. Your documentation remains unclear on several points, including issues beyond those I listed.
Your responses show that further feedback is not worth my time.
I am happy to narrow it further.
I took the time to read the documentation, ask ChatGPT to summarise what I found, and then reduced my follow-up to a simple request:
«Which of points 1–7 are materially wrong?»
That is not the same as asking you to audit “multiple screens” of AI output.
If the answer is “2 and 5 are incorrect”, or even “I do not have time to review it”, that is perfectly fine.
However, dismissing it as “a multiple screens long AI prompt” does not only not answer the question, it comes off as abrasive.
As for the documentation, the confusing parts are exactly those I listed: retention, lifecycle management, browser ingestion, storage limits, deletion, multi-user behaviour, and, most importantly, what Hister actually is and who it is for.
What’s disappointing is not that you disagreed with the AI summary. AIs are idiots.
It that after inviting questions and feedback, your response to a genuine attempt to understand the project is curt dismissal.
The inner workings of Hister may be obvious to you; they are not obvious to others.
The point is that you came here specifically to invite questions and feedback.
“TL;DR” does not encourage the sort of community engagement you ostensibly came here to seek.
Excellent - thanks for clearing that up.
Is there a TTL / max database size per user setting? Say I have 4 users using the server; can I allocate a hard limit of 10GB per user, with 180 day retention rules?
Additionally, is the other parenthetical information materially correct? If not, which points [1 thru to 7] are wrong?
I would like to further recommend Hister but your documentation is somewhat confusing at first blush.
Removed by mod
Both; it has internally reduced thinking tokens AND it’s less verbose, so you get a boost on both prefill and print. I’ve benchmarked identical prompts and tool calls with stock Qwen 35B and Grug 35B; on average, what takes stock 1000 token/s, Grug does in about 200.
Its a simplied front end for people to request shows / movies. Normies find Radarr / Sonarr somewhat perplexing.
Using a sledgehammer to crack a wallnut can be satisfying :)
Besides, the kids are always finding fun and inventive ways to break the TV, so I figured set it up once and then let the bot handle it.
Twelfth Doctor: This is Clara, not my assistant. She’s a… Some other word.
Clara: I’m his carer.
Twelfth Doctor: Yeah, my carer. She cares so I don’t have to.