alot of really small SLMs seem to be too small to even function properly.
however, there may be some unconventional uses for some of them.
one usecase keyword generation. heres what gemma-3-270m-it-qat-Q4_0 gave me
prompt: keywords similar to “construction”
Construction activities
* Construction design
* Construction equipment
* Construction management
* Construction materials
* Construction methodology
* Construction organization
* Construction planning
* Construction process
* Construction project
* Construction project details
* Construction project management
* Construction project management activities
* Construction project management assessment
* Construction project management checklist
* Construction project management document
* Construction project management evaluation
* Construction project management framework
* Construction project management methodology
* Construction project management plan
* Construction project management project
* Construction project management report
* Construction project management review
* Construction project management schedule
* Construction project management software
* Construction project management strategy
* Construction project management system
* Construction project management tool
* Construction project planning
* Construction project scope
* Construction schedule
* Construction strategy
* Construction work
```
90 TPS (but got stuck in a repeating loop)
i think theres alot of other uses. but they arent really LLM answer that people usually reach for an LLM.You must log in or register to comment.
I don’t think these are “keywords similar to construction”… That’d be something like “building”, “contractor”, “builder”, “construction company”, “remodeling”, “roadworks”, …
And your model didn’t really find any of them. I think that output is next to worthless. If you’re a construction company and use this list for your SEO, nobody is ever going to find out you have an excavator and hire you.
well different prompts, at different temps, diff fine tunes , diff quants and quant quality, rag, and lora , are all going to give different results. this is just a single prompt using the model as it is pre-packaged.
you would have to test by running ~100 prompt variations and determine A)no useful value B) unidentifiably wrong answers (primarily if your requirements are unforgiving) C) no predictable/reliable performance whatsoever.
if thats turns out to be the case and its unacceptable…
A) scale up the model size slightly B) spend cash on hardware or service to run the appropriately sized model that accomodates the scale that you need(which could be prohibitively expensive).
I think the more interesting part that for still affordable hardware you could run something locally that is good enough and do not need to refer to commercial models.
For breaking out some simple but tedious processing tasks in scripts they can help save a lot of writing logic and filtering. But it can be a bit rough… I’m having to experiment more than I’d like.
What is the point of that?
Wrap it in a harness. Dump antipatterns. Keep the rest. Call it a ratchet. Loop that. Fail fast. Refine. See what sticks. Design a better harness. Augment that harness. If nothing good comes out, that’s data and that is good.
I am trying to fully understand this. Can you please elaborate further? Are you thinking about a heuristic based harness?
Not sure about keyword generation exactly as OP intends to apply this, but short simple code that needs to be validated on output seems like the better application. Models are tiny and fast, so mmap and run in parallel and race to what passes validation first. Each agent gets a git branch. Losers generate the anti patterns. Human or cheap tokens integrates the results from corpus back into harness. Next generation.
Technologic. Technologic.
I aged a month just reading this
Using it to tag content can make it easier to search later, bookmark management for example
So what’s the advantage of a generated list over “construction”?
Mostly the ratio of effort to convenience
I wouldn’t bother thinking up tags for every article I bookmark
However it is nice if a local model on the device can generate tags automatically
But they seem like iterative synonyms. Hardly unique enough to useful tags for either lumping or splitting.
anything you want it to be.
be creative !I can’t think of single useful function. You might as well have generated nonsense syllables and invited us to attach them to words to create meaning.
I will eventually experiment with small models for various text- and document-processing tasks.
