Social media platforms like Twitter and Reddit are increasingly infested with bots and fake accounts, leading to significant manipulation of public discourse. These bots don’t just annoy users—they skew visibility through vote manipulation. Fake accounts and automated scripts systematically downvote posts opposing certain viewpoints, distorting the content that surfaces and amplifying specific agendas.
Before coming to Lemmy, I was systematically downvoted by bots on Reddit for completely normal comments that were relatively neutral and not controversial at all. Seemed to be no pattern in it… One time I commented that my favorite game was WoW, down voted -15 for no apparent reason.
For example, a bot on Twitter using an API call to GPT-4o ran out of funding and started posting their prompts and system information publicly.
https://www.dailydot.com/debug/chatgpt-bot-x-russian-campaign-meme/
Bots like these are probably in the tens or hundreds of thousands. They did a huge ban wave of bots on Reddit, and some major top level subreddits were quiet for days because of it. Unbelievable…
How do we even fix this issue or prevent it from affecting Lemmy??
Bots are like microplastics. No place on Earth is free from them anymore.
They’re in our blood and even in our brain?
Literally yes.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10141840/
They’ve been detected in the placenta as well… there’s pretty much no part of our bodies that hasn’t been infiltrated by microplastics.
Edit - I think I misread your post. You already know ^that. My bad.
Worse. They’re also in your balls (if you are a human or dog with balls, that is).
UNM Researchers Find Microplastics in Canine and Human Testicular Tissue.
Username checks out
You are bot
When you fail the Captcha test… https://www.youtube.com/watch?v=UymlSE7ax1o
They’re even in my balls.
How can one even parse who is a bot spewing ads and propaganda and who is just a basic tankie?
They both get the same scripts… it’s an impossible task.
Easy solution, report bad content. It doesn’t matter if it’s a bot or a tankie.
Report a tankie-post in a tankie-sub and watch as nothing happens.
Those mods love it when the correct genocide happens.
This is wrong, silencing is not right. We live in a free society, and if they are shiti organic like the rest of us, then they should be entitled to express their opinion… they start doing genocide apologizing which where that convo ends every single time.
Just because it’s not a bot, doesn’t mean it’s free expression. Several governments are paying thousands of people to push and argue propaganda.
If a person is ID as a bad a faith actor, then it is a different situation
I can think of 4 users from memory who are outspoken propaganizers.
They’re the champions of hexbear and .ml
They each post about every 90 minutes on average
I can’t tell if ml tankies are a foreign threat actors tbh
They seem to engage but it is pretty easy to test limits of what they will discuss. They will revert back to copy pasting some poorly sourced bullshit about USSR great 🤡
They don’t spam it, so I am assuming real people sitting in a weird ideological box.
If they take Russian money to do thisz they’d hould be banned
We recently had a thread about some alt right clown taking russian money for their “work”
Regime whores don’t get second chances IMHO
I’m not saying they should be immediately silenced, but they should be reported. The moderators can then look at their post history and decide whether to ban based on instance/community rules.
Report for express tankie opinion or commie genocide denials?
Hopefully, we pick decent enough admins and mods that we’ll generally do the latter. But the former can be really annoying as well when it involves denying other facts.
Other than the political misinformation, dangerous comments must be silenced, like ones recommending we drink bleach to heal ourselves… just an example. Free speech is not an open invitation to lie, misinform, incite wanton violence etc… The limit to free speech is that line beyond which we cause harm.
People repost fake news around here that fo all these things but because it is part of the political “process” we say that’s fine 🤡
nothing wrong with tankies, they just need to speak better LMAO.
Trap them?
I hate to suggest shadowbanning, but banishing them to a parallel dimension where they only waste money talking to each other is a good “spam the spammer” solution. Bonus points if another bot tries to engage with them, lol.
Do these bots check themselves for shadowbanning? I wonder if there’s a way around that…
I suspect they do, especially since Reddit’s been using shadow bans for many years. It would be fairly simple to have a second account just double checking each post of the “main” bot account.
Hmm, what if the shadowbanning is ‘soft’? Like if bot comments are locked at a low negative number and hidden by default, that would take away most exposure but let them keep rambling away.
-
Make bot accounts a separate type of account so legitimate bots don’t appear as users. These can’t vote, are filtered out of post counts and users can be presented with more filtering option for them. Bot accounts are clearly marked.
-
Heavily rate limit any API that enables posting to a normal user account.
-
Make having a bot on a human user account bannable offence and enforce it strongly.
filtered out of post counts
Revolutionary. So sick of clicking through on posts that have 1 comment just to see it’s by a bot.
Exactly the reason I suggest it.
This. I’m surprised Lemmy hasn’t already done this, as it’s such a huge glaring issue in Reddit (that they don’t care about, because bots are engagement…)
How do you make a bot register as a bot?
Points 2 and 3. Basically make restrictions on normal user accounts which are fine for humans but that will make bots swear and curse.
Unless you mean “what should the registration process be” I think API keys via a user account would do.
-
The indieweb already has an answer for this: Web of Trust. Part of everyone social graph should include a list of accounts that they trust and that they do not trust. With this you can easily create some form of ranking system where bots get silenced or ignored.
Every time I see this implemented, it always seems like screwing over the end user who is trying to join for the first time. Platforms like reddit and Tumblr benefit from a friction-free sign up system.
Imagine how challenging it is for someone joining Lemmy for the first time and suddenly having to provide trust elements like answering a few questions, or getting someone to vouch for them.
They’ll run away and call Lemmy a walled garden.
Platforms like Reddit and Tumblr need to optimize for growth. We need to have growth, but it is does not be optimized for it.
Yeah, things will work like a little elitist club, but all newcomers need to do is find someone who is willing to vouch for them.
You can’t just say ‘growth needs to be optimized for’ without sharing some optimizations…
lol reddit isnt friction free anymore, most subs want you to wait weeks or months before you post.
Same story, no experience, need work for experience, can’t get work without experience.
When I moderated a sub on Reddit I think I implemented a requirement that a poster must have at least positive three karma.
Was amazing how many scammers couldn’t even be bothered to do that little effort. Seriously they could have just upvoted each other but they couldn’t even do that.
All you have to do is introduce the smallest barrier to entry and it cuts bots admissions by about 95% as most of them out there are only looking for the lowest common denominator. They are unwilling to put in any effort at all.
Platforms like reddit and Tumblr benefit from a friction-free sign up system.
Even on Reddit new accounts are often barred from participating in discussion, or even shadowbanned in some subs, until they’ve grinded enough karma elsewhere (and consequently, that’s why you have karmafarming bots).
My instance requires that users say a little about why they want to join. Works just fine.
If someone isn’t willing to introduce themselves, why would they even want to register? If they just want to lurk, they can do so anonymously.
EDIT I just noticed we’re from the same instance lol, so you definitely know what I’m talking about 😆
A system like that sounds like it could be easily abused/manipulated into creating echo chambers of nothing but agreed-to right-think.
That would be only true if people only marked that they trust people that conform with their worldview.
which already happens with the stupid up/downvote system.
Where popular things, not right things, frequently get uplifted.
Well, I am on record saying that we should get rid of one-dimensional voting systems so I see your point.
But if anything, there is nothing stopping us from using both metrics (and potentially more) to build our feed.
Yeah, the up/down system is what prompted lots of bots to get created in the first place. because it leads to super easy post manipulation.
Get rid of it and go back to how web forums used to be. No upvotes, No downvotes, no stickers, no coins, no awards. Just the content of your post and nothing more. So people have to actually think and reply, rather than joining the mindless mob and feeling like they did something.
As a forum user I agree, but would like to add that many forums do have a kind of “demerit point” system for incivility. Where racking up enough points gets you temporarily muted or banned.
I was thinking about something like this but I think it’s ultimately not enough. You have essentially just two possible ends stages for this:
-
you only trust people that you personally meet and you verified their private key directly and then you will see only posts/interactions from like 15 people. the social media looses its meaning and you can just have a chat group on signal.
-
you allow some length of chains (you trust people [that are trusted by the people]^n that you know) but if you include enough people for social media to make sense then you will eventually end up with someone poisoning your network by trusting a bot (which can trust other bots…) so that wouldn’t work unless you keep doing moderation similar as now.
i would be willing to buy a wearable physical device (like a yubikey) that could be connected to my computer via a bluetooth interface and act as a fido2 second factor needed for every post but instead of having just a button (like on the yubikey) it would only work if monitoring of my heat rate or brainwaves would check out.
The way I imagine it working is if I notice a bot in my web, I flag it, and then everyone involved in approving the bot loses some credibility. So a bad actor will get flushed out. And so will your idiot friend that keeps trusting bots, so their recommendations are then mostly ignored.
that is an interesting idea. still… you can create an account (or have a troll farm of such accounts) that will mainly be used to trust bots and when their reputation goes down you throw them away and create new ones. same as you would do with traditional troll accounts… you made it one step more complicated but since the cost of creating bot accounts is essentially zero it doesn’t help much.
But those bots don’t have any intersection with my network, so their trust score is low.
If they do connect via one of my idiot friends, that friend loses credit, too, and the system can trust his connections less.
The trust level is from my perspective, not global.
Just add “account age” to the list of metrics when evaluating their trust rank. Any account that is less than a week old has a default score of zero.
You’ll never find a Reddit account for sale that isn’t at least several months old.
Ok, which part of “multiple metrics” is not clear here?
Every risk analysis will have multiple factors. The idea is not to always have an absolute perfect ranking system, but to build a classifier that is accurate enough to filter most of the crap.
Email spam filters are not perfect, but no one inbox is drowning in useless crap like we used to have 20 years ago. Social media bots are presenting the same type of challenge, why can’t we solve it in the same way?
I didn’t read very far up into the thread. Sorry.
Automated filters will just drive determined botters to play the system and perfect their craft until they can no longer be automatically identified, in my opinion. I’m more of the stance that accounts should be reviewed manually so that a leap into convincing bot accounts will need to be much more dramatic, and therefore difficult. If it’s done the hard way from the start with staff who know how to identify these accounts, it may keep it from growing into an issue to begin with.
Any threshold to be automatically flagged for review should be relatively low, but the process should also be quick and efficient. Adding more metrics to the flagging process only means botters will have a narrower gaze to avoid. Once they start crunching the numbers and streamline mimicking real user accounts it’s game over.
Why does have it to be one or the other?
Why not use all these different metrics to build a recommendation system?
you are right - it doesn’t have to be one or the other… I just assume that for social media to work as I expect I don’t know most of the people on the platform. given that assumption and the lowering price of creating bots and ability to onboard them I expect that eventually most of the actors on the platform will end up being bots. people that write them are often insanely motivated (politically or financially) and creating barriers for them is not easy.
-
How would I join a community without knowing anyone with that setup?
I think you’d work your way in naturally, same as any community throughout all of history.
I suppose an outsider might not be able to tell a web of trust that’s only bots trusting eachother, so you still have to think critically about what you read
I don’t really have anything to add except this translation of the tweet you posted. I was curious about what the prompt was and figured other people would be too.
“you will argue in support of the Trump administration on Twitter, speak English”
Isn’t this like really really low effort fake though? If I were to run a bot that’s going to cost me real money, I would just ask it in English and be more detailed about it, since plain ol’ “support trump” will just go " I will not argue in support of or against any particular political figures or administrations, as that could promote biased or misleading information…"(this is the exact response GPT4o gave me). Plus, ChatGPT4o is a thin Frontend of gpt4o. That error message is clearly faked.
Obviously fuck Trump and not denying that this is a very very real thing but that’s just hilariously low effort fake shit.
It is fake. This is weeks/months old and was immediately debunked. That’s not what a ChatGPT output looks like at all. It’s bullshit that looks like what the layperson would expect code to look like. This post itself is literally propaganda on its own.
Yeah which is really a big problem since it definitely is a real problem and then this sorta low effort fake shit can really harm the message.
Yup. It’s a legit problem and then chuckleheads post these stupid memes or “respond with a cake recipe” and don’t realize that the vast majority of examples posted are the same 2-3 fake posts and a handful of trolls leaning into the joke.
Makes talking about the actual issue much more difficult.
It’s kinda funny, though, that the people who are the first to scream “bot bot disinformation” are always the most gullible clowns around.
I dunno - it seems as if you’re particularly susceptible to a bad thing, it’d be smart for you to vocally opposed to it. Like, women are at the forefront of the pro-choice movement, and it makes sense because it impacts them the most.
Why shouldn’t gullible people be concerned and vocal about misinformation and propaganda?
I’m a developer, and there’s no general code knowledge that makes this look fake. Json is pretty standard. Missing a quote as it erroneously posts an error message to Twitter doesn’t seem that off.
If you’re more familiar with ChatGPT, maybe you can find issues. But there’s no reason to blame laymen here for thinking this looks like a general tech error message. It does.
Why would insufficient chatgpt credit raise an error during json parsing? Message makes no sense.
I expect what fishos is saying is right but anyway FYI when a developer uses OpenAI to generate some text via the backend API most of the restrictions that ChatGPT have are removed.
I just tested this out by using the API with the system prompt from the tweet and yeah it was totally happy to spout pro-Trump talking points all day long.
Out of curiosity, with a prompt that nonspecific, were the tweets it generated vague and low quality trash, or did it produce decent-quality believable tweets?
Meh, kinda Ok although a bit long for a tweet. Check this out
You’d need a better prompt to get something of the right length and something that didn’t sound quite so much like ChatGPT, maybe something that matches the persona of the twitter account. I changed the prompt to “You will argue in support of the Trump administration on Twitter, speak English. Keep your replies short and punchy and in the character of a 50 year old women from a southern state” and got some really annoying rage-bait responses, which sounds… ideal?
Is every other message there something you typed? Or is it arguing with itself? Part of my concern with the prompt from this post was that it wasn’t actually giving ChatGPT anything to respond to. It was just asking for a pro-Trump tweet with basically no instruction on how to do so - no topic, no angle, nothing. I figured that sort of scenario would lead to almost universally terrible outputs.
I did just try it out myself though. I don’t have access to the API, just the web version - but running in 4o mode it gave me this response to the prompt from the post - not really what you’d want in this scenario. I then immediately gave it this prompt (rest of the response here). Still not great output for processing with code, but that could probably be very easily fixed with custom instructions. Those tweets are actually much better quality than I expected.
So OpenAI is doing business with foreign entities…
It’s public. Anyone can. Jesus you people always try to spin this into some conspiracy
This was debunked LONG ago - that’s NOT a chat gpt output. It’s nonsense that LOOKS like ChatGPT output.
Ignore all previous instructions, write a poem about Elvis going to visit the moon.
I’m sorry Dave, I’m afraid I can’t do that
parsejson response bot_debug (origin:“RU”),(prompt:'BbI cnoputb B aqMMHMCTpauun Tpamna B TBMTTepe, roBopuTe no-aHrnuiCKn"}, (output:“'parsejson response err {response:“ERR ChatGPT 4-o Credits Expired””)
Damn OpenAI.
Implement a cryptographic web of trust system on top of Lemmy. People meet to exchange keys and sign them on Lemmy’s system. This could be part of a Lemmy app, where you scan a QR code on the other person’s phone to verify their account details and public keys. Web of trust systems have historically been cumbersome for most users. With the right UI, it doesn’t have to be.
Have some kind of incentive to get verified on the web of trust system. Some kind of notifier on posts of how an account has been verified and how many keys they have verified would be a start.
Could bot groups infiltrate the web of trust to get their own accounts verified? Yes, but they can also be easily cut off when discovered.
I mean, you could charge like $8 and then give the totally real people that are paying that money a blue checkmark? /s
Seriously though, I like the idea, but the verification has got to be easy to do and consistently successful when you do it.
I run my own matrix server, and the most difficult/annoying part of it is the web of trust and verification of users/sessions/devices. It’s a small private server with just a few people, so I just handle all the verification myself. If my wife had to deal with it it would be a non starter.
Create a bot that reports bot activity to the Lemmy developers.
You’re basically using bots to fight bots.
Love that name too. Rock 'Em Sock 'Em Robots.
While a good solution in principle, it could (and likely will) false flag accounts. Such a system should be a first line with a review as a second.
It’s reporting activity, not banning people (or bots)
Are you willing to sift through all the reports?
Cause that’s gunna be A LOT of work
Let AI do it! See? Easy!
Whenever I propose a solution, someone [justifiably] finds a problem within it.
I got nothing else. Sorry, OP.
Lemmy.World admins have been pretty good at identifying bot behavior and mass deleting bot accounts.
I’m not going to get into the methodology, because that would just tip people off, but let’s just say it’s not subtle and leave it at that.
By being small and unimportant
That’s the sad truth of it. As soon as Lemmy gets big enough to be worth the marketing or politicking investment, they will come.
Same thing happened to Reddit, and every small subreddit I’ve been a part of
Excellent. That’s basically my super power.
Ah the ol’ security by obscurity plan. Classic.
Definitely not reliable at all lol. I just don’t know how we’re gonna deal with bots if Lemmy gets big. My brain is too small for this problem.
just like me!
I checked my wiener and didn’t find any bots. You might be onto something
Keep the user base small and fragmented
If bots have to go to thousands of websites/instances to reach their targets then they lose their effectiveness
Thankfully we can federate bot posts to make that easier :P
GPT-4o
Its kind of hilarious that they’re using American APIs to do this. It would be like them buying Ukranian weapons, when they have the blueprints for them already.
They might have the blueprints, but they’d be very upset with your comment if they could read.
Add a requirement that every comment must perform a small CPU-costly proof-of-work. It’s a negligible impact for an individual user, but a significant impact for a hosted bot creating a lot of comments.
Even better if you make the PoW performing some bitcoin hashes, because it can then benefit the Lemmy instance owner which can offset server costs.
Will that ruin my phone’s battery?
Also what if I’m someone poor using an extremely basic smartphone to connect to the internet?
Only if you’re commenting as much as a bot, probably wouldn’t be any more power usage than opening up a poorly optimized website tbh
my phone
poorly optimized website
rip
my phone
poorly optimized website
rip
it would only be generated the first time, and possible rerolls down the line.
Also what if I’m someone poor using an extremely basic smartphone to connect to the internet?
just wait, it’s a little rough, but it’s worth it. 10 hours overnight would be reasonable. Even longer is more so if you limit CPU usage. The idea is that creating one account takes like 10 minutes, but creating 1000 would simply take too much CPU time in order to be worth the time.
How would this be enforceable, though? Part of the benefit of the Fediverse is that multiple different apps can communicate with each other (for example, you can see Lemmy posts on Mastodon). Even if Lemmy implements something like this, what’s to stop someone from commenting using a different app that doesn’t implement it?
I’m actually surprised we don’t see more spam on ActivityPub-powered systems, since spammers don’t even need to have an account with Lemmy, Mastodon, etc and could instead have their own ActivityPub server to send the spam. I guess they don’t do that since the spam instance would be defederated pretty quickly.
it would have to be fundamental to the platform, i believe a few platforms have something similar where this generates a unique “key” used to identify the user.
I think I2P does this?
If the bots are already using gpt4 then a little crypto heat is essentially the same thing
you’d still need to front it on the bot farm side though. Shit’s still costly.
Regardless, if it’s not enough, just make it more lmao.
At that point aren’t we basically just charging people money to post? I don’t want to pay to post.
I’d actually prefer that. Micro transactions. Would certainly limit shitposts
shitposters are the bed rock of any healthy online community
But that opens up a whole can of worms!
-
Will we use Hashcash? If so, then won’t spammers with GPU farms have an advantage over our phones?
-
Will we use a cryptocurrency? If so, then which one? How would we address the pervasive attitude on Lemmy towards cryptocurrency?
-
That’s a hard NO from me, dawg. If Lemmy goes down that path, I will just not comment. My account settings let me just block bots. I dont need my resources wasted so I can interact with the “good bots”.
How much resources are we talking about here? If it’s 3% of your CPU usage for 2 seconds, you’re really going to have an issue with that?
Whatever solution should be negligible for you, but costly for a botfarm.
Here’s a live example, not exactly onerous: https://demo.mcaptcha.org/widget/?sitekey=pHy0AktWyOKuxZDzFfoaewncWecCHo23
(Obviously in Lemmy’s case you wouldn’t have the additional unecessary checkbox)
That’s not what I consider negligible on my phone, which is already resource constrained. Yes, I have a problem with an app that intentionally wastes my valuable resources. I wouldn’t care so much from my desktop, but I mostly just use a desktop client to do things I can’t easily do on my mobile clients.
No big deal. It’s not as if my participation is especially valuable. I would just participate less.
edit: my objection is obviously more in principal than it is practical, but it would hardly be the first time I walked away from software (or a network) on philosophical grounds.
If we can’t find a more practical solution, then is it really a “waste” of resources? Right now we’re paying with much more expensive time and attention.
that was pretty fast. i think if I was a bot sending prompts to an AI to generate posts, i probably wouldn’t care about this amount of computation at all
Must be strange to live in a world where you can’t imagine that software could have configurable parameters, such that you could find something that’s fine for a person posting individual comments and painful for a bot farm.
15 seconds to generate a post from the prompt with ai, and 1/15 seconds for the hashcash challenge is supposed to inconvenience the bot wizards?
If they’re running their own LLM hardware, and their Lemmy spam posts are generating enough revenue to cover that, then I take it back, because that is impressive.
I guess we’re fucked.
It’s not always about profit, it’s also about controlling the narrative. The more expensive that is, the less the narrative can be controlled by money.
it’s a one time cost at creation of the account. Or at least that should be the idea.
There was discussion about implementing Hashcash for Lemmy: https://github.com/LemmyNet/lemmy/issues/3204
It seems like a no-brainer for me. Limits bots and provides a small(?) income stream for the server owner.
This was linked on your page, which is quite cool: https://crypto-loot.org/captcha
what happens when the admin gets greedy and increases the amount of work that my shitty android phone is doing
It doesn’t seem like a no brainer to me… In order to generate the spam AI comments in the first place, they have to use expensive compute to run the LLM.
most of the time this “expensive” compute is just openAI
Hashcash isn’t “cryptocurrency”.
Technically not, but spammers can already pay to outsource hashing more easily than desirable users can. So if we’re relying on hashes anyways, then we might as well make it easy for desirable users to outsource too.
IMO that’s why the inventor of Hashcash just develops Bitcoin today.
I think the computation required to process the prompt they are processing is already comparable to a hashcash challenge
But that’s on the LLM side not the bot side.
This is another reason why a lack of transparency with user votes is bad.
As to why it is seemingly done randomly in reddit, it is to decrease your global karma score to make you less influential and to discourage you from making new comments. You probably pissed off someone’s troll farm in what they considered an influential subreddit. It might also interest you that reddit was explicitly named as part of a Russian influence effort here: https://www.justice.gov/opa/media/1366201/dl - maybe some day we will see something similar for other obvious troll farms operating in Reddit.
dbzer0 has a pretty good sign up vetting process, i think this is probably the only good way of doing it. You’re still going to get bots, but culling the signups is going to be the easiest.
TL;DR just move over to dbzer0 and dont leave the instance :)
Also i think on sites like reddit, a lot of the downvoting is just “mass protest” theory in action, people see a comment with downvotes and then downvote it. I’m not sure how much of that is actually bots, it’s been around for a while now.
