Luu Tuyen@lemmy.world to Technology@lemmy.worldEnglish · 17 hours agoTikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAIfortune.comexternal-linkmessage-square70fedilinkarrow-up1373arrow-down15cross-posted to: hackernews
arrow-up1368arrow-down1external-linkTikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAIfortune.comLuu Tuyen@lemmy.world to Technology@lemmy.worldEnglish · 17 hours agomessage-square70fedilinkcross-posted to: hackernews
minus-squareMelvin_Ferd@lemmy.worldlinkfedilinkEnglisharrow-up4·9 hours agoIsn’t there a way to limit requests so that traffic isn’t bringing down your servers
minus-squareMojave@lemmy.worldlinkfedilinkEnglisharrow-up8·7 hours agoThey obfuscate their traffic by randomizing user agents, so it’s either add a global rate limit, or let them ass fuck you
Isn’t there a way to limit requests so that traffic isn’t bringing down your servers
They obfuscate their traffic by randomizing user agents, so it’s either add a global rate limit, or let them ass fuck you