Perplexity was recently accused of scraping sites that had explicitly disallowed LLM crawlers in their robots.txt files. In the wake of…
  • originalucifer@moist.catsweat.com
    12·
    1 month ago

    oh look, an end user whose never had to run a production website on a budget.

    privacy is only 50% of the reasons people block this shit. the rest are resource limitations.