Hey everyone,

I have been scraping a lot of data from a website for the last several months. I have been using 3 phones with unlimited data plans that average around 30 gigs a day of usage per phone.

Recently I just banned from my cell provider due to ToS violation due to automation. Does anyone know of cell providers or somehow I can use this much data without being banned?

A note is that this website will block me after so many requests from a single IP, hence the reason for cell phone plans as I can rotate my IP frequently to avoid being blocked.

I am unable to use VPNs as well as it knows these IPs and will force a log in page.

Any ideas? Maybe starlink lets me rotate IPs?

Or maybe I need to make the phones look more legit to my cell provider?

Hi all, I am building a scraper to scrape articles from a website for research. I have their permission to do so, but they have cloudflare in place, and I keep getting 525 errors (in my browser the site works just fine). Currently I am just using python requests, I was wondering if anyone has any tips on how to mitigate this. Would using selenium and navigating in a browser selenium help? Thanks.