SquishyPillow@burggit.moe to Well That Was Useful!@burggit.moe · edit-21 year agoReddit scraper that works even after the API changescodeberg.orgexternal-linkmessage-square11fedilinkarrow-up19arrow-down10file-textcross-posted to: [email protected]
arrow-up19arrow-down1external-linkReddit scraper that works even after the API changescodeberg.orgSquishyPillow@burggit.moe to Well That Was Useful!@burggit.moe · edit-21 year agomessage-square11fedilinkfile-textcross-posted to: [email protected]
i started working on this after Reddit’s announcement about their API being paywalled. It still works lol
minus-squareMomoNeedsCorrection@burggit.moelinkfedilinkarrow-up1·1 year agoSo you’re telling me I can just set up a scraper to spam the reddit api and use their bandwidth?
minus-squareSquishyPillow@burggit.moeOPlinkfedilinkarrow-up2·1 year agoThis isn’t a tool for a denial-of-service attack, which is what you are describing. It is a tool for stealing reddit data without having to pay for the API.
minus-squareMomoNeedsCorrection@burggit.moelinkfedilinkarrow-up1·1 year agoBut if I steal enough data from reddit in a short period of time, what’s the difference?
minus-squareSquishyPillow@burggit.moeOPlinkfedilinkarrow-up1·1 year agoI guess there is none. I just didn’t build it for DoS purposes heehee To actually cause any meaningful interruption, you are going to need an entire botnet to spam requests at reddit.
minus-squareBurger@burggit.moelinkfedilinkarrow-up1·1 year agoI’m sure they have automated requests detection and will put your IP on a cooldown, API or no API.
minus-squareMomoNeedsCorrection@burggit.moelinkfedilinkarrow-up3·1 year agoI mean given how much janky reddit seems to run at any given moment, I wouldn’t be surprised if it just didn’t work
minus-squareSquishyPillow@burggit.moeOPlinkfedilinkarrow-up2·1 year agoYou would be surprised. As long as you are using an approved User-Agent and are handling cookies correctly, it is pretty easy to scrape endlessly.
So you’re telling me I can just set up a scraper to spam the reddit api and use their bandwidth?
This isn’t a tool for a denial-of-service attack, which is what you are describing. It is a tool for stealing reddit data without having to pay for the API.
But if I steal enough data from reddit in a short period of time, what’s the difference?
I guess there is none. I just didn’t build it for DoS purposes heehee
To actually cause any meaningful interruption, you are going to need an entire botnet to spam requests at reddit.
I’m sure they have automated requests detection and will put your IP on a cooldown, API or no API.
I mean given how much janky reddit seems to run at any given moment, I wouldn’t be surprised if it just didn’t work
You would be surprised. As long as you are using an approved User-Agent and are handling cookies correctly, it is pretty easy to scrape endlessly.