Skip to content
yeongmin's archive
yeongmin's archive

idk what I'm doing

  • Main
  • 📣Minecraft server
  • Making a Reddit scraper (2023)
    • So, about the BAscraper
      • First release of BAScraper
    • stuffs from BAnalyser.ipynb (r/bluearchive recap)
    • Some notes on the new API policy for reddit
    • What counts as a request? (PRAW)
    • Some notes on PRAW
  • Homelabs n’ Servers
yeongmin's archive

idk what I'm doing

Making a Reddit scraper after the 2023 API changes

from r/redditdev

Welp…. on April 2023 this API policy change happned to Reddit. While I did do reddit scraping prior to this(like 1~2 years ago) for funsies and jokes, this still was a bummer.

I primarily used PSAW that is a wrapper for Pushshift.io since it was very easy to use. However, due to the changes, Pushshift.io now only makes it available for reddit mods for big subreddits. So since that’s a gonner the only choice for me is to use the ol’ PRAW again. It has some inherent limitations but thankfully it’s still free if used under the right terms and rate limits(100 requests per min).

But the problem is that there are almost no (that I could find of on google) easy guides on how to use the newest (PRAW 7.7.1 as of writing) PRAW. I mean the docs are good enough and understandable. But Attention deprived people like me or who just wants to get it done quick don’t want to read the docs lol.

So yeah, this is my journey to scraping Reddit using PRAW 7.7.1~! (in progress….) Realized that PRAW can’t do shit. (It could do proper stuffs before but recent updates ruined it 😒) So I’m looking for alternatives!
(btw ONLY scraping comments and submissions. so no moderations or some other stuffs)

Edit: 12/27 Did find some potential ways to get (almost all) posts from subreddits.
PullPush.io – Google CSE – PRAW hybrid style that is! Link to my solution (PushPull.io wrapper that I made)

©2025 yeongmin's archive