How to Scrape Reddit Data

Data API access and terms

Reddit publishes its rules in four documents, and anyone planning to collect Reddit data should read all four. The dates below are the last edits shown when we checked on October 30, 2026.

The Responsible Builder Policy is the gate. It applies to "everyone", developers, moderators, researchers and apps alike, and it sets three restrictions up front: approval is required before accessing Reddit data through the API, you must not misrepresent how or why you access data (which rules out registering several accounts for one use case), and you must not circumvent or exceed access limits. Non-commercial developers are pointed to Devvit, Reddit's own app platform. Commercial use requires "explicit written approval".

The Developer Platform and data access article defines commercial broadly: any use by or for a business, or as part of a monetized product, including apps with ads, paid data services and subscription products. It says Reddit offers both no-fee and paid access, that bulk export is significantly limited by default, and that academic research must go through the Reddit for Researchers program. It also states plainly that Reddit content may not be used to train a model without explicit consent.

Before you write code, decide which bucket you are in: a moderator tool, a non-commercial app, research, or a business use. The answer decides whether the route is Devvit, a Data API application, the researcher program or a data license, and none of them starts with a scraper.

An approved app in Python

Once Reddit approves your app, you get a client ID and secret. The Data API Wiki sets the technical rules:

• Every request must authenticate with a registered OAuth token. Traffic without OAuth "will be blocked".
• The User-Agent must be unique and descriptive, in the form <platform>:<app ID>:<version string> (by /u/<reddit username>). Default agents such as "Python/urllib" are drastically limited, and you must never lie about your User-Agent.
• On the default access tier, each OAuth client ID may make 100 requests a minute, measured as an average across ten minutes, so short bursts are tolerated. Three response headers report where you stand: X-Ratelimit-Used, X-Ratelimit-Remaining, X-Ratelimit-Reset.
• Content deleted on Reddit must be deleted from your copy too, including the author details of deleted accounts. Reddit recommends routinely deleting stored user data and content within 48 hours.

PRAW, the Python Reddit API Wrapper, handles OAuth and the rate-limit headers. Version 8.0.0 shipped in June 2026 with support for Python 3.13 and 3.14 and made several arguments keyword-only; 8.0.3 is current. This script reads the month's top posts from one subreddit with application-only access:

"""Read the month's top posts from one subreddit with PRAW, using an approved app's credentials.

Set REDDIT_CLIENT_ID, REDDIT_CLIENT_SECRET and REDDIT_USERNAME in the environment.
"""
import os

import praw

reddit = praw.Reddit(
    client_id=os.environ["REDDIT_CLIENT_ID"],
    client_secret=os.environ["REDDIT_CLIENT_SECRET"],
    user_agent=f"script:com.example.weeklytop:v0.2 (by /u/{os.environ['REDDIT_USERNAME']})",
)
reddit.read_only = True  # application-only access: no user password involved

top_posts = reddit.subreddit("learnpython").top(time_filter="month", limit=10)
for submission in top_posts:
    print(f"{submission.score:>6} {submission.num_comments:>5} {submission.id} {submission.title[:70]}")

limits = reddit.auth.limits  # PRAW 8 fills these from the X-Ratelimit-* response headers
print(f"remaining={limits['remaining']} used={limits['used']}")

We tested it on October 30, 2026 with PRAW 8.0.3 and Python 3.12 against a local server that mimics Reddit's token and listing endpoints, because our own use does not have Reddit API approval. The script obtained a client-credentials token, printed the listing and reported remaining=997 used=3 from the headers the test server sent. Note that PRAW 8's limits dictionary contains remaining and used only; it no longer exposes a reset timestamp. PRAW also sleeps between requests based on those headers, so a loop over many listings stays inside the limit without extra code.

Keeping deleted content out of your copy

The deletion rule is the one most collectors forget. This job re-checks stored post IDs in batches and lists the ones to remove:

"""Re-check stored Reddit posts and drop the ones that were deleted or removed.

Reddit's Data API terms require removing content that was deleted on Reddit.
Run this on a schedule against the IDs you keep (fullnames such as t3_abc123).
"""
import os

import praw

reddit = praw.Reddit(
    client_id=os.environ["REDDIT_CLIENT_ID"],
    client_secret=os.environ["REDDIT_CLIENT_SECRET"],
    user_agent=f"script:com.example.weeklytop:v0.2 (by /u/{os.environ['REDDIT_USERNAME']})",
)
reddit.read_only = True

stored = ["t3_lab1", "t3_lab2", "t3_lab3", "t3_lab4"]  # load these from your database
still_live = set()
for post in reddit.info(fullnames=stored):  # PRAW batches up to 100 IDs per request
    gone = post.author is None or post.removed_by_category is not None or post.selftext in ("[deleted]", "[removed]")
    if not gone:
        still_live.add(post.fullname)

to_delete = [name for name in stored if name not in still_live]
print(f"keep {sorted(still_live)}")
print(f"delete {to_delete}")

Against the same test server, where two of four posts were marked removed and deleted, it printed keep ['t3_lab1', 't3_lab4'] and delete ['t3_lab2', 't3_lab3']. Run it daily, or more often if you keep comments.

Public JSON and its limits

For years, adding .json to almost any Reddit URL returned the page's data with no account. Many tutorials still teach it. Three of Reddit's own documents now stand against it as a collection method:

1. The Data API Wiki says traffic not using OAuth or login credentials will be blocked, and that the default rate limit will not apply to it.
2. Reddit's robots.txt disallows every path for every crawler and points to the Public Content Policy. The Wiki adds that robots.txt is for search engines, not for Data API users, so it is not a loophole either way.
3. The User Agreement, effective July 1, 2026, prohibits collecting data by any means except as permitted, grants conditional permission to crawl only within robots.txt, and states that scraping without Reddit's prior written consent is prohibited.

The Public Content Policy explains the reasoning. Reddit content is public on purpose and anyone may read it, but the company objects to entities that "amass public data" for commercial gain through scraping or data brokers, and it licenses public content to partners under contract. Reading Reddit in a browser is fine; building a dataset from it without permission is what the rules address.

If you have an old pipeline built on the JSON endpoints, the migration path is the OAuth API with an approved app. The endpoints and response shapes are largely the same, so PRAW or authenticated requests to the OAuth host usually replace the old code with little change.

Browser collection

Driving a browser through Reddit with Playwright or Selenium does not change the rules above; the User Agreement covers collection "by any means (automated or otherwise)". It also creates problems the API avoids:

• Accounts. The Responsible Builder Policy requires apps, including bots and AI agents, to register for an app profile label and keep app accounts separate from personal ones. A personal account driven by a script is the mixed use it prohibits.
• Fragility. Reddit's web interface changes often, and pages load content dynamically. Selectors that work today break without notice.
• Moderation risk. Automated activity on an account can be flagged as spam or inauthentic behavior, and our guide to Reddit network blocks and shadowbans shows how hard those flags are to undo.

There are legitimate browser uses: a moderator reviewing their own community, a QA team testing how a subreddit renders, or a person reading. If what you need is data, the API or a license is the route. If you are building an AI agent that reads Reddit, the policy names bots and AI agents explicitly, so the same approval applies.

When a proxy helps

A proxy changes the network address a request comes from. Measure it against what Reddit limits.

• API rate limits are per OAuth client ID, not per IP. More addresses do not add capacity, and the policy forbids multiple accounts or duplicate requests for the same use case. A proxy cannot help an approved app go faster and should not be used to try.
• Ordinary access problems are network problems. If your browser shows Reddit's "blocked by network security" page on a shared office or cloud network, a normal connection fixes it. People who travel or work from data centers sometimes use a dedicated line for their own browsing; Coronium's mobile proxies are carrier lines with one customer per device. The Reddit proxy page has live stock by country.
• Approved apps on cloud hosts sometimes see their provider's shared ranges throttled for reasons unrelated to them. A stable, dedicated egress for the app's server keeps its traffic separate from its neighbors'. That is the honest network use for an API client: one stable address, identified by its User-Agent, inside its limit.

If you want the Python mechanics of proxy authentication for any of these, see the Requests proxy guide. For the legal background that applies beyond Reddit's own terms, see is web scraping legal.

Frequently asked questions