Dataspryva

Use Reddit Subreddit Profile, Rules & Post Filter Scraper with Python

Scrape full community intelligence for any subreddit: description, subscriber count, active users, weekly activity, posting rules, wiki pages, icons/colors and settings - plus optional recent posts. No API key or login.

1. Choose the data you need

Review the published README for supported collection modes, examples, and limits. The recipe below uses the current published input definition.

2. Prepare a source-based input

This JSON contains published prefills and required-field defaults. Review every value for your task before running it; it is not a promise of successful collection.

{
  "commentSort": "confidence",
  "discoverMode": "search",
  "discoverSort": "relevance",
  "excludeCrossposts": false,
  "excludeDeletedAuthor": false,
  "excludeLocked": false,
  "excludeRemoved": false,
  "excludeSpoilers": false,
  "excludeStickied": false,
  "includeComments": false,
  "includeNSFW": false,
  "includePostDetails": false,
  "includePosts": false,
  "includeRules": true,
  "includeWeeklyStats": false,
  "includeWiki": false,
  "maxCommentDepth": 3,
  "maxCommentsPerPost": 25,
  "maxDiscoverResults": 10,
  "maxItems": 1,
  "maxPosts": 25,
  "maxWikiPages": 20,
  "mode": "communities",
  "onlyOriginalContent": false,
  "onlyWithFlair": false,
  "sort": "hot",
  "subreddits": [
    "Python"
  ],
  "timeFilter": "day"
}

Download input.example.json and save your reviewed copy as input.json.

FieldTypeRequiredDescription
modestringrequiredPick the result you want. 'Communities' returns one profile per subreddit (plus optional posts and comments). 'Discover' finds communities by keyword or browse. 'Posts' returns post records only. 'Comments' returns capped comment trees for post URLs or a subreddit's top posts.
subredditsarrayoptionalSubreddit names to scrape. Accepts bare names, r/ names or full URLs. Use the name as shown in the subreddit URL (Reddit redirects are challenged on some networks). Not needed when 'postUrls' is set or when discovery finds enough communities.
postUrlsarrayoptionalReddit post or comment URLs (or bare post IDs) to scrape in comments mode. When set, these take priority over subreddit top posts. Example: https://www.reddit.com/r/pics/comments/1x1xbpb/
discoverModestringoptionalHow to find extra communities. 'Keyword search' uses the field below. 'Currently popular' and 'Newest' browse Reddit's curated lists (best-effort: those pages expose only a few communities).
discoverQuerystringoptionalSearch Reddit's community directory for this keyword and add the matching communities to the run. Only used with 'Keyword search'. Leave empty to skip discovery.
discoverSortstringoptionalHow to rank keyword-discovered communities: relevance keeps Reddit's order, activity sorts by weekly active users.
maxDiscoverResultsintegeroptionalMaximum number of communities to add from keyword search or browse (1-100).
includeRulesbooleanoptionalAdd posting rules (name, description, priority) to community profiles in communities mode.
includeWikibooleanoptionalFetch the community wiki index and each page's Markdown source. Wiki pages are omitted when the wiki is disabled or empty.
maxWikiPagesintegeroptionalMaximum number of wiki pages to fetch per community when 'Include wiki pages' is on (0-200).
includeWeeklyStatsbooleanoptionalInclude Reddit's weekly active users and weekly contributions from the community header.
includePostsbooleanoptionalAlso scrape the community's recent posts, with the full filter set below.
maxPostsintegeroptionalMaximum posts to collect per subreddit (1-1,000). The global 'Maximum records' cap still applies.
sortstringoptionalHow to order the post feed.
timeFilterstringoptionalTime window used when sorting posts by top or controversial.
includeNSFWbooleanoptionalKeep posts marked NSFW. NSFW-only communities are gated for signed-out visitors and return nothing regardless of this setting.
onlyWithFlairbooleanoptionalDrop posts that have no flair tag.
postTypestringoptionalKeep only one content type. Leave unset to keep all types.
minScoreintegeroptionalKeep posts with a score (upvotes minus downvotes) at or above this value. Leave unset to keep all scores.
maxScoreintegeroptionalKeep posts with a score at or below this value. Useful for finding overlooked posts. Leave unset to keep all scores.
minCommentsintegeroptionalKeep posts with at least this many comments. Leave unset to keep all posts.
maxCommentsintegeroptionalKeep posts with at most this many comments. Leave unset to keep all posts.
excludeStickiedbooleanoptionalDrop posts pinned by moderators (rules, weekly threads, announcements).
minUpvoteRationumberoptionalKeep posts whose upvote ratio (0-1) is at or above this value. Leave unset to keep all ratios.
excludeRemovedbooleanoptionalDrop posts removed by a moderator or Reddit. Deleted post markers are only visible on post detail pages.
excludeCrosspostsbooleanoptionalDrop crossposts from another community, keeping original submissions.
excludeSpoilersbooleanoptionalDrop posts marked as a spoiler by their author.
excludeLockedbooleanoptionalDrop posts whose comments were locked by a moderator.
excludeDeletedAuthorbooleanoptionalDrop posts whose author appears as [deleted] or [removed].
onlyOriginalContentbooleanoptionalKeep only posts the author tagged as original content.
minAwardsintegeroptionalKeep posts with at least this many total awards. Leave unset to keep all posts.
titleContainsstringoptionalKeep posts whose title contains this text (case-insensitive). Leave empty to keep all titles.
contentContainsstringoptionalKeep self posts whose body contains this text (case-insensitive). Requires 'Fetch full post details' so post bodies are available; link/image posts are excluded.
flairContainsstringoptionalKeep posts whose flair text contains this text (case-insensitive). Leave empty to keep all flairs.
authorFlairContainsstringoptionalKeep posts whose author flair text contains this text (case-insensitive).
authorsarrayoptionalKeep posts submitted by one of these usernames (bare or u/-prefixed, case-insensitive).
domainstringoptionalKeep posts whose link domain contains this text (case-insensitive), e.g. youtube.com or self.python.
distinguishedstringoptionalKeep posts distinguished by Reddit as moderator, admin or special, or keep only regular posts. Leave unset to keep all.
postedAfterstringoptionalKeep posts created on or after this date (UTC). When set, the post feed is walked from newest first.
postedBeforestringoptionalKeep posts created on or before this date (end of day, UTC).
includeCommentsbooleanoptionalFetch capped comment trees for every post collected in communities or posts mode. In comments mode this is always on.
maxCommentsPerPostintegeroptionalStop collecting a thread after this many comments (1-10,000). The global 'Maximum records' cap still applies.
commentSortstringoptionalHow Reddit orders the comment tree. 'Confidence' is Reddit's default best-first order; the other four are the verified alternate sorts.
maxCommentDepthintegeroptionalKeep comments up to this nesting depth. 0 keeps top-level comments only; each extra level adds replies (0-100).
includePostDetailsbooleanoptionalOpen each collected post page to fill body, media and detail-only flags. In comments mode this also saves the parent post record. Slower: one extra page per post.
maxItemsintegeroptionalHard cap on all records pushed by the run (communities, posts and comments together). The run stops cleanly when it is reached.

3. Start an Actor run

Set APIFY_TOKEN in your environment. Python 3.10 or newer is sufficient; this recipe uses no third-party packages.

Execution may incur charges. Review current pricing on Apify. The request asks for a 120-second timeout and a charge limit; these settings are subject to Apify’s pricing and do not guarantee an all-inclusive spending cap.

import json
import os
import urllib.request

# Save the schema-derived example as input.json; review and edit it first.
with open("input.json", encoding="utf-8") as stream:
    actor_input = json.load(stream)

token = os.environ["APIFY_TOKEN"]
request = urllib.request.Request(
    "https://api.apify.com/v2/acts/ArU0Lx9NfPdbDYHu8/runs?timeout=120&maxTotalChargeUsd=0.50",
    data=json.dumps(actor_input).encode("utf-8"),
    headers={"Authorization": "Bearer " + token, "Content-Type": "application/json"},
    method="POST",
)
# This starts a potentially billable run. Do not automatically retry this POST.
with urllib.request.urlopen(request, timeout=40) as response:
    run = json.load(response)["data"]
print("Run ID:", run["id"])
print("Status:", run["status"])
# Inspect and download results in Apify Console once the run succeeds.

4. Inspect the actual results

Open the run in Apify Console and wait for a successful completion. Use the published output schema to locate results and the dataset schema to understand declared views. Missing or optional record fields must not be treated as guaranteed values.

Community profiles (with rules, wiki and weekly activity), filtered posts and capped comment trees from Reddit's public pages. Records are distinguished by the `dataType` field.

Questions before you run

Which inputs are required?

The published required fields are mode

What does this guide verify?

The fields, prefills, and schema links are rendered from the published build. This is a usage recipe, not a paid live extraction test. Source availability and collection results can vary.

What if the run request times out?

Check Apify Console before retrying. The first request may already have created a run. Do not automatically repeat a run-creation POST.

Generated from source build Yy4ffYbftredacudh and its README and schemas. Source changes regenerate this guide; no third-party editorial claims are inferred.