Use Reddit Subreddit Profile, Rules & Post Filter Scraper with Python
Scrape full community intelligence for any subreddit: description, subscriber count, active users, weekly activity, posting rules, wiki pages, icons/colors and settings - plus optional recent posts. No API key or login.
1. Choose the data you need
Review the published README for supported collection modes, examples, and limits. The recipe below uses the current published input definition.
2. Prepare a source-based input
This JSON contains published prefills and required-field defaults. Review every value for your task before running it; it is not a promise of successful collection.
{
"commentSort": "confidence",
"discoverMode": "search",
"discoverSort": "relevance",
"excludeCrossposts": false,
"excludeDeletedAuthor": false,
"excludeLocked": false,
"excludeRemoved": false,
"excludeSpoilers": false,
"excludeStickied": false,
"includeComments": false,
"includeNSFW": false,
"includePostDetails": false,
"includePosts": false,
"includeRules": true,
"includeWeeklyStats": false,
"includeWiki": false,
"maxCommentDepth": 3,
"maxCommentsPerPost": 25,
"maxDiscoverResults": 10,
"maxItems": 1,
"maxPosts": 25,
"maxWikiPages": 20,
"mode": "communities",
"onlyOriginalContent": false,
"onlyWithFlair": false,
"sort": "hot",
"subreddits": [
"Python"
],
"timeFilter": "day"
}
Download input.example.json and save your reviewed copy as input.json.
| Field | Type | Required | Description |
|---|---|---|---|
mode | string | required | Pick the result you want. 'Communities' returns one profile per subreddit (plus optional posts and comments). 'Discover' finds communities by keyword or browse. 'Posts' returns post records only. 'Comments' returns capped comment trees for post URLs or a subreddit's top posts. |
subreddits | array | optional | Subreddit names to scrape. Accepts bare names, r/ names or full URLs. Use the name as shown in the subreddit URL (Reddit redirects are challenged on some networks). Not needed when 'postUrls' is set or when discovery finds enough communities. |
postUrls | array | optional | Reddit post or comment URLs (or bare post IDs) to scrape in comments mode. When set, these take priority over subreddit top posts. Example: https://www.reddit.com/r/pics/comments/1x1xbpb/ |
discoverMode | string | optional | How to find extra communities. 'Keyword search' uses the field below. 'Currently popular' and 'Newest' browse Reddit's curated lists (best-effort: those pages expose only a few communities). |
discoverQuery | string | optional | Search Reddit's community directory for this keyword and add the matching communities to the run. Only used with 'Keyword search'. Leave empty to skip discovery. |
discoverSort | string | optional | How to rank keyword-discovered communities: relevance keeps Reddit's order, activity sorts by weekly active users. |
maxDiscoverResults | integer | optional | Maximum number of communities to add from keyword search or browse (1-100). |
includeRules | boolean | optional | Add posting rules (name, description, priority) to community profiles in communities mode. |
includeWiki | boolean | optional | Fetch the community wiki index and each page's Markdown source. Wiki pages are omitted when the wiki is disabled or empty. |
maxWikiPages | integer | optional | Maximum number of wiki pages to fetch per community when 'Include wiki pages' is on (0-200). |
includeWeeklyStats | boolean | optional | Include Reddit's weekly active users and weekly contributions from the community header. |
includePosts | boolean | optional | Also scrape the community's recent posts, with the full filter set below. |
maxPosts | integer | optional | Maximum posts to collect per subreddit (1-1,000). The global 'Maximum records' cap still applies. |
sort | string | optional | How to order the post feed. |
timeFilter | string | optional | Time window used when sorting posts by top or controversial. |
includeNSFW | boolean | optional | Keep posts marked NSFW. NSFW-only communities are gated for signed-out visitors and return nothing regardless of this setting. |
onlyWithFlair | boolean | optional | Drop posts that have no flair tag. |
postType | string | optional | Keep only one content type. Leave unset to keep all types. |
minScore | integer | optional | Keep posts with a score (upvotes minus downvotes) at or above this value. Leave unset to keep all scores. |
maxScore | integer | optional | Keep posts with a score at or below this value. Useful for finding overlooked posts. Leave unset to keep all scores. |
minComments | integer | optional | Keep posts with at least this many comments. Leave unset to keep all posts. |
maxComments | integer | optional | Keep posts with at most this many comments. Leave unset to keep all posts. |
excludeStickied | boolean | optional | Drop posts pinned by moderators (rules, weekly threads, announcements). |
minUpvoteRatio | number | optional | Keep posts whose upvote ratio (0-1) is at or above this value. Leave unset to keep all ratios. |
excludeRemoved | boolean | optional | Drop posts removed by a moderator or Reddit. Deleted post markers are only visible on post detail pages. |
excludeCrossposts | boolean | optional | Drop crossposts from another community, keeping original submissions. |
excludeSpoilers | boolean | optional | Drop posts marked as a spoiler by their author. |
excludeLocked | boolean | optional | Drop posts whose comments were locked by a moderator. |
excludeDeletedAuthor | boolean | optional | Drop posts whose author appears as [deleted] or [removed]. |
onlyOriginalContent | boolean | optional | Keep only posts the author tagged as original content. |
minAwards | integer | optional | Keep posts with at least this many total awards. Leave unset to keep all posts. |
titleContains | string | optional | Keep posts whose title contains this text (case-insensitive). Leave empty to keep all titles. |
contentContains | string | optional | Keep self posts whose body contains this text (case-insensitive). Requires 'Fetch full post details' so post bodies are available; link/image posts are excluded. |
flairContains | string | optional | Keep posts whose flair text contains this text (case-insensitive). Leave empty to keep all flairs. |
authorFlairContains | string | optional | Keep posts whose author flair text contains this text (case-insensitive). |
authors | array | optional | Keep posts submitted by one of these usernames (bare or u/-prefixed, case-insensitive). |
domain | string | optional | Keep posts whose link domain contains this text (case-insensitive), e.g. youtube.com or self.python. |
distinguished | string | optional | Keep posts distinguished by Reddit as moderator, admin or special, or keep only regular posts. Leave unset to keep all. |
postedAfter | string | optional | Keep posts created on or after this date (UTC). When set, the post feed is walked from newest first. |
postedBefore | string | optional | Keep posts created on or before this date (end of day, UTC). |
includeComments | boolean | optional | Fetch capped comment trees for every post collected in communities or posts mode. In comments mode this is always on. |
maxCommentsPerPost | integer | optional | Stop collecting a thread after this many comments (1-10,000). The global 'Maximum records' cap still applies. |
commentSort | string | optional | How Reddit orders the comment tree. 'Confidence' is Reddit's default best-first order; the other four are the verified alternate sorts. |
maxCommentDepth | integer | optional | Keep comments up to this nesting depth. 0 keeps top-level comments only; each extra level adds replies (0-100). |
includePostDetails | boolean | optional | Open each collected post page to fill body, media and detail-only flags. In comments mode this also saves the parent post record. Slower: one extra page per post. |
maxItems | integer | optional | Hard cap on all records pushed by the run (communities, posts and comments together). The run stops cleanly when it is reached. |
3. Start an Actor run
Set APIFY_TOKEN in your environment. Python 3.10 or newer is sufficient; this recipe uses no third-party packages.
Execution may incur charges. Review current pricing on Apify. The request asks for a 120-second timeout and a charge limit; these settings are subject to Apify’s pricing and do not guarantee an all-inclusive spending cap.
import json
import os
import urllib.request
# Save the schema-derived example as input.json; review and edit it first.
with open("input.json", encoding="utf-8") as stream:
actor_input = json.load(stream)
token = os.environ["APIFY_TOKEN"]
request = urllib.request.Request(
"https://api.apify.com/v2/acts/ArU0Lx9NfPdbDYHu8/runs?timeout=120&maxTotalChargeUsd=0.50",
data=json.dumps(actor_input).encode("utf-8"),
headers={"Authorization": "Bearer " + token, "Content-Type": "application/json"},
method="POST",
)
# This starts a potentially billable run. Do not automatically retry this POST.
with urllib.request.urlopen(request, timeout=40) as response:
run = json.load(response)["data"]
print("Run ID:", run["id"])
print("Status:", run["status"])
# Inspect and download results in Apify Console once the run succeeds.
4. Inspect the actual results
Open the run in Apify Console and wait for a successful completion. Use the published output schema to locate results and the dataset schema to understand declared views. Missing or optional record fields must not be treated as guaranteed values.
Community profiles (with rules, wiki and weekly activity), filtered posts and capped comment trees from Reddit's public pages. Records are distinguished by the `dataType` field.
Questions before you run
Which inputs are required?
The published required fields are mode
What does this guide verify?
The fields, prefills, and schema links are rendered from the published build. This is a usage recipe, not a paid live extraction test. Source availability and collection results can vary.
What if the run request times out?
Check Apify Console before retrying. The first request may already have created a run. Do not automatically repeat a run-creation POST.
Generated from source build Yy4ffYbftredacudh and its README and schemas. Source changes regenerate this guide; no third-party editorial claims are inferred.