Use Google Scholar Paper & Citation Search Scraper with Python

Search Google Scholar by keyword and get structured records for papers and citations, with author, venue, year, citation counts, and links to PDFs and related work. Filter by year range, document format, and article type.

1. Choose the data you need

Review the published README for supported collection modes, examples, and limits. The recipe below uses the current published input definition.

2. Prepare a source-based input

This JSON contains published prefills and required-field defaults. Review every value for your task before running it; it is not a promise of successful collection.

{
  "articleType": "any",
  "filter": "all",
  "includeCitationFormats": false,
  "includePaperDetails": false,
  "maxItems": 3,
  "mode": "search",
  "queries": [
    "attention is all you need"
  ],
  "sortBy": "relevance",
  "titleOnly": false
}

Download input.example.json and save your reviewed copy as input.json.

Field	Type	Required	Description
`mode`	string	optional	What to fetch: paper search, citing papers, or public author profiles.
`queries`	array	optional	Keywords or paper titles to search on Google Scholar (mode=search). Each query runs separately.
`clusterIds`	array	optional	Paper cluster IDs or Google Scholar 'cites=' URLs to list citing papers for (mode=citations).
`authorIds`	array	optional	Google Scholar author IDs or profile URLs to export (mode=authorProfile).
`maxItems`	integer	optional	Maximum papers per query or cluster, or papers per author. Google Scholar serves about 10 search results per page.
`sortBy`	string	optional	Sort search results by relevance or date (mode=search).
`filter`	string	optional	Keep only PDF or HTML results; applied to returned rows (mode=search).
`articleType`	string	optional	Return all articles or review articles only (mode=search).
`newerThan`	integer	optional	Only include papers published in this year or later (mode=search).
`olderThan`	integer	optional	Only include papers published in this year or earlier (mode=search).
`author`	string	optional	Return papers written by this author, e.g. 'Geoffrey Hinton' (mode=search).
`publication`	string	optional	Return papers from this journal or venue, e.g. 'Nature' (mode=search).
`titleOnly`	boolean	optional	Match the query against paper titles instead of the full text (mode=search).
`includeCitationFormats`	boolean	optional	Also return MLA, APA, Chicago, Harvard, and Vancouver citations for returned rows, within a bounded budget (mode=search or citations).
`includePaperDetails`	boolean	optional	Fetch journal, volume, pages, publisher, and abstract for each author publication (mode=authorProfile).

3. Start an Actor run

Set APIFY_TOKEN in your environment. Python 3.10 or newer is sufficient; this recipe uses no third-party packages.

Execution may incur charges. Review current pricing on Apify. The request asks for a 120-second timeout and a charge limit; these settings are subject to Apify’s pricing and do not guarantee an all-inclusive spending cap.

import json
import os
import urllib.request

# Save the schema-derived example as input.json; review and edit it first.
with open("input.json", encoding="utf-8") as stream:
    actor_input = json.load(stream)

token = os.environ["APIFY_TOKEN"]
request = urllib.request.Request(
    "https://api.apify.com/v2/acts/XT8wyKa1C9SYFQYTh/runs?timeout=120&maxTotalChargeUsd=0.50",
    data=json.dumps(actor_input).encode("utf-8"),
    headers={"Authorization": "Bearer " + token, "Content-Type": "application/json"},
    method="POST",
)
# This starts a potentially billable run. Do not automatically retry this POST.
with urllib.request.urlopen(request, timeout=40) as response:
    run = json.load(response)["data"]
print("Run ID:", run["id"])
print("Status:", run["status"])
# Inspect and download results in Apify Console once the run succeeds.

4. Inspect the actual results

Open the run in Apify Console and wait for a successful completion. Use the published output schema to locate results and the dataset schema to understand declared views. Missing or optional record fields must not be treated as guaranteed values.

Papers, citing papers, and public author profiles from Google Scholar, with authors, venue, year, citation counts, PDF links, cluster IDs, and researcher metrics.

Questions before you run

Which inputs are required?

No top-level fields are marked required in the published schema. Conditional requirements can still apply; check the README.

What does this guide verify?

The fields, prefills, and schema links are rendered from the published build. This is a usage recipe, not a paid live extraction test. Source availability and collection results can vary.

What if the run request times out?

Check Apify Console before retrying. The first request may already have created a run. Do not automatically repeat a run-creation POST.

Open on Apify Full Actor reference

Generated from source build pdUktx8PmbvFPEYYD and its README and schemas. Source changes regenerate this guide; no third-party editorial claims are inferred.