Following New Economics Papers with RSS

Economics
Web-Scraping
Published

October 4, 2023

The option to subscribe to REPEC’s NEP service is hidden but still available. This post provides a list of available rss feeds.

REPEC is an online database of economics papers, most of which are freely available to read. It is an incredible resource that is itself underrated by students of economics. But even more underrated is the associated New Economics Papers service that notifies subscribers as new papers are released.

There are over a hundred categories which can be followed. Each managed by an academic volunteer who selects relevant papers for distribution. It is the absolute easiest way to stay up to date on new research in economics.

The NEP page includes buttons to subscribe to an email list or Mastodon feed for each category but RSS, the objectively best option, is missing.1

I knew that RSS feeds would be around somewhere and after a little search I finally found them under nep.repec.org/rss/. This page shows an Apache directory index with the RSS files. Unhelpfully they are only titled by ID but by cross-referencing with the main page I was able to generate this list of available feeds.

Code
library("rvest")

# NEP listing for humans
nep <- read_html("https://nep.repec.org/")

# Select IDs and Categories
nep_categories <- nep |>
  html_elements(".nitpo_antem")

id_to_category <- tibble::tibble(
  ID = nep_categories |>
    html_element(".sbore_repcode") |>
    html_text2(),
  Category = nep_categories |>
    html_element(".derep") |>
    html_element(xpath = "a[contains(@href, 'latest')]") |>
    html_text2()
)

# Apache rss index
nep_rss <- read_html("https://nep.repec.org/rss/")

# Select IDs and RSS Feed Links
id_to_feed <- nep_rss |>
  html_element("table") |>
  html_table() |>
  dplyr::select(Name, `Last modified`) |>
  dplyr::mutate(
    ID = stringr::str_sub(Name, 1, stringr::str_locate(Name, "\\.")[, "end"] - 1),
    Feed = paste0("[", ID, "](https://nep.repec.org/rss/", ID, ".rss.xml)"),
    Dormant = lubridate::interval(
      lubridate::ymd_hm(`Last modified`),
      lubridate::today()
    ) / lubridate::years(1) > 1
  ) |>
  dplyr::filter(!Dormant) |>
  dplyr::select(ID, Feed)

# Combine Feed and Category
feed_and_category <- id_to_category |>
  dplyr::inner_join(id_to_feed, dplyr::join_by(ID)) |>
  dplyr::select(Feed, Category)

# Output
feed_and_category
Feed Category
nep-acc Accounting and Auditing
nep-afr Africa
nep-age Economics of Ageing
nep-agr Agricultural Economics
nep-ain Artificial Intelligence
nep-ara MENA - Middle East and North Africa
nep-ban Banking
nep-bec Business Economics
nep-big Big Data
nep-cba Central Banking
nep-cbe Cognitive and Behavioural Economics
nep-cdm Collective Decision-Making
nep-cfn Corporate Finance
nep-cis Confederation of Independent States
nep-cmp Computational Economics
nep-cna China
nep-com Industrial Competition
nep-cse Economics of Strategic Management
nep-cta Contract Theory and Applications
nep-cul Cultural Economics
nep-dcm Discrete Choice Models
nep-dem Demographic Economics
nep-des Economic Design
nep-dev Development
nep-dge Dynamic General Equilibrium
nep-ecm Econometrics
nep-edu Education
nep-eec European Economics
nep-eff Efficiency and Productivity
nep-ene Energy Economics
nep-ent Entrepreneurship
nep-env Environmental Economics
nep-ets Econometric Time Series
nep-eur Microeconomic European Issues
nep-evo Evolutionary Economics
nep-exp Experimental Economics
nep-fdg Financial Development and Growth
nep-fle Financial Literacy and Education
nep-fmk Financial Markets
nep-for Forecasting
nep-gen Gender
nep-geo Economic Geography
nep-ger German Papers
nep-gro Economic Growth
nep-gth Game Theory
nep-hap Economics of Happiness
nep-hea Health Economics
nep-his Business, Economic and Financial History
nep-hme Heterodox Microeconomics
nep-hpe History and Philosophy of Economics
nep-hrm Human Capital and Human Resource Management
nep-ict Information and Communication Technologies
nep-ifn International Finance
nep-ind Industrial Organization
nep-ino Innovation
nep-int International Trade
nep-inv Investment
nep-ipr Intellectual Property Rights
nep-isf Islamic Finance
nep-iue Informal and Underground Economics
nep-knm Knowledge Management and Knowledge Economy
nep-lab Labour Economics
nep-lam Central and South America
nep-law Law and Economics
nep-lma Labor Markets - Supply, Demand, and Wages
nep-ltv Unemployment, Inequality and Poverty
nep-mac Macroeconomics
nep-mfd Microfinance
nep-mic Microeconomics
nep-mig Economics of Human Migration
nep-mkt Marketing
nep-mon Monetary Economics
nep-mst Market Microstructure
nep-net Network Economics
nep-neu Neuroeconomics
nep-nud Nudge and Boosting
nep-opm Open Economy Macroeconomics
nep-pay Payment Systems and Financial Technology
nep-pbe Public Economics
nep-pke Post Keynesian Economics
nep-pol Positive Political Economics
nep-ppm Project, Program and Portfolio Management
nep-pub Public Finance
nep-reg Regulation
nep-res Resource Economics
nep-rmg Risk Management
nep-sbm Small Business Management
nep-sea South East Asia
nep-soc Social Norms and Social Capital
nep-sog Sociology of Economics
nep-spo Sports and Economics
nep-tid Technology and Industrial Dynamics
nep-tra Transition Economics
nep-tre Transport Economics
nep-tur Tourism Economics
nep-upt Utility Models and Prospect Theory
nep-ure Urban and Real Estate Economics

Footnotes

  1. It is mentioned on the page but there are no links.↩︎