Building a Simple RSS Feed Aggregator with Node.js

To build an RSS aggregator, you need to use Node.js to combine axios and rss-parser packages to grab and parse multiple RSS sources. First, initialize the project and install the dependencies, and then define a URL list containing Hacker News, TechCrunch and other sources in aggregator.js. Concurrently obtain and process data from each source through Promise.all, extract the title, link, release time and source, and arrange it in reverse order of time. Then, you can output the console or create a server in Express to return the results in JSON format. Finally, you can add a cache mechanism to avoid frequent requests and improve performance, thereby achieving an efficient and extensible RSS aggregation system.

Building an RSS feed aggregator with Node.js is simpler than it sounds. With just a few packages and a bit of JavaScript, you can pull in content from multiple blogs or news sites and display them in one place. Here's how to build a basic RSS feed aggregator using Node.js.

Why Build an RSS Aggregator?

RSS (Really Simple Syndication) feeds let websites publish frequently updated content in a standardized format. An aggregator pulls these feeds, parses them, and presents the latest articles in a unified way. It's useful for tracking blogs, news, or podcasts without visiting each site individually.

Node.js works well for this because of its strong ecosystem and async capabilities—perfect for fetching and processing multiple feeds.

1. Set Up the Project

Start by initializing a new Node.js project:

 mkdir rss-aggregator
cd rss-aggregator
npm init -y

Install the required dependencies. We'll use axios for HTTP requests and rss-parser to parse RSS feeds:

 npm install axios rss-parser

2. Fetch and Parse RSS Feeds

Create a file called aggregator.js . Here's a basic script that fetches and parses a few feeds:

 const Parser = require('rss-parser');
const parser = new Parser();

async function aggregateFeeds() {
  const feedUrls = [
    'https://hnrss.org/newest', // Hacker News
    'https://feeds.feedburner.com/blogspot/gJZGY', // TechCrunch
    'https://rss.simplecast.com/podcasts/4661/rss' // A podcast or blog
  ];

  const feedPromises = feedUrls.map(async (url) => {
    try {
      const feed = await parser.parseURL(url);
      return feed.items.map(item => ({
        title: item.title,
        link: item.link,
        pubDate: item.pubDate,
        source: feed.title
      }));
    } catch (err) {
      console.error(`Failed to fetch ${url}:`, err.message);
      return [];
    }
  });

  const results = await Promise.all(feedPromises);
  const allItems = results.flat();

  // Sort by date, newest first
  allItems.sort((a, b) => new Date(b.pubDate) - new Date(a.pubDate));

  return allItems;
}

This function:

• Defines a list of RSS feed URLs
• Uses Promise.all to fetch them concurrently
• Maps each feed's items with consistent fields
• Handles errors gracefully (eg, if a feed is unreachable)
• Flattens and sorts all articles by publication date

3. Display or Serve the Results

You can log the results to the console:

 // Add to the bottom of aggregator.js
aggregateFeeds().then(items => {
  items.forEach(item => {
    console.log(`[${item.source}] ${item.title} (${item.pubDate})\n${item.link}\n`);
  });
});

Or, better yet, create a simple Express server to serve the data as JSON:

 npm install express

Create server.js :

 const express = require('express');
const { aggregateFeeds } = require('./aggregator');

const app = express();
const PORT = 3000;

app.get('/feeds', async (req, res) => {
  try {
    const items = await aggregateFeeds();
    res.json(items);
  } catch (err) {
    res.status(500).json({ error: err.message });
  }
});

app.listen(PORT, () => {
  console.log(`RSS Aggregator running on http://localhost:${PORT}`);
});

Run it:

 node server.js

Visit http://localhost:3000/feeds to see your aggregated feed in JSON.

4. Enhance with Caching (Optional)

Fetching feeds on every request isn't efficient. Add simple in-memory caching:

 let cachedFeeds = null;
let lastFetch = 0;
const CACHE_TTL = 10 * 60 * 1000; // 10 minutes

async function getAggregatedFeeds() {
  const now = Date.now();
  if (cachedFeeds && (now - lastFetch) < CACHE_TTL) {
    return cachedFeeds;
  }

  cachedFeeds = await aggregateFeeds();
  lastFetch = now;
  return cachedFeeds;
}

Then use getAggregatedFeeds() in your Express route.

Final Thoughts

You now have a working RSS feed aggregator that:

• Pulls multiple feeds
• Parses and normalizes content
• Serves it via a simple API

From here, you could:

• Add a frontend (React, Vue, or plain HTML)
• Support user-submitted feeds
• Filter by keywords or sources
• Deploy it with PM2 or to platforms like Vercel, Render, or AWS

It's not flashy, but it's functional—and a great starting point for more advanced features.

Basically, with just a little code, you've turned scattered content into a single stream. And that's the whole point of RSS.

The above is the detailed content of Building a Simple RSS Feed Aggregator with Node.js. For more information, please follow other related articles on the PHP Chinese website!