How do I serve another existing website on my local Node.js server?

Issue

So take this example:

• User goes to site (localhost:8080)
• The Node.js webserver makes a GET request to https://example.com
• The Node.js webserver also scrapes the content of that site
• It then responds to the user (on localhost:8080) with the Content-Type, Status code, and content of the original site.

I have done this in such a simple method before previously.

What happened? Every time you make any type of GET request in Node.js (with fetch, axios, http/https) I always get the error AggregateError with multiple errors regarding something like "not being able to reach {github.io’s IPv6 address}", ONLY if I make multiple requests at a time. If I make one of these requests, it works fine.

The reason that it’s GitHub.io’s IP address is because the site I want to mirror onto the server is https://parking-master.github.io/FPS/

This is the code that produces that error (for debug purposes or more info):

app.get("*", async function(req, res) {
  let path = req.url;
  try {
    function request(response) {
      const buffer = Buffer.from(response.data);
      res.setHeader("Content-Type", response.headers["content-type"]);
      res.statusCode = response.status;
      return res.send(buffer);
    }
    try {
      let response = await axios.get(address + path, { responseType: "arraybuffer" });
      request(response);
    } catch(err) {
      console.log("Error caught during request:", err);
    }
  } catch(err) {
    console.log("Error caught during scraping:", err);
  }
});

And this is the error:

AggregateError
at internalConnectMultiple (node:net:1103:18)
at internalConnectMultiple (node:net:1161:5)
at internalConnectMultiple (node:net:1161:5)
at internalConnectMultiple (node:net:1161:5)
at internalConnectMultiple (node:net:1161:5)
at Timeout.internalConnectMultipleTimeout (node:net:1638:3)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7) {code: ‘ENETUNREACH’,[errors]: [Error: connect ENETUNREACH 2606:50c0:8001::153:443 – Local (undefined:undefined)

If anyone knows why this problem occurs, let me know. More information is that this error usually occurs when requesting unique files from the original site (like .mp3, .glb) and not on common files (like .html, .css, .js)

My actual question is, am I doing it the right way, and if not, how do I actually host a webserver that mirrors another website’s pages onto itself?

I had a hard time explaining this question, but I hope everyone understands it.

Note: I am using express.js to host the webserver. It would be great if this could be done using express.js, but I’m willing to use some other type of software.

Solution

What you are trying to implement is reverse proxy. You can install http-proxy-middleware from npm and use it in your node app. Below is the example:

const express = require('express');
const { createProxyMiddleware } = require('http-proxy-middleware');

const app = express();

app.use('/', createProxyMiddleware({
  target: 'http://example.com', // target host
  changeOrigin: true, // needed for virtual hosted sites
}));

app.listen(3000, () => {
  console.log('Server is running on http://localhost:3000');
});

Answered By – Ankur Sharma

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0