Merge pull request #48 from Lissy93/FEAT/vercel-support

[FEAT] Vercel support

.github/README.md

@@ -19,11 +19,12 @@
   - [Mirror](#mirror)
   - [Features](#features)
 - **[Usage](#usage)**
-  - [Developer Setup](#developing)
   - [Deploying, Option#1: Netlify](#deploying---option-1-netlify)
-  - [Deploying, Option#2: Docker](#deploying---option-2-docker)
+  - [Deploying, Option#2: Vercel](#deploying---option-2-vercel)
-  - [Deploying, Option#3: Source](#deploying---option-3-from-source)
+  - [Deploying, Option#3: Docker](#deploying---option-3-docker)
+  - [Deploying, Option#4: Source](#deploying---option-4-from-source)
   - [Configuration Options](#configuring)
+  - [Developer Setup](#developing)
 - **[Community](#community)**
   - [Contributing](#contributing)
   - [Bugs](#reporting-bugs)
@@ -56,11 +57,6 @@ A hosted version can be accessed at: **[web-check.as93.net](https://web-check.as
 ### Mirror
 The source for this repo is mirrored to CodeBerg, available at: **[codeberg.org/alicia/web-check](https://codeberg.org/alicia/web-check)**
 
-### Motivation
-Often when you're looking into a website, there's several things you always initially check.
-
-None of this is hard to find with a series of basic curl commands and NPMAP plus a combination of online tools. But it's just so much easier to have everything done all at once, presented clearly and visible in one place :)
-
 ### Features
 
 <details>
@@ -738,35 +734,35 @@ _Note that not all checks will work for all sites. Sometimes it's not possible t
 
 ## Usage
 
-### Developing
-
-1. Clone the repo, `git clone git@github.com:Lissy93/web-check.git`
-2. Cd into it, `cd web-check`
-3. Install dependencies: `yarn`
-4. Start the dev server, with `yarn dev`
-
-You'll need [Node.js](https://nodejs.org/en) (V 18.16.1 or later) installed, plus [yarn](https://yarnpkg.com/getting-started/install) as well as [git](https://git-scm.com/).
-Some checks also require `chromium`, `traceroute` and `dns` to be installed within your environment. These jobs will just be skipped if those packages aren't present.
-
 ### Deploying - Option #1: Netlify
 
 Click the button below, to deploy to Netlify 👇
 
 [![Deploy to Netlify](https://img.shields.io/badge/Deploy-Netlify-%2330c8c9?style=for-the-badge&logo=netlify&labelColor=1e0e41 'Deploy Web-Check to Netlify, via 1-Click Script')](https://app.netlify.com/start/deploy?repository=https://github.com/lissy93/web-check)
 
-### Deploying - Option #2: Docker
+### Deploying - Option #2: Vercel
 
-Run `docker run -p 8888:3000 lissy93/web-check`, then open `http://localhost:3000`
+Click the button below, to deploy to Vercel 👇
+
+[![Deploy with Vercel](https://img.shields.io/badge/Deploy-Vercel-%23ffffff?style=for-the-badge&logo=vercel&labelColor=1e0e41)](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Flissy93%2Fweb-check&project-name=web-check&repository-name=web-check-fork&demo-title=Web-Check%20Demo&demo-description=Check%20out%20web-check.xyz%20to%20see%20a%20live%20demo%20of%20this%20application%20running.&demo-url=https%3A%2F%2Fweb-check.xyz&demo-image=https%3A%2F%2Fraw.githubusercontent.com%2FLissy93%2Fweb-check%2Fmaster%2F.github%2Fscreenshots%2Fweb-check-screenshot10.png)
+
+### Deploying - Option #3: Docker
+
+Run `docker run -p 3000:3000 lissy93/web-check`, then open `http://localhost:3000`
+
+<details>
+<summary>Docker Options</summary>
 
 You can get the Docker image from:
 - DockerHub: [`lissy93/web-check`](https://hub.docker.com/r/lissy93/web-check)
 - GHCR: [`ghcr.io/lissy93/web-check`](https://github.com/Lissy93/web-check/pkgs/container/web-check)
 - Or build the image yourself by cloning the repo and running `docker build -t web-check .`
 
-### Deploying - Option #3: From Source
+</details>
 
-Follow the instructions in the [Developing](#developing) section above, then run `yarn build` && `yarn serve` to build and serve the application.
+### Deploying - Option #4: From Source
-The following commands will work:
+
+Install the prerequisites listed in the [Developing](#developing) section, then run: 
 
 ```bash
 git clone https://github.com/Lissy93/web-check.git # Grab the code
@@ -776,7 +772,6 @@ yarn build # Build the app for production
 yarn serve # Start the app (API and GUI)
 ```
 
-
 ### Configuring
 
 By default, no configuration is needed.
@@ -807,6 +802,17 @@ Note that keys that are prefixed with `REACT_APP_` are used client-side, and as
 The above can be added into an `.env` file in the projects root, or via the Netlify UI, or by passing directly to the Docker container with the --env flag.
 All variables are optional.
 
+### Developing
+
+1. Clone the repo, `git clone git@github.com:Lissy93/web-check.git`
+2. Cd into it, `cd web-check`
+3. Install dependencies: `yarn`
+4. Start the dev server, with `yarn dev`
+
+You'll need [Node.js](https://nodejs.org/en) (V 18.16.1 or later) installed, plus [yarn](https://yarnpkg.com/getting-started/install) as well as [git](https://git-scm.com/).
+Some checks also require `chromium`, `traceroute` and `dns` to be installed within your environment. These jobs will just be skipped if those packages aren't present.
+
+
 ---
 
 ## Community

.gitignore

@@ -1,27 +1,60 @@
-
+# ------------------------
-# keys
+#   ENVIRONMENT SETTINGS
+# ------------------------
 .env
 
-# dependencies
+# ------------------------
-/node_modules
+#       PRODUCTION
-/.pnp
+# ------------------------
-.pnp.js
+/build/
 
-# logs
+# ------------------------
+#         DEPLOYMENT
+# ------------------------
+.vercel/
+.netlify/
+.webpack/
+.serverless/
+
+# ------------------------
+#         DEPENDENCIES
+# ------------------------
+node_modules/
+.yarn/cache/
+.yarn/unplugged/
+.yarn/build-state.yml
+.yarn/install-state.gz
+.pnpm/
+.pnp.*
+
+# ------------------------
+#            LOGS
+# ------------------------
+logs/
+*.log
 npm-debug.log*
 yarn-debug.log*
 yarn-error.log*
+lerna-debug.log*
+.pnpm-debug.log*
 
-# testing
+# ------------------------
-/coverage
+#          TESTING
+# ------------------------
+coverage/
+.nyc_output/
 
-# production
+# ------------------------
-/build
+#       OS SPECIFIC
-
+# ------------------------
-# Random AWS and Netlify crap
-.netlify
-.serverless
-.webpack
-
-# OS generated files
 .DS_Store
+Thumbs.db
+
+# ------------------------
+#         EDITORS
+# ------------------------
+.idea/
+.vscode/
+*.swp
+*.swo
+

api/_common/middleware.js

@@ -2,8 +2,40 @@ const normalizeUrl = (url) => {
   return url.startsWith('http') ? url : `https://${url}`;
 };
 
+const headers = {
+  'Access-Control-Allow-Origin': process.env.API_CORS_ORIGIN || '*',
+  'Access-Control-Allow-Credentials': true,
+  'Content-Type': 'application/json;charset=UTF-8',
+};
+
 const commonMiddleware = (handler) => {
-  return async (event, context, callback) => {
+  // Vercel
+  const vercelHandler = async (request, response) => {
+    const queryParams = request.query || {};
+    const rawUrl = queryParams.url;
+
+    if (!rawUrl) {
+      return response.status(500).json({ error: 'No URL specified' });
+    }
+
+    const url = normalizeUrl(rawUrl);
+
+    try {
+      const handlerResponse = await handler(url, request);
+      if (handlerResponse.body && handlerResponse.statusCode) {
+        response.status(handlerResponse.statusCode).json(handlerResponse.body);
+      } else {
+        response.status(200).json(
+          typeof handlerResponse === 'object' ? handlerResponse : JSON.parse(handlerResponse)
+        );
+      }
+    } catch (error) {
+      response.status(500).json({ error: error.message });
+    }
+  };
+
+  // Netlify
+  const netlifyHandler = async (event, context, callback) => {
     const queryParams = event.queryStringParameters || event.query || {};
     const rawUrl = queryParams.url;
 
@@ -11,28 +43,40 @@ const commonMiddleware = (handler) => {
       callback(null, {
         statusCode: 500,
         body: JSON.stringify({ error: 'No URL specified' }),
+        headers,
       });
+      return;
     }
 
     const url = normalizeUrl(rawUrl);
 
     try {
-      const response = await handler(url, event, context);
+      const handlerResponse = await handler(url, event, context);
-      if (response.body && response.statusCode) {
+      if (handlerResponse.body && handlerResponse.statusCode) {
-      callback(null, response);
+        callback(null, handlerResponse);
       } else {
         callback(null, {
           statusCode: 200,
-          body: typeof response === 'object' ? JSON.stringify(response) : response,
+          body: typeof handlerResponse === 'object' ? JSON.stringify(handlerResponse) : handlerResponse,
+          headers,
         });
       }
     } catch (error) {
       callback(null, {
         statusCode: 500,
         body: JSON.stringify({ error: error.message }),
+        headers,
       });
     }
   };
+
+  // The format of the handlers varies between platforms
+  // E.g. Netlify + AWS expect Lambda functions, but Vercel or Node needs standard handler
+  const platformEnv = (process.env.PLATFORM || '').toUpperCase(); // Has user set platform manually?
+
+  const nativeMode = (['VERCEL', 'NODE'].includes(platformEnv) || process.env.VERCEL || process.env.WC_SERVER);
+
+  return nativeMode ? vercelHandler : netlifyHandler;
 };
 
 module.exports = commonMiddleware;

api/archives.js

@@ -80,4 +80,5 @@ const getWaybackData = async (url) => {
   }
 };
 
-exports.handler = middleware(getWaybackData);
+module.exports = middleware(getWaybackData);
+module.exports.handler = middleware(getWaybackData);

api/block-lists.js

@@ -94,12 +94,12 @@ const checkDomainAgainstDnsServers = async (domain) => {
   return results;
 };
 
-exports.handler = middleware(async (url) => {
+const handler = middleware(async (url) => {
   const domain = new URL(url).hostname;
   const results = await checkDomainAgainstDnsServers(domain);
-  return {
+  return { blocklists: results };
-    statusCode: 200,
-    body: JSON.stringify({ blocklists: results })
-  };
 });
 
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);
+

api/carbon.js

@@ -48,4 +48,5 @@ const handler = async (url) => {
   }
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/cookies.js

@@ -54,4 +54,5 @@ const handler = async (url) => {
   return { headerCookies, clientCookies };
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/dns-server.js

@@ -1,7 +1,7 @@
 const dns = require('dns');
 const dnsPromises = dns.promises;
 const axios = require('axios');
-const commonMiddleware = require('./_common/middleware');
+const middleware = require('./_common/middleware');
 
 const handler = async (url) => {
   try {
@@ -41,4 +41,5 @@ const handler = async (url) => {
   }
 };
 
-exports.handler = commonMiddleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/dns.js

@@ -51,4 +51,5 @@ const handler = async (url) => {
   }
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/dnssec.js

@@ -1,7 +1,7 @@
 const https = require('https');
-const commonMiddleware = require('./_common/middleware'); // Make sure this path is correct
+const middleware = require('./_common/middleware'); // Make sure this path is correct
 
-const fetchDNSRecords = async (domain, event, context) => {
+const handler = async (domain) => {
   const dnsTypes = ['DNSKEY', 'DS', 'RRSIG'];
   const records = {};
 
@@ -49,4 +49,6 @@ const fetchDNSRecords = async (domain, event, context) => {
   return records;
 };
 
-exports.handler = commonMiddleware(fetchDNSRecords);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);
+

api/features.js

@@ -1,7 +1,7 @@
 const https = require('https');
 const middleware = require('./_common/middleware');
 
-const builtWithHandler = async (url) => {
+const handler = async (url) => {
   const apiKey = process.env.BUILT_WITH_API_KEY;
 
   if (!url) {
@@ -45,4 +45,5 @@ const builtWithHandler = async (url) => {
   }
 };
 
-exports.handler = middleware(builtWithHandler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/firewall.js

@@ -102,4 +102,5 @@ const handler = async (url) => {
   }
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/get-ip.js

@@ -18,4 +18,6 @@ const handler = async (url) => {
   return await lookupAsync(address);
 };
 
-exports.handler = middleware(handler);
+
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/headers.js

@@ -15,4 +15,5 @@ const handler = async (url, event, context) => {
   }
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/hsts.js

@@ -1,18 +1,15 @@
 const https = require('https');
 const middleware = require('./_common/middleware');
 
-exports.handler = middleware(async (url, event, context) => {
+const handler = async (url, event, context) => {
   const errorResponse = (message, statusCode = 500) => {
     return {
       statusCode: statusCode,
       body: JSON.stringify({ error: message }),
     };
   };
-  const hstsIncompatible = (message, statusCode = 200) => {
+  const hstsIncompatible = (message, compatible = false, hstsHeader = null ) => {
-    return {
+    return { message, compatible, hstsHeader };
-      statusCode: statusCode,
-      body: JSON.stringify({ message, compatible: false }),
-    };
   };
 
 
@@ -35,14 +32,7 @@ exports.handler = middleware(async (url, event, context) => {
         } else if (!preload) {
           resolve(hstsIncompatible(`HSTS header does not contain the preload directive.`));
         } else {
-          resolve({
+          resolve(hstsIncompatible(`Site is compatible with the HSTS preload list!`, true, hstsHeader));
-            statusCode: 200,
-            body: JSON.stringify({
-              message: "Site is compatible with the HSTS preload list!",
-              compatible: true,
-              hstsHeader: hstsHeader,
-            }),
-          });
         }
       }
     });
@@ -53,5 +43,8 @@ exports.handler = middleware(async (url, event, context) => {
 
     req.end();
   });
-});
+};
+
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);
 

api/http-security.js

@@ -22,4 +22,5 @@ const handler = async (url) => {
   }
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/legacy-rank.js

@@ -66,5 +66,6 @@ return new Promise((resolve, reject) => {
 });
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);
 

api/linked-pages.js

@@ -1,7 +1,7 @@
 const axios = require('axios');
 const cheerio = require('cheerio');
 const urlLib = require('url');
-const commonMiddleware = require('./_common/middleware');
+const middleware = require('./_common/middleware');
 
 const handler = async (url) => {
   const response = await axios.get(url);
@@ -45,4 +45,5 @@ const handler = async (url) => {
   return { internal: internalLinks, external: externalLinks };
 };
 
-exports.handler = commonMiddleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/mail-config.js

@@ -1,4 +1,4 @@
-const commonMiddleware = require('./_common/middleware');
+const middleware = require('./_common/middleware');
 
 const dns = require('dns').promises;
 const URL = require('url-parse');
@@ -72,4 +72,5 @@ const handler = async (url, event, context) => {
   }
 };
 
-module.exports.handler = commonMiddleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/ports.js

@@ -73,17 +73,12 @@ const handler = async (url, event, context) => {
     return errorResponse('The function timed out before completing.');
   }
 
-  return {
+  return { openPorts, failedPorts };
-    statusCode: 200,
-    body: JSON.stringify({ openPorts, failedPorts }),
-  };
 };
 
 const errorResponse = (message, statusCode = 444) => {
-  return {
+  return { error: message };
-    statusCode: statusCode,
-    body: JSON.stringify({ error: message }),
-  };
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/quality.js

@@ -4,10 +4,6 @@ const middleware = require('./_common/middleware');
 const handler = async (url, event, context) => {
   const apiKey = process.env.GOOGLE_CLOUD_API_KEY;
 
-  if (!url) {
-    throw new Error('URL param is required');
-  }
-
   if (!apiKey) {
     throw new Error('API key (GOOGLE_CLOUD_API_KEY) not set');
   }
@@ -19,4 +15,5 @@ const handler = async (url, event, context) => {
   return response.data;
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/rank.js

@@ -10,7 +10,7 @@ const handler = async (url) => {
       { auth: { username: process.env.TRANCO_USERNAME, password: process.env.TRANCO_API_KEY } }
       : {};
     const response = await axios.get(
-      `https://tranco-list.eu/api/ranks/domain/${domain}`, { timeout: 2000 }, auth,
+      `https://tranco-list.eu/api/ranks/domain/${domain}`, { timeout: 5000 }, auth,
       );
     if (!response.data || !response.data.ranks || response.data.ranks.length === 0) {
       return { skipped: `Skipping, as ${domain} isn't ranked in the top 100 million sites yet.`};
@@ -21,4 +21,6 @@ const handler = async (url) => {
   }
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);
+

api/redirects.js

@@ -24,4 +24,6 @@ const handler = async (url) => {
 };
 
 const middleware = require('./_common/middleware');
-exports.handler = middleware(handler);
+
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/robots-txt.js

@@ -67,4 +67,5 @@ const handler = async function(url) {
   }
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/screenshot.js

@@ -2,7 +2,7 @@ const puppeteer = require('puppeteer-core');
 const chromium = require('chrome-aws-lambda');
 const middleware = require('./_common/middleware');
 
-const screenshotHandler = async (targetUrl) => {
+const handler = async (targetUrl) => {
 
   if (!targetUrl) {
     throw new Error('URL is missing from queryStringParameters');
@@ -58,4 +58,5 @@ const screenshotHandler = async (targetUrl) => {
   }
 };
 
-exports.handler = middleware(screenshotHandler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/security-txt.js

@@ -38,7 +38,7 @@ const isPgpSigned = (result) => {
   return false;
 };
 
-const securityTxtHandler = async (urlParam) => {
+const handler = async (urlParam) => {
 
   let url;
   try {
@@ -69,8 +69,6 @@ const securityTxtHandler = async (urlParam) => {
   return { isPresent: false };
 };
 
-exports.handler = middleware(securityTxtHandler);
-
 async function fetchSecurityTxt(baseURL, path) {
   return new Promise((resolve, reject) => {
     const url = new URL(path, baseURL);
@@ -91,3 +89,6 @@ async function fetchSecurityTxt(baseURL, path) {
     });
   });
 }
+
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/sitemap.js

@@ -1,4 +1,4 @@
-const commonMiddleware = require('./_common/middleware');
+const middleware = require('./_common/middleware');
 
 const axios = require('axios');
 const xml2js = require('xml2js');
@@ -25,10 +25,7 @@ const handler = async (url) => {
         }
 
         if (!sitemapUrl) {
-          return {
+          return { skipped: 'No sitemap found' };
-            statusCode: 404,
-            body: JSON.stringify({ skipped: 'No sitemap found' }),
-          };
         }
 
         sitemapRes = await axios.get(sitemapUrl, { timeout: 5000 });
@@ -40,25 +37,18 @@ const handler = async (url) => {
     const parser = new xml2js.Parser();
     const sitemap = await parser.parseStringPromise(sitemapRes.data);
 
-    return {
+    return sitemap;
-      statusCode: 200,
-      body: JSON.stringify(sitemap),
-    };
   } catch (error) {
     // If error occurs
     console.log(error.message);
     if (error.code === 'ECONNABORTED') {
-      return {
+      return { error: 'Request timed out' };
-        statusCode: 500,
-        body: JSON.stringify({ error: 'Request timed out' }),
-      };
     } else {
-      return {
+      return { error: error.message };
-        statusCode: 500,
-        body: JSON.stringify({ error: error.message }),
-      };
     }
   }
 };
 
-exports.handler = commonMiddleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);
+

api/social-tags.js

@@ -1,4 +1,4 @@
-const commonMiddleware = require('./_common/middleware');
+const middleware = require('./_common/middleware');
 
 const axios = require('axios');
 const cheerio = require('cheerio');
@@ -50,16 +50,9 @@ const handler = async (url) => {
     };
 
     if (Object.keys(metadata).length === 0) {
-      return {
+      return { skipped: 'No metadata found' };
-        statusCode: 200,
-        body: JSON.stringify({ skipped: 'No metadata found' }),
-      };
     }
-
+    return metadata;
-    return {
-      statusCode: 200,
-      body: JSON.stringify(metadata),
-    };
   } catch (error) {
     return {
       statusCode: 500,
@@ -68,4 +61,5 @@ const handler = async (url) => {
   }
 };
 
-exports.handler = commonMiddleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/ssl.js

@@ -1,7 +1,7 @@
 const tls = require('tls');
 const middleware = require('./_common/middleware');
 
-const fetchSiteCertificateHandler = async (urlString) => {
+const handler = async (urlString) => {
   try {
     const parsedUrl = new URL(urlString);
     const options = {
@@ -40,4 +40,5 @@ const fetchSiteCertificateHandler = async (urlString) => {
   }
 };
 
-exports.handler = middleware(fetchSiteCertificateHandler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/status.js

@@ -2,7 +2,7 @@ const https = require('https');
 const { performance, PerformanceObserver } = require('perf_hooks');
 const middleware = require('./_common/middleware');
 
-const checkURLHandler = async (url) => {
+const handler = async (url) => {
   if (!url) {
     throw new Error('You must provide a URL query parameter!');
   }
@@ -55,4 +55,5 @@ const checkURLHandler = async (url) => {
   }
 };
 
-exports.handler = middleware(checkURLHandler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/tech-stack.js

@@ -1,7 +1,7 @@
 const Wappalyzer = require('wappalyzer');
 const middleware = require('./_common/middleware');
 
-const analyzeSiteTechnologies = async (url) => {
+const handler = async (url) => {
   const options = {};
 
   const wappalyzer = new Wappalyzer(options);
@@ -27,4 +27,5 @@ const analyzeSiteTechnologies = async (url) => {
   }
 };
 
-exports.handler = middleware(analyzeSiteTechnologies);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/threats.js

@@ -5,6 +5,9 @@ const middleware = require('./_common/middleware');
 const getGoogleSafeBrowsingResult = async (url) => {
   try {
     const apiKey = process.env.GOOGLE_CLOUD_API_KEY;
+    if (!apiKey) {
+      return { error: 'GOOGLE_CLOUD_API_KEY is required for the Google Safe Browsing check' };
+    }
     const apiEndpoint = `https://safebrowsing.googleapis.com/v4/threatMatches:find?key=${apiKey}`;
 
     const requestBody = {
@@ -63,11 +66,15 @@ const getPhishTankResult = async (url) => {
 }
 
 const getCloudmersiveResult = async (url) => {
+  const apiKey = process.env.CLOUDMERSIVE_API_KEY;
+  if (!apiKey) {
+    return { error: 'CLOUDMERSIVE_API_KEY is required for the Cloudmersive check' };
+  }
   try {
     const endpoint = 'https://api.cloudmersive.com/virus/scan/website';
     const headers = {
       'Content-Type': 'application/x-www-form-urlencoded',
-      'Apikey': process.env.CLOUDMERSIVE_API_KEY,
+      'Apikey': apiKey,
     };
     const data = `Url=${encodeURIComponent(url)}`;
     const response = await axios.post(endpoint, data, { headers });
@@ -92,4 +99,5 @@ const handler = async (url) => {
   }
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/tls.js

@@ -25,4 +25,5 @@ const handler = async (url) => {
   }
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/trace-route.js

@@ -2,7 +2,7 @@ const traceroute = require('traceroute');
 const url = require('url');
 const middleware = require('./_common/middleware');
 
-const executeTraceroute = async (urlString, context) => {
+const handler = async (urlString, context) => {
   // Parse the URL and get the hostname
   const urlObject = url.parse(urlString);
   const host = urlObject.hostname;
@@ -28,4 +28,5 @@ const executeTraceroute = async (urlString, context) => {
   };
 };
 
-exports.handler = middleware(executeTraceroute);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/txt-records.js

@@ -29,4 +29,5 @@ const handler = async (url, event, context) => {
   }
 };
 
-exports.handler = middleware(handler);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);

api/whois.js

@@ -83,7 +83,7 @@ const fetchFromMyAPI = async (hostname) => {
   }
 };
 
-const fetchWhoisData = async (url) => {
+const handler = async (url) => {
   if (!url.startsWith('http://') && !url.startsWith('https://')) {
     url = 'http://' + url;
   }
@@ -106,4 +106,6 @@ const fetchWhoisData = async (url) => {
   };
 };
 
-exports.handler = middleware(fetchWhoisData);
+module.exports = middleware(handler);
+module.exports.handler = middleware(handler);
+

package.json

@@ -20,7 +20,9 @@
     "start": "react-scripts start",
     "build": "react-scripts build",
     "test": "react-scripts test",
-    "eject": "react-scripts eject"
+    "dev:vercel": "PLATFORM='vercel' npx vercel dev",
+    "dev:netlify": "PLATFORM='netlify' npx netlify dev",
+    "dev:node": "npx concurrently --names \"frontend,backend\" \"REACT_APP_API_ENDPOINT='http://localhost:3001/api' npx react-scripts start\" \"PLATFORM='node' DISABLE_GUI='true' PORT='3001' API_CORS_ORIGIN='*' npx nodemon server\""
   },
   "dependencies": {
     "@netlify/functions": "^1.6.0",
@@ -35,12 +37,12 @@
     "@types/react-router-dom": "^5.3.3",
     "@types/react-simple-maps": "^3.0.0",
     "@types/styled-components": "^5.1.26",
-    "aws-serverless-express": "^3.4.0",
     "axios": "^1.4.0",
     "cheerio": "^1.0.0-rc.12",
     "chrome-aws-lambda": "^10.1.0",
     "chromium": "^3.0.3",
     "connect-history-api-fallback": "^2.0.0",
+    "cors": "^2.8.5",
     "csv-parser": "^3.0.0",
     "dotenv": "^16.3.1",
     "flatted": "^3.2.7",

public/error.html

@@ -0,0 +1,39 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Web-Check</title>
+  </head>
+  <body>
+    <section>
+      <a class="t" href="/"><h1><img src="https://i.ibb.co/q1gZN2p/web-check-logo.png" width="48" />Web-Check</h1></a>
+      <p class="moji">😪</p>
+      <p>There was an error finding this route.</p>
+      <span>Docs and Source: <a href="https://github.com/lissy93/web-check">github.com/lissy93/web-check</a></span>
+    </section>
+    <style>
+      body { background: #141d2b; color: #fff; }
+      h1 {
+        color: #9fef00; text-shadow: #0f1620 2px 2px 0px;
+        font-size: 3rem; margin: 1rem auto; flex-wrap: wrap;
+        display: flex; align-items: flex-start; gap: 1rem;
+      }
+      section {
+        display: flex; flex-direction: column; align-items: center; margin: 1rem; gap: 0.5rem;
+        background: #1a2332; box-shadow: #0f1620 4px 4px 0px; border-radius: 8px; padding: 1rem;
+        max-width: 800px; margin: 1rem auto;
+      }
+      p { font-size: 1.2rem; }
+      a { color: #9fef00; font-family: monospace; }
+      a.t { text-decoration: none; margin: 0;}
+      span { opacity: 0.8; font-size: 0.85rem; }
+      h1 {
+        font-family: PTMono, Ubuntu, Fira Sans, Helvetica, sans-serif;
+      }
+      p, span, a, section, div { font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New', monospace; }
+      code { color: #9fef00cc;}
+      .moji { font-size: 8rem; margin: 0; }
+    </style>
+  </body>
+</html>

public/placeholder.html

@@ -0,0 +1,37 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Web-Check</title>
+  </head>
+  <body>
+    <section>
+      <a class="t" href="/"><h1><img src="https://i.ibb.co/q1gZN2p/web-check-logo.png" width="48" />Web-Check</h1></a>
+      <p><!-- CONTENT --></p>
+      <span>Docs and Source: <a href="https://github.com/lissy93/web-check">github.com/lissy93/web-check</a></span>
+    </section>
+    <style>
+      body { background: #141d2b; color: #fff; }
+      h1 {
+        color: #9fef00; text-shadow: #0f1620 2px 2px 0px;
+        font-size: 3rem; margin: 1rem auto; flex-wrap: wrap;
+        display: flex; align-items: flex-start; gap: 1rem;
+      }
+      section {
+        display: flex; flex-direction: column; align-items: center; margin: 1rem; gap: 0.5rem;
+        background: #1a2332; box-shadow: #0f1620 4px 4px 0px; border-radius: 8px; padding: 1rem;
+        max-width: 800px; margin: 1rem auto;
+      }
+      p { font-size: 1.2rem; }
+      a { color: #9fef00; font-family: monospace; }
+      a.t { text-decoration: none; margin: 0;}
+      span { opacity: 0.8; font-size: 0.85rem; }
+      h1 {
+        font-family: PTMono, Ubuntu, Fira Sans, Helvetica, sans-serif;
+      }
+      p, span, a, section, div { font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New', monospace; }
+      code { color: #9fef00cc;}
+    </style>
+  </body>
+</html>

server.js

@@ -1,75 +1,82 @@
 const express = require('express');
-const awsServerlessExpress = require('aws-serverless-express');
 const fs = require('fs');
 const path = require('path');
+const cors = require('cors');
 const historyApiFallback = require('connect-history-api-fallback');
 require('dotenv').config();
 
 const app = express();
 
+const port = process.env.PORT || 3000; // The port to run the server on
 const API_DIR = '/api'; // Name of the dir containing the lambda functions
 const dirPath = path.join(__dirname, API_DIR); // Path to the lambda functions dir
 const guiPath = path.join(__dirname, 'build');
+const placeholderFilePath = path.join(__dirname, 'public', 'placeholder.html');
+const handlers = {}; // Will store list of API endpoints
+process.env.WC_SERVER = 'true'; // Tells middleware to return in non-lambda mode
 
-// Execute the lambda function
+// Enable CORS
-const executeHandler = async (handler, req) => {
+app.use(cors({
-  return new Promise((resolve, reject) => {
+  origin: process.env.API_CORS_ORIGIN || '*',
-    const callback = (err, response) => err ? reject(err) : resolve(response);
+}));
-    const promise = handler(req, {}, callback);
 
-    if (promise && typeof promise.then === 'function') {
+// Read and register each API function as an Express routes
-      promise.then(resolve).catch(reject);
+fs.readdirSync(dirPath, { withFileTypes: true })
-    }
-  });
-};
-
-// Array of all the lambda function file names
-const fileNames = fs.readdirSync(dirPath, { withFileTypes: true })
   .filter(dirent => dirent.isFile() && dirent.name.endsWith('.js'))
-  .map(dirent => dirent.name);
+  .forEach(dirent => {
-
+    const routeName = dirent.name.split('.')[0];
-const handlers = {};
-
-fileNames.forEach(file => {
-  const routeName = file.split('.')[0];
     const route = `${API_DIR}/${routeName}`;
-  const handler = require(path.join(dirPath, file)).handler;
+    const handler = require(path.join(dirPath, dirent.name));
-
     handlers[route] = handler;
 
     app.get(route, async (req, res) => {
       try {
-      const { statusCode = 200, body = '' } = await executeHandler(handler, req);
+        await handler(req, res);
-      res.status(statusCode).json(JSON.parse(body));
       } catch (err) {
         res.status(500).json({ error: err.message });
       }
     });
   });
 
-const timeout = (ms, jobName = null) => {
+  // Create a single API endpoint to execute all lambda functions
-  return new Promise((_, reject) => {
-    setTimeout(() => {
-      reject(new Error(`Timed out after the ${ms/1000} second limit${jobName ? `, when executing the ${jobName} task` : ''}`));
-    }, ms);
-  });
-}
-
   app.get('/api', async (req, res) => {
     const results = {};
     const { url } = req.query;
-  const maxExecutionTime = process.env.API_TIMEOUT_LIMIT || 15000;
+    const maxExecutionTime = process.env.API_TIMEOUT_LIMIT || 20000;
+  
+    const executeHandler = async (handler, req, res) => {
+      return new Promise(async (resolve, reject) => {
+        try {
+          const mockRes = {
+            status: (statusCode) => mockRes,
+            json: (body) => resolve({ body }),
+          };
+          await handler({ ...req, query: { url } }, mockRes);
+        } catch (err) {
+          reject(err);
+        }
+      });
+    };
+  
+    const timeout = (ms, jobName = null) => {
+      return new Promise((_, reject) => {
+        setTimeout(() => {
+          reject(new Error(
+            `Timed out after ${ms/1000} seconds${jobName ? `, when executing ${jobName}` : ''}`
+          ));
+        }, ms);
+      });
+    };
   
     const handlerPromises = Object.entries(handlers).map(async ([route, handler]) => {
       const routeName = route.replace(`${API_DIR}/`, '');
   
       try {
         const result = await Promise.race([
-        executeHandler(handler, { query: { url } }),
+          executeHandler(handler, req, res),
           timeout(maxExecutionTime, routeName)
         ]);
-      results[routeName] = JSON.parse((result || {}).body);
+        results[routeName] = result.body;
-      
       } catch (err) {
         results[routeName] = { error: err.message };
       }
@@ -79,38 +86,63 @@ app.get('/api', async (req, res) => {
     res.json(results);
   });
   
+
 // Handle SPA routing
 app.use(historyApiFallback({
   rewrites: [
-    { from: /^\/api\/.*$/, to: function(context) { return context.parsedUrl.path; } },
+    { from: /^\/api\/.*$/, to: (context) => context.parsedUrl.path },
   ]
 }));
 
 // Serve up the GUI - if build dir exists, and GUI feature enabled
 if (process.env.DISABLE_GUI && process.env.DISABLE_GUI !== 'false') {
-  app.get('*', (req, res) => {
+  app.get('*', async (req, res) => {
-      res.status(500).send(
+    const placeholderContent = await fs.promises.readFile(placeholderFilePath, 'utf-8');
-        'Welcome to Web-Check!<br />Access the API endpoints at '
+    const htmlContent = placeholderContent.replace(
+      '<!-- CONTENT -->', 
+      'Web-Check API is up and running!<br />Access the endpoints at '
       +'<a href="/api"><code>/api</code></a>'
     );
+
+    res.status(500).send(htmlContent);
   });
 } else if (!fs.existsSync(guiPath)) {
-  app.get('*', (req, res) => {
+  app.get('*', async (req, res) => {
-    res.status(500).send(
+    const placeholderContent = await fs.promises.readFile(placeholderFilePath, 'utf-8');
-      'Welcome to Web-Check!<br />Looks like the GUI app has not yet been compiled, '
+    const htmlContent = placeholderContent.replace(
-      +'run <code>yarn build</code> to continue, then restart the server.'
+      '<!-- CONTENT -->', 
+      'Looks like the GUI app has not yet been compiled.<br /> ' +
+      'Run <code>yarn build</code> to continue, then restart the server.'
     );
+    res.status(500).send(htmlContent);
 });
 } else { // GUI enabled, and build files present, let's go!!
   app.use(express.static(guiPath));
 }
 
-// Create serverless express server
+app.use((req, res, next) => {
-const port = process.env.PORT || 3000;
+  res.status(404).sendFile(path.join(__dirname, 'public', 'error.html'));
-const server = awsServerlessExpress.createServer(app).listen(port, () => {
-  console.log(`Server is running on port ${port}`);
 });
 
-exports.handler = (event, context) => {
+// Print nice welcome message to user
-  awsServerlessExpress.proxy(server, event, context);
+const printMessage = () => {
+  console.log(
+    `\x1b[36m\n` +
+    '    __      __   _         ___ _           _   \n' +
+    '    \\ \\    / /__| |__ ___ / __| |_  ___ __| |__\n' +
+    '     \\ \\/\\/ / -_) \'_ \\___| (__| \' \\/ -_) _| / /\n' +
+    '      \\_/\\_/\\___|_.__/    \\___|_||_\\___\\__|_\\_\\\n' +
+    `\x1b[0m\n`,
+    `\x1b[1m\x1b[32m🚀 Web-Check is up and running at http://localhost:${port} \x1b[0m\n\n`,
+    `\x1b[2m\x1b[36m🛟 For documentation and support, visit the GitHub repo: ` +
+    `https://github.com/lissy93/web-check \n`,
+    `💖 Found Web-Check useful? Consider sponsoring us on GitHub ` +
+    `to help fund maintenance & development.\x1b[0m`
+  );
 };
+
+// Create server
+app.listen(port, () => {
+  printMessage();
+});
+