Compare commits

..

96 Commits

Author SHA1 Message Date
Alicia Sykes
ab66def695 Adds link to terminal trove in homepage 2024-03-12 18:39:29 +00:00
Alicia Sykes
68778d3824 Updates Terminal Trove in about page 2024-03-12 16:23:22 +00:00
Alicia Sykes
43851ae0fb Adds Terminal Trove to about page 2024-03-12 16:17:01 +00:00
Alicia Sykes
91a6e6221c docs: Adds Terminal Trove 2024-03-12 15:53:12 +00:00
liss-bot
7f2da1905c docs: Updates contributors list 2024-03-10 02:00:35 +00:00
Alicia Sykes
7583843e80 Merge pull request #106 from Gertje823/master
Added Mimecast as mail provider
2024-03-09 22:22:24 +00:00
Gertje823
e77075764e Added Mimecast as mail provider 2024-03-09 13:49:38 +01:00
Alicia Sykes
195577fe0c Merge pull request #91 from ChrisCarini/patch-1
Adding linux/arm64 platform to docker build-push-action
2024-03-06 00:11:51 +00:00
Alicia Sykes
29398665b0 Merge pull request #100 from epreston/chore-caniuse-lite
chore(deps): update browserlist db
2024-03-03 16:16:45 +00:00
liss-bot
1780b2323d docs: Updates contributors list 2024-03-03 01:59:09 +00:00
Alicia Sykes
1e6802afbf Merge pull request #97 from 0xflotus/patch-1
chore: fixed small error
2024-03-02 11:27:03 +00:00
Ed Preston
2cd68c5b98 chore(deps): update browserlist db
Remove "Browserslist: caniuse-lite is outdated." warning by updating browserlist db.
2024-03-02 14:26:44 +11:00
0xflotus
ac3a70ae0d chore: fixed small error 2024-03-01 17:08:23 +01:00
ChrisCarini
761f9dab81 Install python + increase yarn install timeout
(cherry picked from commit 4e70e1621e7c44dc5e3c51edd0b5f3577b846eed)
2024-02-29 08:36:11 -08:00
ChrisCarini
64fbcb3f7d Merge remote-tracking branch 'origin/FIX/chromium-docker' into patch-1
# Conflicts:
#	Dockerfile
2024-02-29 08:35:56 -08:00
ChrisCarini
2fb7dc9a2b Add linux/arm64/v8 platform 2024-02-29 08:34:57 -08:00
Alicia Sykes
0a1023ce19 Merge pull request #72 from PhiRequiem/patch-1
Update SslCert.tsx
2024-02-28 20:06:25 +00:00
Alicia Sykes
f4dd5d7a31 Remove commented out duplicate line 2024-02-28 20:06:07 +00:00
Alicia Sykes
aff5ea5f52 Merge pull request #74 from brianteeman/typos
Typos
2024-02-28 16:56:05 +00:00
Alicia Sykes
4c4813620d Merge pull request #85 from eltociear/patch-1
Update README.md
2024-02-28 16:55:53 +00:00
Ikko Eltociear Ashimine
b9bc24156b Update README.md
minor fix
2024-02-29 00:27:53 +09:00
Alicia Sykes
13d0e4ac9f Fix typo in about page (#81) 2024-02-26 17:59:42 +00:00
Alicia Sykes
01fb32e43c Merge pull request #78 from robinson/master
multi stage build docker, to minimize the docker image
2024-02-26 17:46:11 +00:00
liss-bot
ec30ef7b8b docs: Updates contributors list 2024-02-25 01:59:26 +00:00
lth
af70930be2 multi stage build docker, to minimize the docker image file 2024-02-19 17:16:29 +01:00
liss-bot
55299f001f docs: Updates contributors list 2024-02-18 01:59:13 +00:00
liss-bot
7e51239c8d docs: Updates contributors list 2024-02-11 01:59:38 +00:00
liss-bot
cb6a008680 docs: Updates contributors list 2024-02-04 02:00:45 +00:00
Brian Teeman
55f30f5537 Update docs.ts 2024-01-28 10:56:56 +00:00
Brian Teeman
cc3ca64f25 Update netlify.toml 2024-01-28 10:55:18 +00:00
liss-bot
8c0cf5f870 docs: Updates contributors list 2024-01-28 01:59:30 +00:00
liss-bot
2f46de124d docs: Updates contributors list 2024-01-21 02:04:01 +00:00
PhiRequiem
0db0b044b2 Update SslCert.tsx
esta línea está repetida
2024-01-18 17:51:22 -06:00
liss-bot
299925d22e docs: Updates contributors list 2024-01-14 02:03:54 +00:00
liss-bot
db9b69fac3 docs: Updates contributors list 2024-01-07 02:03:38 +00:00
liss-bot
be307e6876 docs: Updates contributors list 2023-12-31 02:02:28 +00:00
liss-bot
e44f8e73aa docs: Updates contributors list 2023-12-24 02:02:14 +00:00
liss-bot
9c4335f2af docs: Updates contributors list 2023-12-17 02:02:55 +00:00
liss-bot
44cbe47983 docs: Updates contributors list 2023-12-10 02:02:58 +00:00
liss-bot
5924d89f54 docs: Updates contributors list 2023-12-03 02:02:19 +00:00
liss-bot
b5ec08da8b docs: Updates contributors list 2023-11-26 02:02:21 +00:00
liss-bot
184b962731 docs: Updates contributors list 2023-11-19 02:03:01 +00:00
liss-bot
d6035b8e9c docs: Updates contributors list 2023-11-12 02:01:46 +00:00
liss-bot
2988486a65 docs: Updates contributors list 2023-11-05 02:01:26 +00:00
Alicia Sykes
5616b71564 Updates the readme 2023-10-27 21:23:23 +01:00
liss-bot
5348175b5e docs: Updates contributors list 2023-10-15 03:01:08 +01:00
liss-bot
f9b4edda01 docs: Updates contributors list 2023-10-08 03:00:49 +01:00
liss-bot
50590334be docs: Updates contributors list 2023-09-24 02:59:42 +01:00
liss-bot
33a35b94f5 docs: Updates contributors list 2023-09-17 02:59:36 +01:00
Alicia Sykes
a6711aeb63 Update Dockerfile (#43)
Updates the Dockerfile with changes suggested by @GWnbsp  in https://github.com/Lissy93/web-check/issues/43#issuecomment-1719212234

### Summary of Changes
1. **ARG Statements:** Introduced `ARG` statements for Node.js and Debian versions, making the Dockerfile more customizable.
2. **SHELL Command:** Changed the default shell to Bash with certain options enabled, improving robustness.
3. **Chromium Installation:** Updated Chromium installation to use Google's signing keys and repositories, aiming for more secure and up-to-date packages.
4. **Chromium Version:** Added a step to save Chromium's version into `/etc/chromium-version` for reference.
5. **Directory Creation:** Added a new directory /app/data in the container's filesystem.
6. **CMD Change:** Changed the CMD to start Node.js server (server.js) instead of using yarn serve.
7. **General Cleanup and Comments:** Code has been refactored for better readability and detailed comments have been added for clarity.
8. **Dependency Installation:** Kept yarn install and the removal of the Yarn cache, but the command is more streamlined.
9. **Other Minor Changes:**
  - Added flags like `-qq` and `--no-install-recommends` for quieter and optimized installation.
  - Enhanced cleanup with `rm -rf /var/lib/apt/lists/*.`
2023-09-14 20:59:09 +01:00
Alicia Sykes
f36ac56370 Merge pull request #53 from murrple-1/murrple-1-patch-1
Check the correct header for the `strictTransportPolicy` property - Fixes #52
2023-09-14 20:34:44 +01:00
Murray Christopherson
09e5b5d888 Check the correct header for the strictTransportPolicy property
See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security
2023-09-13 19:29:58 -07:00
Alicia Sykes
ad57aaa7f8 Merge pull request #51 from t3chn0m4g3/master
Allow for screenshot within Docker
2023-09-12 17:37:12 +01:00
Marco Ochse
2bce29e3cb Allow for Screenshot within Docker
Chromium in Docker needs to be startet without sandbox or screenshot will fail.
2023-09-12 11:01:49 +02:00
Alicia Sykes
6b9aad81fd docs: Simplified the configuring docs 2023-09-11 22:05:03 +01:00
liss-bot
afc5b54207 docs: Updates contributors list 2023-09-10 02:58:58 +01:00
Alicia Sykes
71ce9a6623 Merge branch 'master' of github.com:lissy93/web-check 2023-09-09 20:30:48 +01:00
Alicia Sykes
d8bb822a4e Updated TLS components, to refresh when new data comes in 2023-09-09 20:30:09 +01:00
Alicia Sykes
5297b2ffe7 Fixes block-lists job (#50) 2023-09-09 20:29:19 +01:00
Alicia Sykes
79c88a5d9a Updates docs 2023-09-08 20:28:51 +01:00
Alicia Sykes
0e022f97a2 Adds status badges into readme 2023-09-03 22:51:36 +01:00
Alicia Sykes
0d4942738d Update public URL, from web-check.as93.net to web-check.xyz 2023-09-03 20:35:27 +01:00
Alicia Sykes
32d5962dc3 Merge pull request #48 from Lissy93/FEAT/vercel-support
[FEAT] Vercel support
2023-09-03 17:01:06 +01:00
Alicia Sykes
8a7b024e99 Return plain JSON, to be handled by middleware instead 2023-09-03 16:58:46 +01:00
Alicia Sykes
c169a3762d Updates .gitignore, to ignore build assets 2023-09-03 16:28:43 +01:00
Alicia Sykes
d26f5b26a7 Renders error page, if there was an error 2023-09-03 16:23:07 +01:00
Alicia Sykes
c2a937ac8e Adds placeholder and error pages 2023-09-03 16:22:33 +01:00
Alicia Sykes
59203acdfa Update the env check, to determine what handler format to use 2023-09-03 15:32:58 +01:00
Alicia Sykes
63db1dbd85 Removed the need for lambda handler, added status message to server, refactored 2023-09-03 15:30:11 +01:00
Alicia Sykes
e23347a936 Adds cors support to server 2023-09-03 14:11:15 +01:00
Alicia Sykes
d41af54513 Updates dev and deployment docs 2023-09-03 14:06:58 +01:00
Alicia Sykes
73c44e39de Adds Deploy to Vercel section 2023-09-03 12:56:45 +01:00
Alicia Sykes
7dd398a9c3 Merge branch 'master' of github.com:lissy93/web-check 2023-09-03 12:27:49 +01:00
Alicia Sykes
1a95a42853 Updates all API functions, to work on both Vercel & Netlify 2023-09-03 12:27:04 +01:00
Alicia Sykes
3695c82472 Updates middleware, with both AWS and Express support 2023-09-03 12:26:41 +01:00
liss-bot
6a4eb5aa8e docs: Updates contributors list 2023-09-03 02:58:42 +01:00
Alicia Sykes
1630f2a050 Temp remove env vars from param store 2023-09-02 16:51:24 +01:00
Alicia Sykes
b1f8c0144b Use env var for AWS_ACCOUNT_ID 2023-09-02 15:39:26 +01:00
Alicia Sykes
76cce7ef9a Read env vars from AWS secret manager 2023-09-02 15:21:36 +01:00
Alicia Sykes
393dafbf84 Updates .env template 2023-09-02 15:13:20 +01:00
Alicia Sykes
6bd353273a Merge pull request #47 from JinnaBalu/master
Updated Dockerfile for layer cache before COPY
2023-09-01 20:31:30 +01:00
JinnaBalu
8a60b77135 Getting latest before merge Merge branch 'master' of https://github.com/JinnaBalu/web-check 2023-09-01 16:13:20 +05:30
JinnaBalu
ab59afc150 Updated dockerfile for optimization 2023-09-01 16:12:53 +05:30
Alicia Sykes
b314168da1 Add caching for node_modules and rename environment deploy status 2023-08-31 20:46:01 +01:00
Alicia Sykes
33e1adb974 Update AWS workflow to update the deployment state for environments 2023-08-31 20:33:31 +01:00
Alicia Sykes
8688fd23f5 Use native AWS CLI to upload to S3 2023-08-30 20:07:28 +01:00
Alicia Sykes
cd2681fd84 Set src_dir in upload to S3 deploy action 2023-08-30 19:52:09 +01:00
Alicia Sykes
0cb9cedd8c Set dependency version 2023-08-30 19:48:59 +01:00
Alicia Sykes
20762dc3ad Tries switching S3 upload action provider 2023-08-30 19:42:21 +01:00
Alicia Sykes
359c6ca476 Ignore node_modules when uploading to S3 2023-08-30 18:58:02 +01:00
Alicia Sykes
f7573572e5 Remove ACL flag from S3 upload 2023-08-30 18:22:38 +01:00
Alicia Sykes
975c73fd2b Fix tag for invalidating cloudfront cache 2023-08-30 18:11:53 +01:00
Alicia Sykes
acf4f90aee Bump version used for invalidating cloudfront cache 2023-08-30 18:07:40 +01:00
Alicia Sykes
daf6850052 Updates AWS action, to also build + deploy frontend, to S3 + Cloudfront 2023-08-30 18:01:33 +01:00
Alicia Sykes
3aa385cf41 Use NPM for gloabl dep 2023-08-30 15:46:34 +01:00
Alicia Sykes
394b68fa29 Automate deployment to AWS lambda (cheaper than Netlify) 2023-08-30 15:40:48 +01:00
57 changed files with 3755 additions and 2939 deletions

29
.env
View File

@@ -2,11 +2,24 @@
# Be sure to uncomment any line you populate # Be sure to uncomment any line you populate
# Everything is optional, but some features won't work without external API access # Everything is optional, but some features won't work without external API access
# GOOGLE_CLOUD_API_KEY='' # API Keys for external services (backend)
# SHODAN_API_KEY='' GOOGLE_CLOUD_API_KEY=''
# REACT_APP_SHODAN_API_KEY='' TORRENT_IP_API_KEY=''
# WHO_API_KEY='' SECURITY_TRAILS_API_KEY=''
# REACT_APP_WHO_API_KEY='' BUILT_WITH_API_KEY=''
# SECURITY_TRAILS_API_KEY='' URL_SCAN_API_KEY=''
# BUILT_WITH_API_KEY='' TRANCO_USERNAME=''
# CI=false TRANCO_API_KEY=''
CLOUDMERSIVE_API_KEY=''
# API Keys for external services (frontend)
REACT_APP_SHODAN_API_KEY=''
REACT_APP_WHO_API_KEY=''
# Configuration settings
# CHROME_PATH='/usr/bin/chromium' # The path the the Chromium executable
# PORT='3000' # Port to serve the API, when running server.js
# DISABLE_GUI='false' # Disable the GUI, and only serve the API
# API_TIMEOUT_LIMIT='10000' # The timeout limit for API requests, in milliseconds
# API_CORS_ORIGIN='*' # Enable CORS, by setting your allowed hostname(s) here
# REACT_APP_API_ENDPOINT='/api' # The endpoint for the API (can be local or remote)

388
.github/README.md vendored
View File

@@ -9,6 +9,22 @@
</p> </p>
---
<p align="center">
<sup>Kindly supported by:</sup><br>
<a href="https://terminaltrove.com/?utm_campaign=github&utm_medium=referral&utm_content=web-check&utm_source=wcgh">
<img src="https://i.ibb.co/fqL7Y6N/terminal-trove-normal.png" width="400" alt="Terminal Trove">
<br>
<strong>The $HOME of all things in the terminal.</strong>
</a>
<br>
<a href="https://terminaltrove.com/newsletter?utm_campaign=github&utm_medium=referral&utm_content=web-check&utm_source=wcgh">
<sub>Find your next CLI / TUI tool and more at Terminal Trove,</sub>
<br>
<sup>Get updates on new tools on our newsletter.</sup>
</a>
</p>
--- ---
#### Contents #### Contents
@@ -19,11 +35,13 @@
- [Mirror](#mirror) - [Mirror](#mirror)
- [Features](#features) - [Features](#features)
- **[Usage](#usage)** - **[Usage](#usage)**
- [Developer Setup](#developing) - [Deployment](#deployment)
- [Deploying, Option#1: Netlify](#deploying---option-1-netlify) - [Option#1: Netlify](#deploying---option-1-netlify)
- [Deploying, Option#2: Docker](#deploying---option-2-docker) - [Option#2: Vercel](#deploying---option-2-vercel)
- [Deploying, Option#3: Source](#deploying---option-3-from-source) - [Option#3: Docker](#deploying---option-3-docker)
- [Option#4: Source](#deploying---option-4-from-source)
- [Configuration Options](#configuring) - [Configuration Options](#configuring)
- [Developer Setup](#developing)
- **[Community](#community)** - **[Community](#community)**
- [Contributing](#contributing) - [Contributing](#contributing)
- [Bugs](#reporting-bugs) - [Bugs](#reporting-bugs)
@@ -56,18 +74,27 @@ A hosted version can be accessed at: **[web-check.as93.net](https://web-check.as
### Mirror ### Mirror
The source for this repo is mirrored to CodeBerg, available at: **[codeberg.org/alicia/web-check](https://codeberg.org/alicia/web-check)** The source for this repo is mirrored to CodeBerg, available at: **[codeberg.org/alicia/web-check](https://codeberg.org/alicia/web-check)**
### Motivation ### Status
Often when you're looking into a website, there's several things you always initially check.
Build & Deploys: [![Netlify Status](https://api.netlify.com/api/v1/badges/c43453c1-5333-4df7-889b-c1d2b52183c0/deploy-status)](https://app.netlify.com/sites/web-check/deploys)
[![Vercel Status](https://therealsujitk-vercel-badge.vercel.app/?app=web-check-ten)](https://vercel.com/as93/web-check/)
[![🐳 Build + Publish Docker Image](https://github.com/Lissy93/web-check/actions/workflows/docker.yml/badge.svg)](https://github.com/Lissy93/web-check/actions/workflows/docker.yml)
[![🚀 Deploy to AWS](https://github.com/Lissy93/web-check/actions/workflows/deploy-aws.yml/badge.svg)](https://github.com/Lissy93/web-check/actions/workflows/deploy-aws.yml)
<br />
Repo Management & Miscellaneous: [![🪞 Mirror to Codeberg](https://github.com/Lissy93/web-check/actions/workflows/mirror.yml/badge.svg)](https://github.com/Lissy93/web-check/actions/workflows/mirror.yml)
[![💓 Inserts Contributors & Sponsors](https://github.com/Lissy93/web-check/actions/workflows/credits.yml/badge.svg)](https://github.com/Lissy93/web-check/actions/workflows/credits.yml)
None of this is hard to find with a series of basic curl commands and NPMAP plus a combination of online tools. But it's just so much easier to have everything done all at once, presented clearly and visible in one place :)
### Features ### Features
<details> <details open>
<summary><h4>Expand to see all features</h4></summary> <summary><b>Click to expand / collapse section</b></summary>
<sup>**Note** _this list needs updating, many more jobs have been added since..._</sup> <sup>**Note** _this list needs updating, many more jobs have been added since..._</sup>
The following section outlines the core features, and briefly explains why this data might be useful for you to know, as well as linking to further resources for learning more.
<details> <details>
<summary><b>IP Info</b></summary> <summary><b>IP Info</b></summary>
@@ -730,23 +757,13 @@ This may be useful to see what a given website looks like, free of the constrain
</details> </details>
Read more here: **[web-check.as93.net/about](https://web-check.as93.net/about)** Read more here: **[web-check.xyz/about](https://web-check.xyz/about)**
_Note that not all checks will work for all sites. Sometimes it's not possible to determine some information, and the demo instance has some limitations imposed by Netlify for the lambda functions._
--- ---
## Usage ## Usage
### Developing ### Deployment
1. Clone the repo, `git clone git@github.com:Lissy93/web-check.git`
2. Cd into it, `cd web-check`
3. Install dependencies: `yarn`
4. Start the dev server, with `yarn dev`
You'll need [Node.js](https://nodejs.org/en) (V 18.16.1 or later) installed, plus [yarn](https://yarnpkg.com/getting-started/install) as well as [git](https://git-scm.com/).
Some checks also require `chromium`, `traceroute` and `dns` to be installed within your environment. These jobs will just be skipped if those packages aren't present.
### Deploying - Option #1: Netlify ### Deploying - Option #1: Netlify
@@ -754,28 +771,39 @@ Click the button below, to deploy to Netlify 👇
[![Deploy to Netlify](https://img.shields.io/badge/Deploy-Netlify-%2330c8c9?style=for-the-badge&logo=netlify&labelColor=1e0e41 'Deploy Web-Check to Netlify, via 1-Click Script')](https://app.netlify.com/start/deploy?repository=https://github.com/lissy93/web-check) [![Deploy to Netlify](https://img.shields.io/badge/Deploy-Netlify-%2330c8c9?style=for-the-badge&logo=netlify&labelColor=1e0e41 'Deploy Web-Check to Netlify, via 1-Click Script')](https://app.netlify.com/start/deploy?repository=https://github.com/lissy93/web-check)
### Deploying - Option #2: Docker ### Deploying - Option #2: Vercel
Run `docker run -p 8888:3000 lissy93/web-check`, then open `http://localhost:3000` Click the button below, to deploy to Vercel 👇
[![Deploy with Vercel](https://img.shields.io/badge/Deploy-Vercel-%23ffffff?style=for-the-badge&logo=vercel&labelColor=1e0e41)](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Flissy93%2Fweb-check&project-name=web-check&repository-name=web-check-fork&demo-title=Web-Check%20Demo&demo-description=Check%20out%20web-check.xyz%20to%20see%20a%20live%20demo%20of%20this%20application%20running.&demo-url=https%3A%2F%2Fweb-check.xyz&demo-image=https%3A%2F%2Fraw.githubusercontent.com%2FLissy93%2Fweb-check%2Fmaster%2F.github%2Fscreenshots%2Fweb-check-screenshot10.png)
### Deploying - Option #3: Docker
Run `docker run -p 3000:3000 lissy93/web-check`, then open [`localhost:3000`](http://localhost:3000)
<details>
<summary>Docker Options</summary>
You can get the Docker image from: You can get the Docker image from:
- DockerHub: [`lissy93/web-check`](https://hub.docker.com/r/lissy93/web-check) - DockerHub: [`lissy93/web-check`](https://hub.docker.com/r/lissy93/web-check)
- GHCR: [`ghcr.io/lissy93/web-check`](https://github.com/Lissy93/web-check/pkgs/container/web-check) - GHCR: [`ghcr.io/lissy93/web-check`](https://github.com/Lissy93/web-check/pkgs/container/web-check)
- Or build the image yourself by cloning the repo and running `docker build -t web-check .` - Or build the image yourself by cloning the repo and running `docker build -t web-check .`
### Deploying - Option #3: From Source </details>
Follow the instructions in the [Developing](#developing) section above, then run `yarn build` && `yarn serve` to build and serve the application. ### Deploying - Option #4: From Source
The following commands will work:
Install the prerequisites listed in the [Developing](#developing) section, then run:
```bash ```bash
git clone https://github.com/Lissy93/web-check.git # Grab the code git clone https://github.com/Lissy93/web-check.git # Download the code from GitHub
cd web-check # Move into the project directory cd web-check # Navigate into the project dir
yarn install # Install dependencies yarn install # Install the NPM dependencies
yarn build # Build the app for production yarn build # Build the app for production
yarn serve # Start the app (API and GUI) yarn serve # Start the app (API and GUI)
``` ```
---
### Configuring ### Configuring
@@ -783,9 +811,17 @@ By default, no configuration is needed.
But there are some optional environmental variables that you can set to give you access to some additional checks, or to increase rate-limits for some checks that use external APIs. But there are some optional environmental variables that you can set to give you access to some additional checks, or to increase rate-limits for some checks that use external APIs.
Note that keys that are prefixed with `REACT_APP_` are used client-side, and as such they must be scoped correctly with minimum privileges.
**API Keys & Credentials**: **API Keys & Credentials**:
Key | Value
---|---
`GOOGLE_CLOUD_API_KEY` | A Google API key ([get here](https://cloud.google.com/api-gateway/docs/authenticate-api-keys)). This can be used to return quality metrics for a site
`REACT_APP_SHODAN_API_KEY` | A Shodan API key ([get here](https://account.shodan.io/)). This will show associated host names for a given domain
`REACT_APP_WHO_API_KEY` | A WhoAPI key ([get here](https://whoapi.com/)). This will show more comprehensive WhoIs records than the default job
<details>
<summary><small>Full / Upcoming Vals</small></summary>
- `GOOGLE_CLOUD_API_KEY` - A Google API key ([get here](https://cloud.google.com/api-gateway/docs/authenticate-api-keys)). This can be used to return quality metrics for a site - `GOOGLE_CLOUD_API_KEY` - A Google API key ([get here](https://cloud.google.com/api-gateway/docs/authenticate-api-keys)). This can be used to return quality metrics for a site
- `REACT_APP_SHODAN_API_KEY` - A Shodan API key ([get here](https://account.shodan.io/)). This will show associated host names for a given domain - `REACT_APP_SHODAN_API_KEY` - A Shodan API key ([get here](https://account.shodan.io/)). This will show associated host names for a given domain
- `REACT_APP_WHO_API_KEY` - A WhoAPI key ([get here](https://whoapi.com/)). This will show more comprehensive WhoIs records than the default job - `REACT_APP_WHO_API_KEY` - A WhoAPI key ([get here](https://whoapi.com/)). This will show more comprehensive WhoIs records than the default job
@@ -797,15 +833,36 @@ Note that keys that are prefixed with `REACT_APP_` are used client-side, and as
- `BUILT_WITH_API_KEY` - A BuiltWith API key ([get here](https://api.builtwith.com/)). This will show the main features of a site - `BUILT_WITH_API_KEY` - A BuiltWith API key ([get here](https://api.builtwith.com/)). This will show the main features of a site
- `TORRENT_IP_API_KEY` - A torrent API key ([get here](https://iknowwhatyoudownload.com/en/api/)). This will show torrents downloaded by an IP - `TORRENT_IP_API_KEY` - A torrent API key ([get here](https://iknowwhatyoudownload.com/en/api/)). This will show torrents downloaded by an IP
**Configuration Settings**: </details>
- `CHROME_PATH` (e.g. `/usr/bin/chromium`) - The path the the Chromium executable
- `PORT` (e.g. `3000`) - Port to serve the API, when running server.js **Configuration Settings**:
- `DISABLE_GUI` (e.g. `false`) - Disable the GUI, and only serve the API
- `API_TIMEOUT_LIMIT` (e.g. `10000`) - The timeout limit for API requests, in milliseconds Key | Value
- `REACT_APP_API_ENDPOINT` (e.g. `/api`) - The endpoint for the API (can be local or remote) ---|---
`CHROME_PATH` | The path the Chromium executable (e.g. `/usr/bin/chromium`)
`PORT` | Port to serve the API, when running server.js (e.g. `3000`)
`DISABLE_GUI` | Disable the GUI, and only serve the API (e.g. `false`)
`API_TIMEOUT_LIMIT` | The timeout limit for API requests, in milliseconds (e.g. `10000`)
`REACT_APP_API_ENDPOINT` | The endpoint for the API, either local or remote (e.g. `/api`)
All values are optional.
You can add these as environmental variables. Either put them directly into an `.env` file in the projects root, or via the Netlify / Vercel UI, or by passing to the Docker container with the --env flag, or using your own environmental variable management system
Note that keys that are prefixed with `REACT_APP_` are used client-side, and as such they must be scoped correctly with minimum privileges, since may be made visible when intercepting browser <-> server network requests
---
### Developing
1. Clone the repo, `git clone git@github.com:Lissy93/web-check.git`
2. Cd into it, `cd web-check`
3. Install dependencies: `yarn`
4. Start the dev server, with `yarn dev`
You'll need [Node.js](https://nodejs.org/en) (V 18.16.1 or later) installed, plus [yarn](https://yarnpkg.com/getting-started/install) as well as [git](https://git-scm.com/).
Some checks also require `chromium`, `traceroute` and `dns` to be installed within your environment. These jobs will just be skipped if those packages aren't present.
The above can be added into an `.env` file in the projects root, or via the Netlify UI, or by passing directly to the Docker container with the --env flag.
All variables are optional.
--- ---
@@ -831,8 +888,9 @@ For bugs, please outline the steps needed to reproduce, and include relevant inf
### Supporting ### Supporting
The app will remain 100% free and open source. The app will remain 100% free and open source.
But due to the amount of traffic that the hosted instance gets, the lambda function usage is costing about $25/month. Any help with covering the costs via GitHub Sponsorship would be much appreciated. It's thanks to the support of the community that this project is able to be freely available for everyone :) But due to the amount of traffic that the hosted instance gets, the lambda function usage is costing about $25/month.
Any help with covering the costs via GitHub Sponsorship would be much appreciated.
It's thanks to the support of the community that this project is able to be freely available for everyone :)
[![Sponsor Lissy93 on GitHub](https://img.shields.io/badge/Sponsor_on_GitHub-Lissy93-%23ff4dda?style=for-the-badge&logo=githubsponsors&logoColor=ff4dda)](https://github.com/sponsors/Lissy93) [![Sponsor Lissy93 on GitHub](https://img.shields.io/badge/Sponsor_on_GitHub-Lissy93-%23ff4dda?style=for-the-badge&logo=githubsponsors&logoColor=ff4dda)](https://github.com/sponsors/Lissy93)
@@ -851,6 +909,13 @@ Credit to the following users for contributing to Web-Check
<sub><b>Alicia Sykes</b></sub> <sub><b>Alicia Sykes</b></sub>
</a> </a>
</td> </td>
<td align="center">
<a href="https://github.com/liss-bot">
<img src="https://avatars.githubusercontent.com/u/87835202?v=4" width="80;" alt="liss-bot"/>
<br />
<sub><b>Alicia Bot</b></sub>
</a>
</td>
<td align="center"> <td align="center">
<a href="https://github.com/muni106"> <a href="https://github.com/muni106">
<img src="https://avatars.githubusercontent.com/u/65845442?v=4" width="80;" alt="muni106"/> <img src="https://avatars.githubusercontent.com/u/65845442?v=4" width="80;" alt="muni106"/>
@@ -859,17 +924,39 @@ Credit to the following users for contributing to Web-Check
</a> </a>
</td> </td>
<td align="center"> <td align="center">
<a href="https://github.com/gso-trifork-security"> <a href="https://github.com/ChrisCarini">
<img src="https://avatars.githubusercontent.com/u/69247026?v=4" width="80;" alt="gso-trifork-security"/> <img src="https://avatars.githubusercontent.com/u/6374067?v=4" width="80;" alt="ChrisCarini"/>
<br /> <br />
<sub><b>Gustav Soelberg</b></sub> <sub><b>Chris Carini</b></sub>
</a> </a>
</td> </td>
<td align="center"> <td align="center">
<a href="https://github.com/treatmesubj"> <a href="https://github.com/brianteeman">
<img src="https://avatars.githubusercontent.com/u/39680353?v=4" width="80;" alt="treatmesubj"/> <img src="https://avatars.githubusercontent.com/u/1296369?v=4" width="80;" alt="brianteeman"/>
<br /> <br />
<sub><b>John Hupperts</b></sub> <sub><b>Brian Teeman</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/jinnabaalu">
<img src="https://avatars.githubusercontent.com/u/11784253?v=4" width="80;" alt="jinnabaalu"/>
<br />
<sub><b>Jinna Baalu</b></sub>
</a>
</td></tr>
<tr>
<td align="center">
<a href="https://github.com/robinson">
<img src="https://avatars.githubusercontent.com/u/237874?v=4" width="80;" alt="robinson"/>
<br />
<sub><b>Lth</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/abhishekMuge">
<img src="https://avatars.githubusercontent.com/u/49590582?v=4" width="80;" alt="abhishekMuge"/>
<br />
<sub><b>Abhishek Muge</b></sub>
</a> </a>
</td> </td>
<td align="center"> <td align="center">
@@ -880,10 +967,60 @@ Credit to the following users for contributing to Web-Check
</a> </a>
</td> </td>
<td align="center"> <td align="center">
<a href="https://github.com/abhishekMuge"> <a href="https://github.com/PhiRequiem">
<img src="https://avatars.githubusercontent.com/u/49590582?v=4" width="80;" alt="abhishekMuge"/> <img src="https://avatars.githubusercontent.com/u/1323576?v=4" width="80;" alt="PhiRequiem"/>
<br /> <br />
<sub><b>Abhishek Muge</b></sub> <sub><b>PhiRequiem</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/murrple-1">
<img src="https://avatars.githubusercontent.com/u/5559656?v=4" width="80;" alt="murrple-1"/>
<br />
<sub><b>Murray Christopherson</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/t3chn0m4g3">
<img src="https://avatars.githubusercontent.com/u/4318452?v=4" width="80;" alt="t3chn0m4g3"/>
<br />
<sub><b>Marco Ochse</b></sub>
</a>
</td></tr>
<tr>
<td align="center">
<a href="https://github.com/treatmesubj">
<img src="https://avatars.githubusercontent.com/u/39680353?v=4" width="80;" alt="treatmesubj"/>
<br />
<sub><b>John Hupperts</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/eltociear">
<img src="https://avatars.githubusercontent.com/u/22633385?v=4" width="80;" alt="eltociear"/>
<br />
<sub><b>Ikko Eltociear Ashimine</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/Gertje823">
<img src="https://avatars.githubusercontent.com/u/36937387?v=4" width="80;" alt="Gertje823"/>
<br />
<sub><b>Gertje823</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/epreston">
<img src="https://avatars.githubusercontent.com/u/347224?v=4" width="80;" alt="epreston"/>
<br />
<sub><b>Ed Preston</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/0xflotus">
<img src="https://avatars.githubusercontent.com/u/26602940?v=4" width="80;" alt="0xflotus"/>
<br />
<sub><b>0xflotus</b></sub>
</a> </a>
</td></tr> </td></tr>
</table> </table>
@@ -897,15 +1034,51 @@ Huge thanks to these wonderful people, who sponsor me on GitHub, their support h
<table> <table>
<tr> <tr>
<td align="center"> <td align="center">
<a href="https://github.com/emlazzarin"> <a href="https://github.com/koconder">
<img src="https://avatars.githubusercontent.com/u/1141361?v=4" width="80;" alt="emlazzarin"/> <img src="https://avatars.githubusercontent.com/u/25068?u=582657b23622aaa3dfe68bd028a780f272f456fa&v=4" width="80;" alt="koconder"/>
<br /> <br />
<sub><b>Eddy Lazzarin</b></sub> <sub><b>Vincent Koc</b></sub>
</a> </a>
</td> </td>
<td align="center">
<a href="https://github.com/peng1can">
<img src="https://avatars.githubusercontent.com/u/225854?v=4" width="80;" alt="peng1can"/>
<br />
<sub><b>Peng1can</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/alydemah">
<img src="https://avatars.githubusercontent.com/u/652035?u=ac2c04e474da37bfeafcfa25076cc1800997aedb&v=4" width="80;" alt="alydemah"/>
<br />
<sub><b>Aly Mohamed</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/bgadrian">
<img src="https://avatars.githubusercontent.com/u/830001?u=69f115baad2fcd8c14eb05bdbf5cd80f4649a95a&v=4" width="80;" alt="bgadrian"/>
<br />
<sub><b>B.G.Adrian</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/tbjers">
<img src="https://avatars.githubusercontent.com/u/1117052?u=539d96d5e581b3139c75713ce35b89a36626404c&v=4" width="80;" alt="tbjers"/>
<br />
<sub><b>Torgny Bjers</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/emlazzarin">
<img src="https://avatars.githubusercontent.com/u/1141361?u=714e3487a3f2e0df721b01a0133945f075d3ff68&v=4" width="80;" alt="emlazzarin"/>
<br />
<sub><b>Eddy Lazzarin</b></sub>
</a>
</td></tr>
<tr>
<td align="center"> <td align="center">
<a href="https://github.com/AnandChowdhary"> <a href="https://github.com/AnandChowdhary">
<img src="https://avatars.githubusercontent.com/u/2841780?v=4" width="80;" alt="AnandChowdhary"/> <img src="https://avatars.githubusercontent.com/u/2841780?u=747e554b3a7f12eb20b7910e1c87d817844f714f&v=4" width="80;" alt="AnandChowdhary"/>
<br /> <br />
<sub><b>Anand Chowdhary</b></sub> <sub><b>Anand Chowdhary</b></sub>
</a> </a>
@@ -914,93 +1087,121 @@ Huge thanks to these wonderful people, who sponsor me on GitHub, their support h
<a href="https://github.com/shrippen"> <a href="https://github.com/shrippen">
<img src="https://avatars.githubusercontent.com/u/2873570?v=4" width="80;" alt="shrippen"/> <img src="https://avatars.githubusercontent.com/u/2873570?v=4" width="80;" alt="shrippen"/>
<br /> <br />
<sub><b>shrippen</b></sub> <sub><b>Shrippen</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/davidpaulyoung">
<img src="https://avatars.githubusercontent.com/u/3418369?v=4" width="80;" alt="davidpaulyoung"/>
<br />
<sub><b>David Young</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/k-rol">
<img src="https://avatars.githubusercontent.com/u/4050412?v=4" width="80;" alt="k-rol"/>
<br />
<sub><b>Carol Ouellet</b></sub>
</a> </a>
</td> </td>
<td align="center"> <td align="center">
<a href="https://github.com/bile0026"> <a href="https://github.com/bile0026">
<img src="https://avatars.githubusercontent.com/u/5022496?v=4" width="80;" alt="bile0026"/> <img src="https://avatars.githubusercontent.com/u/5022496?u=aec96ad173c0ea9baaba93807efa8a848af6595c&v=4" width="80;" alt="bile0026"/>
<br /> <br />
<sub><b>Zach Biles</b></sub> <sub><b>Zach Biles</b></sub>
</a> </a>
</td></tr> </td>
<tr>
<td align="center"> <td align="center">
<a href="https://github.com/UlisesGascon"> <a href="https://github.com/UlisesGascon">
<img src="https://avatars.githubusercontent.com/u/5110813?v=4" width="80;" alt="UlisesGascon"/> <img src="https://avatars.githubusercontent.com/u/5110813?u=3c41facd8aa26154b9451de237c34b0f78d672a5&v=4" width="80;" alt="UlisesGascon"/>
<br /> <br />
<sub><b>Ulises Gascón</b></sub> <sub><b>Ulises Gascón</b></sub>
</a> </a>
</td> </td>
<td align="center"> <td align="center">
<a href="https://github.com/digitalarche"> <a href="https://github.com/digitalarche">
<img src="https://avatars.githubusercontent.com/u/6546135?v=4" width="80;" alt="digitalarche"/> <img src="https://avatars.githubusercontent.com/u/6546135?u=d033c9c16e8367987aec3f9ff5922bc67dd1eedf&v=4" width="80;" alt="digitalarche"/>
<br /> <br />
<sub><b>Digital Archeology</b></sub> <sub><b>Digital Archeology</b></sub>
</a> </a>
</td> </td>
<td align="center">
<a href="https://github.com/InDieTasten">
<img src="https://avatars.githubusercontent.com/u/7047377?u=8d8f8017628b38bc46dcbf3620e194b01d3fb2d1&v=4" width="80;" alt="InDieTasten"/>
<br />
<sub><b>InDieTasten</b></sub>
</a>
</td></tr>
<tr>
<td align="center"> <td align="center">
<a href="https://github.com/bmcgonag"> <a href="https://github.com/bmcgonag">
<img src="https://avatars.githubusercontent.com/u/7346620?v=4" width="80;" alt="bmcgonag"/> <img src="https://avatars.githubusercontent.com/u/7346620?u=2a0f9284f3e12ac1cc15288c254d1ec68a5081e8&v=4" width="80;" alt="bmcgonag"/>
<br /> <br />
<sub><b>Brian McGonagill</b></sub> <sub><b>Brian McGonagill</b></sub>
</a> </a>
</td> </td>
<td align="center"> <td align="center">
<a href="https://github.com/vlad-timofeev"> <a href="https://github.com/vlad-timofeev">
<img src="https://avatars.githubusercontent.com/u/11474041?v=4" width="80;" alt="vlad-timofeev"/> <img src="https://avatars.githubusercontent.com/u/11474041?u=eee43705b54d2ec9f51fc4fcce5ad18dd17c87e4&v=4" width="80;" alt="vlad-timofeev"/>
<br /> <br />
<sub><b>Vlad Timofeev</b></sub> <sub><b>Vlad Timofeev</b></sub>
</a> </a>
</td> </td>
<td align="center"> <td align="center">
<a href="https://github.com/iJasonWade"> <a href="https://github.com/helixzz">
<img src="https://avatars.githubusercontent.com/u/12824479?v=4" width="80;" alt="iJasonWade"/> <img src="https://avatars.githubusercontent.com/u/12218889?u=d06d0c103dfbdb99450623064f7da3c5a3675fb6&v=4" width="80;" alt="helixzz"/>
<br /> <br />
<sub><b>Jason Ash</b></sub> <sub><b>HeliXZz</b></sub>
</a> </a>
</td> </td>
<td align="center"> <td align="center">
<a href="https://github.com/DRXAquosus"> <a href="https://github.com/mryesiller">
<img src="https://avatars.githubusercontent.com/u/45409262?v=4" width="80;" alt="DRXAquosus"/> <img src="https://avatars.githubusercontent.com/u/24632172?v=4" width="80;" alt="mryesiller"/>
<br /> <br />
<sub><b>DRXAquosus</b></sub> <sub><b>Göksel Yeşiller</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/forwardemail">
<img src="https://avatars.githubusercontent.com/u/32481436?v=4" width="80;" alt="forwardemail"/>
<br />
<sub><b>Forward Email - Open-source & Privacy-focused Email Service (2023)</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/Bastii717">
<img src="https://avatars.githubusercontent.com/u/53431819?u=604977bed6ad6875ada890d0d3765a4cacc2fa14&v=4" width="80;" alt="Bastii717"/>
<br />
<sub><b>Bastii717</b></sub>
</a> </a>
</td></tr> </td></tr>
<tr> <tr>
<td align="center"> <td align="center">
<a href="https://github.com/Bastii717"> <a href="https://github.com/frankdez93">
<img src="https://avatars.githubusercontent.com/u/53431819?v=4" width="80;" alt="Bastii717"/> <img src="https://avatars.githubusercontent.com/u/87549420?v=4" width="80;" alt="frankdez93"/>
<br /> <br />
<sub><b>Bastii717</b></sub> <sub><b>Frankdez93</b></sub>
</a> </a>
</td> </td>
<td align="center"> <td align="center">
<a href="https://github.com/ratty222"> <a href="https://github.com/ratty222">
<img src="https://avatars.githubusercontent.com/u/92832598?v=4" width="80;" alt="ratty222"/> <img src="https://avatars.githubusercontent.com/u/92832598?u=137b65530cbd5f5af9c24cde51baa6cc77cc934b&v=4" width="80;" alt="ratty222"/>
<br /> <br />
<sub><b>Brent</b></sub> <sub><b>Brent</b></sub>
</a> </a>
</td> </td>
<td align="center"> <td align="center">
<a href="https://github.com/jtfinley72"> <a href="https://github.com/hernanpopper">
<img src="https://avatars.githubusercontent.com/u/96497997?v=4" width="80;" alt="jtfinley72"/> <img src="https://avatars.githubusercontent.com/u/104868017?v=4" width="80;" alt="hernanpopper"/>
<br /> <br />
<sub><b>jtfinley72</b></sub> <sub><b>Hernanpopper</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/NixyJuppie">
<img src="https://avatars.githubusercontent.com/u/138570196?u=b102c222487905728b858704962d32759df29ebe&v=4" width="80;" alt="NixyJuppie"/>
<br />
<sub><b>Nixy</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/nrvo">
<img src="https://avatars.githubusercontent.com/u/151435968?u=e1dcb307fd0efdc45cddbe9490a7b956e4da6835&v=4" width="80;" alt="nrvo"/>
<br />
<sub><b>Nrvo</b></sub>
</a>
</td>
<td align="center">
<a href="https://github.com/mezza93">
<img src="https://avatars.githubusercontent.com/u/153599966?v=4" width="80;" alt="mezza93"/>
<br />
<sub><b>Mezza93</b></sub>
</a> </a>
</td></tr> </td></tr>
</table> </table>
@@ -1038,8 +1239,11 @@ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
``` ```
[![View Dependency Licenses & SBOM on FOSSA](https://app.fossa.com/api/projects/git%2Bgithub.com%2FLissy93%2Fweb-check.svg?type=large&issueType=license)](https://app.fossa.com/projects/git%2Bgithub.com%2FLissy93%2Fweb-check?ref=badge_large&issueType=license)
</details> </details>
<!-- License + Copyright --> <!-- License + Copyright -->
<p align="center"> <p align="center">
<i>© <a href="https://aliciasykes.com">Alicia Sykes</a> 2023</i><br> <i>© <a href="https://aliciasykes.com">Alicia Sykes</a> 2023</i><br>
@@ -1048,7 +1252,7 @@ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
<sup>Thanks for visiting :)</sup> <sup>Thanks for visiting :)</sup>
</p> </p>
<!-- Dinosaur --> <!-- Dinosaurs are Awesome -->
<!-- <!--
. - ~ ~ ~ - . . - ~ ~ ~ - .
.. _ .-~ ~-. .. _ .-~ ~-.

128
.github/workflows/deploy-aws.yml vendored Normal file
View File

@@ -0,0 +1,128 @@
name: 🚀 Deploy to AWS
on:
workflow_dispatch:
push:
branches:
- master
tags:
- '*'
paths:
- api/**
- serverless.yml
- package.json
- .github/workflows/deploy-aws.yml
jobs:
deploy-api:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Setup Node.js
uses: actions/setup-node@v1
with:
node-version: 16
- name: Cache node_modules
uses: actions/cache@v2
with:
path: node_modules
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Create GitHub deployment for API
uses: chrnorm/deployment-action@releases/v1
id: deployment_api
with:
token: ${{ secrets.BOT_TOKEN || secrets.GITHUB_TOKEN }}
environment: AWS (Backend API)
ref: ${{ github.ref }}
- name: Install Serverless CLI and dependencies
run: |
npm i -g serverless
yarn
- name: Deploy to AWS
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID }}
run: serverless deploy
- name: Update GitHub deployment status (API)
if: always()
uses: chrnorm/deployment-status@releases/v1
with:
token: ${{ secrets.BOT_TOKEN || secrets.GITHUB_TOKEN }}
state: "${{ job.status }}"
deployment_id: ${{ steps.deployment_api.outputs.deployment_id }}
ref: ${{ github.ref }}
deploy-frontend:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Setup Node.js
uses: actions/setup-node@v1
with:
node-version: 16
- name: Cache node_modules
uses: actions/cache@v2
with:
path: node_modules
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Create GitHub deployment for Frontend
uses: chrnorm/deployment-action@releases/v1
id: deployment_frontend
with:
token: ${{ secrets.BOT_TOKEN || secrets.GITHUB_TOKEN }}
environment: AWS (Frontend Web UI)
ref: ${{ github.ref }}
- name: Install dependencies and build
run: |
yarn install
yarn build
- name: Setup AWS
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: Upload to S3
env:
AWS_S3_BUCKET: 'web-check-frontend'
run: aws s3 sync ./build/ s3://$AWS_S3_BUCKET/ --delete
- name: Invalidate CloudFront cache
uses: chetan/invalidate-cloudfront-action@v2.4
env:
DISTRIBUTION: E30XKAM2TG9FD8
PATHS: '/*'
AWS_REGION: 'us-east-1'
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- name: Update GitHub deployment status (Frontend)
if: always()
uses: chrnorm/deployment-status@releases/v1
with:
token: ${{ secrets.BOT_TOKEN || secrets.GITHUB_TOKEN }}
state: "${{ job.status }}"
deployment_id: ${{ steps.deployment_frontend.outputs.deployment_id }}
ref: ${{ github.ref }}

View File

@@ -67,7 +67,7 @@ jobs:
context: . context: .
file: ./Dockerfile file: ./Dockerfile
push: true push: true
platforms: linux/amd64 platforms: linux/amd64,linux/arm64/v8
tags: | tags: |
${{ env.GHCR_TAG }} ${{ env.GHCR_TAG }}
${{ env.DOCKERHUB_TAG }} ${{ env.DOCKERHUB_TAG }}

69
.gitignore vendored
View File

@@ -1,27 +1,60 @@
# ------------------------
# keys # ENVIRONMENT SETTINGS
# ------------------------
.env .env
# dependencies # ------------------------
/node_modules # PRODUCTION
/.pnp # ------------------------
.pnp.js /build/
# logs # ------------------------
# DEPLOYMENT
# ------------------------
.vercel/
.netlify/
.webpack/
.serverless/
# ------------------------
# DEPENDENCIES
# ------------------------
node_modules/
.yarn/cache/
.yarn/unplugged/
.yarn/build-state.yml
.yarn/install-state.gz
.pnpm/
.pnp.*
# ------------------------
# LOGS
# ------------------------
logs/
*.log
npm-debug.log* npm-debug.log*
yarn-debug.log* yarn-debug.log*
yarn-error.log* yarn-error.log*
lerna-debug.log*
.pnpm-debug.log*
# testing # ------------------------
/coverage # TESTING
# ------------------------
coverage/
.nyc_output/
# production # ------------------------
/build # OS SPECIFIC
# ------------------------
# Random AWS and Netlify crap
.netlify
.serverless
.webpack
# OS generated files
.DS_Store .DS_Store
Thumbs.db
# ------------------------
# EDITORS
# ------------------------
.idea/
.vscode/
*.swp
*.swo

View File

@@ -1,22 +1,62 @@
FROM node:16-buster-slim # Specify the Node.js version to use
ARG NODE_VERSION=16
RUN apt-get update && \ # Specify the Debian version to use, the default is "bullseye"
apt-get install -y chromium traceroute && \ ARG DEBIAN_VERSION=bullseye
chmod 755 /usr/bin/chromium && \
# Use Node.js Docker image as the base image, with specific Node and Debian versions
FROM node:${NODE_VERSION}-${DEBIAN_VERSION} AS build
# Set the container's default shell to Bash and enable some options
SHELL ["/bin/bash", "-euo", "pipefail", "-c"]
# Install Chromium browser and Download and verify Google Chromes signing key
RUN apt-get update -qq --fix-missing && \
apt-get -qqy install --allow-unauthenticated gnupg wget && \
wget --quiet --output-document=- https://dl-ssl.google.com/linux/linux_signing_key.pub | gpg --dearmor > /etc/apt/trusted.gpg.d/google-archive.gpg && \
echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google.list && \
apt-get update -qq && \
apt-get -qqy --no-install-recommends install chromium traceroute python make g++ && \
rm -rf /var/lib/apt/lists/* rm -rf /var/lib/apt/lists/*
# Run the Chromium browser's version command and redirect its output to the /etc/chromium-version file
RUN /usr/bin/chromium --no-sandbox --version > /etc/chromium-version
# Set the working directory to /app
WORKDIR /app
# Copy package.json and yarn.lock to the working directory
COPY package.json yarn.lock ./
# Run yarn install to install dependencies and clear yarn cache
RUN apt-get update && \
yarn install --production --frozen-lockfile --network-timeout 100000 && \
rm -rf /app/node_modules/.cache
# Copy all files to working directory
COPY . .
# Run yarn build to build the application
RUN yarn build --production
# Final stage
FROM node:${NODE_VERSION}-${DEBIAN_VERSION} AS final
WORKDIR /app WORKDIR /app
COPY package.json yarn.lock ./ COPY package.json yarn.lock ./
COPY --from=build /app .
RUN yarn install RUN apt-get update && \
apt-get install -y --no-install-recommends chromium traceroute && \
COPY . . chmod 755 /usr/bin/chromium && \
rm -rf /var/lib/apt/lists/* /app/node_modules/.cache
RUN yarn build
# Exposed container port, the default is 3000, which can be modified through the environment variable PORT
EXPOSE ${PORT:-3000} EXPOSE ${PORT:-3000}
# Set the environment variable CHROME_PATH to specify the path to the Chromium binaries
ENV CHROME_PATH='/usr/bin/chromium' ENV CHROME_PATH='/usr/bin/chromium'
# Define the command executed when the container starts and start the server.js of the Node.js application
CMD ["yarn", "serve"] CMD ["yarn", "serve"]

View File

@@ -2,8 +2,40 @@ const normalizeUrl = (url) => {
return url.startsWith('http') ? url : `https://${url}`; return url.startsWith('http') ? url : `https://${url}`;
}; };
const headers = {
'Access-Control-Allow-Origin': process.env.API_CORS_ORIGIN || '*',
'Access-Control-Allow-Credentials': true,
'Content-Type': 'application/json;charset=UTF-8',
};
const commonMiddleware = (handler) => { const commonMiddleware = (handler) => {
return async (event, context, callback) => { // Vercel
const vercelHandler = async (request, response) => {
const queryParams = request.query || {};
const rawUrl = queryParams.url;
if (!rawUrl) {
return response.status(500).json({ error: 'No URL specified' });
}
const url = normalizeUrl(rawUrl);
try {
const handlerResponse = await handler(url, request);
if (handlerResponse.body && handlerResponse.statusCode) {
response.status(handlerResponse.statusCode).json(handlerResponse.body);
} else {
response.status(200).json(
typeof handlerResponse === 'object' ? handlerResponse : JSON.parse(handlerResponse)
);
}
} catch (error) {
response.status(500).json({ error: error.message });
}
};
// Netlify
const netlifyHandler = async (event, context, callback) => {
const queryParams = event.queryStringParameters || event.query || {}; const queryParams = event.queryStringParameters || event.query || {};
const rawUrl = queryParams.url; const rawUrl = queryParams.url;
@@ -11,28 +43,40 @@ const commonMiddleware = (handler) => {
callback(null, { callback(null, {
statusCode: 500, statusCode: 500,
body: JSON.stringify({ error: 'No URL specified' }), body: JSON.stringify({ error: 'No URL specified' }),
headers,
}); });
return;
} }
const url = normalizeUrl(rawUrl); const url = normalizeUrl(rawUrl);
try { try {
const response = await handler(url, event, context); const handlerResponse = await handler(url, event, context);
if (response.body && response.statusCode) { if (handlerResponse.body && handlerResponse.statusCode) {
callback(null, response); callback(null, handlerResponse);
} else { } else {
callback(null, { callback(null, {
statusCode: 200, statusCode: 200,
body: typeof response === 'object' ? JSON.stringify(response) : response, body: typeof handlerResponse === 'object' ? JSON.stringify(handlerResponse) : handlerResponse,
headers,
}); });
} }
} catch (error) { } catch (error) {
callback(null, { callback(null, {
statusCode: 500, statusCode: 500,
body: JSON.stringify({ error: error.message }), body: JSON.stringify({ error: error.message }),
headers,
}); });
} }
}; };
// The format of the handlers varies between platforms
// E.g. Netlify + AWS expect Lambda functions, but Vercel or Node needs standard handler
const platformEnv = (process.env.PLATFORM || '').toUpperCase(); // Has user set platform manually?
const nativeMode = (['VERCEL', 'NODE'].includes(platformEnv) || process.env.VERCEL || process.env.WC_SERVER);
return nativeMode ? vercelHandler : netlifyHandler;
}; };
module.exports = commonMiddleware; module.exports = commonMiddleware;

View File

@@ -80,4 +80,5 @@ const getWaybackData = async (url) => {
} }
}; };
exports.handler = middleware(getWaybackData); module.exports = middleware(getWaybackData);
module.exports.handler = middleware(getWaybackData);

View File

@@ -94,12 +94,12 @@ const checkDomainAgainstDnsServers = async (domain) => {
return results; return results;
}; };
exports.handler = middleware(async (url) => { const handler = async (url) => {
const domain = new URL(url).hostname; const domain = new URL(url).hostname;
const results = await checkDomainAgainstDnsServers(domain); const results = await checkDomainAgainstDnsServers(domain);
return { return { blocklists: results };
statusCode: 200, };
body: JSON.stringify({ blocklists: results })
}; module.exports = middleware(handler);
}); module.exports.handler = middleware(handler);

View File

@@ -48,4 +48,5 @@ const handler = async (url) => {
} }
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -54,4 +54,5 @@ const handler = async (url) => {
return { headerCookies, clientCookies }; return { headerCookies, clientCookies };
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -1,7 +1,7 @@
const dns = require('dns'); const dns = require('dns');
const dnsPromises = dns.promises; const dnsPromises = dns.promises;
const axios = require('axios'); const axios = require('axios');
const commonMiddleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
const handler = async (url) => { const handler = async (url) => {
try { try {
@@ -41,4 +41,5 @@ const handler = async (url) => {
} }
}; };
exports.handler = commonMiddleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -51,4 +51,5 @@ const handler = async (url) => {
} }
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -1,7 +1,7 @@
const https = require('https'); const https = require('https');
const commonMiddleware = require('./_common/middleware'); // Make sure this path is correct const middleware = require('./_common/middleware'); // Make sure this path is correct
const fetchDNSRecords = async (domain, event, context) => { const handler = async (domain) => {
const dnsTypes = ['DNSKEY', 'DS', 'RRSIG']; const dnsTypes = ['DNSKEY', 'DS', 'RRSIG'];
const records = {}; const records = {};
@@ -49,4 +49,6 @@ const fetchDNSRecords = async (domain, event, context) => {
return records; return records;
}; };
exports.handler = commonMiddleware(fetchDNSRecords); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -1,7 +1,7 @@
const https = require('https'); const https = require('https');
const middleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
const builtWithHandler = async (url) => { const handler = async (url) => {
const apiKey = process.env.BUILT_WITH_API_KEY; const apiKey = process.env.BUILT_WITH_API_KEY;
if (!url) { if (!url) {
@@ -45,4 +45,5 @@ const builtWithHandler = async (url) => {
} }
}; };
exports.handler = middleware(builtWithHandler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -102,4 +102,5 @@ const handler = async (url) => {
} }
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -18,4 +18,6 @@ const handler = async (url) => {
return await lookupAsync(address); return await lookupAsync(address);
}; };
exports.handler = middleware(handler);
module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -15,4 +15,5 @@ const handler = async (url, event, context) => {
} }
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -1,18 +1,15 @@
const https = require('https'); const https = require('https');
const middleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
exports.handler = middleware(async (url, event, context) => { const handler = async (url, event, context) => {
const errorResponse = (message, statusCode = 500) => { const errorResponse = (message, statusCode = 500) => {
return { return {
statusCode: statusCode, statusCode: statusCode,
body: JSON.stringify({ error: message }), body: JSON.stringify({ error: message }),
}; };
}; };
const hstsIncompatible = (message, statusCode = 200) => { const hstsIncompatible = (message, compatible = false, hstsHeader = null ) => {
return { return { message, compatible, hstsHeader };
statusCode: statusCode,
body: JSON.stringify({ message, compatible: false }),
};
}; };
@@ -35,14 +32,7 @@ exports.handler = middleware(async (url, event, context) => {
} else if (!preload) { } else if (!preload) {
resolve(hstsIncompatible(`HSTS header does not contain the preload directive.`)); resolve(hstsIncompatible(`HSTS header does not contain the preload directive.`));
} else { } else {
resolve({ resolve(hstsIncompatible(`Site is compatible with the HSTS preload list!`, true, hstsHeader));
statusCode: 200,
body: JSON.stringify({
message: "Site is compatible with the HSTS preload list!",
compatible: true,
hstsHeader: hstsHeader,
}),
});
} }
} }
}); });
@@ -53,5 +43,8 @@ exports.handler = middleware(async (url, event, context) => {
req.end(); req.end();
}); });
}); };
module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -8,7 +8,7 @@ const handler = async (url) => {
const response = await axios.get(fullUrl); const response = await axios.get(fullUrl);
const headers = response.headers; const headers = response.headers;
return { return {
strictTransportPolicy: headers['strict-transport-policy'] ? true : false, strictTransportPolicy: headers['strict-transport-security'] ? true : false,
xFrameOptions: headers['x-frame-options'] ? true : false, xFrameOptions: headers['x-frame-options'] ? true : false,
xContentTypeOptions: headers['x-content-type-options'] ? true : false, xContentTypeOptions: headers['x-content-type-options'] ? true : false,
xXSSProtection: headers['x-xss-protection'] ? true : false, xXSSProtection: headers['x-xss-protection'] ? true : false,
@@ -22,4 +22,5 @@ const handler = async (url) => {
} }
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -66,5 +66,6 @@ return new Promise((resolve, reject) => {
}); });
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -1,7 +1,7 @@
const axios = require('axios'); const axios = require('axios');
const cheerio = require('cheerio'); const cheerio = require('cheerio');
const urlLib = require('url'); const urlLib = require('url');
const commonMiddleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
const handler = async (url) => { const handler = async (url) => {
const response = await axios.get(url); const response = await axios.get(url);
@@ -45,4 +45,5 @@ const handler = async (url) => {
return { internal: internalLinks, external: externalLinks }; return { internal: internalLinks, external: externalLinks };
}; };
exports.handler = commonMiddleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -1,4 +1,4 @@
const commonMiddleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
const dns = require('dns').promises; const dns = require('dns').promises;
const URL = require('url-parse'); const URL = require('url-parse');
@@ -54,6 +54,11 @@ const handler = async (url, event, context) => {
if (yahooMx.length > 0) { if (yahooMx.length > 0) {
mailServices.push({ provider: 'Yahoo', value: yahooMx[0].exchange }); mailServices.push({ provider: 'Yahoo', value: yahooMx[0].exchange });
} }
// Check MX records for Mimecast
const mimecastMx = mxRecords.filter(record => record.exchange.includes('mimecast.com'));
if (mimecastMx.length > 0) {
mailServices.push({ provider: 'Mimecast', value: mimecastMx[0].exchange });
}
return { return {
mxRecords, mxRecords,
@@ -72,4 +77,5 @@ const handler = async (url, event, context) => {
} }
}; };
module.exports.handler = commonMiddleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -73,17 +73,12 @@ const handler = async (url, event, context) => {
return errorResponse('The function timed out before completing.'); return errorResponse('The function timed out before completing.');
} }
return { return { openPorts, failedPorts };
statusCode: 200,
body: JSON.stringify({ openPorts, failedPorts }),
};
}; };
const errorResponse = (message, statusCode = 444) => { const errorResponse = (message, statusCode = 444) => {
return { return { error: message };
statusCode: statusCode,
body: JSON.stringify({ error: message }),
};
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -4,10 +4,6 @@ const middleware = require('./_common/middleware');
const handler = async (url, event, context) => { const handler = async (url, event, context) => {
const apiKey = process.env.GOOGLE_CLOUD_API_KEY; const apiKey = process.env.GOOGLE_CLOUD_API_KEY;
if (!url) {
throw new Error('URL param is required');
}
if (!apiKey) { if (!apiKey) {
throw new Error('API key (GOOGLE_CLOUD_API_KEY) not set'); throw new Error('API key (GOOGLE_CLOUD_API_KEY) not set');
} }
@@ -19,4 +15,5 @@ const handler = async (url, event, context) => {
return response.data; return response.data;
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -10,7 +10,7 @@ const handler = async (url) => {
{ auth: { username: process.env.TRANCO_USERNAME, password: process.env.TRANCO_API_KEY } } { auth: { username: process.env.TRANCO_USERNAME, password: process.env.TRANCO_API_KEY } }
: {}; : {};
const response = await axios.get( const response = await axios.get(
`https://tranco-list.eu/api/ranks/domain/${domain}`, { timeout: 2000 }, auth, `https://tranco-list.eu/api/ranks/domain/${domain}`, { timeout: 5000 }, auth,
); );
if (!response.data || !response.data.ranks || response.data.ranks.length === 0) { if (!response.data || !response.data.ranks || response.data.ranks.length === 0) {
return { skipped: `Skipping, as ${domain} isn't ranked in the top 100 million sites yet.`}; return { skipped: `Skipping, as ${domain} isn't ranked in the top 100 million sites yet.`};
@@ -21,4 +21,6 @@ const handler = async (url) => {
} }
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -24,4 +24,6 @@ const handler = async (url) => {
}; };
const middleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
exports.handler = middleware(handler);
module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -67,4 +67,5 @@ const handler = async function(url) {
} }
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -2,7 +2,7 @@ const puppeteer = require('puppeteer-core');
const chromium = require('chrome-aws-lambda'); const chromium = require('chrome-aws-lambda');
const middleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
const screenshotHandler = async (targetUrl) => { const handler = async (targetUrl) => {
if (!targetUrl) { if (!targetUrl) {
throw new Error('URL is missing from queryStringParameters'); throw new Error('URL is missing from queryStringParameters');
@@ -21,7 +21,7 @@ const screenshotHandler = async (targetUrl) => {
let browser = null; let browser = null;
try { try {
browser = await puppeteer.launch({ browser = await puppeteer.launch({
args: chromium.args, args: [...chromium.args, '--no-sandbox'], // Add --no-sandbox flag
defaultViewport: { width: 800, height: 600 }, defaultViewport: { width: 800, height: 600 },
executablePath: process.env.CHROME_PATH || await chromium.executablePath, executablePath: process.env.CHROME_PATH || await chromium.executablePath,
headless: chromium.headless, headless: chromium.headless,
@@ -58,4 +58,5 @@ const screenshotHandler = async (targetUrl) => {
} }
}; };
exports.handler = middleware(screenshotHandler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -38,7 +38,7 @@ const isPgpSigned = (result) => {
return false; return false;
}; };
const securityTxtHandler = async (urlParam) => { const handler = async (urlParam) => {
let url; let url;
try { try {
@@ -69,8 +69,6 @@ const securityTxtHandler = async (urlParam) => {
return { isPresent: false }; return { isPresent: false };
}; };
exports.handler = middleware(securityTxtHandler);
async function fetchSecurityTxt(baseURL, path) { async function fetchSecurityTxt(baseURL, path) {
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
const url = new URL(path, baseURL); const url = new URL(path, baseURL);
@@ -91,3 +89,6 @@ async function fetchSecurityTxt(baseURL, path) {
}); });
}); });
} }
module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -1,4 +1,4 @@
const commonMiddleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
const axios = require('axios'); const axios = require('axios');
const xml2js = require('xml2js'); const xml2js = require('xml2js');
@@ -25,10 +25,7 @@ const handler = async (url) => {
} }
if (!sitemapUrl) { if (!sitemapUrl) {
return { return { skipped: 'No sitemap found' };
statusCode: 404,
body: JSON.stringify({ skipped: 'No sitemap found' }),
};
} }
sitemapRes = await axios.get(sitemapUrl, { timeout: 5000 }); sitemapRes = await axios.get(sitemapUrl, { timeout: 5000 });
@@ -40,25 +37,18 @@ const handler = async (url) => {
const parser = new xml2js.Parser(); const parser = new xml2js.Parser();
const sitemap = await parser.parseStringPromise(sitemapRes.data); const sitemap = await parser.parseStringPromise(sitemapRes.data);
return { return sitemap;
statusCode: 200,
body: JSON.stringify(sitemap),
};
} catch (error) { } catch (error) {
// If error occurs // If error occurs
console.log(error.message); console.log(error.message);
if (error.code === 'ECONNABORTED') { if (error.code === 'ECONNABORTED') {
return { return { error: 'Request timed out' };
statusCode: 500,
body: JSON.stringify({ error: 'Request timed out' }),
};
} else { } else {
return { return { error: error.message };
statusCode: 500,
body: JSON.stringify({ error: error.message }),
};
} }
} }
}; };
exports.handler = commonMiddleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -1,4 +1,4 @@
const commonMiddleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
const axios = require('axios'); const axios = require('axios');
const cheerio = require('cheerio'); const cheerio = require('cheerio');
@@ -50,16 +50,9 @@ const handler = async (url) => {
}; };
if (Object.keys(metadata).length === 0) { if (Object.keys(metadata).length === 0) {
return { return { skipped: 'No metadata found' };
statusCode: 200,
body: JSON.stringify({ skipped: 'No metadata found' }),
};
} }
return metadata;
return {
statusCode: 200,
body: JSON.stringify(metadata),
};
} catch (error) { } catch (error) {
return { return {
statusCode: 500, statusCode: 500,
@@ -68,4 +61,5 @@ const handler = async (url) => {
} }
}; };
exports.handler = commonMiddleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -1,7 +1,7 @@
const tls = require('tls'); const tls = require('tls');
const middleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
const fetchSiteCertificateHandler = async (urlString) => { const handler = async (urlString) => {
try { try {
const parsedUrl = new URL(urlString); const parsedUrl = new URL(urlString);
const options = { const options = {
@@ -40,4 +40,5 @@ const fetchSiteCertificateHandler = async (urlString) => {
} }
}; };
exports.handler = middleware(fetchSiteCertificateHandler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -2,7 +2,7 @@ const https = require('https');
const { performance, PerformanceObserver } = require('perf_hooks'); const { performance, PerformanceObserver } = require('perf_hooks');
const middleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
const checkURLHandler = async (url) => { const handler = async (url) => {
if (!url) { if (!url) {
throw new Error('You must provide a URL query parameter!'); throw new Error('You must provide a URL query parameter!');
} }
@@ -55,4 +55,5 @@ const checkURLHandler = async (url) => {
} }
}; };
exports.handler = middleware(checkURLHandler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -1,7 +1,7 @@
const Wappalyzer = require('wappalyzer'); const Wappalyzer = require('wappalyzer');
const middleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
const analyzeSiteTechnologies = async (url) => { const handler = async (url) => {
const options = {}; const options = {};
const wappalyzer = new Wappalyzer(options); const wappalyzer = new Wappalyzer(options);
@@ -27,4 +27,5 @@ const analyzeSiteTechnologies = async (url) => {
} }
}; };
exports.handler = middleware(analyzeSiteTechnologies); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -5,6 +5,9 @@ const middleware = require('./_common/middleware');
const getGoogleSafeBrowsingResult = async (url) => { const getGoogleSafeBrowsingResult = async (url) => {
try { try {
const apiKey = process.env.GOOGLE_CLOUD_API_KEY; const apiKey = process.env.GOOGLE_CLOUD_API_KEY;
if (!apiKey) {
return { error: 'GOOGLE_CLOUD_API_KEY is required for the Google Safe Browsing check' };
}
const apiEndpoint = `https://safebrowsing.googleapis.com/v4/threatMatches:find?key=${apiKey}`; const apiEndpoint = `https://safebrowsing.googleapis.com/v4/threatMatches:find?key=${apiKey}`;
const requestBody = { const requestBody = {
@@ -63,11 +66,15 @@ const getPhishTankResult = async (url) => {
} }
const getCloudmersiveResult = async (url) => { const getCloudmersiveResult = async (url) => {
const apiKey = process.env.CLOUDMERSIVE_API_KEY;
if (!apiKey) {
return { error: 'CLOUDMERSIVE_API_KEY is required for the Cloudmersive check' };
}
try { try {
const endpoint = 'https://api.cloudmersive.com/virus/scan/website'; const endpoint = 'https://api.cloudmersive.com/virus/scan/website';
const headers = { const headers = {
'Content-Type': 'application/x-www-form-urlencoded', 'Content-Type': 'application/x-www-form-urlencoded',
'Apikey': process.env.CLOUDMERSIVE_API_KEY, 'Apikey': apiKey,
}; };
const data = `Url=${encodeURIComponent(url)}`; const data = `Url=${encodeURIComponent(url)}`;
const response = await axios.post(endpoint, data, { headers }); const response = await axios.post(endpoint, data, { headers });
@@ -92,4 +99,5 @@ const handler = async (url) => {
} }
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -25,4 +25,5 @@ const handler = async (url) => {
} }
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -2,7 +2,7 @@ const traceroute = require('traceroute');
const url = require('url'); const url = require('url');
const middleware = require('./_common/middleware'); const middleware = require('./_common/middleware');
const executeTraceroute = async (urlString, context) => { const handler = async (urlString, context) => {
// Parse the URL and get the hostname // Parse the URL and get the hostname
const urlObject = url.parse(urlString); const urlObject = url.parse(urlString);
const host = urlObject.hostname; const host = urlObject.hostname;
@@ -28,4 +28,5 @@ const executeTraceroute = async (urlString, context) => {
}; };
}; };
exports.handler = middleware(executeTraceroute); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -29,4 +29,5 @@ const handler = async (url, event, context) => {
} }
}; };
exports.handler = middleware(handler); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -83,7 +83,7 @@ const fetchFromMyAPI = async (hostname) => {
} }
}; };
const fetchWhoisData = async (url) => { const handler = async (url) => {
if (!url.startsWith('http://') && !url.startsWith('https://')) { if (!url.startsWith('http://') && !url.startsWith('https://')) {
url = 'http://' + url; url = 'http://' + url;
} }
@@ -106,4 +106,6 @@ const fetchWhoisData = async (url) => {
}; };
}; };
exports.handler = middleware(fetchWhoisData); module.exports = middleware(handler);
module.exports.handler = middleware(handler);

View File

@@ -5,7 +5,7 @@
publish = "build" publish = "build"
functions = "api" functions = "api"
# Environmental variables and optioanl secrets # Environmental variables and optional secrets
[build.environment] [build.environment]
# Build configuration env vars (uncomment if you want to conigure these) # Build configuration env vars (uncomment if you want to conigure these)
# CI="false" # Set CI to false, to prevent warnings from exiting the build # CI="false" # Set CI to false, to prevent warnings from exiting the build

View File

@@ -4,7 +4,7 @@
"private": false, "private": false,
"description": "All-in-one OSINT tool for analyzing any website", "description": "All-in-one OSINT tool for analyzing any website",
"repository": "github:lissy93/web-check", "repository": "github:lissy93/web-check",
"homepage": "https://web-check.as93.net", "homepage": "https://web-check.xyz",
"license": "MIT", "license": "MIT",
"author": { "author": {
"name": "Alicia Sykes", "name": "Alicia Sykes",
@@ -20,7 +20,9 @@
"start": "react-scripts start", "start": "react-scripts start",
"build": "react-scripts build", "build": "react-scripts build",
"test": "react-scripts test", "test": "react-scripts test",
"eject": "react-scripts eject" "dev:vercel": "PLATFORM='vercel' npx vercel dev",
"dev:netlify": "PLATFORM='netlify' npx netlify dev",
"dev:node": "npx concurrently --names \"frontend,backend\" \"REACT_APP_API_ENDPOINT='http://localhost:3001/api' npx react-scripts start\" \"PLATFORM='node' DISABLE_GUI='true' PORT='3001' API_CORS_ORIGIN='*' npx nodemon server\""
}, },
"dependencies": { "dependencies": {
"@netlify/functions": "^1.6.0", "@netlify/functions": "^1.6.0",
@@ -35,12 +37,12 @@
"@types/react-router-dom": "^5.3.3", "@types/react-router-dom": "^5.3.3",
"@types/react-simple-maps": "^3.0.0", "@types/react-simple-maps": "^3.0.0",
"@types/styled-components": "^5.1.26", "@types/styled-components": "^5.1.26",
"aws-serverless-express": "^3.4.0",
"axios": "^1.4.0", "axios": "^1.4.0",
"cheerio": "^1.0.0-rc.12", "cheerio": "^1.0.0-rc.12",
"chrome-aws-lambda": "^10.1.0", "chrome-aws-lambda": "^10.1.0",
"chromium": "^3.0.3", "chromium": "^3.0.3",
"connect-history-api-fallback": "^2.0.0", "connect-history-api-fallback": "^2.0.0",
"cors": "^2.8.5",
"csv-parser": "^3.0.0", "csv-parser": "^3.0.0",
"dotenv": "^16.3.1", "dotenv": "^16.3.1",
"flatted": "^3.2.7", "flatted": "^3.2.7",

39
public/error.html Normal file
View File

@@ -0,0 +1,39 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Web-Check</title>
</head>
<body>
<section>
<a class="t" href="/"><h1><img src="https://i.ibb.co/q1gZN2p/web-check-logo.png" width="48" />Web-Check</h1></a>
<p class="moji">😪</p>
<p>There was an error finding this route.</p>
<span>Docs and Source: <a href="https://github.com/lissy93/web-check">github.com/lissy93/web-check</a></span>
</section>
<style>
body { background: #141d2b; color: #fff; }
h1 {
color: #9fef00; text-shadow: #0f1620 2px 2px 0px;
font-size: 3rem; margin: 1rem auto; flex-wrap: wrap;
display: flex; align-items: flex-start; gap: 1rem;
}
section {
display: flex; flex-direction: column; align-items: center; margin: 1rem; gap: 0.5rem;
background: #1a2332; box-shadow: #0f1620 4px 4px 0px; border-radius: 8px; padding: 1rem;
max-width: 800px; margin: 1rem auto;
}
p { font-size: 1.2rem; }
a { color: #9fef00; font-family: monospace; }
a.t { text-decoration: none; margin: 0;}
span { opacity: 0.8; font-size: 0.85rem; }
h1 {
font-family: PTMono, Ubuntu, Fira Sans, Helvetica, sans-serif;
}
p, span, a, section, div { font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New', monospace; }
code { color: #9fef00cc;}
.moji { font-size: 8rem; margin: 0; }
</style>
</body>
</html>

View File

@@ -18,7 +18,7 @@
<meta property="og:title" content="Web Check"> <meta property="og:title" content="Web Check">
<meta property="og:description" content="All-in-one Website OSINT Scanner"> <meta property="og:description" content="All-in-one Website OSINT Scanner">
<meta property="og:image" content="/banner.png"> <meta property="og:image" content="/banner.png">
<meta property="og:url" content="https://web-check.as93.net"> <meta property="og:url" content="https://web-check.xyz">
<meta name="twitter:card" content="summary_large_image"> <meta name="twitter:card" content="summary_large_image">
</head> </head>
<body> <body>

37
public/placeholder.html Normal file
View File

@@ -0,0 +1,37 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Web-Check</title>
</head>
<body>
<section>
<a class="t" href="/"><h1><img src="https://i.ibb.co/q1gZN2p/web-check-logo.png" width="48" />Web-Check</h1></a>
<p><!-- CONTENT --></p>
<span>Docs and Source: <a href="https://github.com/lissy93/web-check">github.com/lissy93/web-check</a></span>
</section>
<style>
body { background: #141d2b; color: #fff; }
h1 {
color: #9fef00; text-shadow: #0f1620 2px 2px 0px;
font-size: 3rem; margin: 1rem auto; flex-wrap: wrap;
display: flex; align-items: flex-start; gap: 1rem;
}
section {
display: flex; flex-direction: column; align-items: center; margin: 1rem; gap: 0.5rem;
background: #1a2332; box-shadow: #0f1620 4px 4px 0px; border-radius: 8px; padding: 1rem;
max-width: 800px; margin: 1rem auto;
}
p { font-size: 1.2rem; }
a { color: #9fef00; font-family: monospace; }
a.t { text-decoration: none; margin: 0;}
span { opacity: 0.8; font-size: 0.85rem; }
h1 {
font-family: PTMono, Ubuntu, Fira Sans, Helvetica, sans-serif;
}
p, span, a, section, div { font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New', monospace; }
code { color: #9fef00cc;}
</style>
</body>
</html>

190
server.js
View File

@@ -1,116 +1,148 @@
const express = require('express'); const express = require('express');
const awsServerlessExpress = require('aws-serverless-express');
const fs = require('fs'); const fs = require('fs');
const path = require('path'); const path = require('path');
const cors = require('cors');
const historyApiFallback = require('connect-history-api-fallback'); const historyApiFallback = require('connect-history-api-fallback');
require('dotenv').config(); require('dotenv').config();
const app = express(); const app = express();
const port = process.env.PORT || 3000; // The port to run the server on
const API_DIR = '/api'; // Name of the dir containing the lambda functions const API_DIR = '/api'; // Name of the dir containing the lambda functions
const dirPath = path.join(__dirname, API_DIR); // Path to the lambda functions dir const dirPath = path.join(__dirname, API_DIR); // Path to the lambda functions dir
const guiPath = path.join(__dirname, 'build'); const guiPath = path.join(__dirname, 'build');
const placeholderFilePath = path.join(__dirname, 'public', 'placeholder.html');
const handlers = {}; // Will store list of API endpoints
process.env.WC_SERVER = 'true'; // Tells middleware to return in non-lambda mode
// Execute the lambda function // Enable CORS
const executeHandler = async (handler, req) => { app.use(cors({
return new Promise((resolve, reject) => { origin: process.env.API_CORS_ORIGIN || '*',
const callback = (err, response) => err ? reject(err) : resolve(response); }));
const promise = handler(req, {}, callback);
if (promise && typeof promise.then === 'function') { // Read and register each API function as an Express routes
promise.then(resolve).catch(reject); fs.readdirSync(dirPath, { withFileTypes: true })
}
});
};
// Array of all the lambda function file names
const fileNames = fs.readdirSync(dirPath, { withFileTypes: true })
.filter(dirent => dirent.isFile() && dirent.name.endsWith('.js')) .filter(dirent => dirent.isFile() && dirent.name.endsWith('.js'))
.map(dirent => dirent.name); .forEach(dirent => {
const routeName = dirent.name.split('.')[0];
const route = `${API_DIR}/${routeName}`;
const handler = require(path.join(dirPath, dirent.name));
handlers[route] = handler;
const handlers = {}; app.get(route, async (req, res) => {
try {
fileNames.forEach(file => { await handler(req, res);
const routeName = file.split('.')[0]; } catch (err) {
const route = `${API_DIR}/${routeName}`; res.status(500).json({ error: err.message });
const handler = require(path.join(dirPath, file)).handler; }
});
handlers[route] = handler; });
app.get(route, async (req, res) => { // Create a single API endpoint to execute all lambda functions
try { app.get('/api', async (req, res) => {
const { statusCode = 200, body = '' } = await executeHandler(handler, req); const results = {};
res.status(statusCode).json(JSON.parse(body)); const { url } = req.query;
} catch (err) { const maxExecutionTime = process.env.API_TIMEOUT_LIMIT || 20000;
res.status(500).json({ error: err.message });
} const executeHandler = async (handler, req, res) => {
}); return new Promise(async (resolve, reject) => {
}); try {
const mockRes = {
const timeout = (ms, jobName = null) => { status: (statusCode) => mockRes,
return new Promise((_, reject) => { json: (body) => resolve({ body }),
setTimeout(() => { };
reject(new Error(`Timed out after the ${ms/1000} second limit${jobName ? `, when executing the ${jobName} task` : ''}`)); await handler({ ...req, query: { url } }, mockRes);
}, ms); } catch (err) {
}); reject(err);
} }
});
app.get('/api', async (req, res) => { };
const results = {};
const { url } = req.query; const timeout = (ms, jobName = null) => {
const maxExecutionTime = process.env.API_TIMEOUT_LIMIT || 15000; return new Promise((_, reject) => {
setTimeout(() => {
const handlerPromises = Object.entries(handlers).map(async ([route, handler]) => { reject(new Error(
const routeName = route.replace(`${API_DIR}/`, ''); `Timed out after ${ms/1000} seconds${jobName ? `, when executing ${jobName}` : ''}`
));
try { }, ms);
const result = await Promise.race([ });
executeHandler(handler, { query: { url } }), };
timeout(maxExecutionTime, routeName)
]); const handlerPromises = Object.entries(handlers).map(async ([route, handler]) => {
results[routeName] = JSON.parse((result || {}).body); const routeName = route.replace(`${API_DIR}/`, '');
} catch (err) { try {
results[routeName] = { error: err.message }; const result = await Promise.race([
} executeHandler(handler, req, res),
timeout(maxExecutionTime, routeName)
]);
results[routeName] = result.body;
} catch (err) {
results[routeName] = { error: err.message };
}
});
await Promise.all(handlerPromises);
res.json(results);
}); });
await Promise.all(handlerPromises);
res.json(results);
});
// Handle SPA routing // Handle SPA routing
app.use(historyApiFallback({ app.use(historyApiFallback({
rewrites: [ rewrites: [
{ from: /^\/api\/.*$/, to: function(context) { return context.parsedUrl.path; } }, { from: /^\/api\/.*$/, to: (context) => context.parsedUrl.path },
] ]
})); }));
// Serve up the GUI - if build dir exists, and GUI feature enabled // Serve up the GUI - if build dir exists, and GUI feature enabled
if (process.env.DISABLE_GUI && process.env.DISABLE_GUI !== 'false') { if (process.env.DISABLE_GUI && process.env.DISABLE_GUI !== 'false') {
app.get('*', (req, res) => { app.get('*', async (req, res) => {
res.status(500).send( const placeholderContent = await fs.promises.readFile(placeholderFilePath, 'utf-8');
'Welcome to Web-Check!<br />Access the API endpoints at ' const htmlContent = placeholderContent.replace(
+'<a href="/api"><code>/api</code></a>' '<!-- CONTENT -->',
); 'Web-Check API is up and running!<br />Access the endpoints at '
+'<a href="/api"><code>/api</code></a>'
);
res.status(500).send(htmlContent);
}); });
} else if (!fs.existsSync(guiPath)) { } else if (!fs.existsSync(guiPath)) {
app.get('*', (req, res) => { app.get('*', async (req, res) => {
res.status(500).send( const placeholderContent = await fs.promises.readFile(placeholderFilePath, 'utf-8');
'Welcome to Web-Check!<br />Looks like the GUI app has not yet been compiled, ' const htmlContent = placeholderContent.replace(
+'run <code>yarn build</code> to continue, then restart the server.' '<!-- CONTENT -->',
'Looks like the GUI app has not yet been compiled.<br /> ' +
'Run <code>yarn build</code> to continue, then restart the server.'
); );
res.status(500).send(htmlContent);
}); });
} else { // GUI enabled, and build files present, let's go!! } else { // GUI enabled, and build files present, let's go!!
app.use(express.static(guiPath)); app.use(express.static(guiPath));
} }
// Create serverless express server app.use((req, res, next) => {
const port = process.env.PORT || 3000; res.status(404).sendFile(path.join(__dirname, 'public', 'error.html'));
const server = awsServerlessExpress.createServer(app).listen(port, () => {
console.log(`Server is running on port ${port}`);
}); });
exports.handler = (event, context) => { // Print nice welcome message to user
awsServerlessExpress.proxy(server, event, context); const printMessage = () => {
console.log(
`\x1b[36m\n` +
' __ __ _ ___ _ _ \n' +
' \\ \\ / /__| |__ ___ / __| |_ ___ __| |__\n' +
' \\ \\/\\/ / -_) \'_ \\___| (__| \' \\/ -_) _| / /\n' +
' \\_/\\_/\\___|_.__/ \\___|_||_\\___\\__|_\\_\\\n' +
`\x1b[0m\n`,
`\x1b[1m\x1b[32m🚀 Web-Check is up and running at http://localhost:${port} \x1b[0m\n\n`,
`\x1b[2m\x1b[36m🛟 For documentation and support, visit the GitHub repo: ` +
`https://github.com/lissy93/web-check \n`,
`💖 Found Web-Check useful? Consider sponsoring us on GitHub ` +
`to help fund maintenance & development.\x1b[0m`
);
}; };
// Create server
app.listen(port, () => {
printMessage();
});

View File

@@ -4,37 +4,46 @@ provider:
name: aws name: aws
runtime: nodejs14.x runtime: nodejs14.x
region: us-east-1 region: us-east-1
# environment:
# GOOGLE_CLOUD_API_KEY: ${ssm:GOOGLE_CLOUD_API_KEY~true, ''}
# TORRENT_IP_API_KEY: ${ssm:TORRENT_IP_API_KEY~true, ''}
# SECURITY_TRAILS_API_KEY: ${ssm:SECURITY_TRAILS_API_KEY~true, ''}
# BUILT_WITH_API_KEY: ${ssm:BUILT_WITH_API_KEY~true, ''}
# URL_SCAN_API_KEY: ${ssm:URL_SCAN_API_KEY~true, ''}
# TRANCO_USERNAME: ${ssm:TRANCO_USERNAME~true, ''}
# TRANCO_API_KEY: ${ssm:TRANCO_API_KEY~true, ''}
# CLOUDMERSIVE_API_KEY: ${ssm:CLOUDMERSIVE_API_KEY~true, ''}
# CHROME_PATH: ${ssm:CHROME_PATH~true, ''}
# API_TIMEOUT_LIMIT: ${ssm:API_TIMEOUT_LIMIT~true, ''}
# API_CORS_ORIGIN: ${ssm:API_CORS_ORIGIN~true, ''}
iamRoleStatements:
- Effect: Allow
Action:
- ssm:GetParameter
Resource:
- arn:aws:ssm:us-east-1:${env:AWS_ACCOUNT_ID}:parameter/GOOGLE_CLOUD_API_KEY
- arn:aws:ssm:us-east-1:${env:AWS_ACCOUNT_ID}:parameter/TORRENT_IP_API_KEY
- arn:aws:ssm:us-east-1:${env:AWS_ACCOUNT_ID}:parameter/SECURITY_TRAILS_API_KEY
- arn:aws:ssm:us-east-1:${env:AWS_ACCOUNT_ID}:parameter/BUILT_WITH_API_KEY
- arn:aws:ssm:us-east-1:${env:AWS_ACCOUNT_ID}:parameter/URL_SCAN_API_KEY
- arn:aws:ssm:us-east-1:${env:AWS_ACCOUNT_ID}:parameter/TRANCO_USERNAME
- arn:aws:ssm:us-east-1:${env:AWS_ACCOUNT_ID}:parameter/TRANCO_API_KEY
- arn:aws:ssm:us-east-1:${env:AWS_ACCOUNT_ID}:parameter/CLOUDMERSIVE_API_KEY
- arn:aws:ssm:us-east-1:${env:AWS_ACCOUNT_ID}:parameter/CHROME_PATH
- arn:aws:ssm:us-east-1:${env:AWS_ACCOUNT_ID}:parameter/API_TIMEOUT_LIMIT
- arn:aws:ssm:us-east-1:${env:AWS_ACCOUNT_ID}:parameter/API_CORS_ORIGIN
functions: functions:
dnssec: archives:
handler: api/dnssec.handler handler: api/archives.handler
events: events:
- http: - http:
path: api/dnssec path: api/archives
method: get method: get
linkedPages: blockLists:
handler: api/linked-pages.handler handler: api/block-lists.handler
events: events:
- http: - http:
path: api/linked-pages path: api/block-lists
method: get
robotsTxt:
handler: api/robots-txt.handler
events:
- http:
path: api/robots-txt
method: get
ssl:
handler: api/ssl.handler
events:
- http:
path: api/ssl
method: get
whois:
handler: api/whois.handler
events:
- http:
path: api/whois
method: get method: get
carbon: carbon:
handler: api/carbon.handler handler: api/carbon.handler
@@ -42,138 +51,206 @@ functions:
- http: - http:
path: api/carbon path: api/carbon
method: get method: get
features:
handler: api/features.handler
events:
- http:
path: api/features
method: get
mailConfig:
handler: api/mail-config.handler
events:
- http:
path: api/mail-config
method: get
screenshot:
handler: api/screenshot.handler
events:
- http:
path: api/screenshot
method: get
status:
handler: api/status.handler
events:
- http:
path: api/status
method: get
cookies: cookies:
handler: api/cookies.handler handler: api/cookies.handler
events: events:
- http: - http:
path: api/cookies path: api/cookies
method: get method: get
getIp:
handler: api/get-ip.handler
events:
- http:
path: api/get-ip
method: get
ports:
handler: api/ports.handler
events:
- http:
path: api/ports
method: get
securityTxt:
handler: api/security-txt.handler
events:
- http:
path: api/security-txt
method: get
techStack:
handler: api/tech-stack.handler
events:
- http:
path: api/tech-stack
method: get
dnsServer: dnsServer:
handler: api/dns-server.handler handler: api/dns-server.handler
events: events:
- http: - http:
path: api/dns-server path: api/dns-server
method: get method: get
headers:
handler: api/headers.handler
events:
- http:
path: api/headers
method: get
quality:
handler: api/quality.handler
events:
- http:
path: api/quality
method: get
sitemap:
handler: api/sitemap.handler
events:
- http:
path: api/sitemap
method: get
traceRoute:
handler: api/trace-route.handler
events:
- http:
path: api/trace-route
method: get
dns: dns:
handler: api/dns.handler handler: api/dns.handler
events: events:
- http: - http:
path: api/dns path: api/dns
method: get method: get
dnssec:
handler: api/dnssec.handler
events:
- http:
path: api/dnssec
method: get
features:
handler: api/features.handler
events:
- http:
path: api/features
method: get
firewall:
handler: api/firewall.handler
events:
- http:
path: api/firewall
method: get
getIp:
handler: api/get-ip.handler
events:
- http:
path: api/get-ip
method: get
headers:
handler: api/headers.handler
events:
- http:
path: api/headers
method: get
hsts: hsts:
handler: api/hsts.handler handler: api/hsts.handler
events: events:
- http: - http:
path: api/hsts path: api/hsts
method: get method: get
httpSecurity:
handler: api/http-security.handler
events:
- http:
path: api/http-security
method: get
legacyRank:
handler: api/legacy-rank.handler
events:
- http:
path: api/legacy-rank
method: get
linkedPages:
handler: api/linked-pages.handler
events:
- http:
path: api/linked-pages
method: get
mailConfig:
handler: api/mail-config.handler
events:
- http:
path: api/mail-config
method: get
ports:
handler: api/ports.handler
events:
- http:
path: api/ports
method: get
quality:
handler: api/quality.handler
events:
- http:
path: api/quality
method: get
rank:
handler: api/rank.handler
events:
- http:
path: api/rank
method: get
redirects: redirects:
handler: api/redirects.handler handler: api/redirects.handler
events: events:
- http: - http:
path: api/redirects path: api/redirects
method: get method: get
robotsTxt:
handler: api/robots-txt.handler
events:
- http:
path: api/robots-txt
method: get
screenshot:
handler: api/screenshot.handler
events:
- http:
path: api/screenshot
method: get
securityTxt:
handler: api/security-txt.handler
events:
- http:
path: api/security-txt
method: get
sitemap:
handler: api/sitemap.handler
events:
- http:
path: api/sitemap
method: get
socialTags: socialTags:
handler: api/social-tags.handler handler: api/social-tags.handler
events: events:
- http: - http:
path: api/social-tags path: api/social-tags
method: get method: get
ssl:
handler: api/ssl.handler
events:
- http:
path: api/ssl
method: get
status:
handler: api/status.handler
events:
- http:
path: api/status
method: get
techStack:
handler: api/tech-stack.handler
events:
- http:
path: api/tech-stack
method: get
threats:
handler: api/threats.handler
events:
- http:
path: api/threats
method: get
tls:
handler: api/tls.handler
events:
- http:
path: api/tls
method: get
traceRoute:
handler: api/trace-route.handler
events:
- http:
path: api/trace-route
method: get
txtRecords: txtRecords:
handler: api/txt-records.handler handler: api/txt-records.handler
events: events:
- http: - http:
path: api/txt-records path: api/txt-records
method: get method: get
whois:
handler: api/whois.handler
events:
- http:
path: api/whois
method: get
plugins: plugins:
# - serverless-webpack - serverless-webpack
- serverless-domain-manager # - serverless-domain-manager
- serverless-offline # - serverless-offline
custom: custom:
webpack: webpack:
webpackConfig: 'api/_common/aws-webpack.config.js' webpackConfig: 'api/_common/aws-webpack.config.js'
includeModules: true includeModules: false
packagerOptions:
noInstall: true
customDomain: # customDomain:
domainName: example.com # domainName: example.com
basePath: 'api' # basePath: 'api'
stage: ${self:provider.stage} # stage: ${self:provider.stage}
createRoute53Record: true # createRoute53Record: true
serverless-offline: # serverless-offline:
prefix: '' # prefix: ''
httpPort: 3000 # httpPort: 3000

View File

@@ -12,7 +12,6 @@ const Container = Styled.main`
background: ${colors.background}; background: ${colors.background};
color: ${colors.textColor}; color: ${colors.textColor};
width: 100vw; width: 100vw;
height: 100vh;
margin: 0; margin: 0;
`; `;

View File

@@ -90,7 +90,6 @@ const SslCertCard = (props: { data: any, title: string, actionButtons: any }): J
{ valid_from && <DataRow lbl="Renewed" val={formatDate(valid_from)} /> } { valid_from && <DataRow lbl="Renewed" val={formatDate(valid_from)} /> }
{ serialNumber && <DataRow lbl="Serial Num" val={serialNumber} /> } { serialNumber && <DataRow lbl="Serial Num" val={serialNumber} /> }
{ fingerprint && <DataRow lbl="Fingerprint" val={fingerprint} /> } { fingerprint && <DataRow lbl="Fingerprint" val={fingerprint} /> }
{ fingerprint && <DataRow lbl="Fingerprint" val={fingerprint} /> }
{ ext_key_usage && <ListRow title="Extended Key Usage" list={getExtendedKeyUsage(ext_key_usage)} /> } { ext_key_usage && <ListRow title="Extended Key Usage" list={getExtendedKeyUsage(ext_key_usage)} /> }
</Card> </Card>

View File

@@ -1,5 +1,5 @@
import { useState } from 'react'; import { useState, useEffect } from 'react';
import { Card } from 'components/Form/Card'; import { Card } from 'components/Form/Card';
import Button from 'components/Form/Button'; import Button from 'components/Form/Button';
import { ExpandableRow } from 'components/Form/Row'; import { ExpandableRow } from 'components/Form/Row';
@@ -29,6 +29,10 @@ const TlsCard = (props: {data: any, title: string, actionButtons: any }): JSX.El
const [cipherSuites, setCipherSuites] = useState(makeCipherSuites(props.data)); const [cipherSuites, setCipherSuites] = useState(makeCipherSuites(props.data));
const [loadState, setLoadState] = useState<undefined | 'loading' | 'success' | 'error'>(undefined); const [loadState, setLoadState] = useState<undefined | 'loading' | 'success' | 'error'>(undefined);
useEffect(() => { // Update cipher suites when data changes
setCipherSuites(makeCipherSuites(props.data));
}, [props.data]);
const updateData = (id: number) => { const updateData = (id: number) => {
setCipherSuites([]); setCipherSuites([]);
setLoadState('loading'); setLoadState('loading');

View File

@@ -1,5 +1,5 @@
import { useState } from 'react'; import { useState, useEffect } from 'react';
import { Card } from 'components/Form/Card'; import { Card } from 'components/Form/Card';
import Button from 'components/Form/Button'; import Button from 'components/Form/Button';
import { ExpandableRow } from 'components/Form/Row'; import { ExpandableRow } from 'components/Form/Row';
@@ -38,6 +38,10 @@ const TlsCard = (props: {data: any, title: string, actionButtons: any }): JSX.El
const [clientSupport, setClientSupport] = useState(makeClientSupport(props.data)); const [clientSupport, setClientSupport] = useState(makeClientSupport(props.data));
const [loadState, setLoadState] = useState<undefined | 'loading' | 'success' | 'error'>(undefined); const [loadState, setLoadState] = useState<undefined | 'loading' | 'success' | 'error'>(undefined);
useEffect(() => {
setClientSupport(makeClientSupport(props.data));
}, [props.data]);
const updateData = (id: number) => { const updateData = (id: number) => {
setClientSupport([]); setClientSupport([]);
setLoadState('loading'); setLoadState('loading');

View File

@@ -1,5 +1,5 @@
import { useState } from 'react'; import { useState, useEffect } from 'react';
import styled from 'styled-components'; import styled from 'styled-components';
import colors from 'styles/colors'; import colors from 'styles/colors';
import { Card } from 'components/Form/Card'; import { Card } from 'components/Form/Card';
@@ -74,6 +74,11 @@ const TlsCard = (props: {data: any, title: string, actionButtons: any }): JSX.El
const [tlsResults, setTlsResults] = useState(makeResults(props.data)); const [tlsResults, setTlsResults] = useState(makeResults(props.data));
const [loadState, setLoadState] = useState<undefined | 'loading' | 'success' | 'error'>(undefined); const [loadState, setLoadState] = useState<undefined | 'loading' | 'success' | 'error'>(undefined);
useEffect(() => {
setTlsRowWata(makeExpandableData(props.data));
setTlsResults(makeResults(props.data));
}, [props.data]);
const updateData = (id: number) => { const updateData = (id: number) => {
setTlsRowWata([]); setTlsRowWata([]);
setLoadState('loading'); setLoadState('loading');

View File

@@ -101,6 +101,18 @@ const Section = styled(StyledCard)`
} }
`; `;
const SponsorshipContainer = styled.div`
display: flex;
justify-content: space-between;
gap: 1rem;
flex-wrap: wrap;
align-items: center;
line-height: 1.5rem;
img {
border-radius: 4px;
}
`;
const makeAnchor = (title: string): string => { const makeAnchor = (title: string): string => {
return title.toLowerCase().replace(/[^\w\s]|_/g, "").replace(/\s+/g, "-"); return title.toLowerCase().replace(/[^\w\s]|_/g, "").replace(/\s+/g, "-");
}; };
@@ -121,6 +133,26 @@ const About = (): JSX.Element => {
<p key={index}>{para}</p> <p key={index}>{para}</p>
))} ))}
<hr /> <hr />
<SponsorshipContainer>
<p>
Web-Check is kindly sponsored
by <a href="https://terminaltrove.com/?utm_campaign=github&utm_medium=referral&utm_content=web-check&utm_source=wcgh">
Terminal Trove
</a>
<br />
The $HOME of all things in the terminal.
<br />
<small>
<a href="https://terminaltrove.com/newsletter?utm_campaign=github&utm_medium=referral&utm_content=web-check&utm_source=wcgh">
Find your next CLI / TUI tool, and get updates to your inbox
</a>
</small>
</p>
<a href="https://terminaltrove.com/?utm_campaign=github&utm_medium=referral&utm_content=web-check&utm_source=wcgh">
<img width="300" alt="Terminal Trove" src="https://i.ibb.co/T1KzVmR/terminal-trove-green.png" />
</a>
</SponsorshipContainer>
<hr />
<p> <p>
Web-Check is developed and maintained by <a href="https://aliciasykes.com">Alicia Sykes</a>. Web-Check is developed and maintained by <a href="https://aliciasykes.com">Alicia Sykes</a>.
It's licensed under the <a href="https://github.com/Lissy93/web-check/blob/master/LICENSE">MIT license</a>, It's licensed under the <a href="https://github.com/Lissy93/web-check/blob/master/LICENSE">MIT license</a>,
@@ -179,6 +211,79 @@ const About = (): JSX.Element => {
))} ))}
</Section> </Section>
<Heading as="h2" size="medium" color={colors.primary}>Deploy your own Instance</Heading>
<Section>
<p>Web-Check is designed to be easily self-hosted.</p>
<Heading as="h3" size="small" color={colors.primary}>Option #1 - Netlify</Heading>
<p>Click the button below to deploy to Netlify</p>
<a href="https://app.netlify.com/start/deploy?repository=https://github.com/lissy93/web-check">
<img src="https://www.netlify.com/img/deploy/button.svg" alt="Deploy to Netlify" />
</a>
<Heading as="h3" size="small" color={colors.primary}>Option #2 - Vercel</Heading>
<p>Click the button below to deploy to Vercel</p>
<a href="https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Flissy93%2Fweb-check&project-name=web-check&repository-name=web-check-fork&demo-title=Web-Check%20Demo&demo-description=Check%20out%20web-check.xyz%20to%20see%20a%20live%20demo%20of%20this%20application%20running.&demo-url=https%3A%2F%2Fweb-check.xyz&demo-image=https%3A%2F%2Fraw.githubusercontent.com%2FLissy93%2Fweb-check%2Fmaster%2F.github%2Fscreenshots%2Fweb-check-screenshot10.png">
<img src="https://vercel.com/button" alt="Deploy with Vercel" />
</a>
<Heading as="h3" size="small" color={colors.primary}>Option #3 - Docker</Heading>
<p>
A Docker container is published to <a href="https://hub.docker.com/r/lissy93/web-check">DockerHub</a>
<br />
Run this command, then open <code>localhost:3000</code>
<pre>docker run -p 3000:3000 lissy93/web-check</pre>
</p>
<Heading as="h3" size="small" color={colors.primary}>Option #4 - Manual</Heading>
<pre>
git clone https://github.com/Lissy93/web-check.git<br />
cd web-check # Move into the project directory<br />
yarn install # Install dependencies<br />
yarn build # Build the app for production<br />
yarn serve # Start the app (API and GUI)<br />
</pre>
<Heading as="h3" size="small" color={colors.primary}>Further Docs</Heading>
<p>
More detailed installation and setup instructions can be found in the
GitHub repository - <a href="https://github.com/lissy93/web-check#readme">github.com/lissy93/web-check</a>
</p>
<Heading as="h3" size="small" color={colors.primary}>Configuring</Heading>
<p>
There are some optional environmental variables you can specify to give you access to some additional Web-Checks.
See the README for full list of options.
</p>
<ul>
<li>
<code>GOOGLE_CLOUD_API_KEY</code>
: <a href="https://cloud.google.com/api-gateway/docs/authenticate-api-keys">A Google API key</a>
<i> Used to return quality metrics for a site</i>
</li>
<li>
<code>REACT_APP_SHODAN_API_KEY</code>
: <a href="https://account.shodan.io/">A Shodan API key</a>
<i> To show associated hosts for a domain</i>
</li>
<li>
<code>REACT_APP_WHO_API_KEY</code>
: <a href="https://whoapi.com/">A WhoAPI key</a>
<i> Allows for more comprehensive WhoIs records</i>
</li>
</ul>
{/*
**Configuration Settings**:
- `CHROME_PATH` (e.g. `/usr/bin/chromium`) - The path the the Chromium executable
- `PORT` (e.g. `3000`) - Port to serve the API, when running server.js
- `DISABLE_GUI` (e.g. `false`) - Disable the GUI, and only serve the API
- `API_TIMEOUT_LIMIT` (e.g. `10000`) - The timeout limit for API requests, in milliseconds
- `REACT_APP_API_ENDPOINT` (e.g. `/api`) - The endpoint for the API (can be local or remote)</p> */}
</Section>
<Heading as="h2" size="medium" color={colors.primary}>API Documentation</Heading> <Heading as="h2" size="medium" color={colors.primary}>API Documentation</Heading>
<Section> <Section>
{/* eslint-disable-next-line*/} {/* eslint-disable-next-line*/}
@@ -218,18 +323,6 @@ const About = (): JSX.Element => {
Neither your IP address, browser/OS/hardware info, nor any other data will ever be collected or logged. Neither your IP address, browser/OS/hardware info, nor any other data will ever be collected or logged.
(You may verify this yourself, either by inspecting the source code or the using developer tools) (You may verify this yourself, either by inspecting the source code or the using developer tools)
</p> </p>
<hr />
<Heading as="h3" size="small" color={colors.primary}>Support</Heading>
<p>
If you've found something that doesn't work as expected, or would like to ask any questions,
you can open a ticket at <a href="https://github.com/lissy93/web-check/issues">github.com/lissy93/web-check/issues</a>
</p>
<hr />
<Heading as="h3" size="small" color={colors.primary}>Sponsor</Heading>
<p>
If you've found this service useful, consider sponsoring me on
GitHub - <a href="https://github.com/sponsors/Lissy93">github.com/sponsors/Lissy93</a> 💖
</p>
</Section> </Section>
</AboutContainer> </AboutContainer>
<Footer /> <Footer />

View File

@@ -20,7 +20,7 @@ const HomeContainer = styled.section`
align-items: center; align-items: center;
height: 100%; height: 100%;
font-family: 'PTMono'; font-family: 'PTMono';
padding: 0 1rem; padding: 1.5rem 1rem 4rem 1rem;
footer { footer {
z-index: 1; z-index: 1;
} }
@@ -34,10 +34,52 @@ const UserInputMain = styled.form`
z-index: 5; z-index: 5;
margin: 1rem; margin: 1rem;
width: calc(100% - 2rem); width: calc(100% - 2rem);
max-width: 50rem; max-width: 60rem;
z-index: 2; z-index: 2;
`; `;
const SponsorCard = styled.div`
background: ${colors.backgroundLighter};
box-shadow: 4px 4px 0px ${colors.bgShadowColor};
border-radius: 8px;
padding: 1rem;
z-index: 5;
margin: 1rem;
width: calc(100% - 2rem);
max-width: 60rem;
z-index: 2;
.inner {
display: flex;
justify-content: space-between;
flex-wrap: wrap;
gap: 1rem;
p {
margin: 0.25rem 0;
}
}
a {
color: ${colors.textColor};
}
img {
border-radius: 0.25rem;
box-shadow: 2px 2px 0px ${colors.fgShadowColor};
transition: box-shadow 0.2s;
margin: 0 auto;
display: block;
width: 200px;
&:hover {
box-shadow: 4px 4px 0px ${colors.fgShadowColor};
}
&:active {
box-shadow: -2px -2px 0px ${colors.fgShadowColor};
}
}
.cta {
font-size: 0.78rem;
a { color: ${colors.primary}; }
}
`;
// const FindIpButton = styled.a` // const FindIpButton = styled.a`
// margin: 0.5rem; // margin: 0.5rem;
// cursor: pointer; // cursor: pointer;
@@ -55,7 +97,7 @@ const ErrorMessage = styled.p`
const SiteFeaturesWrapper = styled(StyledCard)` const SiteFeaturesWrapper = styled(StyledCard)`
margin: 1rem; margin: 1rem;
width: calc(100% - 2rem); width: calc(100% - 2rem);
max-width: 50rem; max-width: 60rem;
z-index: 2; z-index: 2;
.links { .links {
display: flex; display: flex;
@@ -173,6 +215,28 @@ const Home = (): JSX.Element => {
{ errorMsg && <ErrorMessage>{errorMsg}</ErrorMessage>} { errorMsg && <ErrorMessage>{errorMsg}</ErrorMessage>}
<Button styles="width: calc(100% - 1rem);" size="large" onClick={submit}>Analyze!</Button> <Button styles="width: calc(100% - 1rem);" size="large" onClick={submit}>Analyze!</Button>
</UserInputMain> </UserInputMain>
<SponsorCard>
<Heading as="h2" size="small" color={colors.primary}>Sponsored by</Heading>
<div className="inner">
<p>
<a href="https://terminaltrove.com/?utm_campaign=github&utm_medium=referral&utm_content=web-check&utm_source=wcgh">
Terminal Trove
</a> - The $HOME of all things in the terminal.
<br />
<span className="cta">
Get updates on the latest CLI/TUI tools via
the <a className="cta" href="https://terminaltrove.com/newsletter?utm_campaign=github&utm_medium=referral&utm_content=web-check&utm_source=wcgh">
Terminal Trove newsletter
</a>
</span>
</p>
<a href="https://terminaltrove.com/?utm_campaign=github&utm_medium=referral&utm_content=web-check&utm_source=wcgh">
<img width="120" alt="Terminal Trove" src="https://i.ibb.co/NKtYjJ1/terminal-trove-web-check.png" />
</a>
</div>
</SponsorCard>
<SiteFeaturesWrapper> <SiteFeaturesWrapper>
<div className="features"> <div className="features">
<Heading as="h2" size="small" color={colors.primary}>Supported Checks</Heading> <Heading as="h2" size="small" color={colors.primary}>Supported Checks</Heading>

View File

@@ -863,7 +863,7 @@ const Results = (): JSX.Element => {
} }
</Nav> </Nav>
<ProgressBar loadStatus={loadingJobs} showModal={showErrorModal} showJobDocs={showInfo} /> <ProgressBar loadStatus={loadingJobs} showModal={showErrorModal} showJobDocs={showInfo} />
{ address?.includes(window?.location?.hostname || 'web-check.as93.net') && <SelfScanMsg />} { address?.includes(window?.location?.hostname || 'web-check.xyz') && <SelfScanMsg />}
<Loader show={loadingJobs.filter((job: LoadingJob) => job.state !== 'loading').length < 5} /> <Loader show={loadingJobs.filter((job: LoadingJob) => job.state !== 'loading').length < 5} />
<FilterButtons>{ showFilters ? <> <FilterButtons>{ showFilters ? <>
<div className="one-half"> <div className="one-half">

View File

@@ -529,7 +529,7 @@ and the web of connections within a site's architecture.
The results can also help optimizing server responses, configuring redirects, The results can also help optimizing server responses, configuring redirects,
managing cookies, or fine-tuning DNS records for your site.`, managing cookies, or fine-tuning DNS records for your site.`,
`So, weather you're a developer, system administrator, security researcher, penetration `So, whether you're a developer, system administrator, security researcher, penetration
tester or are just interested in discovering the underlying technologies of a given site tester or are just interested in discovering the underlying technologies of a given site
- I'm sure you'll find this a useful addition to your toolbox.`, - I'm sure you'll find this a useful addition to your toolbox.`,
]; ];
@@ -557,7 +557,7 @@ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
export const supportUs = [ export const supportUs = [
"Web-Check is free to use without restriction.", "Web-Check is free to use without restriction.",
"All the code is open source, so you're also free to deploy your own instance, as well as fork, modify and distribute the code in both private and commerical settings.", "All the code is open source, so you're also free to deploy your own instance, as well as fork, modify and distribute the code in both private and commercial settings.",
"Running web-check does cost me a small amount of money each month, so if you're finding the app useful, consider <a href='https://github.com/sponsors/Lissy93'>sponsoring me on GitHub</a> if you're able to. Even just $1 or $2/month would be a huge help in supporting the ongoing project running costs.", "Running web-check does cost me a small amount of money each month, so if you're finding the app useful, consider <a href='https://github.com/sponsors/Lissy93'>sponsoring me on GitHub</a> if you're able to. Even just $1 or $2/month would be a huge help in supporting the ongoing project running costs.",
"Otherwise, there are other ways you can help out, like submitting or reviewing a pull request to the <a href='https://github.com/Lissy93/web-check'>GitHub repo</a>, upvoting us on <a href='https://www.producthunt.com/posts/web-check'>Product Hunt</a>, or by sharing with your network.", "Otherwise, there are other ways you can help out, like submitting or reviewing a pull request to the <a href='https://github.com/Lissy93/web-check'>GitHub repo</a>, upvoting us on <a href='https://www.producthunt.com/posts/web-check'>Product Hunt</a>, or by sharing with your network.",
"But don't feel obliged to do anything, as this app (and all my other projects) will always remain 100% free and open source, and I will do my best to ensure the managed instances remain up and available for as long as possible :)", "But don't feel obliged to do anything, as this app (and all my other projects) will always remain 100% free and open source, and I will do my best to ensure the managed instances remain up and available for as long as possible :)",

4878
yarn.lock

File diff suppressed because it is too large Load Diff