About

TL;DR ExpressJS, VueJS cookbook, with evergreen recipes and templates (CRUD, CI/CD, QA, Testing, Cloud container deployment, Web Components, ES Modules, etc.) to develop applications faster, while reducing the need for rewrite or refactoring due to changes in dependencies.

Latest Version 0.6.11 - Released 2022 Jul 22 0830 +8GMT

NOTE:

• NestJS, ReactJS and Typescript example projects react-admin and nest-admin have been moved to https://github.com/ais-one/cookbook-ts
• solid project (SolidJS frontend) has been removed
• cypress E2E test replaced with playwright

Ask for help and recommend improvements here

Considerations for this project are similar to favv. The difference between them are:

• this repo is more of a cookbook and recipes are constantly being improved and updated
• favv backend is written in Python, they can be used to improve on each other

Features

QUICK START - ON YOUR LOCAL MACHINE

Requirements

• Node 16+ LTS
• Npm 8.3.2+ (using workspaces)
• For Windows, integrate bash shell to cmd shell (when installing git), or use git-bash
• Docker

Updating npm on Windows

npm i -g npm@latest

Optional VS Code Plugins

NOTE Useful plugins if using VS Code:

• Essentials
  • Docker
  • Live Server
  • REST Client
  • SSH FS
  • MongoDB Client (official)
• Recommended
  • SonarLint (requires java)
  • GitLens
• JS Language Specific
  • es6-string-html
  • ESLint
  • Volar (for VueJS)
  • Prettier (disabled)

Chrome Extensions

• Web Server
  • https://chrome.google.com/webstore/detail/web-server-for-chrome/ofhbbkphhbklhfoeikjpcbhemlocgigb/related?hl=en
• SAML / OIDC
  • https://chrome.google.com/webstore/detail/saml-ws-federation-and-oa/hkodokikbjolckghdnljbkbhacbhpnkb?hl=en
• React & Vue Dev tools
  • https://chrome.google.com/webstore/detail/react-developer-tools/fmkadmapgofadopljbjfkapdkoienihi?hl=en
  • https://chrome.google.com/webstore/detail/vuejs-devtools/nhdogjmejiglipccpnnnanhbledajbpd?hl=en
• MetaMask
  • https://chrome.google.com/webstore/detail/metamask/nkbihfbeogaeaoehlefnkodbefgpgknn?hl=en

Other Utilities

• DB clients
  • dbeaver (mac / windows)
  • heidisql (windows)

Download

# clone repo and install backend
git clone https://github.com/ais-one/cookbook.git
cd cookbook

Installing & Updating Dependencies

Install dependencies for all workspaces!

Note

• when doing npm i, it will always install latest version matching your package
• sometimes you need to rebuild, delete all node_modules folders and the package-lock.json file in the root

# https://github.com/npm/cli/issues/708
# https://github.com/npm/cli/issues/2032
npm i # use this
# npm i --legacy-peer-deps # use this if there is peer dependencies issues, but not recommended

Update dependencies for all workspaces!

npm outdated # use this to check for outdated dependencies
npm update --save
npm ls <?package> # use npm ls to check on actual versions installed

Install for single workspace

npm i @vscode/sqlite3 --workspace=@es-labs/node
npm i lorem-ipsum --workspace=@es-labs/node

# updating from 2.2.8 to 3.1.1 - major version change
npm i ant-design-vue@latest --workspace=js-web/vue-vite

ExpressJS Backend Setup & Run - development environment

npm run app --workspace=js-node/expressjs -- development

NOTES

• MongoDB examples needs MongoDB to work. To resolve, chose one of the methods to install MongoDB in docs/backend/mongodb/install.md
• The js-node/expressjs/apps/app-template/config/secret/*.env,js files are not present. So there maybe some console log errors (but it is ok to ignore) and websockets will not work. Quick start is still usable. Use the README.md to fill up

Run migration & app

# create and seed relational db on SQLite, (delete the dev.sqlite file each time before you run this)
# command: npm run knex -- <development / uat / production> <custom app name> <seed / migrate>
npm run knex --workspace=js-node/expressjs -- development app-template migrate
npm run knex --workspace=js-node/expressjs -- development app-template seed

# create and seed MongoDB requires MongoDB - you can skip this but MongoDB examples will not work
# command: npm run mongo -- <development / uat / production> <custom app name> <seed / update>
npm run mongo --workspace=js-node/expressjs -- development app-template seed

# run the backend
# command: npm run app --workspace=js-node/expressjs -- <development / uat / production> <custom app name, default = app-template>
npm run app --workspace=js-node/expressjs -- development # app name implied (implied as app-template if not in env)
npm run app --workspace=js-node/expressjs -- development app-template # or app name specified
npm run app:lint --workspace=js-node/expressjs -- development app-template # to include eslint checks

Visit the following URLs

• http://127.0.0.1:3000/api/healthcheck - app is running normally
• http://127.0.0.1:3000 - Website served by Express with functional samples and demos
• http://127.0.0.1:3000/api-docs - OpenAPI documentation

Note: to generate api docs, visit js-node/openapi-file-joiner and follow readme file, also look at the config properties OPENAPI_PATH and OPENAPI_VALIDATOR in js-node/expressjs/apps/app-template/config/common.env.js.

No bundler frontend

See js-web/vue-nobundler. Served from http://127.0.0.1:3000/native/index.html

Testing

• To run unit & integration test on /api/categories endpoint. E2E testing is Work In Progress
• TO TEST EVERYTHING PLEASE change describe.only(...) to describe(...) in the test scripts in js-node/expressjs/apps/app-template/tests

# run in development only
npm run test --workspace=js-node/expressjs -- development app-template

Long Running Processes

For long running processes such as tcp server (event mode, streaming mode), serial server, kafka producer, consumer, cron-triggered process, etc.

See js-node/README.md

Vite SPA Setup & Run - development environment

1. See js-web/vue-vite/README.md. To setup the configuration files. End-to-end testing example using playwright is here also.

2. Run the following

npm run dev --workspace=js-web/vue-vite

3. Visit

• http://127.0.0.1:8080/ to view application
• http://127.0.0.1:8080/nested/index.html to view another page (vite serving multi page, each page can be an SPA)

4. See js-web/vue-vite/README.md for more information on the vue-vite package

5. E2E Testing

npx playwright install chromium
npx playwright test --browser=chromium

Why No SSR or SSG

• potential slow rendering by server app, added complexity in code, rehydration errors, added complexity in server
• nuxt/nuxt.js#8102
• prefer static sites and lazy loaded SPA for now

SAML, OIDC, OAuth

Refer to link below on how to try out...

• Keycloak README.md
• Refer also to the following files
  • js-node/expressjs/router/saml.js
  • js-node/expressjs/router/oidc.js
  • js-node/expressjs/router/oauth.js requires setup of github account and config setup here
• You can test out on the js-web/vue-vite Signin UI,
  • Use the following username / password credentials
    • for simple Login...
      • just clock on Login button, credentials test / test is already prefilled
      • then click on OTP button, the OTP 111111 is already prefilled
    • for SAML and OIDC... test-user / password
      • redirect to a keycloak IDP
    • for OAuth (Github), you need to configure your github settings (not recommended for starting out)
      • redirect to github login
    • for Mock user login, just click on Mock button

Fido2

Refer to following files for SPA sample (uses fido2-lib in backend)

• js-node/expressjs/router/fido.js
• js-node/expressjs/public/demo-express/fido.html

Configuration

The js-node/expressjs/apps/app-template/config folder contains the config information.

You can override the configurations using <NODE_ENV>.env.js files, e.g. development.env.js or uat.env.js in the folder

Project Strcuture

+- .github/ : github related CICD and automations
+- .husky : git hooks
+- @es-labs/
|  +- esm/ : [shared es modules]
|     +- package.json
|  +- node/ : [shared cjs modules]
|     +- auth/ : authentication
|     +- comms/ : messaging
|     +- services/ : db, mq, etc.
|     +- config.default.js: defaults
|     +- config.js: config loader
|     +- package.json
|     +- traps.js
+- docker-devenv/ : docker for development environment
+- docs/ : documentation
+- js-node/ : nodejs applications (kafka, cron triggered, long running)
|  +- expressjs/ : express backend - See [js-node/expressjs/README.md](js-node/expressjs/README.md) for project structure
|  +- openapi-file-joiner/ : pre-process utility to combine openapi yaml files for use in openapi related packages
|  +- serialserver/
|  +- services/: TCP server, kafka, long running / cron processes
|  +- worker-threads/ : demo on use of worker threads
+- js-web
|  +- vue-nobundler/ : frontend (Vue3 no bundler) - See [js-web/vue-nobundler/README.md](js-web/vue-nobundler/README.md) for Project Structure
|  +- vue-vite/: frontend (Vue3 rollup) - See [js-web/vue-vite/README.md](js-web/vue-nobundler/README.md) for Project Structure
+- .editorconfig
+- .gitignore
+- BACKLOG.md
+- CHANGELOG.md
+- LICENCE
+- package.json
+- README.md

CI/CD & Cloud Deployment

Cloud Services

The following Google Cloud Platform (GCP) services are used

• Container Registry
• Cloud Run
• Cloud Storage
• Mongo Atlas (hosted on GCP)

Refer to doc/deployment/home.md for documentation on deployments

Deployment Using Github Actions

• .github/workflows/manual-gcp-expressjs.yml (Manually deploy js-node/expressjs to GCP CloudRun)
  • selectable inputs
    • environment (uat for now, development does not deploy anything)
    • service (default = app-template)
    • branch
• .github/workflows/manual-gcp-vue-vite.yml (Manually deploy js-web/vue-vite to GCP Cloud Storage)
  • selectable inputs
    • environment (uat for now, development does not deploy anything)
    • csutom_app (to be implemented)
    • branch
• .github/workflows/manual-gh-pages.yml (Manually deploy js-web/vue-vite to Github Pages)
  • selectable inputs
    • environment (uat for now, development does not deploy anything)
    • csutom_app (to be implemented)
    • branch

NOTE config/secret contents will not be in repo for CI/CD (so you can get errors), those should be put in VAULT

Current secrets

• GCP_PROJECT_ID
• GCP_SA_KEY
• VAULT_uat, passed in as VAULT

# do not merge
VAULT="unused"

# connect to a hashicorp vault and obtain secrets to merge
VAULT={ url, token } # base64 encoded

 # pass in secrets, this way is insecure and not a good way to send in secrets
VAULT={ secrets: { ... all your secrets here } } # base64 encoded

VERSION CHANGE NOTES

• v0.6+ Improve organization, graceful exit, logging, project rename, add more nodejs applications, repo name vue-crud-x changed to cookbook
• v0.5+ Improve organization and authentication, add new features
• v0.4+ Improve folders and structure organization, handle CI/CD better
• v0.3+ Reorganize folders and structure, for ease of developing and maintaining multiple applications.
• v0.2+ uses Vuetify 2. Due to breaking changes from Vuetify 1 to 2, CRUD component code was refactored to be more UI framework agnostic (reduce dependencies!), easier to use, improving code quality, documentation and supprting article - VueJS+ExpressJS CRUD & Cookbook
• v0.1 and Vuetify 1 will be supported under the v1 branch. You can refer to the v1 Legacy Article (For Historical Reference)