All Projects → hqoss → node-http-client

hqoss / node-http-client

Licence: MIT license
🔌 A light-weight, performant, composable blueprint for writing consistent and re-usable Node.js HTTP clients

Programming Languages

typescript
32286 projects
javascript
184084 projects - #8 most used programming language

Projects that are alternatives of or similar to node-http-client

ultrafetch
Node-based fetch backed with an RFC-7234 compliant filesystem cache.
Stars: ✭ 30 (+42.86%)
Mutual labels:  fetch, node-fetch
sapper-httpclient
An isomorphic http client for Sapper
Stars: ✭ 48 (+128.57%)
Mutual labels:  fetch, node-fetch
electron-request
Zero-dependency, Lightweight HTTP request client for Electron or Node.js
Stars: ✭ 45 (+114.29%)
Mutual labels:  fetch, https
node-fetch-har
Generate HAR entries for requests made with node-fetch
Stars: ✭ 23 (+9.52%)
Mutual labels:  fetch, node-fetch
wumpfetch
🚀🔗 A modern, lightweight, fast and easy to use Node.js HTTP client
Stars: ✭ 20 (-4.76%)
Mutual labels:  fetch, https
node-fetch-cookies
node-fetch wrapper that adds support for cookie-jars
Stars: ✭ 15 (-28.57%)
Mutual labels:  fetch, node-fetch
Node Fetch
A light-weight module that brings the Fetch API to Node.js
Stars: ✭ 7,176 (+34071.43%)
Mutual labels:  fetch, node-fetch
Bs Fetch
Fetch bindings for BuckleScript
Stars: ✭ 194 (+823.81%)
Mutual labels:  fetch
Vue Video
vue + vue-router + vuex + (fetch->axios)
Stars: ✭ 251 (+1095.24%)
Mutual labels:  fetch
Zl Fetch
A library that makes the Fetch API a breeze
Stars: ✭ 186 (+785.71%)
Mutual labels:  fetch
Sinn
a blog based on of react,webpack3,dva,redux,material-ui,fetch,generator,markdown,nodejs,koa2,mongoose,docker,shell,and async/await 基于react+koa2技术栈的个人开源博客系统
Stars: ✭ 175 (+733.33%)
Mutual labels:  fetch
React Redux
React+Redux 入门示例项目,完整的构建部署流程,可扫下方二维码在线查看
Stars: ✭ 198 (+842.86%)
Mutual labels:  fetch
React Fetches
🐙React Fetches a new way to make requests into your REST API's.
Stars: ✭ 253 (+1104.76%)
Mutual labels:  fetch
Wretch
A tiny wrapper built around fetch with an intuitive syntax. 🍬
Stars: ✭ 2,285 (+10780.95%)
Mutual labels:  fetch
https-aspnetcore-in-docker
ASP.NET Core app on HTTPS in Docker
Stars: ✭ 24 (+14.29%)
Mutual labels:  https
Fetchql
GraphQL client with Fetch
Stars: ✭ 178 (+747.62%)
Mutual labels:  fetch
bfetch
📠 Dynamic fetch displayer that SuperB
Stars: ✭ 114 (+442.86%)
Mutual labels:  fetch
cpp20-internet-client
An HTTP(S) client library for C++20.
Stars: ✭ 34 (+61.9%)
Mutual labels:  https
Svelte Boilerplate
Svelte application boilerplate with Webpack, Babel, PostCSS, Sass, Fetch, Jest, .Env, EsLint.
Stars: ✭ 216 (+928.57%)
Mutual labels:  fetch
Next Blog
基于react(ssr)服务端框架next.js和antd-design搭建的个人博客
Stars: ✭ 214 (+919.05%)
Mutual labels:  fetch

Node.js CI Codacy Badge Codacy Badge GuardRails badge npm

🔌 Node Http Client

A light-weight, performant, composable blueprint for writing consistent and re-usable Node.js HTTP clients.

Extends node-fetch, therefore 100% compatible with the underlying APIs.

Table of contents

🤔 Why use http-client

... as opposed to request or node-fetch?

  • request is/was great, but it has entered maintenance mode.
  • Both node-fetch and request are relatively low-level (in JavaScript terms) implementations and as such lack certain convenience methods/APIs that help design maintainable and consistent HTTP clients. This is especially true in the microservices architecture context, where consistency is paramount.

http-client builds on node-fetch to enable composable and re-usable HTTP client implementations.

  • Enforces a consistent approach to writing HTTP clients.

  • Greatly reduces common boilerplate, expressly

    • authentication,
    • default headers,
    • default options,
    • composing urls,
    • connection pooling,
    • parsing responses, and more.
  • It is written in TypeScript.

Install

npm install @hqoss/http-client
# Additionally, for TypeScript users
npm install @types/node-fetch --save-dev

⚠️ NOTE: The project is configured to target ES2018 and the library uses commonjs module resolution. Read more in the Node version support section.

📝 Usage

SDK-like HTTP client

Let's take a look at how we build a simple SDK-like HTTP Client.

import { HttpClient, Header, RequestInterceptor, ResponseTransformer } from "@hqoss/http-client";

import type { CreateIssueArgs } from "./types";

class GitHubClient extends HttpClient {
  constructor() {
    super({
      baseUrl: "https://api.github.com/",
      baseHeaders: { [Header.Authorization]: `token ${process.env.GITHUB_TOKEN}` },
      baseOptions: { timeout: 2500 },
      // Automatically includes `accept: application/json` and 
      // `content-type: application/json` headers and parses responses to json.
      json: true,
    });
  }

  // Inspired by Apollo's REST Data Source, this lifecycle method
  // can be used to perform useful actions before a request is sent.
  protected willSendRequest: RequestInterceptor = (url, _request) => {
    console.info(`Outgoing request to ${url}`);
  };

  // Mmimics the default behaviour of request, e.g. non-ok responses
  // are rejected rather than resolved.
  protected transformResponse: ResponseTransformer = async (response) => {
    // You need to be careful with 204 No Content, please consider
    // using our pre-built `jsonResponseTransformer` here instead.
    const jsonResponse = await response.json();

    if (response.ok) {
      return jsonResponse;
    } else {
      throw jsonResponse;
    }
  };

  // See https://developer.github.com/v3/issues/#create-an-issue.
  createIssue = ({ ownerId, repoId, ...args }: CreateIssueArgs) =>
    this.post<{ id: string; }>(
      `/repos/${ownerId}/${repoId}/issues`,
      args,
      { timeout: 5000 },
    );

  // See https://developer.github.com/v3/orgs/#get-an-organization.
  getOrganisationById = (organisationId: string) =>
    this.get<{ id: string; name: string; /* ... and more. */ }>(
      `/orgs/${organisationId}`,
      { headers: { [Header.Accept]: "application/vnd.github.surtur-preview+json" } }
    )

}

export default GitHubClient;

Then, in your application(s):

// Initiate the client.
const gitHub = new GitHubClient();

// Use clean pre-configured APIs to perform actions.
const { id } = await gitHub.createIssue({ ownerId: "foo", repoId: "bar", title: "New bug!" });

const { id: orgId, name: orgName } = await gitHub.getOrganisationById("foobar");

Advanced example

When it comes to distributed systems, visibility is hugely important. We can leverage the SDK-like design approach further to ensure we maintain a consistent approach to logging, error handling, as well as code structure.

First, let's hook up to the request lifecycle and log the events we care about.

import {
  Header,
  HttpClient,
  jsonResponseTransformer,
  RequestInterceptor,
  ResponseTransformer,
} from "@hqoss/http-client";
import type { PinoLogger } from "@hqoss/logger";
import { pick } from "lodash";

import { BaseRequestContext } from "../types";

class UsersService extends HttpClient {
  private readonly log: PinoLogger;

  // Suppose each incoming request (from outside) creates and maintains
  // its unique context which carries its own instance of a logger,
  // the request headers, and other useful request data.
  constructor({ log, headers }: BaseRequestContext) {
    super({
      baseUrl: "http://s-users/",
      baseHeaders: pick(headers, [Header.Authorization, Header.IdToken, Header.CorrelationId]),
      json: true,
    });

    this.log = log;
  }

  protected willSendRequest: RequestInterceptor = (url, request) => {
    const { log } = this;
    const { headers } = request;

    log.debug(`Outgoing request to ${url}`);

    if (!(Header.CorrelationId in headers)) {
      log.warn(`Missing ${Header.CorrelationId} header`);
    }
  };

  protected transformResponse: ResponseTransformer = async (response) => {
    const { log } = this;
    const { ok, status, statusText } = response;

    const jsonResponse = await jsonResponseTransformer();

    log.debug(`Received response ${status} ${statusText}`);

    if (ok) {
      return jsonResponse;
    } else {
      log.error(jsonResponse);

      throw jsonResponse;
    }
  };

  // ... define APIs as shown in previous examples.

}

Then, we make sure we construct our client(s) and pass in a unique logger instance with the correct correlation id passed in as metadata. We can simply attach the resulting context to the request object itself, making it available in all subsequent request handlers.

import { PinoLogger } from "@hqoss/logger";

import { UsersService } from "./httpClients";
import type { BaseRequestContext, RequestContext } from "./types";

// ... initialise express, basic middleware etc.

// Build a unique context for every request.
app.use((req, res, next) => {
  const { headers } = req;

  // Get or generate a request correlation id.
  const correlationId = headers[Header.CorrelationId] || generateUUID();

  // Always send the correlation id back to the client.
  res.setHeader(Header.CorrelationId, correlationId);

  // Initialise the logger, each log within this request will
  // carry the same correlaiton id to make tracing easy.
  const log = new PinoLogger({ correlationId });

  // Construct base request context.
  const baseCtx: BaseRequestContext = {
    correlationId,
    headers,
    log,
  };

  // Construct HTTP Clients.
  const clients = {
    users: new UsersService(baseCtx),
    // ... define the rest here.
  };

  // Construct full request context and assign it to the request.
  const ctx: RequestContext = {
    ...baseCtx,
    clients,
    // ... add services, etc. here.
  };

  // Makes `ctx` available in all subsequent handlers.
  Object.assign(req, { ctx });

  next();
});

// ... other server setup, routing, etc.

Finally, we can use our client(s) in the handlers through accessing ctx.

import type { Request } from "express";

import type { RequestContext } from "../types";

app.get("/users/:userId", async (req, res, next) => {
  const {
    ctx: { clients },
    params: { userId },
  } = req as Request & { ctx: RequestContext };

  try {
    const user = await clients.users.getUser(userId);

    if (user) {
      res.status(200).send(user);
    } else {
      res.sendStatus(404);
    }
  } catch (error) {
    next(error);
  }
});

Gotchas and useful know-how

  1. json mode is not the default. It needs to be enabled explicitly in the constructor.
import { HttpClient } from "@hqoss/http-client";

class UsersService extends HttpClient {
  constructor() {
    super({ baseUrl: "http://s-users/", json: true });
  }
}
  1. Non-ok responses are not rejected by default. You can mimic this behaviour in the transformResponse lifecycle method.
import { HttpClient, jsonResponseTransformer, ResponseTransformer } from "@hqoss/http-client";

class UsersService extends HttpClient {
  constructor() {
    // You still need `json: true` to let the client know what request
    // headers to configure.
    super({ baseUrl: "http://s-users/", json: true });
  }

  // Mmimics the default behaviour of request, e.g. non-ok responses
  // are rejected rather than resolved.
  protected transformResponse: ResponseTransformer = async (response) => {
    // You can also simply `await response.json()`, however that does not
    // guarantee correctly handling 204 No Content responses.
    const jsonResponse = await jsonResponseTransformer();

    if (response.ok) {
      return jsonResponse;
    } else {
      throw jsonResponse;
    }
  };
}

API Docs

See full API Documentation here.

⚠️ WARNING: Unlike request, http-client (using node-fetch under the hood) does NOT reject non-ok responses by default as per the whatwg spec. You can, however, mimic this behaviour with a custom responseTransformer (see example above).

⚡️ Performance

We ship the default HttpClient with a pre-configured (Node.js) Agent, which may lead to a huge increase in throughput.

For reference, we performed a number of benchmarks comparing the out-of-the-box request, node-fetch, and http-client clients. To fetch a list of 100 users from one service to another (see diagram below), these were the results:

| wrk | -HTTP-> | Server A -> HttpClient | -HTTP-> | Server B -> data in memory |
  • Default request setup (used by most projects): 10,893 requests in 30.08s; 362.19 requests/sec
  • Default node-fetch setup (used by many projects): 8,632 requests in 30.08s; 286.98 requests/sec
  • Default http-client setup: 71,359 requests in 30.10s; 2,370.72 requests/sec

Please note that these benchmarks were run through wrk, each lasting 30 seconds, using 5 threads and keeping 500 connections open.

This is the default Agent configuration, which can easily be overriden in the HttpClient constructor. You can simply provide your own Agent instance in baseOptions.

const opts = {
  keepAlive: true,
  maxSockets: 64,
  keepAliveMsecs: 5000,
};

Core design principles

  • Code quality; This package may end up being used in mission-critical software, so it's important that the code is performant, secure, and battle-tested.

  • Developer experience; Developers must be able to use this package with no significant barriers to entry. It has to be easy-to-find, well-documented, and pleasant to use.

  • Modularity & Configurability; It's important that users can compose and easily change the ways in which they consume and work with this package.

Node version support

The project is configured to target ES2018. In practice, this means consumers should run on Node 12 or higher, unless additional compilation/transpilation steps are in place to ensure compatibility with the target runtime.

Please see https://node.green/#ES2018 for reference.

Why ES2018

Firstly, according to the official Node release schedule, Node 12.x entered LTS on 2019-10-21 and is scheduled to enter Maintenance on 2020-10-20. With the End-of-Life scheduled for April 2022, we are confident that most users will now be running 12.x or higher.

Secondly, the 7.3 release of V8 (ships with Node 12.x or higher) includes "zero-cost async stack traces".

From the release notes:

We are turning on the --async-stack-traces flag by default. Zero-cost async stack traces make it easier to diagnose problems in production with heavily asynchronous code, as the error.stack property that is usually sent to log files/services now provides more insight into what caused the problem.

Testing

Ava and Jest were considered. Jest was chosen as it is very easy to configure and includes most of the features we need out-of-the-box.

Further investigation will be launched in foreseeable future to consider moving to Ava.

We prefer using Nock over mocking.

TODO

A quick and dirty tech debt tracker before we move to Issues.

  • Write a Contributing guide
  • Complete testing section, add best practices
  • Describe scripts and usage, add best practices
  • Add typespec and generate docs
  • Describe security best practices, e.g. npm doctor, npm audit, npm outdated, ignore-scripts in .npmrc, etc.
  • Add "Why should I use this" section
  • Implement and document support for basic auth
  • Document willSendRequest and reponseTransformer
  • Library architectural design (+ diagram?)
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].