All Projects → mitmplay → mitm-play

mitmplay / mitm-play

Licence: AGPL-3.0 License
Man in the middle using Playwright

Programming Languages

javascript
184084 projects - #8 most used programming language
Svelte
593 projects
CSS
56736 projects

Projects that are alternatives of or similar to mitm-play

Recorder
A browser extension that generates Cypress, Playwright and Puppeteer test scripts from your interactions 🖱 ⌨
Stars: ✭ 277 (+2030.77%)
Mutual labels:  firefox, chromium, puppeteer, playwright
Qawolf
🐺 Create browser tests 10x faster
Stars: ✭ 2,912 (+22300%)
Mutual labels:  firefox, webkit, puppeteer, playwright
Secret Agent
The web browser that's built for scraping.
Stars: ✭ 151 (+1061.54%)
Mutual labels:  mitm, chromium, mitmproxy, puppeteer
Playwright Python
Python version of the Playwright testing and automation library.
Stars: ✭ 5,583 (+42846.15%)
Mutual labels:  firefox, chromium, webkit, playwright
Playwright
Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
Stars: ✭ 31,513 (+242307.69%)
Mutual labels:  firefox, chromium, webkit, playwright
playwright-demos
playwright for scrapping and UI testing / automate testing workflows
Stars: ✭ 65 (+400%)
Mutual labels:  chromium, webkit, puppeteer, playwright
mps
MPS is a high-performance HTTP(S) proxy library that supports forward proxies, reverse proxies, man-in-the-middle proxies, tunnel proxies, Websocket proxies. MPS 是一个高性能HTTP(s)中间代理库,它支持正向代理、反向代理、中间人代理、隧道代理、Websocket代理
Stars: ✭ 64 (+392.31%)
Mutual labels:  mitm, man-in-the-middle, mitmproxy
Pokemon Go Mitm
🎁 Pokemon Go MITM Proxy - Intercepts the traffic between your Pokemon Go app and their servers, decodes the protocol and gives you a handy tool to enrich your own game experience on the fly.
Stars: ✭ 475 (+3553.85%)
Mutual labels:  mitm, man-in-the-middle, mitmproxy
Proxy.py
⚡⚡⚡Fast, Lightweight, Pluggable, TLS interception capable proxy server focused on Network monitoring, controls & Application development, testing, debugging
Stars: ✭ 1,291 (+9830.77%)
Mutual labels:  mitm, man-in-the-middle, mitmproxy
Playwright Go
Playwright for Go a browser automation library to control Chromium, Firefox and WebKit with a single API.
Stars: ✭ 272 (+1992.31%)
Mutual labels:  firefox, chromium, webkit
Playwright Sharp
.NET version of the Playwright testing and automation library.
Stars: ✭ 459 (+3430.77%)
Mutual labels:  firefox, chromium, webkit
LInkedIn-Reverese-Lookup
🔎Search LinkedIn profile by email address📧
Stars: ✭ 20 (+53.85%)
Mutual labels:  chromium, puppeteer
ESP-Bug
ESP8266 based WiFi implant to remotely track the presence of certain people or devices via a simple web interface
Stars: ✭ 78 (+500%)
Mutual labels:  mitm, man-in-the-middle
browser-automation-api
Browser automation API for repetitive web-based tasks, with a friendly user interface. You can use it to scrape content or do many other things like capture a screenshot, generate pdf, extract content or execute custom Puppeteer, Playwright functions.
Stars: ✭ 24 (+84.62%)
Mutual labels:  puppeteer, playwright
simplechrome
Webrecorders DevTools Protocol Automation Library
Stars: ✭ 16 (+23.08%)
Mutual labels:  chromium, puppeteer
throughout
🎪 End-to-end testing made simple (using Jest and Puppeteer)
Stars: ✭ 16 (+23.08%)
Mutual labels:  chromium, puppeteer
browser-pool
A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.
Stars: ✭ 71 (+446.15%)
Mutual labels:  puppeteer, playwright
FlareSolverrSharp
FlareSolverr .Net / Proxy server to bypass Cloudflare protection
Stars: ✭ 62 (+376.92%)
Mutual labels:  chromium, puppeteer
mugshot
Framework independent visual testing library
Stars: ✭ 126 (+869.23%)
Mutual labels:  puppeteer, playwright
PassivesScan
《被动扫描资源汇总》
Stars: ✭ 27 (+107.69%)
Mutual labels:  mitm, mitmproxy

Man in the middle

Using Playwright to intercept traffic in between and do lots of stuff for Developer to exercise

mitm-play in action

mitm-play

Installation

npm install -g mitm-play

Execute mitm-play command with demo route, or add -h to see help screen:

mitm-play -Gdr
Example
// create file: ~/user-route/keybr.com/index.js & add this content:
const css = `
.Body-header,.Body-aside {
  display: none !important;
}`;
 
const route = {
  url: 'https://keybr.com',
  tags: [],
  'mock:no-ads': {
    'doubleclick.net': '',
    'a.pub.network': '',
    'google.+.com': '',
  },
  css: {
    'GET:no-ads/assets/[a-z0-9]+': `=>${css}`
  },
}
module.exports = route;
// create file: ~/user-route/_global_/index.js & add this content:
const route = {
  tags: [],
  'config:no-logs': {
    logs: {
      'referer-reqs': false,
      'no-namespace': false,
    }
  }
}
module.exports = route;
# 1st run will be to save all cli option to 'default'
mitm-play keyb --delete --save  # --OR--
mitm-play keyb -ds

# next run should be simple as:
mitm-play

Routing definition having remove-ads tag, it will be shown on chrome dev-tools "mitm-play" "tags" as an option to enabled / disabled rules. You can see the togling process on this video.

Features

Feature payload note
screenshot ---------- DOM specific rules for taking screenshot
noproxy ---------- array ..of [domain] - will serve directly
proxy ---------- array ..of [domain] - will serve using proxy
noskip ---------- array ..of [domain] - forces to noskip
skip ---------- array ..of [domain] - browser will handle it
request request modify reqs object - call to remote server
mock response mock resp object - no call to remote server
cache response 1st call save to local - next call, read from cache
log response save/log reqs/resp to local - call to remote server
response modify resp based on contentType - call remote server
=>> * html - response handler (replace / update + JS + ws)
=>> * json - response handler (replace / update)
=>> * css - response handler (replace / update)
=>> * js - response handler (replace / update)
response response modify resp object - call to remote server

Concept

Mitm intercept is hierarchical checking routes.

First check is to match domain on the url with route-folder as a domain namespace.

Next check is to match full-url with regex-routing of each section/rule. the regex-routing having two type:

  • An Array [ nosocket, nonproxy, proxy, noskip, skip ]
  • Object Key:
    1. General [ request, mock, cache, log, response ]
    2. Specific to content-type [ html, json, css, js ]

if match, then based on the route section / rules meaning, the next process can be carry over, detail explanations will be on the title of: "Route Section".

Structure Object of Routes:
/**
 * Folder structure:
 * = user-route       // folder
 * \=== abc.com       // folder
 *    |--- index.js   // file
 *    \--- index.json // autogenerated file
*/
{
  'abc.com': { // route-folder mapped to object as namespace
    request: { // sections can be: skip, proxy, <etc...> 
      '/assets/main.js': {     // regex-routing
        request(reqs, match) { // handler 
          const {body} = reqs;
          ...
          return {body}
        }
      }
    }
  }
}

If the process of checking is not match, then it will fallback to _global_ namespace for checking, and the operation is the same as mention in above paragraph: 'Next check...'.

Usually html page load with several assets (image, js & css) that not belong to the same domain, and to match those type of assets, it use browser headers attributes: origin or referer, in which will scoping to the same namespace.

Object & function

Detail structure of Object and Function shared accros Section

Objects

match
/**
 * match: {
 *   tags       : {},
 *   route      : {}, 
 *   contentType: {}, 
 *   workspace  : '',/undefined,
 *   namespace  : '', 
 *   pathname   : '', 
 *   hidden     : true,/false 
 *   search     : '',
 *   host       : '',
 *   arr        : [],
 *   url        : '',
 *   key        : '',
 *   log        : '',
 *   typ        : ''
 * }
*/
request
/**
 * reqs/request: {
 *   url        : '',
 *   method     : 'GET',/PUT/POST/DELETE 
 *   headers    : {}, 
 *   oriRef     : '',
 *   body       : '',/null,
 *   browserName: 'chromium',/webkit/firefox
 * }
*/
response
/**
 * resp/response: {
 *   url    : '',
 *   status : 200,/302/400/500/etc.. 
 *   headers: {},
 *   body   : '',
 * }
*/

Functions

file(reqs, match)
/**
 * arguments:
 * - <reqs: object>
 * - <match: object>
 * 
 * return: <filename: string>/false
 * 
 * False value indicate skiping rule
*/
file(reqs, match) {
  match.path = 'some/path' // superseded match.route.path
  ...
  return 'common.js'; //return {path: 'some/path', file: 'common.js'}
},
request(reqs, match)
/**
 * arguments:
 * - <reqs: object>
 * - <match: object>
 * 
 * return: <reqs: object>
*/
request(reqs, match) {
  const {headers} = reqs;
  headers['new-header'] = 'with some value';
  ...
  return {headers};
},
response(resp, reqs, match)
/**
 * arguments:
 * - <resp: object>
 * - <reqs: object>
 * - <match: object>
 * 
 * return: <resp: object>
*/
response(reqs, reqs, match) {
  const {headers} = reqs;
  headers['new-header'] = 'with some value';
  ...
  return {headers};
},

Route Section

on each route you can add section supported:

Skeleton
route = {
  url:     '',
  urls:    {},
  title:   '',
  jsLib:   [],
  workspace: '',
  screenshot: {}, //user interaction rules & DOM-element observer
  nosocket:[],
  noproxy: [], 
  proxy:   [], //request with proxy
  noskip:  [], //start routing rules
  skip:    [],
  request: {},
  mock:    {}, 
  cache:   {},
  response:{},
  html:    {},
  json:    {},
  css:     {},
  js:      {},
  log:     {}, //end routing rules
}

the execution order as documented start with `skip`, end with `response`, no need to implement all of routing rules.

Title, url, urls, workspace & jsLib

Title: provide basic information about this route.

Url: when user enter cli with 1st args, it will try to find in url, then open the browser with that location.

Urls: additional search urls key, the 1st args can be split by [,], if find more than one, it will open multi tabs.

workspace: will be use as the base folder for file option in Mock and Cache.

lib: inject js library into html which having websocket, it can be [jquery.js, faker.js, chance.js, log-patch.js]

route = {
  title: 'Amazon - amazon',
  url:  'https://www.amazon.com/b?node=229189',
  urls: {
    ama1: 'https://www.amazon.com/b?node=229100',
    ama2: 'https://www.amazon.com/b?node=229111',
  },
  workspace: '~/Projects',
  jsLib: ['chance.js'],
};
// cli: mitm-play ama -dpsr='.' 
// search: 'ama' and it will open two browser tabs
Screenshot

Capture/Screeshot when user click specific DOM-Element match with selector or state-change, like DOM-Element getting insert or remove and match selector inside observer key.

Below example show three selector in observer:

  • '.field.error' -> filename: field-error -> state: insert or remove
  • '.input.focus' -> filename: input -> state: insert or remove
  • '.panel.error' -> filename: panel-error -> state: insert

Caveat: observer is an experimental feature, take it as a grain salt, expectation of selector should be like toggling and it need a unique match to the DOM-Element, please do test on chrome-devtools before reporting a bug.

Caveat 2: this Screenshot rule(s), required successful injection of websocket client to html document, if it not success (error can be seen on chrome dev-tools),might be content-security-policy restriction.

Caveat 3: process screenshot sometime take times and for SPA, transition between page usually instantly and it lead to capturing next page, even if the trigger come from button in previouse page, there is a CLI option: -z/--lazy to delay click action for about ~400ms

screenshot: {
  selector: '[type=button],[type=submit],button,a', //click event
  observer: {
    /***
     * selector must be uniq, represent not in the dom 
     * state change couse element tobe insert or remove,
     * or can be just class change 
    */
    '.field.error': 'field-error:insert,remove',
    '.input.focus': 'input:insert,remove',
    '.panel.error': 'panel-error:insert',
  },
  at: 'sshot', //'^sshot' part of log filename
},

at is a partion of filename and having a simple rule attach on it. Guess what is it?.

Nosocket

No WebSocket Injection to html, mitm-play will process further.

nosocket: ['sso'],
Noproxy

if proxy config was set to all request/response, noproxy will exclude it from proxy. Example below will set domain nytimes.com with direct access and the rest will go thru proxy.

// HTTP_PROXY env need to be set, cli: --proxy .. --noproxy ..
noproxy: ['nytimes.com'],
proxy:   ['.+'],
Proxy

Certain domain will go thru proxy, proxy & noproxy will make sanse if command line contains -x/--proxy

// HTTP_PROXY env need to be set, cli: --proxy ..
proxy: [
  'google-analytics.com',
],
Noskip

Forces to some domains not to be skip it

noskip: ['wp-admin'],
skip  : ['.+'], // put it in on global routes
Skip

Skipping back url to the browser if partion of url match text in array of skip section, mitm-play will not process further.

skip: ['.+'],
Request

Manipulate request with request function

request: {
  'GET:/disqus.com/embed/comments/': {
    request(reqs, match) {
      const {headers} = reqs;
      headers['new-header'] = 'with some value';
      ...
      return {headers};
    },
    session: true, // optional - set session id
  }
},
Mock

Mock the response.

Basic rule:

Replace response body with the matcher value

mock: {'2mdn.net': ''},

Manipulate response with response function

mock: {
  'mitm-play/twitter.js': {
    file(reqs, match) {
      match.path = 'some/path' // superseded match.route.path
      ...
      return 'filename' //return {path: 'some/path', file: 'filename'}
    },
    response(resp, reqs, match) {
      const {body} = resp
      ...
      return {body} // {status, headers, body} or false to skip
    },
    log: true, // optional - enable logging
    ws: true,  // inject web socket (html)
  },
},

Below is the logic of file getting translate combine with path or workspace, if workspace exists, and file value not start with dot(.), it will use workspace (ie: ${workspace}/${file}) and the path will be ignore.

mock: {
  'mitm-play/twitter.js': {
    file: 'relative/to/workspace/file.html', // --OR--
    // file: '../relative/to/route/folder/file.html',
    // file: './relative/to/route/folder/file.html',
    // file: '~/relative/to/home/folder/file.html',
    // file: (reqs, match) => 'filename'
  },
},

Concatenation of JS code js at the end of the mock body

const unregisterJS = () => {
  ...
  console.log('unregister service worker')
};

mock: {
  'mitm-play/twitter.js': {
    js: [unregisterJS],
  },
},

If both options are defined: response, js, js will be ignored.

Cache

Save the first request to your local disk so next request will serve from there.

cache: {
  'amazon.com': {
    contentType: ['javascript', 'image'], //required!
    querystring: true, // hash of unique file-cache
    hidden: true,      // optional - no consolo.log
    log: true,         // optional - enable logging
    path: './api',     // optional cache file-path
    file: ':1.png',    // optional cache file-name
    tags: 'js-img',    // optional route by tags
    at: 'mycache',     // part of filename
  }
},

logic for file is the same as in mock, if workspace exists and file value not start with dot(.), it will use workspace (ie: ${workspace}/${file}) and the path will be ignore.

cache: {
  'amazon.com': {
    file: 'relative/to/workspace/file.html', // --OR--
    // file: '../relative/to/route/folder/file.html',
    // file: './relative/to/route/folder/file.html',
    // file: '~/relative/to/home/folder/file.html',
    // file: (reqs, match) => 'filename'
  },
},

cache support response function, it means the result can be manipulate first before send to the browser.

cache: {
  'amazon.com': {
    contentType: ['json'], //required! 
    response(resp, reqs, match) {
      const {body} = resp;
      ...
      return {body} // {status, headers, body} or false to skip
    },    
  }
},
Response

Manipulate response with response function

response: {
  '.+': {
    response(resp) {
      const {headers} = resp;
      headers['new-header'] = 'with some value';
      ...
      return {headers};
    },
    tags: 'all-response',
  }
},
Html

Manipulate the response.

Basic rule:

Replace response body with some value

html: {'twitter.net': ''},

Insert js script element into specific area in html document:

  • el: 'head'   // default, no need to add el key
  • el: 'body'
html: {
  'https://keybr.com/': {
    // el: 'head', // JS at <head> 
    js: [()=>console.log('Injected on Head')],
  },
},

Insert <script src="..."></script> into <head> section

html: {
  'https://keybr.com/': {
    src: ['http://localhost:/myscript.js'],
    ws: true, // inject web socket
  },
},

Manipulate response with response function

html: {
  'https://keybr.com/': {
    response(resp, reqs, match) {
      const {body} = resp;
      ....
      return {body} // {status, headers, body} or false to skip
    },
    tags: 'response' // enable/disable route by tags
    hidden: true, // optional - no consolo.log
  },
},
Json

Manipulate the response.

Basic rule:

Replace response body with some value

json: {'twitter.net': '{}'},

Manipulate response with response function

json: {
  'twitter.com/home': {
    response(resp, reqs, match) {
      const {body} = resp;
      ....
      return {body} // {status, headers, body} or false to skip
    },
    tags: 'json-manipulate',
  },
},
Css

Manipulate the response.

Basic rule:

Replace response body with some value -or- add to the end of response body by adding FAT arrow syntax =>${style}

const style = 'body: {color: red}';
...
css: {'twitter.net': style}, //or `=>${style}`

Manipulate response with response function

css: {
  'twitter.com/home': {
    response(resp, reqs, match) {
      const {body} = resp;
      ....
      return {body} // {status, headers, body} or false to skip
    },
    tags: 'css-manipulate',
  },
},
Js

Manipulate the response.

Basic rule:

Replace response body with some value -or- add to the end of response body by adding FAT arrow syntax =>${style}

const code = 'alert(0);'
...
js: {'twitter.net': code}, //or `=>${code}`

Manipulate response with response function

js: {
  'twitter.com/home': {
    response(resp, reqs, match) {
      const {body} = resp;
      ....
      return {body} // {status, headers, body} or false to skip
    },
    tags: 'js-manipulate',
  },
},
Log

Save the response to your local disk. by default contentType json will log complete request / response, for different type default log should be response payload.

Special usacase like google-analytic will send contentType of gif with [GET] request, and response payload is not needed, there is an option log to force log with json complete request / response.

log: {
  'amazon.com': {
    contentType: ['json'],
    tags: 'json-bo' // optional route by tags
    at: 'myjson',   // part of log filename
  },
  'google-analytics.com/collect': {
    contentType: ['gif'],
    log: true,      // '<remove>'
  }
},

log support response function, it means the result can be manipulate first before send to the browser or save to logs file.

log: {
  'amazon.com': {
    contentType: ['json'], //required! 
    response(resp, reqs, match) {
      const {body} = resp;
      ...
      return {body} // {status, headers, body} or false to skip
    },
  }
},

_global_ Route

A special route to handle global scope (without namespace)

Common route rules
_global_ = {
  jsLib:   [],
  skip:    [], //start routing rules
  proxy:   [], //request with proxy
  noproxy: [], 
  nosocket:[],
  request: {},
  mock:    {}, 
  cache:   {},
  log:     {},
  html:    {},
  json:    {},
  css:     {},
  js:      {},
  response:{}, //end routing rules
}
Args & flag

Two additional Section only appear in _global_

args, flag and it can be served as a section-tags

_global_ = {
  args: { // part of cli options
    activity,  // rec/replay cache activity*
    cookie,    // reset cookies expire date*
    fullog,    // show detail logs on each rule*
    lazyclick, // delay ~700ms click action*
    nosocket,  // no websocket injection to html page*
    nohost,    // set logs without host name*
    nourl,     // set logs without URL*
    csp,       // relax CSP unblock websocket*
  }
}
_global_ = {
  flag: { // toggle to show/hide from console.log()
    'referer-reqs': true,
    'no-namespace': true,
    'ws-broadcast': false, // true if --verbose/--debug
    'ws-connect': false,   // true if --verbose/--debug
    'ws-message': false,   // true if --verbose/--debug
    'frame-load': false,   // true if --verbose/--debug
    'page-load': false,    // true if --verbose/--debug
    'mitm-mock': false,    // true if --verbose/--debug
    'file-log': false,     // true if --verbose/--debug
    'file-md': false,      // true if --verbose/--debug  
    silent:   false,       // true: hide all
    skip:     false,
    nosocket: true,
    request:  true,
    mock:     true,
    cache:    true,
    log:      true,
    html:     true,
    json:     true,
    css:      true,
    js:       true,
    response: true,
  }
}

~/.mitm-play

By default all save file are on the ~/.mitm-play profile folder.

HTTP_PROXY

mitm-play support env variable HTTP_PROXY and NO_PROXY if your system required proxy to access internet. Please check on CLI Options > -x --proxy section for detail explanation.

CLI Options

when entering CLI commands, mitm-play support two kind of arguments:

mitm-play [args] [-options]
  • args:
    • 1st for searching url/urls
    • 2nd for loading profile
  • options.
# syntax
$ mitm-play [args] [-options]

# create 'secure' profile with -s/--save option # OR
$ mitm-play yahoo --lazyclick --incognito -s='secure'
$ mitm-play yahoo -zts='secure'

# search yahoo route and use 'secure' profile & add -k/--cookie option 
$ mitm-play yahoo secure -k

# if no profile, fallback to 'default'
$ mitm-play yahoo --cookie
-h --help

To show all the options Command Line Interface (CLI). this option can be arbitrary position on cli, the result should be always display this messages:

$ mitm-play -h  <OR>
$ mitm-play --help

  Usage: mitm-play [args] [options]

  args:
    1st for searching url/urls
    2nd for loading profile

  options:
    -h --help          show this help
    -u --url           go to specific url
    -s --save          save as default <profl>
    -r --route         userscript folder routes
    -a --activity      rec/replay cache activity*
    -b --basic         login to http authentication
    -c --clear         clear/delete cache & log(s)
    -d --devtools      show chrome devtools on start
    -e --device        resize to mobile screen device
    -f --fullog        show detail logs on each rule* 
    -i --insecure      accept insecure cert in nodejs env 
    -n --nosocket      no websocket injection to html page*
    -o --offline       console log withount new-line
    -k --cookie        reset cookies expire date*
    -l --lazylog       delay ~500ms print console.log
    -g --group         create cache group/rec
    -p --csp           relax CSP unblock websocket*
    -t --incognito     set chromium incognito
    -w --worker        enable service worker
    -x --proxy         a proxy request
    -z --lazyclick     delay ~700ms click action*

    -D --debug         show ws messages
    -G --nogpu         set chromium without GPU
    -H --nohost        set logs without host name*
    -K --dark          set chrome devtools to dark mode
    -L --showsql       show sqlite generated commands 
    -N --nice          JSON cache save as human readable
    -R --redirect      set redirection: true/false/manual
    -Q --nosql         disabling persist data using sqlite
    -S --session       sqlite session from requst header
    -U --nourl         set logs without URL*
    -V --verbose       show more detail of console log
    -X --proxypac      set chromium proxypac

    -C --chromium      run chromium browser
    -F --firefox       run firefox browser
    -W --webkit        run webkit browser

  * _global_.config.args
    
  v0.9.xxx
-u --url

Open Browser to specific URL

$ mitm-play -u='https://google.com'  <OR>
$ mitm-play --url='https://google.com'
-s --save

Save CLI options with default or named so later time you don't need to type long CLI options

$ mitm-play -s  <OR>
$ mitm-play --save
  <OR>
$ mitm-play -s='google'  <OR>
$ mitm-play --save='google'
-r --route

Specify which folder contains routes config

$ mitm-play -r='../user-route'  <OR>
$ mitm-play --route='../user-route'
-a --activity

Flag the caching with sequences, they are three mode of activity:

  • rec:activity to record cache w/ seq, all cache always recorded
  • mix:activity to record cache w/ seq, non seq behave as std cache
  • play:activity to replay cache w/ seq, non seq behave as std cache

Tag activity need to be add to html - rule to indicate the point when sequences cached will be start.

$ mitm-play -a='rec:activity'  <OR>
$ mitm-play --activity='rec:activity'

The first step is to record the flow and do the navigation

$ mitm-play -a='rec:activity'

Next step is to replay the flow

$ mitm-play -a='play:activity'
-b --basic

When page required HTTP Authentication, this parameters will be passs to the newly created Page Context with login and password supplied to this params

$ mitm-play -b='MYCREAD:MYPASSWORD'  <OR>
$ mitm-play --basic='MYCREAD:MYPASSWORD'
-c --clear

Delete logs or cache, can be all or specific one

$ mitm-play -c  <OR>
$ mitm-play --clear
  <OR>
$ mitm-play -c='log'  <OR>
$ mitm-play --clear='log'
  <OR>
$ mitm-play -c='cache'  <OR>
$ mitm-play --clear='cache'
-d --devtools

Show chrome devtools on start up on ech tabs

$ mitm-play -d  <OR>
$ mitm-play --devtools
-e --device

Resize screen to specific mobile device (still buggy)

$ mitm-play -e  <OR>
$ mitm-play --device
  <OR>
$ mitm-play -e='iPhone 11 Pro'  <OR>
$ mitm-play --device='iPhone 11 Pro'
-i --insecure

Set NodeJS to operate within insecure / no https checking

$ mitm-play -i  <OR>
$ mitm-play --insecure
-n --nosocket

If only the params with no value, it will act as No Injection on HTML Page, meaning no open websocket on the page

$ mitm-play -n  <OR>
$ mitm-play --nosocket

if params contain value off ie: -n=off, there will be Injection into HTML Page with no open websocket connection, this options is to get alternative for macros automation tobe send via [POST] request.

$ mitm-play -n=off  <OR>
$ mitm-play --nosocket=off
-o --offline

change console.log to print the logs only when the log-message is unique from the previous log

$ mitm-play -o  <OR>
$ mitm-play --offline
-k --cookie

Set proper cache retriver with an update expiry of the cookies

$ mitm-play -k  <OR>
$ mitm-play --cookie
-l --lazylog

Delay console log ~500ms or you can provide value in milisecond.

$ mitm-play -l  <OR>
$ mitm-play --lazylog
  <OR>
$ mitm-play -l=400  <OR>
$ mitm-play --lazylog=400
-g --group

Add group name to file cache/logs, if necessary when large capturing is done and difficult to check the files.

There is an option at on the rules of cache/log for additional filename grouping path.

$ mitm-play -g='mygroup'  <OR>
$ mitm-play --group='mygroup'
-t --incognito

By Default program will run in normal browser, adding this option will result in Incognito mode.

$ mitm-play -t  <OR>
$ mitm-play --incognito
-w --worker

enable service worker, current release playwirght cannot intercept request that came from service worker.

$ mitm-play -w  <OR>
$ mitm-play --worker
-x --proxy

Some traffict with domain match to proxy section will use proxy.

this option serving two kind of needs:

  1. if --proxy without value, mitm-play traffict will get thru proxy. Proxy configuration will get from ENV variable.
  2. if --proxy with string domain, all (mitm-play or browser) traffict will get thru proxy. (ie: --proxy='http://username:[email protected]')
$ mitm-play -x  <OR>
$ mitm-play --proxy
  <OR>
$ mitm-play -x='http://username:[email protected]'  <OR>
$ mitm-play --proxy='http://username:[email protected]'
-z --lazyclick

Delay click action ~700ms or you can provide value in milisecond, to provide enough time for screenshot to be taken

$ mitm-play -z  <OR>
$ mitm-play --lazyclick
  <OR>
$ mitm-play -z=400  <OR>
$ mitm-play --lazyclick=400

--csp

Update CSP header on Html Page injected with wws-client.js to unblock Websocket communication

$ mitm-play --csp
-D --debug

More information will be shown in console.log (ex: websocket), including info from DEBUG=pw:api

$ mitm-play -D  <OR>
$ mitm-play --debug
-G --nogppu

Necessary option for Macbook owner.

Options can be added with value -G=all to disabled all gpu (might hang notebook)

$ mitm-play -G  <OR>
$ mitm-play --nogpu
-H --nohost

set logs without host name

$ mitm-play -H  <OR>
$ mitm-play --nohost
-K --dark

set chrome devtools to dark mode, this option effected only when theme set to System preference.

$ mitm-play -K  <OR>
$ mitm-play --dark
-L --showsql

To switch on / show sqlite generated syntax.

$ mitm-play -L  <OR>
$ mitm-play --showsql
-R --redirect

Change mechanism of redirection

$ mitm-play -R  <OR>
$ mitm-play --redirect
-U --nourl

set logs without URL

$ mitm-play -U  <OR>
$ mitm-play --nourl
-V --verbose

Add additional info in console.log

$ mitm-play -V  <OR>
$ mitm-play --verbose
-X --proxypac

When network on your having a proxypac settings, might be usefull to use the same. This option only in Chromium

$ mitm-play -X='w3proxy.netscape.com:8080'  <OR>
$ mitm-play --proxypac='w3proxy.netscape.com:8080'

-C --chromium

Launch Chromium browser

$ mitm-play -C  <OR>
$ mitm-play --chromium

Preset either chrome or msedge

If in the system having stock browser of chrome or msedge

  • chrome
  • msedge
  • chrome-dev
  • msedge-dev
  • chrome-beta
  • msedge-beta
$ mitm-play -C="chrome"  <OR> 
$ mitm-play --chromium="chrome"

Can be a path to Chrome installation ie on MAC

$ mitm-play -C="/Applications/Google\ Chrome.app"  <OR> 
$ mitm-play --chromium="/Applications/Google\ Chrome.app"
-F --firefox

Launch Firefox browser

$ mitm-play -F  <OR>
$ mitm-play --firefox
-W --webkit

Launch Webkit browser

$ mitm-play -W  <OR>
$ mitm-play --webkit

Macros

When creating rule for specific website site (ie: autologin to gmail), inside folder you can add macros.js to contains what automation need to be run

Example
# folder
./accounts.google.com/index.js
./accounts.google.com/_macros_/macros.js
// .../_macros_/macros.js
module.exports = () => {
  const observeOnce = async function() {
    console.log('Getting execute one time')
  }
  return {
    '^/signin/v2/identifier?'() {
      console.log('login to google account...!');
      window.mitm.autofill = [
        '#identifierId => myemailname',
        '#identifierId -> press ~> Enter',
      ];
    },
    '^/signin/v2/challenge/pwd?'() {
      window.mitm.autofill = [
        'input[type="password"] => password',
        'input[type="password"] -> press ~> Enter',
      ];
      // executed when DOM changes, use MutationObserver event
      // postfix "Once" indicate one-time execution
      return observeOnce
    }
  }
}
// will be send to playwright to execute when user click button "Autofill"
window.mitm.autofill = [...]

// it will run on interval 500ms
window.mitm.autointerval = () => {...};

// additinal buttons to be visible on the page top-right
// buttons can be toggle show / hide by clicking [Ctrl] + [SHIFT]
window.mitm.autobuttons = {
  'one|blue'() {console.log('one')},
  'two|green'() {console.log('two')}
}

// A macro keys can be set as a hotkey!
window.mitm.macrokeys = {...}

Macro Keys

A hot keys that can be press on specific page and it will do similar thing with a macro from mechanical keyboard, except its generated from injected mitm-play macros.js,

Example below show a defined macro keys: code:KeyA or code:KeyP & To activate, it need to press combination buttons of Ctrl + Alt + KeyA/KeyP.

list of event.code : https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent/code/code_values

Example
// .../_macros_/macros.js
module.exports = () => {
  return {
    '^/signin/v2/identifier?'() {
      window.mitm.macrokeys = {
        'code:KeyA'() {
          alert('Alert KeyA')
        }
      }
      // -- OR --
      window.mitm.fn.hotKeys({
        'code:KeyP'() {
          // chance is a javascript faker defined in jsLib
          const name = chance.email().split('@')[0];
          return [
            `=> ${name}@mailinator.com`,
            '-> press ~> Enter',
          ]  
        }
      })
    }
  }
}    

Automation commands return from KeyP function don't include selector, means it will run from current input focused.

Variations

Recomended macro keys

Combination Ctrl + Alt + ... will work on Mac/Windows.

Suport all event.code & lowercase event.key

window.mitm.macrokeys = {
  'key:a'()          { console.log('key in: Ctrl + Alt + a') }, // take presedance over code:KeyA
  'key:ab'()         { console.log('key in: Ctrl + Alt + ab') },// take presedance over code:KeyA:KeyB
  'code:KeyA'()      { console.log('key in: Ctrl + Alt + KeyA') },
  'code:KeyA:KeyB'() { console.log('key in: Ctrl + Alt + KeyA:KeyB') },
}

Feature to provide shortcut with option of _keys as condition logic.

if Shift key pressed, it will serve as saving the key into windows.mitm.lastKey._keys.

Ie: how to type shortcut: KeyL with same keys: 'one' save to windows.mitm.lastKey._keys:

* press:   `Ctrl + Alt + Shift + one`, then
* release: `Shift` and press: `KeyL`
// complete press/release on oneliner
* press: `Ctrl + Alt + Shift + one` release: `Shift` press: `KeyL`

Not Recomended macro keys - may conflict with reserved keys on OS/Chrome

Conflict with Chrome shortcut keys or in Windows conflict with Ctrl + J

Suport all event.code & event.key

window.mitm.macrokeys = {
  'key:<a>'()          { console.log('key in: .... + Ctrl + a') }, // take presedance over code:KeyA
  'key:<A>'()          { console.log('key in: .... + Ctrl + A') }, // take presedance over code:KeyA
  'key:<aA>'()         { console.log('key in: .... + Ctrl + aA') },// take presedance over code:KeyA:KeyA
  'code:<KeyA>'()      { console.log('key in: .... + Ctrl + KeyA') },
  'code:<KeyA:KeyA>'() { console.log('key in: .... + Ctrl + KeyA:KeyA') },
}

Not Recomended macro keys - may conflict with reserved keys on OS/Chrome

In windows conflict with Alt + D, unless need to combine with Shift ie: Shift + Alt + D

Suport all event.code & event.key

window.mitm.macrokeys = {
  'key:{a}'()          { console.log('key in: .... + Alt + a') }, // take presedance over code:KeyA
  'key:{A}'()          { console.log('key in: .... + Alt + A') }, // take presedance over code:KeyA
  'key:{aA}'()         { console.log('key in: .... + Alt + aA') },// take presedance over code:KeyA:KeyA
  'code:{KeyA}'()      { console.log('key in: .... + Alt + KeyA') },
  'code:{KeyA:KeyA}'() { console.log('key in: .... + Alt + KeyA:KeyA') },
}

Persistent

isomorphic - persistent is currently implement as a global function under namespace mitm.fn.sql....:

mitm.fn.sqlList - retriving records

when params is a string, should be sql like statement where condition (no need to put quote) with an option of orderby, the order orientation need to be added after fieldname with colon either :a for asc and :d for desc, other type is an object params with combination of keys:

  • _where_ - string sql like statement as state above
  • _limit_ + _offset_ - number for pagination result set
  • _pages_ - boolean to calculate how many pagination pages
await mitm.fn.sqlList()
// (*sqlite sqlList*)
// select * from `kv` []

await mitm.fn.sqlList('(hst like %o%) orderby hst id:d')
// (*sqlite sqlList where:(hst LIKE ?) orderby:hst asc, id desc, ["%o%"]*)
// select * from `kv` where (hst LIKE ?) order by `hst` asc, `id` desc [ '%o%' ]

await mitm.fn.sqlList('(hst like %o%) && id=20 orderby hst id:d')
// (*sqlite sqlList where:(hst LIKE ?) AND id = ? orderby:hst asc, id desc, ["%o%","20"]*)
// select * from `kv` where (hst LIKE ?) AND id = ? order by `hst` asc, `id` desc [ '%o%', '20' ]

await mitm.fn.sqlList('(hst like %o%) && (id=20 || id=21) orderby hst id:d')
// (*sqlite sqlList where:(hst LIKE ?) AND (id = ? OR id = ?) orderby:hst asc, id desc, ["%o%","20","21"]*)
// select * from `kv` where (hst LIKE ?) AND (id = ? OR id = ?) order by `hst` asc, `id` desc [ '%o%', '20', '21' ]

await mitm.fn.sqlList({
  _where_:'(hst like %o%) orderby dtu:d',
  _limit_: 15,
  _offset_: 0,
  _pages_: true
})
// (*sqlite sqlList where:{"_where_":"(hst like %o%) orderby dtu:d","_limit_":15,"_offset_":0,"_pages_":true}*)
// select count(`id`) as `ttl` from `kv` where (hst LIKE ?) order by `dtu` desc [ '%o%' ]
// select * from `kv` where `id` in (select `id` from `kv` where (hst LIKE ?) order by `dtu` desc limit ?) [ '%o%', 15 ]
mitm.fn.sqlDel - delete record(s)

parameters is required, the string parameters having same rules as sqlList excluding orderby

await mitm.fn.sqlDel('(hst like %o%) && app=WOW')
// (*sqlite sqlDel where:(hst LIKE ?) AND app = ?, ["%o%","WOW"]*)
// delete from `kv` where (hst LIKE ?) AND app = ? [ '%o%', 'WOW' ]

await mitm.fn.sqlDel({_hold_:'id>1 orderby hst:d', _limit_: 15})
// (*sqlite sqlDel where:{"_hold_":"id>1 orderby hst:d","_limit_":15}*)
// delete from `kv` where `id` in (select `id` from `kv` where id > ? order by `hst` desc limit ? offset ?) [ '1', -1, 15 ]

await mitm.fn.sqlDel({id:1, _hold_:'id>1 orderby hst:d', _limit_: 15})
// (*sqlite sqlDel where:{"id":1,"_hold_":"id>1 orderby hst:d","_limit_":15}*)
// delete from `kv` where `id` in (select `id` from `kv` where id > ? order by `hst` desc limit ? offset ?) or (`id` = ?) [ '1', -1, 15, 1 ]
mitm.fn.sqlUpd - update record(s)

parameters is required, an object literal at minimum should be 2 field and the first field either id or _where_ to indentify record that need to be updated.

await mitm.fn.sqlUpd({id:14, app: 'LOL2'})
// (*sqlite sqlUpd set:{"id":14,"app":"LOL2"}*)
// update `kv` set `app` = ?, `dtu` = CURRENT_TIMESTAMP where `id` = ? [ 'LOL2', 14 ]

await mitm.fn.sqlUpd({_upd_:'id<10', app: 'below10'})
// (*sqlite sqlUpd set:{"_upd_":"id<10","app":"below10"}*)
// update `kv` set `app` = ?, `dtu` = CURRENT_TIMESTAMP where id < ? [ 'below10', '10' ]
mitm.fn.sqlIns - add a new record

parameters is required, an object literal. it will serve two purpose: first just insert a record or second to delete record(s) before insert with _hold_, _limit_, _del_ keys.

await mitm.fn.sqlIns({hst: 'demo2', grp: 'group2', typ: 'type2', name: 'name2', meta: 'meta2', data: 'data2'})
// (*sqlite sqlIns set:{"hst":"demo2","grp":"group2","typ":"type2","name":"name2","meta":"meta2","data":"data2"}*)
// insert into `kv` (`data`, `dtc`, `dtu`, `grp`, `hst`, `meta`, `name`, `typ`) values (?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?) [ 'data2', 'group2', 'demo2', 'meta2', 'name2', 'type2' ]

await mitm.fn.sqlIns({
  _hold_:'id>1 orderby hst:d',  
  hst: 'demo3', grp: 'group3', typ: 'type3', name: 'name3', meta: 'meta3', data: 'data3'
})
// (*sqlite sqlIns set:{"_hold_":"id>1 orderby hst:d","hst":"demo3","grp":"group3","typ":"type3","name":"name3","meta":"meta3","data":"data3"}*)
// delete from `kv` where `id` in (select `id` from `kv` where id > ? order by `hst` desc limit ? offset ?) [ '1', -1, 1 ]
// insert into `kv` (`data`, `dtc`, `dtu`, `grp`, `hst`, `meta`, `name`, `typ`) values (?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?) [ 'data3', 'group3', 'demo3', 'meta3', 'name3', 'type3' ]

await mitm.fn.sqlIns({
  _hold_:'id>1 orderby hst:d', _limit_: 15,  
  hst: 'demo4', grp: 'group4', typ: 'type4', name: 'name4', meta: 'meta4', data: 'data4'
})
// (*sqlite sqlIns set:{"_hold_":"id>1 orderby hst:d","_limit_":15,"hst":"demo4","grp":"group4","typ":"type4","name":"name4","meta":"meta4","data":"data4"}*)
// delete from `kv` where `id` in (select `id` from `kv` where id > ? order by `hst` desc limit ? offset ?) [ '1', -1, 15 ]
// insert into `kv` (`data`, `dtc`, `dtu`, `grp`, `hst`, `meta`, `name`, `typ`) values (?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?) [ 'data4', 'group4', 'demo4', 'meta4', 'name4', 'type4' ]

await mitm.fn.sqlIns({
  _hold_:'id>1 orderby hst:d', _limit_: 15, _del_:'id<10', 
  hst: 'demo5', grp: 'group5', typ: 'type5', name: 'name5', meta: 'meta5', data: 'data5'
})
// (*sqlite sqlIns set:{"_hold_":"id>1 orderby hst:d","_limit_":15,"_del_":"id<10","hst":"demo5","grp":"group5","typ":"type5","name":"name5","meta":"meta5","data":"data5"}*)
// delete from `kv` where id < ? [ '10' ]
// delete from `kv` where `id` in (select `id` from `kv` where id > ? order by `hst` desc limit ? offset ?) [ '1', -1, 15 ]
// insert into `kv` (`data`, `dtc`, `dtu`, `grp`, `hst`, `meta`, `name`, `typ`) values (?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?) [ 'data5', 'group5', 'demo5', 'meta5', 'name5', 'type5' ]

There are three tables available: kv(default), log & cache. log & cache are preserved, not yet used.

ws__send

Create socket custom command and later it can be use to update/manipulate object, it utilize ws_send function with built-in random keys to make command send to BE is unique

// from browser CLI terminal 
ws__send('ping', 'hi', d=>console.log(`result ${d}`)) // >>> ws-message: `ping:G2kGPCYj{"data":"pong hi!"}`

// example of socket custom command built in for the purpose of testing and validate the custom command
window.mitm.wsrun.$ping = ({ data }) => { // it become: window.mitm.wsrun.$ping
  return `pong ${data}!`
},

User Route

User-route are available on this repo: https://github.com/mitmplay/user-route and it should be taken as an experiment to test mitm-play functionality.

If you think you have a nice routing want to share, you can create a PR to the user-route or add a link to your repo.

Use Cases

Reduce Internet usage

There are several strategy to reduce internet usage, user commonly use different tools to achieve, either install new browser (ie: Brave) or install Add Blocker (ie: uBlock). Using mitm-play, developer can controll which need to be pass, blocked or cached.

Cache any reguest with content type: font, image, javascript, css, if url contains cached busting, it may miss the cached, you can experiment by turning off querystring to false.

cache: {
  '.+': {
    contentType: ['font','image','javascript','css'],
    querystring: true,
  }
},

Block/Mock unnecessary javascript with an empty result, be careful to not block UX or content navigation.

mock: {
  'block/w/empty.js': '',
  'some/url/with/adv.js': {
    response(resp, reqs, match) {
      const {body} = resp;
      ...
      return {body: '/* content is blocked! */'}
    },
  },
},
Simplify Developer workflow

as developer sometime we need to get access to lots website in which some of the page need to be automated fill in and submit to the next page. With Macros it can be done!

Early Stage

Expect to have some rule changed as feature/fix code are incrementally committed.

.

Goodluck!,

-wh.

Known Limitation

Issue or Limitation on Playwright:

  • Support route aka request interception for service workers #1090
    • Workaround to disable service worker #6045
  • Route handler to support redirects #3993 / Disallow intercepting redirects #2617
    • or alternative intercept response is implemented #1774
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].