All Projects → badoo → Liveprof

badoo / Liveprof

Licence: mit
A performance monitoring system for running on live sites

Projects that are alternatives of or similar to Liveprof

Vcprofiler
An accurate and simple tool uses KVO to measure the time cost of every view controller.
Stars: ✭ 61 (-70.39%)
Mutual labels:  performance, profiler
Jvm Profiler
JVM Profiler Sending Metrics to Kafka, Console Output or Custom Reporter
Stars: ✭ 1,558 (+656.31%)
Mutual labels:  performance, profiler
Xpedite
A non-sampling profiler purpose built to measure and optimize performance of ultra low latency/real time systems
Stars: ✭ 89 (-56.8%)
Mutual labels:  performance, profiler
Nanoscope
An extremely accurate Android method tracing tool.
Stars: ✭ 1,023 (+396.6%)
Mutual labels:  performance, profiler
Visualvm
VisualVM is an All-in-One Java Troubleshooting Tool
Stars: ✭ 2,084 (+911.65%)
Mutual labels:  performance, profiler
Profimp
Python import profiler
Stars: ✭ 52 (-74.76%)
Mutual labels:  performance, profiler
Easy profiler
Lightweight profiler library for c++
Stars: ✭ 1,594 (+673.79%)
Mutual labels:  performance, profiler
Pyinstrument
🚴 Call stack profiler for Python. Shows you why your code is slow!
Stars: ✭ 3,870 (+1778.64%)
Mutual labels:  performance, profiler
Orbit
C/C++ Performance Profiler
Stars: ✭ 2,291 (+1012.14%)
Mutual labels:  performance, profiler
Tickprofiler
Profile your minecraft server: Find which entities and tile entities are making your server slow
Stars: ✭ 119 (-42.23%)
Mutual labels:  performance, profiler
Profiler
Firefox Profiler — Web app for Firefox performance analysis
Stars: ✭ 546 (+165.05%)
Mutual labels:  performance, profiler
Tracy
C++ frame profiler
Stars: ✭ 3,115 (+1412.14%)
Mutual labels:  performance, profiler
Pprof
pprof is a tool for visualization and analysis of profiling data
Stars: ✭ 4,990 (+2322.33%)
Mutual labels:  performance, profiler
Phpspy
Low-overhead sampling profiler for PHP 7+
Stars: ✭ 1,105 (+436.41%)
Mutual labels:  performance, profiler
Spark
spark is a performance profiling plugin/mod for Minecraft clients, servers and proxies.
Stars: ✭ 287 (+39.32%)
Mutual labels:  performance, profiler
Optick
C++ Profiler For Games
Stars: ✭ 1,364 (+562.14%)
Mutual labels:  performance, profiler
Liveprof Ui
An aggregator and web interface for Live Profiler
Stars: ✭ 118 (-42.72%)
Mutual labels:  performance, profiler
Hotspot
The Linux perf GUI for performance analysis.
Stars: ✭ 2,415 (+1072.33%)
Mutual labels:  performance, profiler
Myperf4j
High performance Java APM. Powered by ASM. Try it. Test it. If you feel its better, use it.
Stars: ✭ 2,281 (+1007.28%)
Mutual labels:  performance, profiler
Aws Lambda Power Tuning
AWS Lambda Power Tuning is an open-source tool that can help you visualize and fine-tune the memory/power configuration of Lambda functions. It runs in your own AWS account - powered by AWS Step Functions - and it supports three optimization strategies: cost, speed, and balanced.
Stars: ✭ 3,040 (+1375.73%)
Mutual labels:  performance

Live Profiler

logo

Live Profiler is a system-wide performance monitoring system in use at Badoo that is built on top of XHProf or its forks (Uprofiler or Tideways). Live Profiler continually gathers function-level profiler data from production tier by running a sample of page requests under XHProf.

Live profiler UI then aggregates the profile data corresponding to individual requests by various dimensions such a time, page type, and can help answer a variety of questions such as: What is the function-level profile for a specific page? How expensive is function "foo" across all pages, or on a specific page? What functions regressed most in the last day/week/month? What is the historical trend for execution time of a page/function? and so on.

Here is a plugin for PhpStorm to see the method performance directly in IDE.

liveprof.org shows all features and can be used for test purposes.

Build Status codecov Scrutinizer Code Quality GitHub license

System Requirements

  • PHP version 5.4 or later / hhvm version 3.25.0 or later
  • One of XHProf, Uprofiler or Tideways to profile and collect the data. You can use other profiler which returns data in the follow format:
    $data = [
        [
            'parent_method==>child_method' => [
                'param' => 'value' 
            ]
        ]  
    ];
    
  • Database extension to save profiling results to the database.

Installation

  1. You can install Live Profiler via Composer:
php composer.phar require badoo/liveprof
  1. Prepere a storage for results depends on mode

[save data in database] If you use DB mode you need to prepare a database server. You can use any driver described here or implement the custom one. You need run a script to configure database. This script creates "details" table:

LIVE_PROFILER_CONNECTION_URL=mysql://db_user:[email protected]_mysql:3306/Profiler?charset=utf8 php vendor/badoo/liveprof/bin/install.php

[save data in files] It's also possible to save profiling result into files. To do it prepare a directory with write permissions.

[send data to demo site] You need to visit liveprof.org , sign in and copy API key.

  1. Init a profiler before working code in the project entry point (usually public/index.php).

Usage

There is an example of usage a profiler with default parameters:

<?php
include 'vendor/autoload.php';

\Badoo\LiveProfiler\LiveProfiler::getInstance()->start();
// Code is here

There is an example how to test Live Profiler without any extension and database. You can use a build-in profiler compatible with XHProf and liveprof.org as UI:

<?php
include 'vendor/autoload.php';

\Badoo\LiveProfiler\LiveProfiler::getInstance()
     ->setMode(\Badoo\LiveProfiler\LiveProfiler::MODE_API)
     ->setApiKey('70366397-97d6-41be-a83c-e9e649c824e1') // a key for guest
     ->useSimpleProfiler() // Use build-in profiler instead of XHProf or its forks
     ->setApp('Demo') // Some unique app name
     ->start();
     
// Code is here
// start a timer before each inportant method
\Badoo\LiveProfiler\SimpleProfiler::getInstance()->startTimer(__METHOD__); // any string can be used as a timer tag
// stop the timer before the end of the method
\Badoo\LiveProfiler\SimpleProfiler::getInstance()->endTimer(__METHOD__); // any string can be used as a timer tag

There is a full list of methods you can use to change options:

<?php

// Start profiling
\Badoo\LiveProfiler\LiveProfiler::getInstance()
    ->setMode(\Badoo\LiveProfiler\LiveProfiler::MODE_DB) // optional, MODE_DB - save profiles to db, MODE_FILES - save profiles to files, MODE_API - send profiles to http://liveprof.org/ 
    ->setConnectionString('mysql://db_user:[email protected]_mysql:3306/Profiler?charset=utf8') // optional, you can also set the connection url in the environment variable LIVE_PROFILER_CONNECTION_URL
    ->setPath('/app/data/') // optional, path to save profiles, you can also set the file path in the environment variable LIVE_PROFILER_PATH
    ->setApiKey('api_key') // optional, api key to send profiles and see demo, you can get it on http://liveprof.org/ 
    ->setApp('Site1') // optional, current app name to use one profiler in several apps, "Default" by default
    ->setLabel('users') // optional, the request name, by default the url path or script name in cli
    ->setDivider(700) // optional, profiling starts for 1 of 700 requests with the same app and label, 1000 by default
    ->setTotalDivider(7000) // optional, profiling starts for 1 of 7000 requests with forces label "All", 10000 by default
    ->setLogger($Logger) // optional, a custom logger implemented \Psr\Log\LoggerInterface
    ->setConnection($Connection) // optional, a custom instance of \Doctrine\DBAL\Connection if you can't use the connection url
    ->setDataPacker($DatePacker) // optional, a class implemented \Badoo\LiveProfiler\DataPackerInterface to convert array into string
    ->setStartCallback($profiler_start_callback) // optional, set it if you use custom profiler
    ->setEndCallback($profiler_profiler_callback) // optional, set it if you use custom profiler
    ->useXhprof() // optional, force use xhprof as profiler
    ->useTidyWays() // optional, force use TidyWays as profiler
    ->useUprofiler() // optional, force use uprofiler as profiler
    ->useSimpleProfiler() // optional, force use internal profiler
    ->useXhprofSample() // optional, force use xhprof in sampling mode
    ->start();

If you want to change the Label during running (for instance, after you got some information in the router or controller) you can call:

<?php

$number = random_int(0, 100);
$current_label = \Badoo\LiveProfiler\LiveProfiler::getInstance()->getLabel();
\Badoo\LiveProfiler\LiveProfiler::getInstance()->setLabel($current_label . $number);

if you don't want to save profiling result you can reset it anytime:

<?php

\Badoo\LiveProfiler\LiveProfiler::getInstance()->reset();

After script ends it will call \Badoo\LiveProfiler\LiveProfiler::getInstance()->end(); on shutdown, but you can call it explicitly after working code.

Environment Variables

LIVE_PROFILER_CONNECTION_URL: url for the database connection

LIVE_PROFILER_PATH: path to save profiles in \Badoo\LiveProfiler\LiveProfiler::MODE_FILES mode

LIVE_PROFILER_API_URL: api url to send profiles in \Badoo\LiveProfiler\LiveProfiler::MODE_API mode and see demo on liveprof.org

Work flow

Live profiler allows to run profiling with custom frequency (for instance 1 of 1000 requests) grouped by app name ('Default' by default) and custom label (by default it's the url path or script name).

It's important to calculate the request divider properly to have enough data for aggregation. You should divide daily request count to have approximately 1 profile per minute. For instance, if you have 1M requests a day for the page /users the divider should be 1000000/(60*24) = 694, so divider = 700 is enough.

Also you have to calculate a total divider for all request profiling. It's important to control the whole system health. It can be calculated the same way as a divider calculation for particularly request, but in this case you should use count of all daily requests. For instance, if you have 10M requests a day - total_divider=10000000/(60*24) = 6940, so total_divider = 7000 is enough.

The profiler automatically detects which profiler extension you have (xhprof, uprofiler or tidyways). You have to set profiler callbacks if you use other profiler.

You can run the test script in the docker container with xhprof extension and look at the example of the server configuration in Dockerfile:

docker build -t badoo/liveprof .
docker run badoo/liveprof

or you can build a docker container with xhprof using sampling:

docker build -f DockerfileSamples -t badoo/liveprof .
docker run badoo/liveprof

or you can build a docker container with tideways extension:

docker build -f DockerfileTidyWays -t badoo/liveprof .
docker run badoo/liveprof

or uprofiler extension:

docker build -f DockerfileUprofiler -t badoo/liveprof .
docker run badoo/liveprof

or latest hhvm with included xhprof extension:

docker build -f DockerfileHHVM -t badoo/liveprof .
docker run badoo/liveprof

or if you want to use API with included xhprof extension:

docker build -f DockerfileUseApi -t badoo/liveprof .
docker run badoo/liveprof

If your server has php version 7.0 or later it's better to use Tideways as profiler.

Steps to install tideways extension:

git clone https://github.com/tideways/php-profiler-extension.git
cd php-profiler-extension
phpize
./configure
make
make install
echo "extension=tideways_xhprof.so" >> /usr/local/etc/php/conf.d/20-tideways_xhprof.ini
echo "xhprof.output_dir='/tmp/xhprof'" >> /usr/local/etc/php/conf.d/20-tideways_xhprof.ini

Steps to install uprofiler:

git clone https://github.com/FriendsOfPHP/uprofiler.git
cd uprofiler/extension/
phpize
./configure
make
make install
echo "extension=uprofiler.so" >> /usr/local/etc/php/conf.d/20-uprofiler.ini
echo "uprofiler.output_dir='/tmp/uprofiler'" >> /usr/local/etc/php/conf.d/20-uprofiler.ini

Tests

Install Live Profiler with dev requirements:

php composer.phar require --dev badoo/liveprof

In the project directory, run:

vendor/bin/phpunit

License

This project is licensed under the MIT open source license.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].