All Projects → Wscats → Media Tutorial

Wscats / Media Tutorial

流处理,TCP和UDP,WebRTC和Blob

Programming Languages

javascript
184084 projects - #8 most used programming language

Projects that are alternatives of or similar to Media Tutorial

Kalm.js
The socket manager
Stars: ✭ 155 (+229.79%)
Mutual labels:  webrtc, tcp, udp
Impulse
💣 Impulse Denial-of-service ToolKit
Stars: ✭ 538 (+1044.68%)
Mutual labels:  tcp, udp
Leaf
A lightweight and fast proxy utility tries to include any useful features.
Stars: ✭ 530 (+1027.66%)
Mutual labels:  tcp, udp
Elixir Socket
Socket wrapping for Elixir.
Stars: ✭ 642 (+1265.96%)
Mutual labels:  tcp, udp
Yasio
A multi-platform support c++11 library with focus on asio (asynchronous socket I/O) for any client application.
Stars: ✭ 483 (+927.66%)
Mutual labels:  tcp, udp
Libzt
ZeroTier Sockets - Put a network stack in your app
Stars: ✭ 486 (+934.04%)
Mutual labels:  tcp, udp
Libnet
A portable framework for low-level network packet construction
Stars: ✭ 640 (+1261.7%)
Mutual labels:  tcp, udp
Logstash Logger
Ruby logger that writes logstash events
Stars: ✭ 442 (+840.43%)
Mutual labels:  tcp, udp
Media Server
WebRTC Media Server
Stars: ✭ 821 (+1646.81%)
Mutual labels:  media, webrtc
Objecttransport
Send and Receive objects over TCP or UDP
Stars: ✭ 39 (-17.02%)
Mutual labels:  tcp, udp
Gensio
A library to abstract stream I/O like serial port, TCP, telnet, UDP, SSL, IPMI SOL, etc.
Stars: ✭ 30 (-36.17%)
Mutual labels:  tcp, udp
Udp2raw Tunnel
A Tunnel which Turns UDP Traffic into Encrypted UDP/FakeTCP/ICMP Traffic by using Raw Socket,helps you Bypass UDP FireWalls(or Unstable UDP Environment)
Stars: ✭ 4,839 (+10195.74%)
Mutual labels:  tcp, udp
Cnp3
Computer Networking : Principles, Protocols and Practice (first and second edition, third edition is being written on https://github.com/cnp3/ebook)
Stars: ✭ 471 (+902.13%)
Mutual labels:  tcp, udp
Proxy admin free
Proxy是高性能全功能的http代理、https代理、socks5代理、内网穿透、内网穿透p2p、内网穿透代理、内网穿透反向代理、内网穿透服务器、Websocket代理、TCP代理、UDP代理、DNS代理、DNS加密代理,代理API认证,全能跨平台代理服务器。
Stars: ✭ 487 (+936.17%)
Mutual labels:  tcp, udp
Scriptcommunicator serial Terminal
Scriptable cross-platform data terminal which supports: serial port, UDP, TCP, SPI, I2C and CAN.
Stars: ✭ 462 (+882.98%)
Mutual labels:  tcp, udp
Blinksocks
A framework for building composable proxy protocol stack.
Stars: ✭ 587 (+1148.94%)
Mutual labels:  tcp, udp
Tools
C# 工具箱,提供Socket(TCP、UDP协议)、Redis、activemq、数据库访问等技术的封装实现
Stars: ✭ 34 (-27.66%)
Mutual labels:  tcp, udp
Networker
A simple to use TCP and UDP networking library for .NET. Compatible with Unity.
Stars: ✭ 408 (+768.09%)
Mutual labels:  tcp, udp
Hp Socket
High Performance TCP/UDP/HTTP Communication Component
Stars: ✭ 4,420 (+9304.26%)
Mutual labels:  tcp, udp
Parallec
Fast Parallel Async HTTP/SSH/TCP/UDP/Ping Client Java Library. Aggregate 100,000 APIs & send anywhere in 20 lines of code. Ping/HTTP Calls 8000 servers in 12 seconds. (Akka) www.parallec.io
Stars: ✭ 777 (+1553.19%)
Mutual labels:  tcp, udp

BAQ

createObjectURL错误Failed to execute 'createObjectURL' on 'URL'

window.URL.createObjectURL(data)

var binaryData = [];
binaryData.push(data);
window.URL.createObjectURL(new Blob(binaryData, {type: "application/zip"}));

navigator

  • navigator.mediaDevices.getDisplayMedia屏幕源
  • navigator.mediaDevices.getUserMedia摄像头源
<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>Document</title>
</head>

<body>
    <style>
        video {
            width: 1000px;
            height: 500px
        }
    </style>
    <video autoplay id="video">Video stream not available.</video>
    <script>
        let video = document.getElementById('video');
        navigator.mediaDevices.getDisplayMedia({
                video: true,
                audio: true
            })
            .then(stream => {
                // we have a stream, attach it to a feedback video element
                console.log(stream);
                
                video.srcObject = stream;
            }, error => {
                console.log("Unable to acquire screen capture", error);
            });
    </script>
</body>

</html>

MediaRecorder

MediaRecorder录制视频或者音频的步骤

  • navigator.mediaDevices.getUserMedia(constraints).then(function(stream) {}获取音视频的stream
  • var mediaRecorder = new MediaRecorder(stream)实例化mediaRecordermediaRecorder会不断接受navigator.mediaDevices.getUserMedia(constraints).then(function(stream) {}所提供的stream,并且它自身有多个方法
  • mediaRecorder.start()点击事件触发开始录制
  • mediaRecorder.onstop = function(e) {}mediaRecorder监听停止事件var blob = new Blob(chunks, { 'type' : 'audio/ogg; codecs=opus' });chunks = [];这里的chunks会从无到有,随着音视频流被监听,不断增加,注意onstop会触发两次,一开始未录制和暂停都会触发
  • mediaRecorder.ondataavailable = function(e) {chunks.push(e.data);}不断监听stream,并往chunks数组添加音视频流的数据
  • mediaRecorder.stop()点击事件触发暂停录制
  • var audioURL = URL.createObjectURL(blob);audio.src = audioURL;console.log("recorder stopped");blob生成一个url挂载到audio或者video标签上面去播放
var constraints = { audio: true };
var chunks = [];

navigator.mediaDevices.getUserMedia(constraints)
.then(function(stream) {

var mediaRecorder = new MediaRecorder(stream);

record.onclick = function() {
    mediaRecorder.start();
    console.log(mediaRecorder.state);
}

stop.onclick = function() {
    mediaRecorder.stop();
    console.log(mediaRecorder.state);
}

mediaRecorder.onstop = function(e) {
    audio.setAttribute('controls', '');
    audio.controls = true;
    var blob = new Blob(chunks, { 'type' : 'audio/ogg; codecs=opus' });
    chunks = [];
    var audioURL = URL.createObjectURL(blob);
    audio.src = audioURL;
    console.log("recorder stopped");
}

mediaRecorder.ondataavailable = function(e) {
    chunks.push(e.data);
}
})
.catch(function(err) {
    console.log('The following error occurred: ' + err);
})

webkitRTCPeerConnection

localPeerConnection = new RTCPeerConnection(null);
localPeerConnection.onicecandidate = function(event) {
    if (event.candidate) {
        remotePeerConnection.addIceCandidate(new RTCIceCandidate(event.candidate)); //answer方接收ICE
    }
};

remotePeerConnection = new RTCPeerConnection(null);
remotePeerConnection.onicecandidate = function(event) {
    if (event.candidate) {
        localPeerConnection.addIceCandidate(new RTCIceCandidate(event.candidate)); //offer方接收ICE
    }
};

remotePeerConnection.onaddstream = function gotRemoteStream(event) {
    recordedVideo.srcObject = event.stream;
};

localPeerConnection.addStream(recordStream);
localPeerConnection.createOffer(function (description) { //description是offer方的  SD  ==>  传输的内容
    localPeerConnection.setLocalDescription(description);
    remotePeerConnection.setRemoteDescription(description); //answer方接收offer的SD
    remotePeerConnection.createAnswer(function (description) {
        remotePeerConnection.setLocalDescription(description); //answer方设置本身自己的SD
        localPeerConnection.setRemoteDescription(description); //offer接收answer方的SD
    }, function (error) {
        console.log(error)
    }); //answer方发送自己的SD
}, function (error) {
    console.log(error)
});

Blob

Blob Binary Large Object的缩写,代表二进制类型的大对象

new Blob(blobParts, [options]);

用法

  • blobParts:数组类型,数组中的每一项连接起来构成Blob对象的数据,数组中的每项元素可以是ArrayBuffer, ArrayBufferView, Blob, DOMString 。

  • options:可选项,字典格式类型,可以指定如下两个属性:

    • type,默认值为 "",它代表了将会被放入到blob中的数组内容的MIME类型。
    • endings,默认值为"transparent",用于指定包含行结束符\n的字符串如何被写入。 它是以下两个值中的一个: "native",表示行结束符会被更改为适合宿主操作系统文件系统的换行符; "transparent",表示会保持blob中保存的结束符不变。
var data1 = "a";
var data2 = "b";
var data3 = "<div style='color:red;'>This is a blob</div>";
var data4 = {
    "name": "abc"
};

var blob1 = new Blob([data1]);
var blob2 = new Blob([data1, data2]);
var blob3 = new Blob([data3]);
var blob4 = new Blob([JSON.stringify(data4)]);
var blob5 = new Blob([data4]);
var blob6 = new Blob([data3, data4]);

console.log(blob1); //输出:Blob {size: 1, type: ""}
console.log(blob2); //输出:Blob {size: 2, type: ""}
console.log(blob3); //输出:Blob {size: 44, type: ""}
console.log(blob4); //输出:Blob {size: 14, type: ""}
console.log(blob5); //输出:Blob {size: 15, type: ""}
console.log(blob6); //输出:Blob {size: 59, type: ""}

slice方法

Blob对象有一个slice方法,返回一个新的Blob对象,包含了源Blob对象中指定范围内的数据。

slice([start], [end], [contentType])
  • start: 可选,代表Blob里的下标,表示第一个会被会被拷贝进新的Blob的字节的起始位置。如果传入的是一个负数,那么这个偏移量将会从数据的末尾从后到前开始计算。
  • end: 可选,代表的是Blob的一个下标,这个下标-1的对应的字节将会是被拷贝进新的Blob的最后一个字节。如果你传入了一个负数,那么这个偏移量将会从数据的末尾从后到前开始计算。
  • contentType: 可选,给新的Blob赋予一个新的文档类型。这将会把它的type属性设为被传入的值。它的默认值是一个空的字符串。
var data = "abcdef";
var blob1 = new Blob([data]);
var blob2 = blob1.slice(0, 3);

console.log(blob1); //输出:Blob {size: 6, type: ""}
console.log(blob2); //输出:Blob {size: 3, type: ""}
let href = URL.createObjectURL(blob2); //浏览器可以直接打开href连接看输出
console.log(href); //abc

URL.createObjectURL()

URL.createObjectURL()静态方法会创建一个DOMString,其中包含一个表示参数中给出的对象的URL。这个URL的生命周期和创建它的窗口中的document绑定。这个新的URL对象表示指定的File对象或 Blob对象。

objectURL = URL.createObjectURL(blob);

URL.revokeObjectURL()

URL.revokeObjectURL()静态方法用来释放一个之前通过调用URL.createObjectURL()创建的已经存在的URL对象。当你结束使用某个URL对象时,应该通过调用这个方法来让浏览器知道不再需要保持这个文件的引用了。

window.URL.revokeObjectURL(objectURL);

分割上传

目前,Blob对象大多是运用在,处理大文件分割上传(利用Blobslice方法),处理图片canvas跨域(避免增加crossOrigin = "Anonymous",生成当前域名的url,然后URL.revokeObjectURL()释放,createjs有用到),以及隐藏视频源路径等等。

function upload(blobOrFile) {
    var xhr = new XMLHttpRequest();
    xhr.open('POST', '/server', true);
    xhr.onload = function (e) {
        // ...
    };
    xhr.send(blobOrFile);
}

document.querySelector('input[type="file"]').addEventListener('change', function (e) {
var blob = this.files[0];
const BYTES_PER_CHUNK = 1024 * 1024; // 1MB chunk sizes.
const SIZE = blob.size;
var start = 0;
var end = BYTES_PER_CHUNK;
while (start < SIZE) {
    upload(blob.slice(start, end));
    start = end;
    end = start + BYTES_PER_CHUNK;
}
}, false);

下载

var xhr = new XMLHttpRequest();
xhr.open('GET', '/path/to/image.png', true);
xhr.responseType = 'blob';
xhr.send()

xhr.onload = function (e) {
    if (this.status == 200) {
        var blob = this.response;

        var img = document.createElement('img');
        var URL = window.URL || window.webkitURL; //兼容处理
        var objectUrl = URL.createObjectURL(blob);
        img.onload = function (e) {
            window.URL.revokeObjectURL(img.src); // 释放 url.
        };

        img.src = objectUrl;
        document.body.appendChild(img);
        // ...
    }
};

xhr.send();
代码 作用 类型
navigator.mediaDevices.getUserMedia(constraints).then(function(stream){}; 根据音视频源头生成stream MediaStream
mediaRecorder = new MediaRecorder(window.stream, options) 根据音视频源头把stream交给mediaRecorder处理 MediaRecorder
mediaRecorder.ondataavailable = function (event) {recordedBlobs.push(event.data);} 根据stream获取每一片段的BlobEvent并存入recordedBlobs数组 BlobEvent
let buffer = new Blob(recordedBlobs, {type: "video/webm"}); 得到Blob格式的音视频流 Blob
test.src = window.URL.createObjectURL(buffer); 可用window.URL.createObjectURL方法处理Blob并配合video或者audio标签播放 src
recordStream = test.captureStream(); Blob转化为MediaStream MediaStream
new RTCPeerConnection(null).addStream(recordStream); recordStream交给RTCPeerConnection处理 MediaStream
mediaRecorder = new MediaRecorder(window.stream, options); // 设置音频录入源、格式
mediaRecorder.ondataavailable = function (event) {
    // 这个会不断接受BlobEvent
    console.log(event);
    if (event.data && event.data.size > 0) {
        recordedBlobs.push(event.data);
    }
}; // 存放获取的数据
let buffer = new Blob(recordedBlobs, {
    type: "video/webm"
});
console.log(buffer);
test.src = window.URL.createObjectURL(buffer);
recordStream = test.captureStream();

参考文档

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].