javascript - HTML5 video to canvas playing very slow - Stack Overflow

admin管理员组

文章数量:1295923

I've built this HTML5 video player that I am loading into a canvas to manipulate and back onto a canvas to display it. The video starts out quite slow and the frame rate only gets worse each time it is played. All I am currently manipulating in the video now is the color value when the video is paused, but will eventually be using real time manipulation throughout videos that will be posted in the future.

I used the below tutorial to learn this trick

Here is the relevant code, but there may possibly be interference ing from elsewhere so feel free to check the source code at the link at the bottom

var v = document.getElementById('video');
var color = "#DA7AC1";
var processes={
  timerCallback:function() {
    if (this.v2.paused || this.v2.ended) {
      return;
    }
        this.ctxIn.drawImage(this.v2,0,0,this.width,this.height);
        this.pixelScan();
        var self=this;
        setTimeout(function() {
          self.timerCallback();
        }, 0);
  },
  doLoad:function(){
    this.v2=document.getElementById("video");
    this.cIn=document.getElementById("cIn");
    this.ctxIn=this.cIn.getContext("2d");
    this.cOut=document.getElementById("cOut");
    this.ctxOut=this.cOut.getContext("2d");
    var self=this;
    this.v2.addEventListener("playing", function() {
      self.width=self.v2.videoWidth;
      self.height=self.v2.videoHeight;
      cIn.width=self.v2.videoWidth;
      cIn.height=self.v2.videoHeight;
      cOut.width=self.v2.videoWidth;
      cOut.height=self.v2.videoHeight;
      self.timerCallback();
    }, false);
  },
  pixelScan: function() {
    var frame = this.ctxIn.getImageData(0,0,this.width,this.height);
    for(var i=0; i<frame.data.length;i+=4) {
      var grayscale=frame.data[i]*.3+frame.data[i+1]*.59+frame.data[i+2]*.11;
      frame.data[i]=grayscale;
      frame.data[i+1]=grayscale;
      frame.data[i+2]=grayscale;
    }
    this.ctxOut.putImageData(frame,0,0);
    return;
  }
}

/

Any and all help would be greatly appreciated! Thanks!

I've built this HTML5 video player that I am loading into a canvas to manipulate and back onto a canvas to display it. The video starts out quite slow and the frame rate only gets worse each time it is played. All I am currently manipulating in the video now is the color value when the video is paused, but will eventually be using real time manipulation throughout videos that will be posted in the future.

I used the below tutorial to learn this trick https://www.youtube./watch?v=zjQzP3mOXdc

Here is the relevant code, but there may possibly be interference ing from elsewhere so feel free to check the source code at the link at the bottom

var v = document.getElementById('video');
var color = "#DA7AC1";
var processes={
  timerCallback:function() {
    if (this.v2.paused || this.v2.ended) {
      return;
    }
        this.ctxIn.drawImage(this.v2,0,0,this.width,this.height);
        this.pixelScan();
        var self=this;
        setTimeout(function() {
          self.timerCallback();
        }, 0);
  },
  doLoad:function(){
    this.v2=document.getElementById("video");
    this.cIn=document.getElementById("cIn");
    this.ctxIn=this.cIn.getContext("2d");
    this.cOut=document.getElementById("cOut");
    this.ctxOut=this.cOut.getContext("2d");
    var self=this;
    this.v2.addEventListener("playing", function() {
      self.width=self.v2.videoWidth;
      self.height=self.v2.videoHeight;
      cIn.width=self.v2.videoWidth;
      cIn.height=self.v2.videoHeight;
      cOut.width=self.v2.videoWidth;
      cOut.height=self.v2.videoHeight;
      self.timerCallback();
    }, false);
  },
  pixelScan: function() {
    var frame = this.ctxIn.getImageData(0,0,this.width,this.height);
    for(var i=0; i<frame.data.length;i+=4) {
      var grayscale=frame.data[i]*.3+frame.data[i+1]*.59+frame.data[i+2]*.11;
      frame.data[i]=grayscale;
      frame.data[i+1]=grayscale;
      frame.data[i+2]=grayscale;
    }
    this.ctxOut.putImageData(frame,0,0);
    return;
  }
}

http://coreytegeler./ethan/

Any and all help would be greatly appreciated! Thanks!

• javascript
• html
• video
• canvas

Share Improve this question Follow asked Jan 17, 2014 at 23:11 TomelowerTomelower 49533 gold badges88 silver badges1717 bronze badges

Add a ment  | 

3 Answers 3

Sorted by: Reset to default 8

Reason 1

Try to adjust your timer avoiding 0 as timeout value:

setTimeout(function() {
      self.timerCallback();
}, 34);

34ms is plenty as video frame rate is typically never more than 30 FPS (NTSC) or 25 FPS (PAL), ie 1000 / 30. If you use 0 you risk stacking up your calls which means the browser will be busy trying to empty the event queue.

If you use anything lower than 33-34ms you end up having the same frame processed twice or more which of course is unnecessary (your video is actually 29.97 FPS/NTSC so you might want to consider keeping 34ms).

Reason 2

The video resolution is also full HD (1920x1080) which is a bit too much for canvas and JS to process in real-time (for a typcial consumer puter). Try to reduce the video size so a normal spec'ed puter will be able to process the data.

Reason 3 (in part)

You don't need two on-screen canvases or even an on-screen video. Try to create these tags dynamically and not inserting them into the DOM. Use a single canvas on-screen and draw the result to that (you can putImageData from one canvas to another).

Reason 4 (in part)

Ideally, replace setTimeout with a requestAnimationFrame approach as this improves the synchronization and efficiency considerably. You can implement a toggle to reduce the FPS to for example 30 as you don't need to process each frame twice (ref. 30 FPS video frame rate).

Update

To create these elements dynamically (ref reason 3) you can do something like this:

var canvas = document.createElement('canvas'),
    video = document.createElement('video'),
    ctx = canvas.getContext('2d');

video.preload = 'auto';
video.addEventListener('canplay', start, false);

if (video.canPlayType('video/mp4')) {
    video.src = 'videoUrl.mp4';

} else if ...etc.

Then when the video has loaded enough data (on metadata or canplay) you set the off-screen (and on-screen) canvas element to the size of the video:

canvas.width = video.videoWidth;
canvas.height = video.videoHeight;

Then when playing process its buffer and copy to the on-screen canvas you defined before.

You don't have have an off-screen canvas - I merely mention this as you in your original code used and in and out canvas IIRC. You can simply use a single on-screen canvas and the off-screen video and draw to the video frame to the canvas, process it and put back the processed data. Should work fine too in this case.

I ran a profile in chrome and it points to line 46 as taking up the most CPU.

setTimeout(function() {
    self.timerCallback();
}, 0);

Perhaps increasing the timeout will stop it from lagging.

I had the same issues and tried a number of fixes. I was using Premier Elements which didn't export to mp4 and using HandBrake to convert the format. I also Tried FFMpeg to do the conversion, but neither worked.

What I did was switch to Kdenlive as my video editor, it exported directly to MP4, and that video worked perfectly.

So, if you are have this slow render issue, it is probably an issues with the video encoding. Easiest fix is to get a high quality video editor like Premier Pro, Final Cut, or Kdenlive. Kdenlive is free but it has a huge learning curve and poor public documentation.

本文标签： javascriptHTML5 video to canvas playing very slowStack Overflow