blob - Download large data stream (> 1Gb) using javascript - Stack Overflow

admin管理员组

文章数量:1194547

I was wondering if it was possible to stream data from javascript to the browser's downloads manager.

Using webrtc, I stream data (from files > 1Gb) from a browser to the other. On the receiver side, I store into memory all this data (as arraybuffer ... so the data is essentially still chunks), and I would like the user to be able to download it.

Problem : Blob objects have a maximum size of about 600 Mb (depending on the browser) so I can't re-create the file from the chunks. Is there a way to stream these chunks so that the browser downloads them directly ?

I was wondering if it was possible to stream data from javascript to the browser's downloads manager.

Using webrtc, I stream data (from files > 1Gb) from a browser to the other. On the receiver side, I store into memory all this data (as arraybuffer ... so the data is essentially still chunks), and I would like the user to be able to download it.

Problem : Blob objects have a maximum size of about 600 Mb (depending on the browser) so I can't re-create the file from the chunks. Is there a way to stream these chunks so that the browser downloads them directly ?

• javascript
• blob
• webrtc

Share Improve this question Follow asked Mar 5, 2017 at 22:32 BrianBrian 30711 gold badge22 silver badges88 bronze badges 15

• 1 bugs.chromium.org/p/chromium/issues/detail?id=375297 seems to be the most appropriate example of what i'm currently facing (this example uses multiple files, but the same error occurs with 1 big file) ... It's from one year ago, so I would be glad to be proven wrong, but I think it hasn't been fixed yet ? – Brian Commented Mar 5, 2017 at 23:05
• Was able to create and read var b = new Blob([new Uint8Array(500*1024*1024)], {type: 'application/octet-string'}); though var b = new Blob([new Uint8Array(500*1024*1024), new Uint8Array(500*1024*1024)], {type: 'application/octet-string'}); logged Uncaught RangeError: Array buffer allocation failed – guest271314 Commented Mar 5, 2017 at 23:12
• I have the same problem; I was planning to break up the stream into smaller files and offer each as a separate download. – Meirion Hughes Commented Mar 5, 2017 at 23:15
• 1 The limit should be 2GB on Chrome 57 which is 9 days from now. – zer00ne Commented Mar 5, 2017 at 23:35
• 3 @Brian See StreamSaver.js – guest271314 Commented Mar 6, 2017 at 1:24

 |  Show 10 more comments

2 Answers 2

Sorted by: Reset to default 10

if you want to fetch a large file blob from an api or url, you can use streamsaver.

npm install streamsaver

then you can do something like this

import { createWriteStream } from 'streamsaver';

export const downloadFile = (url, fileName) => {
  return fetch(url).then(res => {
    const fileStream = createWriteStream(fileName);
    const writer = fileStream.getWriter();
    if (res.body.pipeTo) {
      writer.releaseLock();
      return res.body.pipeTo(fileStream);
    }

    const reader = res.body.getReader();
    const pump = () =>
      reader
        .read()
        .then(({ value, done }) => (done ? writer.close() : writer.write(value).then(pump)));

    return pump();
  });
};

and you can use it like this:

const url = "http://urltobigfile";
const fileName = "bigfile.zip";

downloadFile(url, fileName).then(() => { alert('done'); });

Following @guest271314's advice, I added StreamSaver.js to my project, and I successfully received files bigger than 1GB on Chrome. According to the documentation, it should work for files up to 15GB but my browser crashed before that (maximum file size was about 4GB for me).

Note I: to avoid the Blob max size limitation, I also tried to manually append data to the href field of a <a></a> but it failed with files of about 600MB ...

Note II: as amazing as it might seem, the basic technique using createObjectURL works perfectly fine on Firefox for files up to 4GB !!

本文标签： blobDownload large data stream (gt 1Gb) using javascriptStack Overflow