admin管理员组文章数量:1277896
For the educational purposes, I wanted to create file chunks upload. How do you guys know when all of the chunks are uploaded?
I tried to move chunks from temp and renaming them so they are in correct order, and then with the last chunk merge them together. However the last piece sent is not the last piece received, I guess. So fopen() on chunks fails since they're not created yet, and I get final file with the size exactly the size of the last chunk.
I believe I could send chunks one by one using .onload event on xhr, that way I wouldn't have to even move them from PHP temp, but I'm wondering if there are different solutions.
Some basic code to please you:
function upload(file) {
var BYTES_PER_CHUNK = parseInt(2097152, 10),
size = file.size,
NUM_CHUNKS = Math.max(Math.ceil(SIZE / BYTES_PER_CHUNK), 1),
start = 0, end = BYTES_PER_CHUNK, num = 1;
var chunkUpload = function(blob) {
var fd = new FormData();
var xhr = new XMLHttpRequest();
fd.append('upload', blob, file.name);
fd.append('num', num);
fd.append('num_chunks', NUM_CHUNKS);
xhr.open('POST', '/somedir/upload.php', true);
xhr.send(fd);
}
while (start < size) {
chunkUpload(file.slice(start, end));
start = end;
end = start + BYTES_PER_CHUNK;
num++;
}
}
And PHP:
$target_path = ROOT.'/upload/';
$tmp_name = $_FILES['upload']['tmp_name'];
$filename = $_FILES['upload']['name'];
$target_file = $target_path.$filename;
$num = $_POST['num'];
$num_chunks = $_POST['num_chunks'];
move_uploaded_file($tmp_name, $target_file.$num);
if ($num === $num_chunks) {
for ($i = 1; $i <= $num_chunks; $i++) {
$file = fopen($target_file.$i, 'rb');
$buff = fread($file, 2097152);
fclose($file);
$final = fopen($target_file, 'ab');
$write = fwrite($final, $buff);
fclose($final);
unlink($target_file.$i);
}
}
For the educational purposes, I wanted to create file chunks upload. How do you guys know when all of the chunks are uploaded?
I tried to move chunks from temp and renaming them so they are in correct order, and then with the last chunk merge them together. However the last piece sent is not the last piece received, I guess. So fopen() on chunks fails since they're not created yet, and I get final file with the size exactly the size of the last chunk.
I believe I could send chunks one by one using .onload event on xhr, that way I wouldn't have to even move them from PHP temp, but I'm wondering if there are different solutions.
Some basic code to please you:
function upload(file) {
var BYTES_PER_CHUNK = parseInt(2097152, 10),
size = file.size,
NUM_CHUNKS = Math.max(Math.ceil(SIZE / BYTES_PER_CHUNK), 1),
start = 0, end = BYTES_PER_CHUNK, num = 1;
var chunkUpload = function(blob) {
var fd = new FormData();
var xhr = new XMLHttpRequest();
fd.append('upload', blob, file.name);
fd.append('num', num);
fd.append('num_chunks', NUM_CHUNKS);
xhr.open('POST', '/somedir/upload.php', true);
xhr.send(fd);
}
while (start < size) {
chunkUpload(file.slice(start, end));
start = end;
end = start + BYTES_PER_CHUNK;
num++;
}
}
And PHP:
$target_path = ROOT.'/upload/';
$tmp_name = $_FILES['upload']['tmp_name'];
$filename = $_FILES['upload']['name'];
$target_file = $target_path.$filename;
$num = $_POST['num'];
$num_chunks = $_POST['num_chunks'];
move_uploaded_file($tmp_name, $target_file.$num);
if ($num === $num_chunks) {
for ($i = 1; $i <= $num_chunks; $i++) {
$file = fopen($target_file.$i, 'rb');
$buff = fread($file, 2097152);
fclose($file);
$final = fopen($target_file, 'ab');
$write = fwrite($final, $buff);
fclose($final);
unlink($target_file.$i);
}
}
2 Answers
Reset to default 8Sorry for my previous ments, I misunderstood a question. This quiestion is interesting and fun to play with.
The expression you are looking for is this:
$target_path = ROOT.'/upload/';
$tmp_name = $_FILES['upload']['tmp_name'];
$filename = $_FILES['upload']['name'];
$target_file = $target_path.$filename;
$num = $_POST['num'];
$num_chunks = $_POST['num_chunks'];
move_uploaded_file($tmp_name, $target_file.$num);
// count ammount of uploaded chunks
$chunksUploaded = 0;
for ( $i = 1, i <= $num; $i++ ) {
if ( file_exists( $target_file.$i ) ) {
++$chunksUploaded;
}
}
// and THAT's what you were asking for
// when this triggers - that means your chunks are uploaded
if ($chunksUploaded === $num_chunks) {
/* here you can reassemble chunks together */
for ($i = 1; $i <= $num_chunks; $i++) {
$file = fopen($target_file.$i, 'rb');
$buff = fread($file, 2097152);
fclose($file);
$final = fopen($target_file, 'ab');
$write = fwrite($final, $buff);
fclose($final);
unlink($target_file.$i);
}
}
And this must be mentioned:
Point of fragility of my version - is when you expect files
'tmp-1',
'tmp-2',
'tmp-3'
but, let's assume that after sending 'tmp-2' we were interrupted - that tmp-2 pollutes tmp folder, and it will interfere with future uploads with the same filename - that would be a sleeping bomb.
To counter that - you must find a way to change tmp to something more original.
'tmp-ABCew-1',
'tmp-ABCew-2',
'tmp-ABCew-3'
is a bit better - where 'ABCew' could be called 'chunksSessionId' - you provide it when sending your POST, you make it randomly. Still, collisions are possible - as space of random names depletes. You could add time to equation - for example - you can see that
'tmp-ABCew-2016-03-17-00-11-22--1',
'tmp-ABCew-2016-03-17-00-11-22--2',
'tmp-ABCew-2016-03-17-00-11-22--3'
Is much more collision-resistant but it is difficult to implement - a whole can of worms here - client date and time is controlled by client and could be spoofed - this data is unreliable.
So making tmp-name unique is a plex task. Designing a system that makes it reliable - is an interesting problem ^ ^ You can play with that.
index.html
<!DOCTYPE HTML>
<html lang="tr-TR">
<head>
<meta charset="UTF-8">
<title>3 Parçalı dosya Yükleme Testi
CHUNK Partial file Upload Test</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link href="https://cdn.jsdelivr/npm/[email protected]/dist/css/bootstrap.min.css" rel="stylesheet">
<script src="https://cdn.jsdelivr/npm/[email protected]/dist/js/bootstrap.bundle.min.js"></script>
<script src="https://ajax.googleapis./ajax/libs/jquery/3.7.1/jquery.min.js"></script>
</head>
<body>
<input type="file" id="f" />
<script src="script3.js"></script>
</body>
</html>
script3.js
(function () {
if (f.files.length) {
processFile();
}
f.addEventListener('change', processFile, false);
// MD5_js: https://stackoverflow./questions/14733374/how-to-generate-an-md5-hash-from-a-string-in-javascript-node-js
var MD5 = function (d) {
var r = M(V(Y(X(d), 8 * d.length)));
return r.toLowerCase();
};
function M(d) {
for (var _, m = '0123456789ABCDEF', f = '', r = 0; r < d.length; r++)
(_ = d.charCodeAt(r)), (f += m.charAt((_ >>> 4) & 15) + m.charAt(15 & _));
return f;
}
function X(d) {
for (var _ = Array(d.length >> 2), m = 0; m < _.length; m++) _[m] = 0;
for (m = 0; m < 8 * d.length; m += 8) _[m >> 5] |= (255 & d.charCodeAt(m / 8)) << m % 32;
return _;
}
function V(d) {
for (var _ = '', m = 0; m < 32 * d.length; m += 8) _ += String.fromCharCode((d[m >> 5] >>> m % 32) & 255);
return _;
}
function Y(d, _) {
(d[_ >> 5] |= 128 << _ % 32), (d[14 + (((_ + 64) >>> 9) << 4)] = _);
for (var m = 1732584193, f = -271733879, r = -1732584194, i = 271733878, n = 0; n < d.length; n += 16) {
var h = m,
t = f,
g = r,
e = i;
(f = md5_ii(
(f = md5_ii(
(f = md5_ii(
(f = md5_ii(
(f = md5_hh(
(f = md5_hh(
(f = md5_hh(
(f = md5_hh(
(f = md5_gg(
(f = md5_gg(
(f = md5_gg(
(f = md5_gg(
(f = md5_ff(
(f = md5_ff(
(f = md5_ff(
(f = md5_ff(
f,
(r = md5_ff(
r,
(i = md5_ff(
i,
(m = md5_ff(
m,
f,
r,
i,
d[n + 0],
7,
-680876936
)),
f,
r,
d[n + 1],
12,
-389564586
)),
m,
f,
d[n + 2],
17,
606105819
)),
i,
m,
d[n + 3],
22,
-1044525330
)),
(r = md5_ff(
r,
(i = md5_ff(
i,
(m = md5_ff(
m,
f,
r,
i,
d[n + 4],
7,
-176418897
)),
f,
r,
d[n + 5],
12,
1200080426
)),
m,
f,
d[n + 6],
17,
-1473231341
)),
i,
m,
d[n + 7],
22,
-45705983
)),
(r = md5_ff(
r,
(i = md5_ff(
i,
(m = md5_ff(
m,
f,
r,
i,
d[n + 8],
7,
1770035416
)),
f,
r,
d[n + 9],
12,
-1958414417
)),
m,
f,
d[n + 10],
17,
-42063
)),
i,
m,
d[n + 11],
22,
-1990404162
)),
(r = md5_ff(
r,
(i = md5_ff(
i,
(m = md5_ff(
m,
f,
r,
i,
d[n + 12],
7,
1804603682
)),
f,
r,
d[n + 13],
12,
-40341101
)),
m,
f,
d[n + 14],
17,
-1502002290
)),
i,
m,
d[n + 15],
22,
1236535329
)),
(r = md5_gg(
r,
(i = md5_gg(
i,
(m = md5_gg(m, f, r, i, d[n + 1], 5, -165796510)),
f,
r,
d[n + 6],
9,
-1069501632
)),
m,
f,
d[n + 11],
14,
643717713
)),
i,
m,
d[n + 0],
20,
-373897302
)),
(r = md5_gg(
r,
(i = md5_gg(
i,
(m = md5_gg(m, f, r, i, d[n + 5], 5, -701558691)),
f,
r,
d[n + 10],
9,
38016083
)),
m,
f,
d[n + 15],
14,
-660478335
)),
i,
m,
d[n + 4],
20,
-405537848
)),
(r = md5_gg(
r,
(i = md5_gg(
i,
(m = md5_gg(m, f, r, i, d[n + 9], 5, 568446438)),
f,
r,
d[n + 14],
9,
-1019803690
)),
m,
f,
d[n + 3],
14,
-187363961
)),
i,
m,
d[n + 8],
20,
1163531501
)),
(r = md5_gg(
r,
(i = md5_gg(
i,
(m = md5_gg(m, f, r, i, d[n + 13], 5, -1444681467)),
f,
r,
d[n + 2],
9,
-51403784
)),
m,
f,
d[n + 7],
14,
1735328473
)),
i,
m,
d[n + 12],
20,
-1926607734
)),
(r = md5_hh(
r,
(i = md5_hh(
i,
(m = md5_hh(m, f, r, i, d[n + 5], 4, -378558)),
f,
r,
d[n + 8],
11,
-2022574463
)),
m,
f,
d[n + 11],
16,
1839030562
)),
i,
m,
d[n + 14],
23,
-35309556
)),
(r = md5_hh(
r,
(i = md5_hh(
i,
(m = md5_hh(m, f, r, i, d[n + 1], 4, -1530992060)),
f,
r,
d[n + 4],
11,
1272893353
)),
m,
f,
d[n + 7],
16,
-155497632
)),
i,
m,
d[n + 10],
23,
-1094730640
)),
(r = md5_hh(
r,
(i = md5_hh(
i,
(m = md5_hh(m, f, r, i, d[n + 13], 4, 681279174)),
f,
r,
d[n + 0],
11,
-358537222
)),
m,
f,
d[n + 3],
16,
-722521979
)),
i,
m,
d[n + 6],
23,
76029189
)),
(r = md5_hh(
r,
(i = md5_hh(
i,
(m = md5_hh(m, f, r, i, d[n + 9], 4, -640364487)),
f,
r,
d[n + 12],
11,
-421815835
)),
m,
f,
d[n + 15],
16,
530742520
)),
i,
m,
d[n + 2],
23,
-995338651
)),
(r = md5_ii(
r,
(i = md5_ii(
i,
(m = md5_ii(m, f, r, i, d[n + 0], 6, -198630844)),
f,
r,
d[n + 7],
10,
1126891415
)),
m,
f,
d[n + 14],
15,
-1416354905
)),
i,
m,
d[n + 5],
21,
-57434055
)),
(r = md5_ii(
r,
(i = md5_ii(
i,
(m = md5_ii(m, f, r, i, d[n + 12], 6, 1700485571)),
f,
r,
d[n + 3],
10,
-1894986606
)),
m,
f,
d[n + 10],
15,
-1051523
)),
i,
m,
d[n + 1],
21,
-2054922799
)),
(r = md5_ii(
r,
(i = md5_ii(
i,
(m = md5_ii(m, f, r, i, d[n + 8], 6, 1873313359)),
f,
r,
d[n + 15],
10,
-30611744
)),
m,
f,
d[n + 6],
15,
-1560198380
)),
i,
m,
d[n + 13],
21,
1309151649
)),
(r = md5_ii(
r,
(i = md5_ii(
i,
(m = md5_ii(m, f, r, i, d[n + 4], 6, -145523070)),
f,
r,
d[n + 11],
10,
-1120210379
)),
m,
f,
d[n + 2],
15,
718787259
)),
i,
m,
d[n + 9],
21,
-343485551
)),
(m = safe_add(m, h)),
(f = safe_add(f, t)),
(r = safe_add(r, g)),
(i = safe_add(i, e));
}
return Array(m, f, r, i);
}
function md5_cmn(d, _, m, f, r, i) {
return safe_add(bit_rol(safe_add(safe_add(_, d), safe_add(f, i)), r), m);
}
function md5_ff(d, _, m, f, r, i, n) {
return md5_cmn((_ & m) | (~_ & f), d, _, r, i, n);
}
function md5_gg(d, _, m, f, r, i, n) {
return md5_cmn((_ & f) | (m & ~f), d, _, r, i, n);
}
function md5_hh(d, _, m, f, r, i, n) {
return md5_cmn(_ ^ m ^ f, d, _, r, i, n);
}
function md5_ii(d, _, m, f, r, i, n) {
return md5_cmn(m ^ (_ | ~f), d, _, r, i, n);
}
function safe_add(d, _) {
var m = (65535 & d) + (65535 & _);
return (((d >> 16) + (_ >> 16) + (m >> 16)) << 16) | (65535 & m);
}
function bit_rol(d, _) {
return (d << _) | (d >>> (32 - _));
}
var MD5_tuz = "'KV,`Z?(>p:_^xk7^}b}Q|b=#0Pf~v++ ?$.}Eoruu%:Ohmt&LfP^lxe6(G+HqXT7'";
function processFile(e) {
var file = f.files[0];
var size = file.size;
var fname = file.name;
var start = 0;
console.log('a Boyutunda Dosya Gönderiliyor : ' + size);
var imzaAyrac = '_' + MD5(size + fname + MD5_tuz) + '_'; // unique signature for each file
console.log('x1 md5 imzaAyrac: ' + imzaAyrac);
upload(file, imzaAyrac, size);
console.log('h islem bitti **The transaction is finished.**..... ');
}
function upload(file, imzaAyrac, size) {
var BYTES_PER_CHUNK = parseInt(2097152, 10), //2mb
//size = file.size,
NUM_CHUNKS = Math.max(Math.ceil(size / BYTES_PER_CHUNK), 1), // Chunk file count
start = 0,
end = BYTES_PER_CHUNK,
num = 1;
console.log('b Pasca Dosya Gönderiliyor...**Pasca File Sending** : ' + size);
var chunkUpload = function (blob) {
var fd = new FormData();
var xhr = new XMLHttpRequest();
console.log('c Parca Dosya Gönderiliyor**Sending Partial Files** (blob): ' + blob);
console.log('d Parca Dosya Gönderiliyor**Sending Partial Files** (num) : ' + num);
console.log('e Parca Dosya Gönderiliyor**Sending Partial Files** (NUM_CHUNKS): ' + NUM_CHUNKS);
fd.append('upload', blob, file.name);
fd.append('num', num);
fd.append('num_chunks', NUM_CHUNKS);
fd.append('imzaAyrac', imzaAyrac);
// xhr.open('POST', '/somedir/upload.php', true);
xhr.open('POST', 'https://2024.umutdemir..tr/tjs/upload3.php', true);
xhr.send(fd);
console.log('f Pasca Dosya Gönderildi **Pasca File Sent** ');
};
while (start < size) {
console.log('g Dosya Parcalama islemleri **File Shredding operations**');
chunkUpload(file.slice(start, end));
start = end;
end = start + BYTES_PER_CHUNK;
num++;
}
}
})();
upload3.php
<?php
// We keep instant log in Log_ud_file.txt file
$Log_ud_file = fopen("uploads/Log_ud_file.txt", "a") or die("Dosya acilamiyor! **The file cannot be opened!**");
$Log = "\n\r\n\r Deneme Başladi **Testing Started**............\n";
fwrite($Log_ud_file, $Log);
$target_path = 'uploads/';
$tmp_name = $_FILES['upload']['tmp_name'];
$filename = $_FILES['upload']['name']; // example: phpservermon-3.5.2.zip
$target_file = $target_path.$filename; // uploads/phpservermon-3.5.2.zip
$num = $_POST['num']; // example: 2 #kacinci dosya parcasi? ** Which piece of file?
$imzaAyrac = $_POST['imzaAyrac']; // _be8c0150d43dc5dbb2134fd2f009e621_
$num_chunks = $_POST['num_chunks']; // example: 3 #dosya toplam kac parca olacak **How many parts will the file have in total?
$target_file2 = $target_path.$imzaAyrac.$filename."_".$num."_.temp";
$Log = "Degiskenler okundu ** Variables read\n";
fwrite($Log_ud_file, $Log);
if ( file_exists( $target_path.$imzaAyrac.$filename."_".$num."_.temp" ) ) {
$chunksUploaded++;
$Log = "Dosya zaten var **The file already exists : $target_file2 \n";
fwrite($Log_ud_file, $Log);
}else{
//_be8c0150d43dc5dbb2134fd2f009e621_phpservermon-3.5.2.zip_2_.temp
move_uploaded_file( $tmp_name, $target_file2 );
$Log = "Dosya yeni olusturuldu **The file has just been created: $target_file2 \n";
fwrite($Log_ud_file, $Log);
}
$Log = "Parca dosya tasindi ** Partial file moved: $num \n";
fwrite($Log_ud_file, $Log);
// Yuklenecek Parca Miktarini say
// Count the Amount of Parts to Be Loaded
$chunksUploaded = 0;
for ( $i = 1; $i <= $num; $i++ ) {
if ( file_exists( $target_path.$imzaAyrac.$filename."_".$i."_.temp" ) ) { // "_be8c0150d43dc5dbb2134fd2f009e621_phpservermon-3.5.2.zip_2_.temp" varsa 2. paket tamadir ** If there is, the 2nd package is plete.
$chunksUploaded++;
$Log = "Parca dosyalardan kaci yuklendi? **How many of the partial files were uploaded? (for) : $chunksUploaded \n";
fwrite($Log_ud_file, $Log);
$Log = "Parca dosyalardan Yuklenen **Uploaded from partial files (for) : $target_path.$imzaAyrac.$filename.'_'.$i.'_.temp' \n";
fwrite($Log_ud_file, $Log);
}
}
// Tum dosya parcalari indi mi?
// Have all the file parts been downloaded?
$Log = "Parca dosyalarTamam mı? **Partial files OK? $chunksUploaded == $num_chunks \n";
fwrite($Log_ud_file, $Log);
if ($chunksUploaded == $num_chunks) {
$Log = "\n\r Parca dosyalarTamam EVET **Part filesOK YES \n";
fwrite($Log_ud_file, $Log);
/* parçaları bir araya getirebiliriz **we can put the pieces together */
for ($i = 1; $i <= $num_chunks; $i++) {
$Log = "Parca dosyalari Birlestir **Merge track files.... \n";
fwrite($Log_ud_file, $Log);
$file = fopen($target_path.$imzaAyrac.$filename."_".$i."_.temp", 'rb'); // "_be8c0150d43dc5dbb2134fd2f009e621_phpservermon-3.5.2.zip_2_.temp"
$buff = fread($file, 2097152);
fclose($file);
$Log = "Parca dosyalari Birlestir **Merge track files : $target_path.$imzaAyrac.$filename.'_'.$i.'_.temp' \n";
fwrite($Log_ud_file, $Log);
$final = fopen($target_file, 'ab');
$write = fwrite($final, $buff);
fclose($final);
$Log = "Parca dosyalari Birlestir Son **Merge part files Finish: $target_file \n";
fwrite($Log_ud_file, $Log);
unlink($target_path.$imzaAyrac.$filename."_".$i."_.temp");
$Log = "Parca dosyalar SiLiNDi **Partial files deleted: $target_path.$imzaAyrac.$filename.'_'.$i.'_.temp' \n";
fwrite($Log_ud_file, $Log);
}
}
$Log = "Deneme ** TEST END Bitti..............................................\n";
fwrite($Log_ud_file, $Log);
fclose($Log_ud_file);
本文标签： javascriptMerging file chunks in PHPStack Overflow
版权声明：本文标题：javascript - Merging file chunks in PHP - Stack Overflow 内容由网友自发贡献，该文观点仅代表作者本人， 转载请联系作者并注明出处：http://www.betaflare.com/web/1741250568a2365708.html， 本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容，一经查实，本站将立刻删除。
发表评论