Node.js Javascript heap out of memory even with 64 GiB --max-old-space-size - Stack Overflow

admin管理员组

文章数量:1123157

I am having an issue of my Javascript application on our deployment server. It processes a few large CSVs (about 3 GB in size) by means of loading into memory (it is necessary because there are interdependencies between them).

However, the application crashes with follow:

[3910795:0x7f2d1c144000]   153589 ms: Scavenge (interleaved) 15.0 (17.9) -> 14.4 (16.1) MB, pooled: 2 MB, 487.30 / 0.00 ms  (average mu = 0.962, current mu = 0.962) allocation failure; 


<--- Last few GCs --->

[3910795:0x7f35941ef000]   118015 ms: Scavenge (interleaved) 10.3 (14.1) -> 9.6 (14.1) MB, pooled: 2 MB, 0.37 / 0.00 ms  (average mu = 0.988, current mu = 0.988) task; 
[3910795:0x7f35941ef000]   151219 ms: Mark-Compact (reduce) 9.8 (14.1) -> 9.5 (11.1) MB, pooled: 0 MB, 32232.90 / 0.01 ms  (+ 151.2 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 32418 ms) (average mu = 0.778, c

<--- JS stacktrace --->


<--- JS stacktrace --->

FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0xe3811e node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
 1: 0xe3811e node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
 2: 0x125faf0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 2: 0x125faf0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 3: 0x125fdc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 4: 0x148d805  [node]
 3: 0x125fdc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 5: 0x14fadce  [node]
 4: 0x148d805  [node]
 5: 0x14fadce  [node]
 6: 0x14a4ab0 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::internal::GarbageCollectionReason, char const*) [node]
 7: 0x14a6629  [node]
 8: 0x14a9ab8  [node]
 9: 0x1ce6351  [node]

I am already running the application with node --max-semi-space-size=1024 --max-old-space-size=65536 and the memory allocation has never reached 64 GiB, while on my local workstation it doesn't crash even I have only allocated 32 GiB to max-old-space-size.

How can I prevent it from crashing?

本文标签： Nodejs Javascript heap out of memory even with 64 GiB maxoldspacesizeStack Overflow