Node.js: Count the number of lines in a file


Question

I have large text files, which range between 30MB and 10GB. How can I count the number of lines in a file using Node.js?

I have these limitations:

  • The entire file does not need to be written to memory
  • A child process is not required to perform the task
1
26
5/20/2013 3:13:13 AM

Accepted Answer

solution without using wc:

var i;
var count = 0;
require('fs').createReadStream(process.argv[2])
  .on('data', function(chunk) {
    for (i=0; i < chunk.length; ++i)
      if (chunk[i] == 10) count++;
  })
  .on('end', function() {
    console.log(count);
  });

it's slower, but not that much you might expect - 0.6s for 140M+ file including node.js loading & startup time

>time node countlines.js video.mp4 
619643

real    0m0.614s
user    0m0.489s
sys 0m0.132s

>time wc -l video.mp4 
619643 video.mp4
real    0m0.133s
user    0m0.108s
sys 0m0.024s

>wc -c video.mp4
144681406  video.mp4
29
9/17/2012 5:31:42 AM

You could do this as the comments suggest using wc

var exec = require('child_process').exec;

exec('wc /path/to/file', function (error, results) {
    console.log(results);
});

Licensed under: CC-BY-SA with attribution
Not affiliated with: Stack Overflow
Icon