javascript - how to import csv file into postgresql using node js? - Stack Overflow

admin管理员组

文章数量:1425964

I am new to node js.

I have a csv file in my local system that I want to upload it local PostgreSQL Database using node js.

I am trying the following code:

var csv = require('csv-stream');
var request = require('request');
var fs = require('fs');

// All of these arguments are optional.
var options = {
    delimiter : '\t', // default is ,
    endLine : '\n', // default is \n,
// by default read the first line and use values found as columns 
   // columns : ['Settlement Ref No.', 'Order Type','Fulfilment Type','Seller SKU','wsn'],
    escapeChar : '"', // default is an empty string
    enclosedChar : '"' // default is an empty string
}

var csvStream = csv.createStream(options);
 fs.createReadStream('C:\\Users\\YAM\\Documents\\fk_starchi.csv').pipe(csvStream)
    .on('error',function(err){
        console.error(err);
    })
    .on('data',function(data){
        // outputs an object containing a set of key/value pair representing a line found in the csv file.
       // console.log(data);
    })
    .on('column',function(key,value){
        // outputs the column name associated with the value found
      // console.log('#' + key + ' = ' + value);
        console.log('# '   + value);

    })

Its reading data . now i want to import it on postgrsql database.

Where can I get a tutorial or any other help to do this.

I am new to node js.

I have a csv file in my local system that I want to upload it local PostgreSQL Database using node js.

I am trying the following code:

var csv = require('csv-stream');
var request = require('request');
var fs = require('fs');

// All of these arguments are optional.
var options = {
    delimiter : '\t', // default is ,
    endLine : '\n', // default is \n,
// by default read the first line and use values found as columns 
   // columns : ['Settlement Ref No.', 'Order Type','Fulfilment Type','Seller SKU','wsn'],
    escapeChar : '"', // default is an empty string
    enclosedChar : '"' // default is an empty string
}

var csvStream = csv.createStream(options);
 fs.createReadStream('C:\\Users\\YAM\\Documents\\fk_starchi.csv').pipe(csvStream)
    .on('error',function(err){
        console.error(err);
    })
    .on('data',function(data){
        // outputs an object containing a set of key/value pair representing a line found in the csv file.
       // console.log(data);
    })
    .on('column',function(key,value){
        // outputs the column name associated with the value found
      // console.log('#' + key + ' = ' + value);
        console.log('# '   + value);

    })

Its reading data . now i want to import it on postgrsql database.

Where can I get a tutorial or any other help to do this.

• javascript
• node.js
• nodes

Share Improve this question Follow edited May 7, 2015 at 9:45 asked May 7, 2015 at 7:48 user3946530user3946530 9

• Why you need node if postgres has a built-in function? Referer: stackoverflow./questions/2987433/… – vanadium23 Commented May 7, 2015 at 7:51
• I also have to create a file dialog, that will not create in postgresql – user3946530 Commented May 7, 2015 at 7:53
• May be you can write query COPY zip_codes FROM '/path/to/csv/ZIP_CODES.txt' DELIMITER ',' CSV; without csv module? – vanadium23 Commented May 7, 2015 at 7:54
• it should be http path mycsv./file.csv like that – Rajat Modi Commented May 7, 2015 at 7:59
• @RajatModi if i need the local path then what changes should i do? – user3946530 Commented May 7, 2015 at 8:15

 |  Show 4 more ments

1 Answer 1

Sorted by: Reset to default 2

I understand you want to import this cvs file into Postgres.

There's two steps. Reading the file. Writing the data.

1) Reading the file you've done with csv-stream. I don't quite understand what the column event does, but it looks like the 'data' event is where to start. So add your code there.

2) Writing the data.

There's two routes for this:

a) Quick and dirty. In the 'data' event, craft the SQL using strings, then run them with a thin library like node-postgres.

var sql = 'INSERT INTO table VALUES (' data.this + ',' + data.that + ',' + data.theotherthing + ');';

Check out this example for a structure to start. You're already familiar with streams, so you'll just need to manage the callbacks.

You're csv-stream will produce SQL statements faster than postgress will handle them, so you could run into 1000's of simultaneous requests. You might want to + the query strings together in batches, and/or use through2 to query, wait, then query.

The reason NOT to do this is someone could put a SQL injection into the CSV and trash your database.

b) The smart way to do this (especially if dealing with unknown CSV's) is to use an ORM like sequelize.

There isn't a copy/paste and done. A good place to start is reading their homepage.

本文标签： javascripthow to import csv file into postgresql using node jsStack Overflow