NodeJS Bulk Insert MySQL Using Sequelize with Chunk Data

Question

I want to bulk insert MySQL with Sequelize ORM and NodeJS (NestJS). I'm curious if there is a best way to handle very large data. I thought of "chunk" like Laravel did. In Laravel we can do it like this:

$chunks = $insert_data->chunk(500);

foreach ($chunks as $chunk) {
   \DB::table('items_details')->insert($chunk->toArray());
}

Has anyone implemented this with sequelize and NodeJS. Thanks

Anatoly · Accepted Answer · 2022-04-07 17:35:58Z

1

I suppose it would be almost the same (using lodash to get chunks):

const _ = require('lodash');

const chunks = _.chunk(insert_data, 500);

for (const chunk of chunks) {
  await ItemDetails.bulkCreate(chunk);
}

P.S. Depending on what you want to achieve you either use a transaction to insert all chunks or none of them if there will is an error while inserting any of chunks OR use try/catch to bypass or continue inserting other chunks after failed one.

answered Apr 7, 2022 at 17:35

Anatoly

23.2k3 gold badges33 silver badges46 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

NodeJS Bulk Insert MySQL Using Sequelize with Chunk Data

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related