I am running my Node JS backend using typeorm ORM.
Coming from Entity Framework, it was very easy to seed the db with a few lines such as
Database.SetInitializer(new DbInitializer());
Where the DbInitializer class would contain all the seeding info.
Is there a similar approach to seed the database in TypeOrm?
If not, what is the recommended way of doing it?
1) Create a new migration with the data insertion statements?
2) Create a task where you instantiate and save entities?
Unfortunately, there is no officially released solution from TypeORM (at the time this answer was being published).
But there is a nice workaround we can use:
create another connection inside ormconfig.js file and specify another
folder for "migrations" - in fact our seeds
generate and run your seeds with -c <connection name>. That's it!
Sample ormconfig.js:
module.exports = [
{
...,
migrations: [
'src/migrations/*.ts'
],
cli: {
migrationsDir: 'src/migrations',
}
},
{
name: 'seed',
...,
migrations: [
'src/seeds/*.ts'
],
cli: {
migrationsDir: 'src/seeds',
}
}
]
Sample package.json:
{
...
scripts: {
"seed:generate": "ts-node typeorm migration:generate -c seed -n ",
"seed:run": "ts-node typeorm migration:run -c seed",
"seed:revert": "ts-node typeorm migration:revert -c seed",
},
...
}
For those who are using TypeORM with Nest.js, here is a solution to perform your seeding programatically, from within your code.
Rough idea:
We create a dedicated "seeding module" containing a "seeding middleware" that is responsible for conducting the seeding and ensuring that all seeding is done before any request is answered.
For any request that arrives, the seeding middleware intercepts it and postpones it until it is confirmed that seeding is done.
If the db has been seeded, the "seeding middleware" passes the request to the next middleware.
To speed things up, the "seeding middleware" keeps a "seeding complete" flag as state in memory to avoid any further db-checks after the seeding has occurred.
Implementation:
For this to work, first create a module that registers a middleware that listens to all incoming requests:
// file: src/seeding/SeedingModule.ts
#Module({})
export class SeedingModule implements NestModule {
configure(consumer: MiddlewareConsumer) {
consumer
.apply(SeedingMiddleware)
.forRoutes('*')
}
}
Now create the middleware:
// file: src/seeding/SeedingMiddleware.ts
import { Injectable, NestMiddleware } from '#nestjs/common';
import { Request, Response } from 'express';
import { EntityManager } from 'typeorm';
import { SeedingLogEntry } from './entities/SeedingLogEntry.entity';
#Injectable()
export class SeedingMiddleware implements NestMiddleware {
// to avoid roundtrips to db we store the info about whether
// the seeding has been completed as boolean flag in the middleware
// we use a promise to avoid concurrency cases. Concurrency cases may
// occur if other requests also trigger a seeding while it has already
// been started by the first request. The promise can be used by other
// requests to wait for the seeding to finish.
private isSeedingComplete: Promise<boolean>;
constructor(
private readonly entityManager: EntityManager,
) {}
async use(req: Request, res: Response, next: Function) {
if (await this.isSeedingComplete) {
// seeding has already taken place,
// we can short-circuit to the next middleware
return next();
}
this.isSeedingComplete = (async () => {
// for example you start with an initial seeding entry called 'initial-seeding'
// on 2019-06-27. if 'initial-seeding' already exists in db, then this
// part is skipped
if (!await this.entityManager.findOne(SeedingLogEntry, { id: 'initial-seeding' })) {
await this.entityManager.transaction(async transactionalEntityManager => {
await transactionalEntityManager.save(User, initialUsers);
await transactionalEntityManager.save(Role, initialRoles);
// persist in db that 'initial-seeding' is complete
await transactionalEntityManager.save(new SeedingLogEntry('initial-seeding'));
});
}
// now a month later on 2019-07-25 you add another seeding
// entry called 'another-seeding-round' since you want to initialize
// entities that you just created a month later
// since 'initial-seeding' already exists it is skipped but 'another-seeding-round'
// will be executed now.
if (!await this.entityManager.findOne(SeedingLogEntry, { id: 'another-seeding-round' })) {
await this.entityManager.transaction(async transactionalEntityManager => {
await transactionalEntityManager.save(MyNewEntity, initalSeedingForNewEntity);
// persist in db that 'another-seeding-round' is complete
await transactionalEntityManager.save(new SeedingLogEntry('another-seeding-round'));
});
}
return true;
})();
await this.isSeedingComplete;
next();
}
}
Finally here is the entity that we use to record in our db that a seeding of a certain type has occured. Make sure to register it as entity in your TypeOrmModule.forRoot call.
// file: src/seeding/entities/Seeding.entity.ts
import { Entity, PrimaryColumn, CreateDateColumn } from 'typeorm';
#Entity()
export class Seeding {
#PrimaryColumn()
public id: string;
#CreateDateColumn()
creationDate: Date;
constructor(id?: string) {
this.id = id;
}
}
An alternative seeding solution using lifecycle events:
with Nest.js, you can also implement the OnApplicationBootstrap interface (see lifecycle events) instead of going for a middleware-based solution to handle your seedings. The onApplicationBootstrap method will "called once the application has fully started and is bootstrapped". This approach, however, in contrast to a middleware-solution, will not allow you to seed your db in a multi-tenant environment where db-schemas for different tenants will be created at runtime and seeding needs to be conducted several times at runtime for different tenants after they are created.
I would love to see such functionality as well (and we're not alone), but at the moment, there's no official feature for seeding.
in lack of such a built-in feature, I think the next best thing would be to create a migration script named 0-Seed (so it precedes any other migration scripts you might have) and have the seed data populated there.
#bitwit has created a snippet that may come in handy for you; it's a function that reads the data from yaml files, which you can incorporate in the seed migration script.
after some research, however, I found another interesting approach: bind an after_create event to the table, and initialize the data in the listener.
I haven't implemented this, so I'm not sure it can be done directly with TypeORM.
In Nest.js, this is what B12Toaster's alternative solution using OnApplicationBootstrap could look like.
src/seeding.service.ts
import { Injectable, Logger } from '#nestjs/common';
import { EntityManager } from 'typeorm';
import { UserEntity} from 'src/entities/user.entity';
import { RoleEntity } from 'src/entities/role.entity';
import { userSeeds } from 'src/seeds/user.seeds';
import { roleSeeds } from 'src/seeds/role.seeds';
#Injectable()
export class SeedingService {
constructor(
private readonly entityManager: EntityManager,
) {}
async seed(): Promise<void> {
// Replace with your own seeds
await Promise.all([
this.entityManager.save(UserEntity, userSeeds),
this.entityManager.save(RoleEntity, roleSeeds),
]);
}
}
src/app.module.ts
import { Module, OnApplicationBootstrap } from '#nestjs/common'
import { TypeOrmModule } from '#nestjs/typeorm';
import { getConnectionOptions } from 'typeorm';
#Module({
imports: [
TypeOrmModule.forRootAsync({
useFactory: async () =>
Object.assign(await getConnectionOptions(), {
autoLoadEntities: true,
}),
}),
TypeOrmModule.forFeature([
CompanyOrmEntity,
ProductOrmEntity,
]),
],
providers: [
SeedingService,
...
],
...
})
export class AppModule implements OnApplicationBootstrap {
constructor(
private readonly seedingService: SeedingService,
) {}
async onApplicationBootstrap(): Promise<void> {
await this.seedingService.seed();
}
}
Looks like a module is being built for this, typeorm-seeding.
While using a initial migration for seeding also works its not very useful for testing where a freshly seeded DB may be needed for tests to pass. Once you start creating more migrations you can't drop, sync and run migrations without errors. This could be solved by being able to run migration:run for a single migration file but with the CLI you currently can't. My solution was a light weight script that accesses the QueryRunner object through a typeorm connection:
// testSeed.ts
import { ConnectionOptions, createConnection, QueryRunner } from "typeorm";
import { config } from "../config";
import { DevSeed } from "./DevSeed";
createConnection(config.typeOrmConfig as ConnectionOptions).then(async connection => {
let queryRunner = connection.createQueryRunner("master");
// runs all seed SQL commands in this function.
await DevSeed(queryRunner);
await queryRunner.release();
return connection.close();
});
Then run node ./dist/path/to/testSeed.js
Also for NestJS you can use the nestjs-console package to execute tasks. This way you'll have access to entities, services, repositories, etc. I like this better than the middleware solution proposed by #B12Toaster because you don't need to maintain it as production code.
Create a seed command as shown below, then simply: yarn console seed.
There's a working example here (runs in the CI): https://github.com/thisismydesign/nestjs-starter/tree/ee7abf6d481b1420708e87dea3cb99ca110cc168
Along these lines:
src/console.ts
import { BootstrapConsole } from 'nestjs-console';
import { AppModule } from 'src/server/app/app.module';
const bootstrap = new BootstrapConsole({
module: AppModule,
useDecorators: true,
});
bootstrap.init().then(async (app) => {
try {
await app.init();
await bootstrap.boot();
app.close();
process.exit(0);
} catch (e) {
app.close();
process.exit(1);
}
});
src/console/seed.service.ts
import { Inject } from '#nestjs/common';
import { Console, Command } from 'nestjs-console';
import { UsersService } from 'src/users/users.service';
#Console()
export class SeedService {
constructor(
#Inject(UsersService) private usersService: UsersService,
) {}
#Command({
command: 'seed',
description: 'Seed DB',
})
async seed(): Promise<void> {
await this.seedUsers();
}
async seedUsers() {
await this.usersService.create({ name: 'Joe' });
}
}
package.json
{
"scripts": {
"console": "ts-node -r tsconfig-paths/register src/console.ts",
I have modified #B12Toaster's answer (For seeding to the database in NestJs) to be able to take an array of objects to seed. His answer helped considerably and I was also looking for a way to make it take many DB objects at once. Here's the little modification made to the seedingMiddleware.ts
// file: src/seeding/SeedingMiddleware.ts
import { Injectable, NestMiddleware } from '#nestjs/common';
import { Request, Response } from 'express';
import { TxnCategory } from 'src/txn-categories/entities/txn-category.entity';
import { init_categories } from 'src/txn-categories/entities/txn_cat-seed-data';
import { init_services } from 'src/txn-services/entities/txn-serv-seed-data';
import { TxnService } from 'src/txn-services/entities/txn-service.entity';
import { EntityManager } from 'typeorm';
import { Seeding } from './entities/seeding.entity';
#Injectable()
export class SeedingMiddleware implements NestMiddleware {
// to avoid roundtrips to db we store the info about whether
// the seeding has been completed as boolean flag in the middleware
// we use a promise to avoid concurrency cases. Concurrency cases may
// occur if other requests also trigger a seeding while it has already
// been started by the first request. The promise can be used by other
// requests to wait for the seeding to finish.
private isSeedingComplete: Promise<boolean>;
constructor(private readonly entityManager: EntityManager) {}
async use(req: Request, res: Response, next: any) {
if (await this.isSeedingComplete) {
// seeding has already taken place,
// we can short-circuit to the next middleware
return next();
}
this.isSeedingComplete = (async () => {
// for example you start with an initial seeding entry called 'initial-seeding'
// if 'init-txn-cats' and 'init-txn-serv' already exists in db, then this
// part is skipped
// MODIFIED
if (
!(await this.entityManager.findOne(Seeding, {
id: 'init-txn-cats',
}))
) {
await this.entityManager.transaction(
async (transactionalEntityManager) => {
for (let i = 0; i < init_categories.length; i++) {
await transactionalEntityManager.save(
TxnCategory,
init_categories[i],
);
}
await transactionalEntityManager.save(new Seeding('init-txn-cats'));
},
);
}
// MODIFIED
if (
!(await this.entityManager.findOne(Seeding, {
id: 'init-txn-serv',
}))
) {
await this.entityManager.transaction(
async (transactionalEntityManager) => {
for (let i = 0; i < init_services.length; i++) {
await transactionalEntityManager.save(
TxnService,
init_services[i],
);
}
await transactionalEntityManager.save(new Seeding('init-txn-serv'));
},
);
}
return true;
})();
await this.isSeedingComplete;
next();
}
}
Then the array of DB objects referenced would be like so:
// file: src/txn-categories/entities/txn_cat-seed-data.ts
export const init_categories = [
{
id: 1,
category_name: 'name 1',
category_code: 'cat_code_1',
enabled: true,
},
{
id: 2,
category_name: 'name 2',
category_code: 'cat_code_2',
enabled: true,
},
{
id: 3,
category_name: 'name 3',
category_code: 'cat_code_3',
enabled: true,
},
// etc
];
Same format for the src/txn-services/entities/txn-serv-seed-data.ts file.
Every other thing in B12Toaster's answer remains the same hence you'll still have your module and entity files as below:
SeedingModule:
// file: src/seeding/SeedingModule.ts
#Module({})
export class SeedingModule {
configure(consumer: MiddlewareConsumer) {
consumer
.apply(SeedingMiddleware)
.forRoutes('*')
}
}
SeedingEntity:
// file: src/seeding/entities/Seeding.entity.ts
import { Entity, PrimaryColumn, CreateDateColumn } from 'typeorm';
#Entity()
export class Seeding {
#PrimaryColumn()
public id: string;
#CreateDateColumn()
creationDate: Date;
constructor(id?: string) {
this.id = id;
}
}
Cheers!
The simplest and most efficient way to do it is by creating a new migration file as follows:
import { MigrationInterface, QueryRunner } from 'typeorm';
export class <Class Name> implements MigrationInterface {
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.connection
.createQueryBuilder()
.insert()
.into('table_name', ['columns_1','column_2',...])
.values([
{
columns_1: value,
},
{
column_2: value
}
])
.execute();
}
}
Run the following code and it will work like a charm!
so heres how I am seeding data from an sql file with insert statement.
Heres my whole migration file after adding the seed
import { MigrationInterface, QueryRunner } from 'typeorm';
import * as path from 'path';
import * as fs from 'fs';
let insertPermissionQueries = fs
.readFileSync(path.resolve(__dirname, '../../scripts/sql/insert.sql'))
.toString()
.replace(/(\r\n|\n|\r)/gm, ' ') // remove newlines
.replace(/\s+/g, ' '); // excess white space
export class init1591103087130 implements MigrationInterface {
name = 'init1591103087130';
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`CREATE TABLE "public"."RoleTemp" ("idx" uuid NOT NULL DEFAULT uuid_generate_v4(), "name" text, "created_on" TIMESTAMP DEFAULT now(), "is_active" boolean DEFAULT true, "role_type" text, "created_by" uuid NOT NULL, "status" text, "alias" text, "operation" text, "rejection_reason" text, "id" SERIAL NOT NULL, "is_obsolete" boolean NOT NULL DEFAULT false, "modified_on" TIMESTAMP DEFAULT CURRENT_TIMESTAMP, "role_id" integer, CONSTRAINT "UQ_835baad60041a3413f9ef95bc07" UNIQUE ("idx"), CONSTRAINT "PK_a76dd0012be252eefbdd4a2a589" PRIMARY KEY ("id"))`,
);
await queryRunner.query(
`CREATE UNIQUE INDEX "RoleTemp_idx_key" ON "public"."RoleTemp" ("idx") `,
);
await queryRunner.query(
`CREATE TABLE "public"."PermissionRoleTemp" ("idx" uuid NOT NULL DEFAULT uuid_generate_v4(), "created_on" TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, "permission_base_name" text, "id" SERIAL NOT NULL, "is_obsolete" boolean NOT NULL DEFAULT false, "modified_on" TIMESTAMP DEFAULT CURRENT_TIMESTAMP, "role_id" integer NOT NULL, "permission_id" integer NOT NULL, CONSTRAINT "PK_c1f2648a18ac911e096f08c187d" PRIMARY KEY ("id"))`,
);
await queryRunner.query(
`CREATE TABLE "public"."Permission" ("idx" uuid NOT NULL DEFAULT uuid_generate_v4(), "base_name" text NOT NULL, "url" text NOT NULL, "method" text NOT NULL, "created_on" TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, "is_active" boolean NOT NULL DEFAULT true, "permission_type" text, "alias" text NOT NULL, "id" SERIAL NOT NULL, "is_obsolete" boolean NOT NULL DEFAULT false, "modified_on" TIMESTAMP DEFAULT CURRENT_TIMESTAMP, CONSTRAINT "PK_28657fa560adca66b359c18b952" PRIMARY KEY ("id"))`,
);
await queryRunner.query(
`CREATE TABLE "public"."PermissionRole" ("idx" uuid NOT NULL DEFAULT uuid_generate_v4(), "created_on" TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, "is_active" boolean NOT NULL DEFAULT true, "permission_base_name" text NOT NULL, "id" SERIAL NOT NULL, "is_obsolete" boolean NOT NULL DEFAULT false, "modified_on" TIMESTAMP DEFAULT CURRENT_TIMESTAMP, "role_id" integer NOT NULL, "permission_id" integer NOT NULL, CONSTRAINT "PK_b5e2271c229f65f17ee93677a0f" PRIMARY KEY ("id"))`,
);
await queryRunner.query(
`CREATE TABLE "public"."UserRole" ("idx" uuid NOT NULL DEFAULT uuid_generate_v4(), "created_on" TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, "is_active" boolean NOT NULL DEFAULT true, "id" SERIAL NOT NULL, "is_obsolete" boolean NOT NULL DEFAULT false, "modified_on" TIMESTAMP DEFAULT CURRENT_TIMESTAMP, "role_id" integer NOT NULL, "company_user_id" integer NOT NULL, CONSTRAINT "PK_431fc1ec3d46ac513ef3701604e" PRIMARY KEY ("id"))`,
);
await queryRunner.query(
`CREATE TABLE "public"."UsersTemp" ("idx" uuid DEFAULT uuid_generate_v1(), "username" text, "first_name" text, "middle_name" text, "last_name" text, "password" text, "email" text, "address" text, "phone_number" text, "phone_ext" text, "company_idx" uuid, "is_superadmin" boolean NOT NULL DEFAULT false, "operation" text, "created_by" text, "status" text, "rejection_reason" text, "created_on" TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, "is_active" boolean NOT NULL DEFAULT true, "id" SERIAL NOT NULL, "is_obsolete" boolean NOT NULL DEFAULT false, "modified_on" TIMESTAMP DEFAULT CURRENT_TIMESTAMP, "role_id" integer, "user_id" integer, CONSTRAINT "PK_9d3fbcec3cc0b054324f93da038" PRIMARY KEY ("id"))`,
);
await queryRunner.query(
`CREATE TABLE "public"."Role" ("idx" uuid NOT NULL DEFAULT uuid_generate_v4(), "name" text NOT NULL, "alias" text NOT NULL, "created_on" TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, "is_active" boolean NOT NULL DEFAULT true, "role_type" text, "created_by" uuid NOT NULL, "id" SERIAL NOT NULL, "is_obsolete" boolean NOT NULL DEFAULT false, "modified_on" TIMESTAMP DEFAULT CURRENT_TIMESTAMP, CONSTRAINT "UQ_c9a53325a7642edb5f9bd44f5aa" UNIQUE ("idx"), CONSTRAINT "PK_422113329ddec949e76c7943c56" PRIMARY KEY ("id"))`,
);
await queryRunner.query(
`CREATE UNIQUE INDEX "Role_idx_key" ON "public"."Role" ("idx") `,
);
await queryRunner.query(
`CREATE TABLE "public"."Users" ("idx" uuid NOT NULL DEFAULT uuid_generate_v4(), "username" text NOT NULL, "first_name" text NOT NULL, "middle_name" text, "last_name" text NOT NULL, "password" text NOT NULL, "email" text, "address" text, "phone_number" text, "phone_ext" text, "company_idx" uuid, "created_on" TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, "is_active" boolean NOT NULL DEFAULT true, "is_superadmin" boolean NOT NULL DEFAULT false, "id" SERIAL NOT NULL, "is_obsolete" boolean NOT NULL DEFAULT false, "modified_on" TIMESTAMP DEFAULT CURRENT_TIMESTAMP, "role_id" integer NOT NULL, CONSTRAINT "PK_ac3c96e3c912cbda773b7c7edc9" PRIMARY KEY ("id"))`,
);
await queryRunner.query(
`CREATE TABLE "public"."CompanyUser" ("id" SERIAL NOT NULL, "is_obsolete" boolean NOT NULL DEFAULT false, "modified_on" TIMESTAMP DEFAULT CURRENT_TIMESTAMP, "idx" uuid NOT NULL DEFAULT uuid_generate_v4(), "company_idx" uuid, "created_on" TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, "is_active" boolean NOT NULL DEFAULT true, "user_id" integer, CONSTRAINT "PK_4a915d69bf079a8e5dd10784cc3" PRIMARY KEY ("id"))`,
);
await queryRunner.query(
`ALTER TABLE "public"."RoleTemp" ADD CONSTRAINT "FK_d304588d17c9349ca6e7ebee5d3" FOREIGN KEY ("role_id") REFERENCES "public"."Role"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "public"."PermissionRoleTemp" ADD CONSTRAINT "FK_7e7cdde853500f56b3db43fc258" FOREIGN KEY ("role_id") REFERENCES "public"."RoleTemp"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "public"."PermissionRoleTemp" ADD CONSTRAINT "FK_0068d3de1c59050561d35f17544" FOREIGN KEY ("permission_id") REFERENCES "public"."Permission"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "public"."PermissionRole" ADD CONSTRAINT "FK_5b57492441a568bc7562fbbaa5b" FOREIGN KEY ("role_id") REFERENCES "public"."Role"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "public"."PermissionRole" ADD CONSTRAINT "FK_1951a810af06342fcd4530ec61c" FOREIGN KEY ("permission_id") REFERENCES "public"."Permission"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "public"."UserRole" ADD CONSTRAINT "FK_fb09d73b0dd011be81a272e1efa" FOREIGN KEY ("role_id") REFERENCES "public"."Role"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "public"."UserRole" ADD CONSTRAINT "FK_b221977a41587e58d7c58e16db0" FOREIGN KEY ("company_user_id") REFERENCES "public"."CompanyUser"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "public"."UsersTemp" ADD CONSTRAINT "FK_6d74dfaddaa94e1bba0c8c12a2f" FOREIGN KEY ("role_id") REFERENCES "public"."Role"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "public"."UsersTemp" ADD CONSTRAINT "FK_e5b2930fe35042dab17945bb131" FOREIGN KEY ("user_id") REFERENCES "public"."Users"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "public"."Users" ADD CONSTRAINT "FK_34be125e29cee0e71d58456aed7" FOREIGN KEY ("role_id") REFERENCES "public"."Role"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "public"."CompanyUser" ADD CONSTRAINT "FK_1354e3e408b5ffdebe476a6fbd2" FOREIGN KEY ("user_id") REFERENCES "public"."Users"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(insertPermissionQueries);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "public"."CompanyUser" DROP CONSTRAINT "FK_1354e3e408b5ffdebe476a6fbd2"`,
);
await queryRunner.query(
`ALTER TABLE "public"."Users" DROP CONSTRAINT "FK_34be125e29cee0e71d58456aed7"`,
);
await queryRunner.query(
`ALTER TABLE "public"."UsersTemp" DROP CONSTRAINT "FK_e5b2930fe35042dab17945bb131"`,
);
await queryRunner.query(
`ALTER TABLE "public"."UsersTemp" DROP CONSTRAINT "FK_6d74dfaddaa94e1bba0c8c12a2f"`,
);
await queryRunner.query(
`ALTER TABLE "public"."UserRole" DROP CONSTRAINT "FK_b221977a41587e58d7c58e16db0"`,
);
await queryRunner.query(
`ALTER TABLE "public"."UserRole" DROP CONSTRAINT "FK_fb09d73b0dd011be81a272e1efa"`,
);
await queryRunner.query(
`ALTER TABLE "public"."PermissionRole" DROP CONSTRAINT "FK_1951a810af06342fcd4530ec61c"`,
);
await queryRunner.query(
`ALTER TABLE "public"."PermissionRole" DROP CONSTRAINT "FK_5b57492441a568bc7562fbbaa5b"`,
);
await queryRunner.query(
`ALTER TABLE "public"."PermissionRoleTemp" DROP CONSTRAINT "FK_0068d3de1c59050561d35f17544"`,
);
await queryRunner.query(
`ALTER TABLE "public"."PermissionRoleTemp" DROP CONSTRAINT "FK_7e7cdde853500f56b3db43fc258"`,
);
await queryRunner.query(
`ALTER TABLE "public"."RoleTemp" DROP CONSTRAINT "FK_d304588d17c9349ca6e7ebee5d3"`,
);
await queryRunner.query(`DROP TABLE "public"."CompanyUser"`);
await queryRunner.query(`DROP TABLE "public"."Users"`);
await queryRunner.query(`DROP INDEX "public"."Role_idx_key"`);
await queryRunner.query(`DROP TABLE "public"."Role"`);
await queryRunner.query(`DROP TABLE "public"."UsersTemp"`);
await queryRunner.query(`DROP TABLE "public"."UserRole"`);
await queryRunner.query(`DROP TABLE "public"."PermissionRole"`);
await queryRunner.query(`DROP TABLE "public"."Permission"`);
await queryRunner.query(`DROP TABLE "public"."PermissionRoleTemp"`);
await queryRunner.query(`DROP INDEX "public"."RoleTemp_idx_key"`);
await queryRunner.query(`DROP TABLE "public"."RoleTemp"`);
}
}
I used a simpler method with migrations here is my code i believe it should be simpler so run this in your migrations
import { MigrationInterface, QueryRunner } from 'typeorm';
const tableName = 'foo';
const columnName = 'foo_column';
const features = ['foo_content_1', 'foo_content_2'];
export class seedIntoPermissionsTable1638518166717 implements MigrationInterface {
public async up(queryRunner: QueryRunner): Promise<void> {
await Promise.all(features.map((feature) => queryRunner.query(`INSERT INTO ${tableName} (${columnName}) VALUES ('${feature}')`)));
}
public async down(queryRunner: QueryRunner): Promise<void> {
await Promise.all(features.map((feature) => queryRunner.query(`DELETE FROM ${tableName} WHERE ${columnName}='${feature}';`)));
}
}
This is what i tend to use for my seeders.
You can also give https://github.com/tada5hi/typeorm-extension a try for populating the database with (fake) data.
Related
Is there a simple way to seed data in typeORM v.0.3.6 with DataSource ? typeorm-seeding seems to use Connection which is deprecated.
Found this package
https://www.npmjs.com/package/typeorm-extension
It has seeding feature and supports typeorm 0.3.x
Simple seed example:
import { Seeder, SeederFactoryManager } from 'typeorm-extension';
import { DataSource } from 'typeorm';
import { CategoryEntity } from 'src/entities/category.entity';
export default class CategorySeeder implements Seeder {
public async run(
dataSource: DataSource,
factoryManager: SeederFactoryManager
): Promise<any> {
const repository = dataSource.getRepository(CategoryEntity);
await repository.insert([
{
name: "Cats"
},
{
name: "Dogs"
}
]);
}
}
run seeds with npx typeorm-extension seed
I've just solved the same issue. So, you need to create one more connection in your config file through the DataSource (if you already connected through DataSource, than no need), my DataSource connection looks like this:
export const MigrationAppDataSource = new DataSource({
type: "postgres",
host: process.env.DB_HOST,
port: parseInt(process.env.DB_PORT),
username: process.env.DB_USERNAME,
password: process.env.DB_PASSWORD,
database: process.env.DB_DATABASE,
entities: ["../**/*.entity.ts"],
migrations: ["dist/migrations/*{.ts,.js}"],
synchronize: false,
});
Also have to mention: you have to set synchronize to false in both connections (if you have 2+ ofc).
The next step is try to create simple migration. My package.json snippet for creating a simple seed-migration:
"typeorm": "typeorm-ts-node-commonjs",
"db:createMigration": "typeorm migration:create",
Remember to enter a path for seed-migration, so your code should look like this:
npm run db:createMigration src/migrations/SeedName
If everything is cool, then you have to change the timestamp of this migration and insert the seed data you need with SQL code mine is:
export class Seed2617378125500 implements MigrationInterface {
name = "Seed2617378125500";
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`INSERT INTO package_entity (id, created_at, name, description, price) VALUES(1, '20062022', 'Creative Direction', '', '450')`,
);
await queryRunner.query(
`INSERT INTO project_type (id, created_at, name) VALUES(1, '20062022', 'Animation')`,
);
await queryRunner.query(
`INSERT INTO project_type_packages_package_entity ("projectTypeId", "packageEntityId") VALUES(1, 3)`,
);
await queryRunner.query(
`INSERT INTO project_type_packages_package_entity ("projectTypeId", "packageEntityId") VALUES(1, 4)`,
);
await queryRunner.query(
`INSERT INTO project_type_packages_package_entity ("projectTypeId", "packageEntityId") VALUES(1, 5)`,
);
await queryRunner.query(
`INSERT INTO project_type_packages_package_entity ("projectTypeId", "packageEntityId") VALUES(1, 6)`,
);
await queryRunner.query(
`INSERT INTO project_type_packages_package_entity ("projectTypeId", "packageEntityId") VALUES(1, 11)`,
);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`DELETE * FROM project_type`);
await queryRunner.query(`DELETE * FROM package_entity`);
}
}
TIP: My seed is including the many-to-many connection and i was struggling trying to understand how to pass values in many-to-many connection columns. So you need to pass values in middle-table, which is created during initialization of your connected columns.
The next step for me is generate initial migration to create DB itslef:
"db:migrate": "npm run build && node --require ts-node/register ./node_modules/typeorm/cli.js -d src/config/configuration.ts migration:generate",
Remember to provide the same path as your seed-migration:
npm run db:migrate src/migrations/MigrationName
Also have to mention: you have to provide -d path/to/configWithDataSource in the command generating migrations and the command running migrations.
When my initial generation is generated and seeds are also done, i simply run a command to run migrations (you don't need to enter the path there, because it takes path from your DataSource file), mine:
"db:migrationRun": "npm run build && npx typeorm-ts-node-commonjs migration:run -d src/config/configuration.ts"
Enjoy!
If you have some questions - feel free to ask me :)
I used "typeorm-seeding": "^1.6.1" and the following were the steps I took.
Create an ormconfig.ts file with the content below
require('dotenv').config();
module.exports = {
type: "postgres",
host: process.env.DB_HOST,
port: 5432,
username: process.env.DB_USERNAME,
password: process.env.DB_PASSWORD,
database: process.env.DB_NAME,
entities: ["src/db/entities/**/*.ts"],
seeds: ['src/db/seeds/**/*{.ts,.js}'],
factories: ['src/db/factories/**/*{.ts,.js}'],
}
You have your entity defined in src/db/entities according to the above ormconfig.ts and assuming you have your User.ts entity defined there with the following content
#Entity()
export class User {
#PrimaryGeneratedColumn("uuid")
id: string
#Column()
firstName: string
#Column()
lastName: string
#IsEmail()
#Column({
unique: true,
})
email: string
#Column()
password: string
}
Create your seeder file src/db/seeds/createUser.ts
import { Factory, Seeder } from 'typeorm-seeding'
import { DataSource } from 'typeorm'
import { User } from '../entities/User'
import * as bcrypt from "bcrypt"
export default class CreateUsers implements Seeder {
public async run(factory: Factory, datasource: DataSource): Promise<any> {
const salt = 10
const password = await bcrypt.hash("password", salt)
await datasource
.createQueryBuilder()
.insert()
.into(User)
.values([
{ firstName: 'Timber', lastName: 'Saw', email: "example#gmail.com", password },
{ firstName: 'Phantom', lastName: 'Lancer', email: "example#gmail.com", password},
])
.execute()
}
}
With the command npm run seed:run then your seeder is done
You can check Typeorm Seeding Link for more information.
In the project setting, I have 2 entities: Organization and Issue. One such "organization" has many "issues" belonging to it.
Issue has a column named status and the values are "Done", "In Progress", "Rejected", etc.
Given an organizationId, I am trying to find the organization with all its issues, except for those whose status is 'Done'.
Organization:
class Organization extends BaseEntity {
... other code
#OneToMany(
() => Issue,
issue => issue.dstOrg,
)
receivedIssues: Issue[];
}
Issue:
class Issue extends BaseEntity {
... other code
#Column('varchar')
status: IssueStatus;
}
Some helper code:
type EntityConstructor = typeof Organization | typeof User | typeof Issue | ...
const findEntityOrThrow = async <T extends EntityConstructor>(
Constructor: T,
id: number | string,
options?: FindOneOptions,
): Promise<InstanceType<T>> => {
const instance = await Constructor.findOne(id, options);
if (!instance) {
throw new EntityNotFoundError(Constructor.name);
}
return instance;
};
If the query does not limit the status of Issues:
const organizationId = 1;
const organization = await findEntityOrThrow(Organization, organizationId, {
relations: ['receivedIssues'],
});
console.log(organization.receivedIssues)
It works well. organization now contains a receivedIssues field and it contains all the issues.
However, the code that does the complete query fails:
const organizationId = 1;
const organization = await findEntityOrThrow(Organization, organizationId, {
relations: ['receivedIssues'],
where: {
receivedIssues: {
status: Not('Done')
}
}
});
console.log(organization.receivedIssues)
This throws an error:
EntityColumnNotFound: No entity column "receivedIssues" was found.
Why am I missing?
Second Question:
If I do not use the helper function and use findOne() directly:
const organization = await Organization.findOne(organizationId, {
relations: ['receivedIssues'],
where: {
receivedIssues: {
status: Not('Done')
}
}
})
const allReceivedIssues = organization.receivedIssues;
I get this error:
src/controllers/organizations.ts:71:29 - error TS2532: Object is possibly 'undefined'.
71 const allReceivedIssues = organization.receivedIssues;
How can I fix this one if I want to use findOne() directly instead of the helper function?
Below are my original entities:
#Entity()
export class User {
#PrimaryGeneratedColumn()
id: number;
#Column()
firstName: string;
#Column()
lastName: string;
#Column()
age: number;
}
#Entity()
export class Company {
#PrimaryGeneratedColumn()
id: number;
#Column()
name: string;
}
Below is the migration script that I generated using typeorm migration:generate:
export class schemaCreation1627513975044 implements MigrationInterface {
name = "schemaCreation1627513975044";
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`CREATE TABLE "company" ("id" SERIAL NOT NULL, "name" character varying NOT NULL, CONSTRAINT "PK_056f7854a7afdba7cbd6d45fc20" PRIMARY KEY ("id"))`
);
await queryRunner.query(
`CREATE TABLE "user" ("id" SERIAL NOT NULL, "firstName" character varying NOT NULL, "lastName" character varying NOT NULL, "age" integer NOT NULL, CONSTRAINT "PK_cace4a159ff9f2512dd42373760" PRIMARY KEY ("id"))`
);
const user = new User();
user.age = 1;
user.firstName = "first";
user.lastName = "user";
await queryRunner.manager.save(user);
const company = new Company();
company.name = "company";
await queryRunner.manager.save(company);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`DROP TABLE "user"`);
await queryRunner.query(`DROP TABLE "company"`);
}
}
I run the script, the schema is created and is initialized with data.
Later during the development of my app, I updated the User entity such that it now has a FK to Company and so the entities have been updated as follows:
#Entity()
export class User {
#PrimaryGeneratedColumn()
id: number;
#Column()
firstName: string;
#Column()
lastName: string;
#Column()
age: number;
// Adding FK to Company
#ManyToOne(() => Company, (company) => company.users)
#JoinColumn({ name: "company_id" })
company: Company;
#Column({
name: "company_id",
})
companyId: number;
}
#Entity()
export class Company {
#PrimaryGeneratedColumn()
id: number;
#Column()
name: string;
#OneToMany(() => User, (user) => user.company)
users: User[];
}
And below is the migration script I created, using typeorm migration:create to update the existing users:
export class addCompanyToUser1627514375000 implements MigrationInterface {
name = "addCompanyToUser1627514375000";
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`ALTER TABLE "user" ADD "company_id" integer`);
await queryRunner.query(
`ALTER TABLE "user" ADD CONSTRAINT "FK_9e70b5f9d7095018e86970c7874" FOREIGN KEY ("company_id") REFERENCES "company"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`
);
const targetCompany = await queryRunner.manager
.createQueryBuilder(Company, "c")
.where("c.name = :name", { name: "company" })
.getOne();
await queryRunner.manager
.createQueryBuilder(User, "u")
.update({
companyId: targetCompany.id,
})
.execute();
await queryRunner.query(
'ALTER TABLE "user" ALTER COLUMN "company_id" SET NOT NULL'
);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "user" DROP CONSTRAINT "FK_9e70b5f9d7095018e86970c7874"`
);
await queryRunner.query(`ALTER TABLE "user" DROP COLUMN "company_id"`);
}
}
I run the new migration script, and the schema for user has been updated to include a company_id column. All of this work fine if the first script was run independently of the second script.
The Error
If, on the other hand, I run the scripts all at once, as in the case where I'm trying to stand up a new environment, I get the following error:
query failed: INSERT INTO "user"("firstName", "lastName", "age", "company_id") VALUES ($1, $2, $3, DEFAULT) RETURNING "id" -- PARAMETERS: ["first","user",1]
error: error: column "company_id" of relation "user" does not exist
at Parser.parseErrorMessage (/Users/junior/Source/typeorm-migration-issue/node_modules/pg-protocol/dist/parser.js:287:98)
at Parser.handlePacket (/Users/junior/Source/typeorm-migration-issue/node_modules/pg-protocol/dist/parser.js:126:29)
at Parser.parse (/Users/junior/Source/typeorm-migration-issue/node_modules/pg-protocol/dist/parser.js:39:38)
at Socket.<anonymous> (/Users/junior/Source/typeorm-migration-issue/node_modules/pg-protocol/dist/index.js:11:42)
at Socket.emit (events.js:200:13)
at addChunk (_stream_readable.js:290:12)
at readableAddChunk (_stream_readable.js:271:11)
at Socket.Readable.push (_stream_readable.js:226:10)
at TCP.onStreamRead (internal/stream_base_commons.js:166:17) {
length: 127,
name: 'error',
severity: 'ERROR',
code: '42703',
detail: undefined,
hint: undefined,
position: '52',
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: undefined,
table: undefined,
column: undefined,
dataType: undefined,
constraint: undefined,
file: 'parse_target.c',
line: '1034',
routine: 'checkInsertTargets'
}
I understand the error, but I don't know what the proper solution is. The core problem is the original shape of the User entity and corresponding schema that is generated, in schemaCreation1627513975044, does not match the current shape of User
I'm facing this issue in a much larger project where the schema has evolved over time and migration scripts were created that insert data based on the shape of the original Entity.
What would be the best way to handle this scenario?
Repo
See typeorm-migration-issue for working example.
yarn install
yarn build
yarn typeorm migration:run
I have 2 migrations Users and Posts and Posts has a onDelete: 'CASCADE'. For some reason when I delete a User with a Posts it throws an error saying:
"Cannot delete or update a parent row: a foreign key constraint fails (`development_db`.`posts`, CONSTRAINT `posts_ibfk_1` FOREIGN KEY (`userId`) REFERENCES `users` (`id`))",
but I already set my Posts entity to onDelete: 'CASCADE'. What trips me off is when I add ON DELETE CASCADE on my posts migration the cascade delete works even though I removed onDelete: 'CASCADE' on my posts model. Any idea? so what's the use of onDelete in typeorm when you can set it on migration and not on the entity but still works.
USER Migration:
/* eslint-disable class-methods-use-this */
import { MigrationInterface, QueryRunner } from 'typeorm';
export class UsersTable1575433829516 implements MigrationInterface {
public async up(queryRunner: QueryRunner): Promise<any> {
await queryRunner.query(`
CREATE TABLE users (
id INT AUTO_INCREMENT,
username VARCHAR(50) NOT NULL,
email VARCHAR(50) NOT NULL UNIQUE,
password VARCHAR(255) NOT NULL,
createdAt DATETIME NOT NULL,
PRIMARY KEY(id)
);
`);
}
public async down(queryRunner: QueryRunner): Promise<any> {
await queryRunner.dropTable('users', true);
}
}
POST Migration:
/* eslint-disable class-methods-use-this */
import { MigrationInterface, QueryRunner } from 'typeorm';
export class PostsTable1581617587575 implements MigrationInterface {
public async up(queryRunner: QueryRunner): Promise<any> {
await queryRunner.query(`
CREATE TABLE posts (
id INT AUTO_INCREMENT,
title VARCHAR(255) NOT NULL,
body TEXT(65000) NOT NULL,
createdAt DATETIME NOT NULL,
updatedAt DATETIME,
PRIMARY KEY(id),
userId INT NOT NULL,
FOREIGN KEY (userId)
REFERENCES users (id)
);
`);
}
public async down(queryRunner: QueryRunner): Promise<any> {
await queryRunner.dropTable('posts', true);
}
}
I was running into this same error message ("Cannot delete or update a parent row: a foreign key constraint fails") because I had put the onDelete: 'CASCADE' in the #OneToMany decorator, instead of the #ManyToOne decorator, where it needed to be in order to generate the migration correctly.
Here's what didn't work for me:
#Entity()
export class User {
//...
#OneToMany(() => Post, (post) => post.user, { onDelete: 'CASCADE' })
public posts: Post[];
//...
}
Here's what worked for me:
#Entity()
export class Post {
// ...
#ManyToOne(() => User, (user) => user.posts, { onDelete: 'CASCADE' })
public user: User;
// ...
}
I'm guessing this is the issue, because based on one of your comments, it sounds like you're coming from the Rails world. In Rails, you specify the 'cascade delete' on the 'One' side (has_many :posts, dependent: :destroy). In contrast, TypeORM seems to need it on the 'Many' side.
Just add to your migration the property ON DELETE, like this !
/* eslint-disable class-methods-use-this */
import { MigrationInterface, QueryRunner } from 'typeorm';
export class PostsTable1581617587575 implements MigrationInterface {
public async up(queryRunner: QueryRunner): Promise<any> {
await queryRunner.query(`
CREATE TABLE posts (
id INT AUTO_INCREMENT,
title VARCHAR(255) NOT NULL,
body TEXT(65000) NOT NULL,
createdAt DATETIME NOT NULL,
updatedAt DATETIME,
PRIMARY KEY(id),
userId INT NOT NULL,
FOREIGN KEY (userId)
REFERENCES users (id)
ON DELETE CASCADE
);
`);
}
public async down(queryRunner: QueryRunner): Promise<any> {
await queryRunner.dropTable('posts', true);
}
}
I have an Entity Transaction and an Entity Integration
#Entity()
export default class Transaction {
#PrimaryGeneratedColumn()
public id_trans?: number;
#OneToOne(type => Integration, i => i.transaction, { nullable: true })
public integration?: Integration;
}
and
#Entity()
export default class Integration {
#PrimaryGeneratedColumn()
public id_cust?: number;
#OneToOne(type => Transaction, t => t.integration)
#JoinColumn({ referencedColumnName: 'id_trans', name: 'int_id_module' })
public transaction?: Transaction;
}
I tried to query the Transaction by using a Integration property as filter.
const id_api = 10;
const transaction = await repository.find({
where: { integration: { int_id_api: id_api} },
relations: ['integration', 'customer'],
});
but it returns the entire table of Transactions, even if the integration.int_id_api is different from id_api property
What am I doing wrong. What should I do to get this query working ?
You need to add a primary key to all your entities. From the doc:
Each entity must have at least one primary key column. This is a
requirement and you can't avoid it. To make a column a primary key,
you need to use #PrimaryColumn decorator.
Assuming you update the Integration entity like so
#Entity()
export default class Integration {
#PrimaryColumn()
public id_inte: number;
#OneToOne(type => Transaction, t => t.integration)
#JoinColumn({ referencedColumnName: 'id_trans', name: 'int_id_module' })
public transaction?: Transaction;
}
You should be able to find transactions like so
const transaction = await repository.find({
where: { integration: { id_inte: some_id } },
relations: ['integration', 'customer'],
});