How to connect to remote oracle database using typeorm in nestjs? - oracle

I was wondering how to connect to remote oracle database from nestjs using typeorm.
I installed typeorm and oracle package using following command.
npm i --save #nestjs/typeorm typeorm oracle
npm install oracledb --save
and then tried configuring in app.module.ts using TypeOrmModule.forRoot but it was not succesfull.
Here are my configuration settings.
TypeOrmModule.forRoot({
type: 'oracle',
host: 'ip of hostname',
port: port number,
username: 'username',
password: 'password',
serviceName: 'servicename',
synchronize: false,
entities: []
})
Can anybody help me out what am I missing? Also would like to know how can I execute the query once this connection is succesfully? If any example that would be helpfull.

Got it.
one missing thing was database name.
Added
database: 'databasename' in above configuration and it worked.
But, still my question is how to use this connection in service to fetch/push the data from/to oracle databse?

If you provide a name in your connection details you should be able to refer to the database connection using that. Otherwise, if no name is provided I believe it assigns it the name 'default'.
Basically these are the steps you should perform to use the database connection: (examples below each)
Create a model - this is how TypeORM knows to create a table.
export class Photo {
id: number
name: string
description: string
filename: string
views: number
isPublished: boolean
}
Create an Entity. - this should match your model, with the appropriate decorators. At minimum you should have the #Entity() decorator before your class definition and #Column() before each field.
import { Entity, Column } from "typeorm"
#Entity()
export class Photo {
#Column()
id: number
#Column()
name: string
#Column()
description: string
#Column()
filename: string
#Column()
views: number
#Column()
isPublished: boolean
}
Create your data source - looks like you have already done this. But I would give it a name field and you will need to pass your entities into the entity array you have.
const AppDataSource = new DataSource({
type: "postgres",
name: "photos",
host: "localhost",
port: 5432,
username: "root",
password: "admin",
database: "test",
entities: [Photo],
synchronize: true,
logging: false,
})
Then you can use repositories to manage data in the database:
const photo = new Photo()
photo.name = "Me and Bears"
photo.description = "I am near polar bears"
photo.filename = "photo-with-bears.jpg"
photo.views = 1
photo.isPublished = true
const photoRepository = AppDataSource.getRepository(Photo)
await photoRepository.save(photo)
console.log("Photo has been saved")
const savedPhotos = await photoRepository.find()
console.log("All photos from the db: ", savedPhotos)
For more details I would spend some time reading through the typeORM website, all the examples I pulled were from there:
https://typeorm.io/

Related

TypeORM EntityMetadataNotFoundError: No metadata for \"Score\" was found

My project has 2 seperate connection datasources. One is to a readonly mysql database, the other is a postgres database.
Everything on the mysql database works fine. But I'm having issues with the postgres database.
Whenever I run the createQueryBuilder to getMany I get the following error.
EntityMetadataNotFoundError: No metadata for \"Score\" was found.
In my index.ts file where I spin up my server, I initialize the postgres db datasouce
pgDataSource.initialize();
And here is pgDataSource
import 'reflect-metadata';
import 'dotenv-safe/config';
import { DataSource } from 'typeorm';
import path from 'path';
import Score from '../entities/Score';
const pgDataSource = new DataSource({
name: 'favourites',
type: 'postgres',
url: process.env.DATABASE_URL,
logging: true,
synchronize: true,
migrations: [path.join(__dirname, './migrations/*')],
entities: [Score],
});
export const Manager = pgDataSource.manager;
export const ScoreRepository = pgDataSource.getRepository(Score);
export default pgDataSource;
At this point, everything seems fine, it creates a table in my database with the Score entity, so the connection exists and it acknowledges that there is an Entity as it creates the corresponding table.
Here is my Score entity
import { ObjectType, Field, Int } from 'type-graphql';
import { Entity, PrimaryGeneratedColumn, Column, BaseEntity } from 'typeorm';
#ObjectType()
#Entity({ database: 'pp_favourites', name: 'score' })
class Score extends BaseEntity {
#Field()
#PrimaryGeneratedColumn()
id!: number;
#Field(() => Int)
#Column()
propertyId!: number;
}
export default Score;
I have a GraphQL resolver for the mysql database and a resolver for postgres database.
However, when I run this query
await ScoreRepository.createQueryBuilder().getMany()
In a Query decorator function, I get the No metadata for \"Score\" was found. error
Although, if I run it through a useMiddleware decorator, it strangely works?
This is what my mysql resolver looks like with the query on the ScoreRespository
import { Resolver, Query } from 'type-graphql';
import Shortlist from '../entities/Shortlist';
import { ShortlistRepository } from '../datasources/ppDataSource';
import Property from '../entities/Property';
import { ScoreRepository } from '../datasources/pgDataSource';
#Resolver(Shortlist)
class ShortlistResolver {
#Query(() => [Property], { nullable: true })
async getShortlist(): Promise<Property[]> {
await ScoreRepository.createQueryBuilder().getMany()
}
}
export default ShortlistResolver;
Perhaps, also worth mentioning, as this could potentially be the issue. But in my index file where i spin up my server. I setup my ApolloServer which uses the now deprecated getConnectionfunction from typeorm
const apolloServer = new ApolloServer({
schema: await buildSchema({
resolvers: [ShortlistResolver, ScoreResolver],
validate: true,
}),
context: ({ req, res }) => ({
req,
res,
}),
plugins: [
// dataloaders with TypeormLoader in type-graphql-dataloader
ApolloServerLoaderPlugin({
typeormGetConnection: getConnection, // for use with TypeORM
}),
// use older playground
ApolloServerPluginLandingPageGraphQLPlayground({
settings: {
'request.credentials': 'include',
},
}),
],
});
I've tried
Adding #Resolver(Score) as a decorator to the ShortlistResolver
Removing the dist folder
creating a new database and new connection
Logged Manager.connection.entityMetadatas from pgDataSource which shows there are no entity's, despite being defined.
Any help would be greatly appreciated. I don't really know if the issue lies with TypeORM or GraphQL.

"Error: Unknown dialect undefined" when deploying Strapi 4.0 app to Heroku

I created a Strapi app using the latest version, the new 4.0 and I wanted to deploy it to Heroku. I did follow the Strapi documentation in order to do so, like explained in this page. Now I'm getting an error that I don't understand, I guess it has something to do with postgres. This is the error
2021-12-18T15:26:26.658380+00:00 app[web.1]: [2021-12-18 15:26:26.656] debug: ⛔️ Server wasn't able to start properly.
2021-12-18T15:26:26.659122+00:00 app[web.1]: [2021-12-18 15:26:26.658] error: Unknow dialect undefined
2021-12-18T15:26:26.659123+00:00 app[web.1]: Error: Unknow dialect undefined
2021-12-18T15:26:26.659123+00:00 app[web.1]: at getDialectClass (/app/node_modules/#strapi/database/lib/dialects/index.js:12:13)
2021-12-18T15:26:26.659123+00:00 app[web.1]: at getDialect (/app/node_modules/#strapi/database/lib/dialects/index.js:19:23)
2021-12-18T15:26:26.659124+00:00 app[web.1]: at new Database (/app/node_modules/#strapi/database/lib/index.js:38:20)
2021-12-18T15:26:26.659124+00:00 app[web.1]: at Function.Database.init (/app/node_modules/#strapi/database/lib/index.js:84:33)
2021-12-18T15:26:26.659125+00:00 app[web.1]: at Strapi.bootstrap (/app/node_modules/#strapi/strapi/lib/Strapi.js:347:30)
2021-12-18T15:26:26.659125+00:00 app[web.1]: at Strapi.load (/app/node_modules/#strapi/strapi/lib/Strapi.js:410:16)
2021-12-18T15:26:26.659125+00:00 app[web.1]: at async Strapi.start (/app/node_modules/#strapi/strapi/lib/Strapi.js:161:9)
Apart from doing what is explained in the docs I linked, I just added a few collections using the UI in development mode. How can I fix this error and deploy to Heroku this new 4.0 version of Strapi?
I had a similar issue when I was connecting pg locally and then realised my connection config was incorrect. When I replaced it with v4 template it worked.
path: config/database.js
module.exports = ({ env }) => ({
defaultConnection: "default",
connection: {
client: "postgres",
connection: {
host: env("DATABASE_HOST", "localhost"),
port: env.int("DATABASE_PORT", 5432),
database: env("DATABASE_NAME", "bank"),
username: env("DATABASE_USERNAME", "postgres"),
password: env("DATABASE_PASSWORD", "0000"),
schema: env("DATABASE_SCHEMA", "public"),
},
}
});
And for Heorku as the article suggested:
path: config/env/production/database.js:
const parse = require('pg-connection-string').parse;
const config = parse(process.env.DATABASE_URL);
module.exports = ({ env }) => ({
connection: {
client: 'postgres',
connection: {
host: config.host,
port: config.port,
database: config.database,
user: config.user,
password: config.password,
ssl: {
rejectUnauthorized: false
},
},
debug: false,
},
});
PS: make sure you add pg-connection-string to your dependencies before pushing to heroku
getDialectClass - from the error log
const getDialectClass = client => {
switch (client) {
case 'postgres':
return require('./postgresql');
case 'mysql':
return require('./mysql');
case 'sqlite':
return require('./sqlite');
default:
throw new Error(`Unknow dialect ${client}`);
}
};
where the client is - db.config.connection
So if you have been following previous solutions - which talked about a connections object etc ( like this )
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'postgres',
host: config.host,
port: config.port,
database: config.database,
username: config.user,
password: config.password,
ssl: {
rejectUnauthorized: false,
},
},
options: {
ssl: true,
},
},
},
});
db.config.connection would return undefined. & so it would fail
Configuring the DB with - https://docs.strapi.io/developer-docs/latest/setup-deployment-guides/configurations/required/databases.html#configuration-structure
Works fine
If you are using typescript in Strapi, we may encounter this issue too as there is a issue affecting Strapi typescript capabilities.
I can resolve the problem when I downgraded Strapi to 4.3.2 from 4.3.8
You can check this method using heroku/cli from strapi docs here, but it's the same thing, i'm using version 4.0.2, this methods works on older versions V3, i think the docs needs update, most of it is deprecated, for example "fetching only data (example: posts/project etc..) related to the authenticated user who created it", most of resources related to this topic in docs and forum are deprecated solutions that don't work in V4.
For the deployment, i have the same issue, i tried a lot of solutions but it didn't work for me to, but i managed to get the server running like this:
but i got this problem when i visit the "/admin" page:
Strapi is really interesting, let's hope they update the docs soon and someone respond with a solution here.
I fixed this issue by using database url directly. Not sure why the old way does not work in Strapi v4.
module.exports = ({ env }) => ({
defaultConnection: "default",
connection: {
client: "postgres",
connection: {
connectionString: process.env.DATABASE_URL,
ssl: {
rejectUnauthorized: false
}
},
}
});
In case someone meets the same problem as me.
In my situation, I am using vim in windows as my text editor. After editing the "config/database.js" file. It left a "database.js~" file in that directory. Then, this filename causes the same problems: "Error: Unknown dialect undefined".
I guess it's a bug in strapi. After removing that file, everything works!
So, the solutions for me. I add this line to my vim config file:
set backupdir=~/.vimbackup
Then create a directory named '.vimbackup' in "C:/User/[me]/" .
Strapi has once again changed the database configuration structure and even their deployment docs use the old and incorrect example.
Here is the most up to date example: Strapi Database Configuration
In strapi v4, you need to use like:
module.exports = ({ env }) => ({
connection: {
client: "postgres",
connection: {
host: env("DATABASE_HOST", "localhost"),
port: env.int("DATABASE_PORT", 5432),
database: env("DATABASE_NAME", "db-name"),
user: env("DATABASE_USERNAME", "postgres"),
password: env("DATABASE_PASSWORD", "password"),
schema: env("DATABASE_SCHEMA", "public"),
},
}
});
it's look like there is a problem in the new strapi version , when i try to create project with typescript support i faced same error , i don't know if you are using typescript in your project but these two solutions solved my problem;
solution (1) :- create compile config ts files with tsc $ tsc .\config\database.ts .\config\server.ts ./ .\config\admin.ts
solution (2) :- downgrade your strapi to 4.3.2 version in the package.json file.
solution (3) :- don't use #latest when creating strapi app instate use 4.3.2 npx create-strapi-app#4.3.2 test-strapi-verion-with-ts --ts
you can use one of theme

Symfony4: How to toggle entities using dotenv and multiple Entity Manager?

How it is possible through dotenv select different entity path with same name of entities.
Originally, We had application in Symfony 3 for people meetings on events of our organization. Then we decided to offer this application to our partners.
One of the partners asked us to customize the application for them with their data and specifications. We basically created a new instance of Symfony application with copy of database tables (with different prefix), changes in Entities to reflect new db table names, and some translation text changes.
It seems that other partners will follow this trend of customized instances.
Therefore, I am trying to update core application to Symfony 4 and I am trying to use multiple Entity Managers and dotenv to differentiate between partners database tables, as described bellow.
In a nutshell, I am trying to use multiple Entity Manager to switch db table names by prefix.
.env
###> doctrine/doctrine-bundle ###
# Format described at http://docs.doctrine-project.org/projects/doctrine-dbal/en/latest/reference/configuration.html#connecting-using-a-url
# For an SQLite database, use: "sqlite:///%kernel.project_dir%/var/data.db"
# Configure your db driver and server_version in config/packages/doctrine.yaml
DATABASE_URL=oci8://user:pass#127.0.0.1:1521/XE
EM_TYPE=OpenAccess
###< doctrine/doctrine-bundle ###
Only in security.yaml working env good
security:
encoders:
App\Entity\%env(EM_TYPE)%\Osoba:
providers:
our_db_provider:
entity:
class: App\Entity\%env(EM_TYPE)%\Osoba
property: username
When i tried get %env(EM_TYPE)% in default_entity_manager, then give error You have requested a non-existent service "doctrine.orm.%env(EM_TYPE)%_entity_manager". doctrine.yaml
parameters:
env(DATABASE_URL): ''
doctrine:
dbal:
default_connection: '%env(EM_TYPE)%'
connections:
MeetingTool:
driver: 'oci8'
charset: UTF8
schema_filter: /^MT_/
url: '%env(resolve:DATABASE_URL)%'
OpenAccess:
driver: 'oci8'
charset: UTF8
schema_filter: /^OA6_/
url: '%env(resolve:DATABASE_URL)%'
orm:
auto_generate_proxy_classes: true
default_entity_manager: '%env(EM_TYPE)%'
entity_managers:
MeetingTool:
connection: MeetingTool
mappings:
Main:
is_bundle: false
type: annotation
dir: '%kernel.project_dir%/src/Entity/MeetingTool'
prefix: 'App\Entity\MeetingTool'
alias: App2
OpenAccess:
connection: OpenAccess
mappings:
App:
is_bundle: false
type: annotation
dir: '%kernel.project_dir%/src/Entity/OpenAccess'
prefix: 'App\Entity\OpenAccess'
alias: OpenAccess
and biggest problem is use, how i targeting right entity here? for example loginController.php
<?php
namespace App\Controller;
use Doctrine\ORM\EntityManagerInterface;
use App\Entity\OpenAccess\LogPrihlaseni;
//use App\Entity\MeetingTool\LogPrihlaseni;
class LoginController extends AbstractController {
private $entityManager;
public function __construct(EntityManagerInterface $entityManager)
{
$this->entityManager = $entityManager;
}
public function logPrihlaseni() {
$log = new LogPrihlaseni();

Doctrine migrations: create code for mysql and postgresql

I am using Doctrine ORM 2.6.1 in a Symfony 3.4.4 project.
Some of my instances work on a MySQL database, some on Postgresql, and a few installations even access a MicosoftSQL server. This works fine without any special changes to my project or entities, I only have to configure the corresponding connection parameters.
But: if I create migrations, only statements compatible with the current database connection are created in the migration file.
I develop with a postgres-conncection, so I only produce postgresql-statements, like:
class Version20180430083616 extends AbstractMigration
{
public function up(Schema $schema)
{
// this up() migration is auto-generated, please modify it to your needs
$this->abortIf($this->connection->getDatabasePlatform()->getName() !== 'postgresql', 'Migration can only be executed safely on \'postgresql\'.');
$this->addSql('DELETE FROM document_category');
$this->addSql('DROP SEQUENCE document_category_id_seq CASCADE');
$this->addSql('DROP TABLE document_category');
}
public function down(Schema $schema)
{
// this down() migration is auto-generated, please modify it to your needs
$this->abortIf($this->connection->getDatabasePlatform()->getName() !== 'postgresql', 'Migration can only be executed safely on \'postgresql\'.');
//...
}
}
My Question: How can I tell the migrations bundle to create statements for each platform, like:
class Version20180430083616 extends AbstractMigration
{
public function up(Schema $schema)
{
// this up() migration is auto-generated, please modify it to your needs
if($this->connection->getDatabasePlatform()->getName() == 'postgresql'){
$this->addSql('DELETE FROM document');
$this->addSql('DELETE FROM document_category');
$this->addSql('DROP SEQUENCE document_category_id_seq CASCADE');
$this->addSql('DROP TABLE document_category');
} else if($this->connection->getDatabasePlatform()->getName() == 'mysql'){
...
} else if ($this->connection->getDatabasePlatform()->getName() == 'mssql') { // MicrosoftSQL ?
...
}
}
}
Edit:
So, I think the only solution to my problem is to define multiple database connections and entity managers, and to always create a distinct migration for each connection type. According to this article, I can define several connections as:
I found a doable solution:
inf config.yml I define one connection and one EntityManager per database type:
doctrine:
dbal:
default_connection: pgdb
connections:
pgdb:
driver: pdo_pgsql
host: db
port: 5432
name: pgdb
user: postgres
password: example
charset: utf8
mapping_types:
enum: string
mysql:
driver: pdo_mysql
host: mysqlhost
port: 3306
name: mydb
dbname: mydb
user: root
password: xxx
charset: utf8mb4
default_table_options:
collate: utf8mb4_unicode_ci
mapping_types:
enum: string
mssql:
driver: pdo_sqlsrv
host: mssqlhost
port: 1433
name: msdb
dbname: testdb
user: sa
password: xxx
charset: utf8
mapping_types:
enum: string
orm:
auto_generate_proxy_classes: false
proxy_dir: '%kernel.cache_dir%/doctrine/orm/Proxies'
proxy_namespace: Proxies
entity_managers:
default:
connection: pgdb
naming_strategy: doctrine.orm.naming_strategy.underscore
mappings:
AppBundle: ~
my:
connection: mydb
naming_strategy: doctrine.orm.naming_strategy.underscore
mappings:
AppBundle: ~
ms:
connection: msdb
naming_strategy: doctrine.orm.naming_strategy.underscore
mappings:
AppBundle: ~
Then, I can issue the diff-command 3 times instead of only once
$ bin/console doctrine:migrations:diff --em=default
$ bin/console doctrine:migrations:diff --em=my
$ bin/console doctrine:migrations:diff --em=ms
This creates three migrations each starting with a fence line:
$this->abortIf($this->connection->getDatabasePlatform()->getName() !== 'mssql', 'Migration can only be executed safely on \'mssql\'.');
in which I exchange abortIf by skipIf, such that the migration process is not aborted if the current migration if for a different database type, but just skipped:
$this->skipIf($this->connection->getDatabasePlatform()->getName() !== 'mssql', 'Migration can only be executed safely on \'mssql\'.');
I hope this helps somebody.

How to Access Athena QueryString From CloudFormation in Lambda?

AWS-loaded question, but does anyone know what the proper way to access an Athena Query String (in CloudFormation) in Lambda?
I have set up both the Athena NamedQuery and the Lambda in CloudFormation. Abstracting out some of the more project-specific details, the general form I have is:
MyQuery:
Type: AWS::Athena::NamedQuery
Properties:
Database: "mydatabase"
Name: "DataQuery"
QueryString: SELECT * FROM mydatabase
MyLambda:
Type: AWS::Serverless::Function
Properties:
Handler: 'handlers.migration_handler'
Runtime: python3.6
CodeUri:
Bucket: BATS::SAM::CodeS3Bucket
Key: BATS::SAM::CodeS3Key
Description: Migrates data from output of Athena query to S3
Policies:
- AmazonS3FullAccess
- AWSLambdaExecute
- AmazonAthenaFullAccess
Environment:
Variables:
MY_QUERY:
Ref: MyQuery
When I'm writing the handler for the lambda, I want to call:
athena_client = boto3.client('athena')
response = athena_client.start_query_execution(
QueryString = os.environ['MY_QUERY']
ResultConfiguration = {'OutputLocation: 's3://my-bucket'}
)
However, QueryString needs to be a string, so this currently isn't working. I want to access the QueryString property in MY_QUERY, and I feel like I'm so close but I'm not quite sure how to get that last step. Any help here would be greatly appreciated.
Figured it out yesterday (or more specifically, my teammate figured it out).
Boto3 happens to have another method called get_named_query(NamedQueryId), and that returns a dictionary in the form of:
{
'NamedQuery': {
'Name': 'string',
'Description': 'string',
'Database': 'string',
'QueryString': 'string',
'NamedQueryId': 'string'
}
Thus, my code worked when I modified my lambda handler to:
athena_client = boto3.client('athena')
query_info = athena_client.get_named_query(
NamedQueryId = os.environ['MY_QUERY']
)
response = athena_client.start_query_execution(
QueryString = query_info['NamedQuery']['QueryString']
ResultConfiguration = {'OutputLocation: 's3://my-bucket'}
)

Resources