How to correct convert byte[] column from spring 3.2 #Column annotation to hibernate mapping - spring

I use in hibernate.cfg.xml:
<property name="hibernate.dialect">org.hibernate.dialect.H2Dialect</property>
<property name="hibernate.hbm2ddl.auto">create-drop</property>
Works correctly:
#Column(columnDefinition = "LONGBLOB", nullable = false, updatable = true)
protected byte[] content;
in table (with data from import.sql. In import.sql "content" is of type longblob):
COLUMN_NAME
TYPE_NAME
IS_NULLABLE
DECIMAL_DIGITS
COLUMN_SIZE
COLUMN_DEF
REMARKS
DATA_TYPE
BUFFER_LENGTH
NUM_PREC_RADIX
NULLABLE
SQL_DATA_TYPE
SQL_DATETIME_SUB
CHAR_OCTET_LENGTH
ORDINAL_POSITION
CONTENT
BLOB
NO
0
2147483647
null
2004
2147483647
10
0
2004
0
2147483647
4
without import.sql (empty db):
COLUMN_NAME
TYPE_NAME
IS_NULLABLE
DECIMAL_DIGITS
COLUMN_SIZE
COLUMN_DEF
REMARKS
DATA_TYPE
BUFFER_LENGTH
NUM_PREC_RADIX
NULLABLE
SQL_DATA_TYPE
SQL_DATETIME_SUB
CHAR_OCTET_LENGTH
ORDINAL_POSITION
CONTENT
BLOB
NO
0
2147483647
null
2004
2147483647
10
0
2004
0
2147483647
3
When I do a db import (using h2-1.4.200.jar), in the table the field contains the type:
"CONTENT" LONGBLOB NOT NULL
Does not work properly:
<property name="content" type="byte[]" update="true">
<column name="content" sql-type="LONGBLOB" not-null="true"/></property>
in table (with data from import.sql. In import.sql "content" is of type longblob):
COLUMN_NAME
TYPE_NAME
IS_NULLABLE
DECIMAL_DIGITS
COLUMN_SIZE
COLUMN_DEF
REMARKS
DATA_TYPE
BUFFER_LENGTH
NUM_PREC_RADIX
NULLABLE
SQL_DATA_TYPE
SQL_DATETIME_SUB
CHAR_OCTET_LENGTH
ORDINAL_POSITION
CONTENT
BLOB
NO
0
2147483647
null
2004
2147483647
10
0
2004
0
2147483647
4
Resulting in invalid characters: �cw�%��6�ţ��ٟ�{^y���.
without import.sql (empty db):
COLUMN_NAME
TYPE_NAME
IS_NULLABLE
DECIMAL_DIGITS
COLUMN_SIZE
COLUMN_DEF
REMARKS
DATA_TYPE
BUFFER_LENGTH
NUM_PREC_RADIX
NULLABLE
SQL_DATA_TYPE
SQL_DATETIME_SUB
CHAR_OCTET_LENGTH
ORDINAL_POSITION
CONTENT
BLOB
NO
0
2147483647
null
2004
2147483647
10
0
2004
0
2147483647
2
How to write correct hibernate mapping similar to #Column Spring annotation?
I tried using (no effect):
<property name="hibernate.jdbc.use_streams_for_binary">false</property>
and
<property name="hibernate.jdbc.use_streams_for_binary">true</property>

Related

Grails 4 oracle connection stuck

When the application starts up it is validating the connection. This is the last log entry while it sits there waiting.
2020-07-29 15:01:20.743 INFO --- [ main] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.Oracle12cDialect
It makes a connection in the DB and tries to run this.
SELECT 'NUMBER' AS type_name
, 2 AS data_type
, 38 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 1 AS fixed_prec_scale
, 0 AS auto_increment
, 'NUMBER' AS local_type_name
, -84 AS minimum_scale
, 127 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'CHAR' AS type_name
, 1 AS data_type
, 2000 AS precision
, '''' AS literal_prefix
, '''' AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 1 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'CHAR' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'NCHAR' AS type_name
, -15 AS data_type
, 2000 AS precision
, '''' AS literal_prefix
, '''' AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 1 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'NCHAR' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'VARCHAR2' AS type_name
, 12 AS data_type
, 4000 AS precision
, '''' AS literal_prefix
, '''' AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 1 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'VARCHAR2' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'NVARCHAR2' AS type_name
, -9 AS data_type
, 4000 AS precision
, '''' AS literal_prefix
, '''' AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 1 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'nVARCHAR2' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'DATE' AS type_name
, 93 AS data_type
, 7 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'DATE' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'DATE' AS type_name
, 92 AS data_type
, 7 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'DATE' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'TIMESTAMP' AS type_name
, 93 AS data_type
, 11 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'TIMESTAMP' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'TIMESTAMP WITH TIME ZONE' AS type_name
, -101 AS data_type
, 13 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'TIMESTAMP WITH TIME ZONE' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'TIMESTAMP WITH LOCAL TIME ZONE' AS type_name
, -102 AS data_type
, 11 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'TIMESTAMP WITH LOCAL TIME ZONE' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'INTERVALYM' AS type_name
, -103 AS data_type
, 5 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'INTERVALYM' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'INTERVALDS' AS type_name
, -104 AS data_type
, 4 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'INTERVALDS' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'RAW' AS type_name
, -3 AS data_type
, 2000 AS precision
, '''' AS literal_prefix
, '''' AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'RAW' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'LONG' AS type_name
, -1 AS data_type
, 2147483647 AS precision
, '''' AS literal_prefix
, '''' AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 1 AS case_sensitive
, 0 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'LONG' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'LONG RAW' AS type_name
, -4 AS data_type
, 2147483647 AS precision
, '''' AS literal_prefix
, '''' AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 0 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'LONG RAW' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'NUMBER' AS type_name
, -7 AS data_type
, 1 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, '(1)' AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 1 AS fixed_prec_scale
, 0 AS auto_increment
, 'NUMBER' AS local_type_name
, -84 AS minimum_scale
, 127 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'NUMBER' AS type_name
, -6 AS data_type
, 3 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, '(3)' AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 1 AS fixed_prec_scale
, 0 AS auto_increment
, 'NUMBER' AS local_type_name
, -84 AS minimum_scale
, 127 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'NUMBER' AS type_name
, 5 AS data_type
, 5 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, '(5)' AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 1 AS fixed_prec_scale
, 0 AS auto_increment
, 'NUMBER' AS local_type_name
, -84 AS minimum_scale
, 127 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'NUMBER' AS type_name
, 4 AS data_type
, 10 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, '(10)' AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 1 AS fixed_prec_scale
, 0 AS auto_increment
, 'NUMBER' AS local_type_name
, -84 AS minimum_scale
, 127 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'NUMBER' AS type_name
, -5 AS data_type
, 38 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 1 AS fixed_prec_scale
, 0 AS auto_increment
, 'NUMBER' AS local_type_name
, -84 AS minimum_scale
, 127 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'FLOAT' AS type_name
, 6 AS data_type
, 63 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 1 AS fixed_prec_scale
, 0 AS auto_increment
, 'FLOAT' AS local_type_name
, -84 AS minimum_scale
, 127 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'REAL' AS type_name
, 7 AS data_type
, 63 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 3 AS searchable
, 0 AS unsigned_attribute
, 1 AS fixed_prec_scale
, 0 AS auto_increment
, 'REAL' AS local_type_name
, -84 AS minimum_scale
, 127 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'BLOB' AS type_name
, 2004 AS data_type
, -1 AS precision
, NULL AS literal_prefix
, NULL AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 0 AS case_sensitive
, 0 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'BLOB' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'CLOB' AS type_name
, 2005 AS data_type
, -1 AS precision
, '''' AS literal_prefix
, '''' AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 1 AS case_sensitive
, 0 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'CLOB' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'NCLOB' AS type_name
, 2011 AS data_type
, -1 AS precision
, '''' AS literal_prefix
, '''' AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 1 AS case_sensitive
, 0 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'NCLOB' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'REF' AS type_name
, 2006 AS data_type
, 0 AS precision
, '''' AS literal_prefix
, '''' AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 1 AS case_sensitive
, 0 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'REF' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'ARRAY' AS type_name
, 2003 AS data_type
, 0 AS precision
, '''' AS literal_prefix
, '''' AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 1 AS case_sensitive
, 0 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'ARRAY' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
UNION
SELECT 'STRUCT' AS type_name
, 2002 AS data_type
, 0 AS precision
, '''' AS literal_prefix
, '''' AS literal_suffix
, NULL AS create_params
, 1 AS nullable
, 1 AS case_sensitive
, 0 AS searchable
, 0 AS unsigned_attribute
, 0 AS fixed_prec_scale
, 0 AS auto_increment
, 'STRUCT' AS local_type_name
, 0 AS minimum_scale
, 0 AS maximum_scale
, NULL AS sql_data_type
, NULL AS sql_datetime_sub
, 10 AS num_prec_radix
FROM DUAL
ORDER BY data_type
I think it never finishes. Grails just hangs waiting. If I kill the oracle session then it continues on and the app loads. I am running DEBUG logging and the next log entry is this.
2020-07-29 15:01:37.142 DEBUG --- [ main] o.h.e.j.e.i.LobCreatorBuilderImpl : HHH000424: Disabling contextual LOB creation as createClob() method threw error : java.lang.reflect.InvocationTargetException
Environment details
Grails 4.0.4
Oracle DB version 12.1.0.2.0
JDBC 4.1
Oracle JDBC 12c drivers
application.yml
---
grails:
profile: web
codegen:
defaultPackage: btpg
gorm:
reactor:
# Whether to translate GORM events into Reactor events
# Disabled by default for performance reasons
events: false
info:
app:
name: '#info.app.name#'
version: '#info.app.version#'
grailsVersion: '#info.app.grailsVersion#'
spring:
jmx:
unique-names: true
main:
banner-mode: "off"
groovy:
template:
check-template-location: false
devtools:
restart:
additional-exclude:
- '*.gsp'
- '**/*.gsp'
- '*.gson'
- '**/*.gson'
- 'logback.groovy'
- '*.properties'
management:
endpoints:
enabled-by-default: false
---
grails:
mime:
disable:
accept:
header:
userAgents:
- Gecko
- WebKit
- Presto
- Trident
types:
all: '*/*'
atom: application/atom+xml
css: text/css
csv: text/csv
form: application/x-www-form-urlencoded
html:
- text/html
- application/xhtml+xml
js: text/javascript
json:
- application/json
- text/json
multipartForm: multipart/form-data
pdf: application/pdf
rss: application/rss+xml
text: text/plain
hal:
- application/hal+json
- application/hal+xml
xml:
- text/xml
- application/xml
urlmapping:
cache:
maxsize: 1000
controllers:
defaultScope: singleton
converters:
encoding: UTF-8
views:
default:
codec: html
gsp:
encoding: UTF-8
htmlcodec: xml
codecs:
expression: html
scriptlet: html
taglib: none
staticparts: none
management:
endpoints:
jmx:
unique-names: true
---
hibernate:
cache:
queries: false
use_second_level_cache: false
use_query_cache: false
dataSource:
pooled: false
jmxExport: false
driverClassName: oracle.jdbc.OracleDriver
username: 'xxx'
password: 'xxx'
environments:
development:
dataSource:
dbCreate: none
url: jdbc:oracle:thin:#10.10.99.110:1521:SMPL
properties:
jmxEnabled: false
initialSize: 5
maxActive: 50
minIdle: 5
maxIdle: 25
maxWait: 10000
maxAge: 600000
timeBetweenEvictionRunsMillis: 5000
minEvictableIdleTimeMillis: 60000
validationQuery: SELECT 1 from dual;
validationQueryTimeout: 15
validationInterval: 15000
testOnBorrow: true
testWhileIdle: true
testOnReturn: false
jdbcInterceptors: ConnectionState
defaultTransactionIsolation: 2 # TRANSACTION_READ_COMMITTED
test:
dataSource:
dbCreate: none
url: jdbc:h2:mem:testDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
production:
dataSource:
dbCreate: none
url: jdbc:h2:./prodDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
properties:
jmxEnabled: true
initialSize: 5
maxActive: 50
minIdle: 5
maxIdle: 25
maxWait: 10000
maxAge: 600000
timeBetweenEvictionRunsMillis: 5000
minEvictableIdleTimeMillis: 60000
validationQuery: SELECT 1
validationQueryTimeout: 3
validationInterval: 15000
testOnBorrow: true
testWhileIdle: true
testOnReturn: false
jdbcInterceptors: ConnectionState
defaultTransactionIsolation: 2 # TRANSACTION_READ_COMMITTED
Any help is greatly appreciated.
You need to specify the datasource dialect in your yaml file. The bolded line is what you are missing.
dataSource:
pooled: false
jmxExport: false
driverClassName: oracle.jdbc.OracleDriver
################ Add this line ###############
dialect: org.hibernate.dialect.Oracle12cDialect
username: 'xxx'
password: 'xxx'

Dynamically retrieve JSON into datatable ORacle 12c version 1

I need to convert JSON into data table (key value columns) in Oracle 12c v12.1.0.2
So for eg: there is a JSON string like {"ID": 10, "Description": "TestJSON", "status":"New"}
I need this converted to :
Column1 Column2
------------------------------------
ID 10
Description TestJSON
status New
Now my JSON string could change the number of attributes and hence I require to keep the conversion dynamic.
Any help would be appreciated.
Thanks!
You can implement it using the hierarchy query and REGEXP as follows:
SQL> with dataa(d) as
2 (select '["ID": 10, "Description": "TestJSON", "status":"New"]' from dual)
3 --
4 -- Your query starts from here
5 SELECT TRIM('"' FROM REGEXP_SUBSTR(STR,'[^:]+',1,1)) AS COL1,
6 TRIM('"' FROM TRIM(REGEXP_SUBSTR(STR,'[^:]+',1,2))) AS COL2
7 FROM
8 (SELECT t.d, trim(REGEXP_SUBSTR(trim(']' from trim('[' from T.D)), '[^,]+', 1, COLUMN_VALUE)) as str
9 FROM DATAA T
10 CROSS JOIN TABLE ( CAST(MULTISET(
11 SELECT LEVEL FROM DUAL CONNECT BY
12 LEVEL <= LENGTH(REGEXP_REPLACE(T.D, '[^,]+')) + 1
13 ) AS SYS.ODCINUMBERLIST) ) LEVELS);
COL1 COL2
--------------- --------------------
ID 10
Description TestJSON
status New
SQL>

Oracle - Selecting not null result of calculation

For every row of my data set, there exist data for the only one of the two options for calculation and the other columns are Null.
My goal is to find simplest way to select not null result of calculation for each row. Expected result:
ROW_NUM result
-------- -------
1 4.5
2 4.56
My code:
With DATASET AS (
-- column 1 is just for row number,
-- column 2 and 3 for caculation option1,
--- columns 4~6 for caculation option2
SELECT 1 ROW_NUM, NULL time1, NULL qty1, 2 time2_1, 2.5 time2_2, 1 qty2
FROM DUAL
UNION
SELECT 2 ROW_NUM, 4.56 time1, 1 qty1, NULL time2_1, NULL time2_2, NULL qty2
FROM DUAL
)
SELECT ROW_NUM, time1/qty1 OPTION1, (time2_1+time2_2)/qty2 OPTION2
FROM DATASET;
Result:
ROW_NUM OPTION1 OPTION2
-------- ------- ---------
1 4.5
2 4.56
You can decode and use different representation when null:
SELECT ROW_NUM, decode(time1/qty1,null,(time2_1+time2_2)/qty2,time1/qty1) result FROM DATASET;
Or nvl
SELECT ROW_NUM, nvl(time1/qty1,(time2_1+time2_2)/qty2,time1/qty1) result FROM DATASET;
NVL lets you replace null (returned as a blank) with a string in the results of a query.
use COALESCE function as following:
With DATASET AS (
--each row contain information for either option1 or 2
SELECT *
FROM
(
--column 1 is just for row number, column 2 and 3 for caculation option1, columns 4~6 for caculation option2
SELECT 1 ROW_NUM, NULL time1, NULL qty1 , 2 time2_1 , 2.5 time2_2, 1 qty2 FROM DUAL
UNION
SELECT 2 ROW_NUM, 4.56 time1 , 1 qty1 , NULL time2_1 , NULL time2_2 , NULL qty2 FROM DUAL
)
)SELECT ROW_NUM, coalesce(time1/qty1,(time2_1+time2_2)/qty2) as result FROM DATASET;
db<>fiddle demo
Cheers!!

How to get rows with most recent nulls?

For the following:
col1 col2
1 null
3 null
5 100
6 200
10 null
11 201
How do I get the most recent col2==null rows using LINQ? This would be only col1= (1,3). These values are coming from a database.
The result set would be:
col1 col2
1 null
3 null
I'm not so familiar with Linq-To-Sql, so i'm not sure if it is supported, but try:
var query = db.TableName
.Where(r1 => r1.Col2 == null
&& r1.Col1 < db.TableName
.Where(r2 => r2.Col1 != null)
.Select(r2 => r2.Col1)
.OrderBy(col1 => col1)
.FirstOrDefault());
At least it should be the LINQ equivalent of this (working) sql-query: http://sqlfiddle.com/#!6/439be/6/0

How to generate model from database using Dapper?

I am coming from PetaPoco camp. PetaPoco has a T4 template which generates model from the database. Is anything similar available for Dapper?
I installed Dapper using NuGet and added SqlHelper.cs, but I didn't find anything which generates model from the database.
I've just recently written a sql query to do the job for myself. And updating it with extra types when i need. Just replace the table name where it says ####.
To make alot of tables i created a temp stored procedure to call. eg.
exec createTablePOCO(#tableName)
SELECT
'public ' + a1.NewType + ' ' + a1.COLUMN_NAME + ' {get;set;}'
,*
FROM (
/*using top because i'm putting an order by ordinal_position on it.
putting a top on it is the only way for a subquery to be ordered*/
SELECT TOP 100 PERCENT
COLUMN_NAME,
DATA_TYPE,
IS_NULLABLE,
CASE
WHEN DATA_TYPE = 'varchar' THEN 'string'
WHEN DATA_TYPE = 'datetime' AND IS_NULLABLE = 'NO' THEN 'DateTime'
WHEN DATA_TYPE = 'datetime' AND IS_NULLABLE = 'YES' THEN 'DateTime?'
WHEN DATA_TYPE = 'int' AND IS_NULLABLE = 'YES' THEN 'int?'
WHEN DATA_TYPE = 'int' AND IS_NULLABLE = 'NO' THEN 'int'
WHEN DATA_TYPE = 'smallint' AND IS_NULLABLE = 'NO' THEN 'Int16'
WHEN DATA_TYPE = 'smallint' AND IS_NULLABLE = 'YES' THEN 'Int16?'
WHEN DATA_TYPE = 'decimal' AND IS_NULLABLE = 'NO' THEN 'decimal'
WHEN DATA_TYPE = 'decimal' AND IS_NULLABLE = 'YES' THEN 'decimal?'
WHEN DATA_TYPE = 'numeric' AND IS_NULLABLE = 'NO' THEN 'decimal'
WHEN DATA_TYPE = 'numeric' AND IS_NULLABLE = 'YES' THEN 'decimal?'
WHEN DATA_TYPE = 'money' AND IS_NULLABLE = 'NO' THEN 'decimal'
WHEN DATA_TYPE = 'money' AND IS_NULLABLE = 'YES' THEN 'decimal?'
WHEN DATA_TYPE = 'bigint' AND IS_NULLABLE = 'NO' THEN 'long'
WHEN DATA_TYPE = 'bigint' AND IS_NULLABLE = 'YES' THEN 'long?'
WHEN DATA_TYPE = 'tinyint' AND IS_NULLABLE = 'NO' THEN 'byte'
WHEN DATA_TYPE = 'tinyint' AND IS_NULLABLE = 'YES' THEN 'byte?'
WHEN DATA_TYPE = 'char' THEN 'string'
WHEN DATA_TYPE = 'timestamp' THEN 'byte[]'
WHEN DATA_TYPE = 'varbinary' THEN 'byte[]'
WHEN DATA_TYPE = 'bit' AND IS_NULLABLE = 'NO' THEN 'bool'
WHEN DATA_TYPE = 'bit' AND IS_NULLABLE = 'YES' THEN 'bool?'
WHEN DATA_TYPE = 'xml' THEN 'string'
END AS NewType
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = '####'
ORDER BY ORDINAL_POSITION
) as a1
Calling the stored procedure from a cursor
If you combine the sp mattritchies mentioned (see answer above) and call it from a cursor you can generate the poco class for every table in your database
USE YourDataBaseName
GO
DECLARE #field1 nvarchar(400)
DECLARE cur CURSOR LOCAL for
SELECT TABLE_NAME FROM information_schema.tables
OPEN cur
FETCH NEXT FROM cur INTO #field1 --, #field2
WHILE ##FETCH_STATUS = 0 BEGIN
exec Helper_CreatePocoFromTableName #field1 -- , #field2
fetch next from cur into #field1 -- , #field2
END
close cur
deallocate cur
Stored Procedure mattritchies mentioned
I took the sql from mattritchies answer (see above) and created the stored procedure he mentioned and modified it a bit so that it adds the class name as well. If you put Management Studio into Text-Output-Mode and remove the output of the column names you get copy paste text for all classes:
CREATE PROCEDURE [dbo].[Helper_CreatePocoFromTableName]
#tableName varchar(100)
AS
BEGIN
SET NOCOUNT ON;
-- Subquery to return only the copy paste text
Select PropertyColumn from (
SELECT 1 as rowNr, 'public class ' + #tableName + ' {' as PropertyColumn
UNION
SELECT 2 as rowNr, 'public ' + a1.NewType + ' ' + a1.COLUMN_NAME + ' {get;set;}' as PropertyColumn
-- ,* comment added so that i get copy pasteable output
FROM
(
/*using top because i'm putting an order by ordinal_position on it.
putting a top on it is the only way for a subquery to be ordered*/
SELECT TOP 100 PERCENT
COLUMN_NAME,
DATA_TYPE,
IS_NULLABLE,
CASE
WHEN DATA_TYPE = 'varchar' THEN 'string'
WHEN DATA_TYPE = 'nvarchar' THEN 'string'
WHEN DATA_TYPE = 'datetime' AND IS_NULLABLE = 'NO' THEN 'DateTime'
WHEN DATA_TYPE = 'datetime' AND IS_NULLABLE = 'YES' THEN 'DateTime?'
WHEN DATA_TYPE = 'smalldatetime' AND IS_NULLABLE = 'NO' THEN 'DateTime'
WHEN DATA_TYPE = 'datetime2' AND IS_NULLABLE = 'NO' THEN 'DateTime'
WHEN DATA_TYPE = 'smalldatetime' AND IS_NULLABLE = 'YES' THEN 'DateTime?'
WHEN DATA_TYPE = 'datetime2' AND IS_NULLABLE = 'YES' THEN 'DateTime?'
WHEN DATA_TYPE = 'int' AND IS_NULLABLE = 'YES' THEN 'int?'
WHEN DATA_TYPE = 'int' AND IS_NULLABLE = 'NO' THEN 'int'
WHEN DATA_TYPE = 'smallint' AND IS_NULLABLE = 'NO' THEN 'Int16'
WHEN DATA_TYPE = 'smallint' AND IS_NULLABLE = 'YES' THEN 'Int16?'
WHEN DATA_TYPE = 'decimal' AND IS_NULLABLE = 'NO' THEN 'decimal'
WHEN DATA_TYPE = 'decimal' AND IS_NULLABLE = 'YES' THEN 'decimal?'
WHEN DATA_TYPE = 'numeric' AND IS_NULLABLE = 'NO' THEN 'decimal'
WHEN DATA_TYPE = 'numeric' AND IS_NULLABLE = 'YES' THEN 'decimal?'
WHEN DATA_TYPE = 'money' AND IS_NULLABLE = 'NO' THEN 'decimal'
WHEN DATA_TYPE = 'money' AND IS_NULLABLE = 'YES' THEN 'decimal?'
WHEN DATA_TYPE = 'bigint' AND IS_NULLABLE = 'NO' THEN 'long'
WHEN DATA_TYPE = 'bigint' AND IS_NULLABLE = 'YES' THEN 'long?'
WHEN DATA_TYPE = 'tinyint' AND IS_NULLABLE = 'NO' THEN 'byte'
WHEN DATA_TYPE = 'tinyint' AND IS_NULLABLE = 'YES' THEN 'byte?'
WHEN DATA_TYPE = 'char' THEN 'string'
WHEN DATA_TYPE = 'timestamp' THEN 'byte[]'
WHEN DATA_TYPE = 'varbinary' THEN 'byte[]'
WHEN DATA_TYPE = 'bit' AND IS_NULLABLE = 'NO' THEN 'bool'
WHEN DATA_TYPE = 'bit' AND IS_NULLABLE = 'YES' THEN 'bool?'
WHEN DATA_TYPE = 'xml' THEN 'string'
END AS NewType
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = #tableName
ORDER BY ORDINAL_POSITION
) AS a1
UNION
SELECT 3 as rowNr, '} // class ' + #tableName
) as t Order By rowNr asc
END
P.S.: I would have done it as an edit suggestion to his answers but my experience is that often edit suggestions get rejected.
Update
User chris-w-mclean suggested the following changes (see his suggested-edit) which i have not tried myself:
Replace SELECT 1 as rowNr, 'public class ' with SELECT 1.0 as rowNr, 'public class '
Replace SELECT 2 as rowNr, 'public ' with SELECT 2 + a1.ORDINAL_POSITION/1000 as rowNr, 'public '
Replace SELECT TOP 100 PERCENT COLUMN_NAME, with SELECT COLUMN_NAME,
add between IS_NULLABLE, CASE this line cast(ORDINAL_POSITION as float) as ORDINAL_POSITION,
remove ORDER BY ORDINAL_POSITION
change SELECT 3 as to SELECT 3.0 as
Try this version I optimized a bit, so that the result doesn't need to be piped to Text output. Instead, the PRINT statement allows the output to be copy/pasted easily. I've also removed the subquery and added declarations for nvarchar/ntext types.
This is for a single table, but it can be converted to a stored proc to use one of the cursor suggestions above.
SET NOCOUNT ON
DECLARE #tbl as varchar(255)
SET #tbl = '####'
DECLARE #flds as varchar(8000)
SET #flds=''
SELECT -1 as f0, 'public class ' + #tbl + ' {' as f1 into #tmp
INSERT #tmp
SELECT
ORDINAL_POSITION,
' public ' +
CASE
WHEN DATA_TYPE = 'varchar' THEN 'string'
WHEN DATA_TYPE = 'nvarchar' THEN 'string'
WHEN DATA_TYPE = 'text' THEN 'string'
WHEN DATA_TYPE = 'ntext' THEN 'string'
WHEN DATA_TYPE = 'char' THEN 'string'
WHEN DATA_TYPE = 'xml' THEN 'string'
WHEN DATA_TYPE = 'datetime' AND IS_NULLABLE = 'NO' THEN 'DateTime'
WHEN DATA_TYPE = 'datetime' AND IS_NULLABLE = 'YES' THEN 'DateTime?'
WHEN DATA_TYPE = 'int' AND IS_NULLABLE = 'YES' THEN 'int?'
WHEN DATA_TYPE = 'int' AND IS_NULLABLE = 'NO' THEN 'int'
WHEN DATA_TYPE = 'smallint' AND IS_NULLABLE = 'NO' THEN 'Int16'
WHEN DATA_TYPE = 'smallint' AND IS_NULLABLE = 'YES' THEN 'Int16?'
WHEN DATA_TYPE = 'decimal' AND IS_NULLABLE = 'NO' THEN 'decimal'
WHEN DATA_TYPE = 'decimal' AND IS_NULLABLE = 'YES' THEN 'decimal?'
WHEN DATA_TYPE = 'numeric' AND IS_NULLABLE = 'NO' THEN 'decimal'
WHEN DATA_TYPE = 'numeric' AND IS_NULLABLE = 'YES' THEN 'decimal?'
WHEN DATA_TYPE = 'money' AND IS_NULLABLE = 'NO' THEN 'decimal'
WHEN DATA_TYPE = 'money' AND IS_NULLABLE = 'YES' THEN 'decimal?'
WHEN DATA_TYPE = 'bigint' AND IS_NULLABLE = 'NO' THEN 'long'
WHEN DATA_TYPE = 'bigint' AND IS_NULLABLE = 'YES' THEN 'long?'
WHEN DATA_TYPE = 'tinyint' AND IS_NULLABLE = 'NO' THEN 'byte'
WHEN DATA_TYPE = 'tinyint' AND IS_NULLABLE = 'YES' THEN 'byte?'
WHEN DATA_TYPE = 'timestamp' THEN 'byte[]'
WHEN DATA_TYPE = 'varbinary' THEN 'byte[]'
WHEN DATA_TYPE = 'bit' AND IS_NULLABLE = 'NO' THEN 'bool'
WHEN DATA_TYPE = 'bit' AND IS_NULLABLE = 'YES' THEN 'bool?'
END + ' ' + COLUMN_NAME + ' {get;set;}'
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = #tbl
INSERT #tmp SELECT 999, '}'
SELECT #flds=#flds + f1 +'
' from #tmp order by f0
DROP TABLE #tmp
PRINT #flds
Dapper itself provides few extension methods (Query, Execute) for the connection object and does not have "model generator." Perhaps some other framework can be used to generate POCO's based on the db schema.
Update:
Database tables to C# POCO classes T4 template
<## template language="C#" debug="True" #>
<## assembly name="System" #>
<## assembly name="System.Data" #>
<## assembly name="System.Core" #>
<## assembly name="System.Xml" #>
<## assembly name="Microsoft.SqlServer.ConnectionInfo" #>
<## assembly name="Microsoft.SqlServer.Management.Sdk.Sfc" #>
<## assembly name="Microsoft.SqlServer.Smo" #>
<## import namespace="System" #>
<## import namespace="System.Text" #>
<## import namespace="System.Xml" #>
<## import namespace="Microsoft.SqlServer.Management.Smo" #>
<## import namespace="System.Data.SqlClient" #>
<## import namespace="Microsoft.SqlServer.Management.Common" #>
namespace Namespace
{
<#
var databaseName = "testDb";
var serverConnection = new SqlConnection(
#"Data Source=.\SQLEXPRESS; Integrated Security=true; Initial Catalog=" + databaseName);
var svrConnection = new ServerConnection(serverConnection);
Server srv = new Server(svrConnection);
foreach (Table table in srv.Databases[databaseName].Tables)
{
#>
class <#= table.Name #>
{
<#
foreach (Column col in table.Columns)
{
#>
public <#= GetNetDataType(col.DataType.Name) #> <#= col.Name #> { get; set; }
<#
}
#>
}
<# }
#>
}
<#+
public static string GetNetDataType(string sqlDataTypeName)
{
switch (sqlDataTypeName.ToLower())
{
case "bigint":
return "Int64";
case "binary":
return "Byte[]";
case "bit":
return "bool";
case "char":
return "char";
case "cursor":
return string.Empty;
case "datetime":
return "DateTime";
case "decimal":
return "Decimal";
case "float":
return "Double";
case "int":
return "int";
case "money":
return "Decimal";
case "nchar":
return "string";
case "numeric":
return "Decimal";
case "nvarchar":
return "string";
case "real":
return "single";
case "smallint":
return "Int16";
case "text":
return "string";
case "tinyint":
return "Byte";
case "varbinary":
return "Byte[]";
case "xml":
return "string";
case "varchar":
return "string";
case "smalldatetime":
return "DateTime";
case "image":
return "byte[]";
default:
return string.Empty;
}
}
#>
My approach is to:
Use <dynamic> to fetch some rows without type
Serialize these rows to JSON
Copy the JSON string from the console (or using the debugger)
Paste this into a JSON to C# model generator (e.g. https://app.quicktype.io/).
I.e.:
var persons = connection.Query<dynamic>("SELECT * FROM Persons");
var serializedPerson = JsonConvert.Serialize(persons.First());
Console.WriteLine(serializedPerson);
This one is for Oracle. It's probably not complete, but it's worked for me thus far.
SELECT
'public ' || A.NewType || ' ' || REPLACE(INITCAP(REPLACE(A.COLUMN_NAME, '_', ' ')), ' ', '') || ' {get;set;}' GET_SET
, A.*
FROM
(
SELECT
COLUMN_NAME,
DATA_TYPE,
NULLABLE,
CASE
WHEN DATA_TYPE = 'VARCHAR2' THEN 'string'
WHEN DATA_TYPE = 'VARCHAR' THEN 'string'
WHEN DATA_TYPE = 'DATE' AND NULLABLE = 'N' THEN 'DateTime'
WHEN DATA_TYPE = 'DATE' AND NULLABLE = 'Y' THEN 'DateTime?'
WHEN DATA_TYPE = 'INT' AND NULLABLE = 'N' THEN 'int?'
WHEN DATA_TYPE = 'INT' AND NULLABLE = 'Y' THEN 'int'
WHEN DATA_TYPE = 'DECIMAL' AND NULLABLE = 'N' THEN 'decimal'
WHEN DATA_TYPE = 'DECIMAL' AND NULLABLE = 'Y' THEN 'decimal?'
WHEN DATA_TYPE = 'NUMBER' AND NULLABLE = 'N' THEN 'decimal'
WHEN DATA_TYPE = 'NUMBER' AND NULLABLE = 'Y' THEN 'decimal?'
WHEN DATA_TYPE = 'NUMBER2' AND NULLABLE = 'N' THEN 'decimal'
WHEN DATA_TYPE = 'NUMBER2' AND NULLABLE = 'Y' THEN 'decimal?'
WHEN DATA_TYPE = 'CHAR' THEN 'string'
WHEN DATA_TYPE = 'CHAR2' THEN 'string'
WHEN DATA_TYPE = 'timestamp' THEN 'byte[]'
WHEN DATA_TYPE = 'CLOB' THEN 'byte[]'
ELSE '??'
END AS NewType
FROM USER_TAB_COLUMNS
WHERE TABLE_NAME = 'FIN_GLOBAL_CR_NUM_A'
ORDER BY COLUMN_ID
) A
Here's dapper-pocos I made for generating POCOs for Dapper. The solution uses SQL Server's "sp_HELP" and "sp_describe_first_result_set". Give it the name of a stored procedure, or give it a select statement, and it will generate the related POCOs for use with Dapper. The app just passes the stored procedure or select statement to sp_Help and sp_describe_first_result_set, and maps the results to C# data types.
I know it is an old topic,but there is another simple option can choose.
You can use PocoClassGenerator: Mini Dapper's POCO Class Generator (Support Dapper Contrib)
Support current DataBase all tables and views generate POCO class code
Support Dapper.Contrib
Support mutiple RDBMS : sqlserver,oracle,mysql,postgresql
mini and faster (only in 5 seconds generate 100 tables code)
Use appropriate dialect schema table SQL for each database query
DEMO
POCOGenerator Generate Class By Dynamic SQL | .NET Fiddle
POCO Class Generator GenerateAllTables | .NET Fiddle
DataTable POCO Class Generator | .NET Fiddle
GetStart
👇First : Copy&Paste PocoClassGenerator.cs Code to your project or LINQPad.
or Install from NuGet
PM> install-package PocoClassGenerator
👇Second : Use Connection to call GenerateAllTables and then print it.
using (var connection = Connection)
{
Console.WriteLine(connection.GenerateAllTables());
}
Support Dapper Contrib POCO Class
Just call method with GeneratorBehavior.DapperContrib
using (var conn = GetConnection())
{
var result = conn.GenerateAllTables(GeneratorBehavior.DapperContrib);
Console.WriteLine(result);
}
The Online Demo : POCO Dapper Contrib Class Generator GenerateAllTables | .NET Fiddle
Generate Comment
using (var conn = GetConnection())
{
var result = conn.GenerateAllTables(GeneratorBehavior.Comment);
Console.WriteLine(result);
}
Generate View
using (var conn = GetConnection())
{
var result = conn.GenerateAllTables(GeneratorBehavior.View);
Console.WriteLine(result);
}
Generate View and Comment and Dapper.Contrib
using (var conn = GetConnection())
{
var result = conn.GenerateAllTables(GeneratorBehavior.View | GeneratorBehavior.Comment | GeneratorBehavior.DapperContrib);
Console.WriteLine(result);
}
Generate one class by sql
Generate one class
using (var connection = Connection)
{
var classCode = connection.GenerateClass("select * from Table");
Console.WriteLine(classCode);
}
Specify class name
using (var connection = Connection)
{
var classCode = connection.GenerateClass("with EMP as (select 1 ID,'WeiHan' Name,25 Age) select * from EMP", className: "EMP");
Console.WriteLine(classCode);
}
DataTablePocoClass
Code at DataTablePocoClassGenerator.cs
var dt = new DataTable();
dt.TableName = "TestTable";
dt.Columns.Add(new DataColumn() { ColumnName = "ID", DataType = typeof(string) });
var result = dt.GenerateClass();
var expect =
#"public class TestTable
{
public string ID { get; set; }
}";
Assert.Equal(expect, result);
I had exactly the same requirement to generate objects from a database while handling CRUD reliably and efficiently in Dapper and took a different approach of preparing a replacement for Dapper's own Dapper.Contrib with support of Entity Framework schema definition so that scaffolding a database (models, relations, keys) can be done using Entity Framework tools like described for example here, sample below:
dotnet add package Microsoft.EntityFrameworkCore.SqlServer
dotnet add package Microsoft.EntityFrameworkCore.Design
dotnet ef dbcontext scaffold "Server=.\;Database=AdventureWorksLT2012;Trusted_Connection=True;" Microsoft.EntityFrameworkCore.SqlServer -o Model
Above dependencies can be removed from the project after scaffolding.
Currently Dapper.SqlGenerator is successfully working in production. It does not produce any overhead over Dapper in terms of performance, sometimes reducing time to generate a query by other means.
Keep in mind there are 2 separate nuget packages - Dapper.SqlGenerator for purely SQL Code generation from EF (Core) scaffolded models and Dapper.SqlGenerator.Async to run CRUD queries against the database using Dapper.
TLDR; You can use Entity Framework (Core) to scaffold model from database and use Dapper.SqlGenerator to generate CRUD queries on generated objects.
I've seen where people use a hybrid project, using EF to scaffold the database, but I had to dapperize the output from that. For the recommended tools, I'm sure they are good, but I shy away from installing special software, until I had a stab at writing my own solution.
That said, here's a small CLI program(for my needs) that may be useful. Disclaimer, I'm not a seasoned C# programmer, so forgive anything that may be off kilter.
using System;
using System.Collections.Generic;
using System.Data.SqlClient;
using System.IO;
using System.Linq;
using Dapper;
namespace Pocos
{
public class TAB {
public string TABLE_NAME { get; set; }
}
public class COL {
public string COLUMN_NAME { get; set; }
public int? ORIDINAL_POSITIONS { set; get; }
public string DATA_TYPE { get; set; }
public string CHARACTER_MAXIMUM_LENGTH { get; set; }
public string NUMERIC_PRECISION { get; set; }
public string NUMERIC_SCALE { get; set; }
}
class Program {
static void Main(string[] args) {
string sConnect = "Server=LT6-MARKL;Database=PKDEM815;UID=PKDEM815;Password=PKDEM815";
IEnumerable tables;
IEnumerable columns;
List lines;
using ( var conn = new SqlConnection(sConnect))
tables = conn.Query("SELECT * FROM INFORMATION_SCHEMA.TABLES ORDER BY TABLE_NAME");
// Roll through each table of the database and generate an .cs file, as a POCO
foreach (TAB t in tables.OrderBy(t => t.TABLE_NAME)) {
lines = new List();
lines.Add("using System;");
lines.Add("using System.Collections.Generic;");
lines.Add("using System.Configuration;");
lines.Add("using System.Data.SqlClient;");
lines.Add("using Dapper;");
lines.Add("using Dapper.Contrib.Extensions;");
lines.Add("");
lines.Add("namespace PKDataLayer.Models {");
lines.Add("");
lines.Add("\t[Table(\"" + t.TABLE_NAME + "\")]");
lines.Add("\tpublic class " + t.TABLE_NAME + " {");
lines.Add("");
using (var conn2 = new SqlConnection(sConnect)) {
columns = conn2.Query("SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = '"+ t.TABLE_NAME +"' ORDER BY ORDINAL_POSITION");
foreach( COL c in columns) {
if (t.TABLE_NAME + "_KEY" == c.COLUMN_NAME || t.TABLE_NAME + "_SEQNUM" == c.COLUMN_NAME)
lines.Add("\t\t[Key]");
// SELECT DISTINCT DATA_TYPE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME IN ( SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES )
if (c.DATA_TYPE == "char" || c.DATA_TYPE == "varchar")
lines.Add("\t\tpublic string " + c.COLUMN_NAME + " { get; set; }");
if (c.DATA_TYPE == "int")
lines.Add("\t\tpublic int " + c.COLUMN_NAME + " { get; set; }");
if (c.DATA_TYPE == "datetime")
lines.Add("\t\tpublic DateTime? " + c.COLUMN_NAME + " { get; set; }");
if (c.DATA_TYPE == "decimal" || c.DATA_TYPE == "numeric")
lines.Add("\t\tpublic decimal? " + c.COLUMN_NAME + " { get; set; }");
}
}
lines.Add("\t}");
lines.Add("}");
Console.WriteLine("Creating POCO for " + t.TABLE_NAME);
using (TextWriter tw = new StreamWriter( t.TABLE_NAME + ".cs" ))
foreach (String s in lines)
tw.WriteLine(s);
}
}
}
}
This might not work with VS2010, but if you've been updating your Version this should work.
My way of generating Models from a Database is with Ef Core Power Tools, a little Add-On that uses Ef Core 6.
Go to Extensions in Visual Studio and install it. After that, you can right-click on your Project and select Reverse Engineer under EF Core Power Tools.
From there, you connect to the Database, select the Tables to be reverse-engineered and select EntityTypes only
You can be as specific as you want, e.g. specify the output path (DbModels in my case). Click on OK.
Then, your Models should pop up and you're free to use these Models in your Dapper-Code.
I'm the author of a POCO-Generator template called CodegenCS.POCO.
The link above contains C# and PowerShell Scripts which allow you to build complete POCOs (ready to use in Dapper) which also include override bool Equals(), override int GetHashCode(), and (for those who like it) it includes full ActiveRecord CRUD queries (Insert/Update).
Check out this example POCO from Northwind database. If you like it, it's very easy to use the templates:
Edit the connection string and paths in RefreshSqlServerSchema.csx and invoke it through PowerShell script RefreshSqlServerSchema.ps1
This will Extract the Schema of a SQL Server database into a JSON file
Edit the paths and the POCOs Namespace in GenerateSimplePOCOs.csx and invoke it through PowerShell script GenerateSimplePOCOs.ps1
This will Read the JSON Schema and build the POCOs.
The generator script is very simple to understand and customize.
So based on some SQL here and there. I've build a "simple" select that generate class(es) for table(s) or a schema(s)!
What's nice about it, is that it will:
Keep the column order
Adding namespaces
Add the [Table] Attributes on the class.
Add the [Key] on the primary key(s) (might not be supported by dapper.contrib if 2+ PK)
The function below can be used like so:
select *
from dbo.TableToClass('schema','null or table name') m
where m.TableName not in('unwantedtablename')
order by m.TableSchema asc
, m.TableName asc
, m.ClassOrder asc
, m.ColumnOrder asc;
When you copy paste from SSMS, it might remove the TAB. Here is the code:
CREATE or alter function dbo.TableToClass(
#schema varchar(250)
, #table varchar(250)
)
returns table
as
return
/*
--USE IT LIKE: the order by is necessary on the query
select *
from dbo.TableToClass('schemaName', 'null or table name') m
order by
m.tableSchema asc
, m.tableName asc
, m.ClassOrder asc
, m.columnOrder asc
*/
with typeConversion as(
Select
typeConversion.sqlType
,typeConversion.isNullable
,typeConversion.cSharpType
from
( values
('nvarchar', 'YES', 'string'), ('nvarchar','NO', 'string')
,('varchar', 'YES', 'string'), ('varchar', 'NO', 'string')
,('char', 'YES', 'string'), ('char', 'NO', 'string')
,('datetime', 'YES', 'DateTime?'), ('datetime', 'NO', 'DateTime')
,('datetime2', 'YES', 'DateTime?'), ('datetime2', 'NO', 'DateTime')
,('date', 'YES', 'DateTime?'), ('date', 'NO', 'DateTime')
,('datetimeoffset', 'YES', 'DateTimeOffset?'), ('datetimeoffset', 'NO', 'DateTimeOffset')
,('time', 'YES', 'TimeSpan?'), ('timestamp', 'NO', 'TimeSpan')
,('bigint', 'YES', 'long?'), ('bigint', 'NO', 'long')
,('int', 'YES', 'int?'), ('int', 'NO', 'int')
,('smallint', 'YES', 'Int16?'), ('smallint','NO', 'Int16')
,('decimal', 'YES', 'decimal?'), ('decimal', 'NO', 'decimal')
,('numeric', 'YES', 'decimal?'), ('numeric', 'NO', 'decimal')
,('money', 'YES', 'decimal?'), ('money', 'NO', 'decimal')
,('tinyint', 'YES', 'byte?'), ('tinyint', 'NO', 'byte')
,('varbinary', 'YES', 'byte[]'), ('varbinary', 'NO', 'byte[]?')
,('bit', 'YES', 'bool?'), ('bit', 'NO', 'bool')
,('xml', 'YES', 'string'), ('xml', 'NO', 'string')
) typeConversion(sqlType, isNullable, cSharpType)
), columnInfo as (
select
colInfo.TABLE_SCHEMA
, colInfo.TABLE_NAME
, concat(colInfo.TABLE_SCHEMA, '.', colInfo.TABLE_NAME) FullTableName
, colInfo.COLUMN_NAME
, colInfo.ORDINAL_POSITION
, typeConversion.sqlType
, typeConversion.csharpType
,case
(
Select top 1 pk.CONSTRAINT_TYPE
from INFORMATION_SCHEMA.KEY_COLUMN_USAGE AS columnUsage
join INFORMATION_SCHEMA.TABLE_CONSTRAINTS AS pk
--on pk.CONSTRAINT_TYPE = 'PRIMARY KEY'
on pk.TABLE_SCHEMA = colInfo.TABLE_SCHEMA
and pk.TABLE_NAME = colInfo.TABLE_NAME
and pk.CONSTRAINT_NAME = columnUsage.CONSTRAINT_NAME
where
columnUsage.TABLE_SCHEMA = colInfo.TABLE_SCHEMA
and columnUsage.TABLE_NAME = colInfo.TABLE_NAME
and columnUsage.COLUMN_NAME = colInfo.COLUMN_NAME
)
when 'PRIMARY KEY' then (
case (select COLUMNPROPERTY(OBJECT_ID(concat(colInfo.TABLE_SCHEMA ,'.',colInfo.TABLE_NAME)),colInfo.COLUMN_NAME,'isidentity'))
when 1 then 'PK IDENTITY'
else 'PK'
end
)
when 'FOREIGN KEY' then 'FK'
else 'COL'
end as ColumnType
from INFORMATION_SCHEMA.COLUMNS as colInfo
left join typeConversion on typeConversion.sqlType = colInfo.DATA_TYPE and typeConversion.isNullable = colInfo.IS_NULLABLE
where
/************ SET PARAMETER / CONDITION HERE **************/
( --SCHEMA
'True' = (
case
--when #schema is null then return 'True'
when colInfo.TABLE_SCHEMA = coalesce(#schema, 'dbo') then 'True'
else 'False'
end
)
And -- SET SCHEMA NAME HERE (might be dbo for default)
'True' = ( --Table
case
when #table is null then 'True'
when colInfo.TABLE_NAME = #table then 'True'
else 'False'
end
)
)
), classBuilder2_StartFile as (
select top 1
concat(
'using System;', char(10)
,'using Dapper;', char(10)
,'using Dapper.Contrib;', char(10)
,'using Dapper.Contrib.Extensions;', char(10)
, char(10)
,'namespace MYPROJECTNAMESPACE.',c.TABLE_SCHEMA, '.Models.Db; ', char(10)
) as txt
, 'Using & Namespace' as ClassPart
, 5 as ClassOrder
, c.TABLE_SCHEMA as tableSchema
from columnInfo c
), classBuilder2_StartClass as(
select distinct
concat(
char(10)
, '[Table("',c.FullTableName,'")]', char(10)
, 'public partial class ', c.TABLE_NAME, char(10)
, '{'
) as txt
, 'Class name' as ClassPart
, 17 as ClassOrder
, c.TABLE_NAME as tableName
, c.TABLE_SCHEMA as tableSchema
from columnInfo c
), classBuilder2_Properties as(
select
concat(
case c.ColumnType --Column Attribute for dapper.
when 'PK' then
concat(Char(9),'[ExplicitKey]', char(10)) --Dapper: After insert return 0
when 'PK IDENTITY'then
concat(Char(9),'[Key]', char(10)) -- Dapper: After inser return actual PK
else ''
end
, Char(9), char(9), 'public ', c.csharpType,' ', c.COLUMN_NAME, ' { get; set; }'
) as txt
, ORDINAL_POSITION as columnOrder
, 'Property' as ClassPart
, 30 as ClassOrder
, c.COLUMN_NAME as columnName
, c.TABLE_NAME as tableName
, c.TABLE_SCHEMA as tableSchema
from columnInfo c
), classBuilder2_EndClass as(
select distinct
concat(
char(9), '}'
) as txt
, 'End of C# Class' as ClassPart
, 111 as ClassOrder
, c.TABLE_NAME as tableName
, c.TABLE_SCHEMA as tableSchema
from columnInfo c
), classBuilder2_EndFile as(
select top 1
concat(
char(10),char(10)
) as txt
, 'End of C# Class' as ClassPart
, 120 as ClassOrder
, 'ZZZZZ' as tableName
, 'ZZZZZ' as tableSchema
from columnInfo c
), classBuilder_merge as(
select txt, ClassPart, ClassOrder, 'AAA_SCHEMA' as tableName, tableSchema, 'N/A' as columnName, 0 as columnOrder
from classBuilder2_StartFile
union all
select txt, ClassPart, ClassOrder, tableName, tableSchema, 'N/A' as columnName, 0 as columnOrder
from classBuilder2_StartClass
union all
select txt, ClassPart, ClassOrder, tableName, tableSchema, columnName, columnOrder
from classBuilder2_Properties
union all
select txt, ClassPart, ClassOrder, tableName, tableSchema, 'N/A'as columnNam, 0 as columnOrder
from classBuilder2_EndClass
union all
select txt, ClassPart, ClassOrder, tableName, tableSchema, 'N/A'as columnNam, 0 as columnOrder
from classBuilder2_EndFile
), finalSelect as(
--AFTER SQL Server 2016 (13.x) and later. If before that, remove CROSS APPLY STRING_SPLIT
select top 100 percent
lines.*
, m.tableSchema
, m.tableName
, m.ClassOrder
, m.columnOrder
from classBuilder_merge m
CROSS APPLY STRING_SPLIT(m.txt, char(10)) lines
order by --
m.tableSchema asc
, m.tableName asc
, m.ClassOrder asc
, m.columnOrder asc
)
select * from finalSelect;

Resources