Different paramaters in one liquibase .properties file possible? - jenkins-pipeline

Hey all I am wondering if its possible to place different .properties inside just one .properties file so that I do not have to make a seprate .praperties file for each of my database?
This site here is doing what I would like to do but it doesnt explain how to go about sending those contexts to the .properties file. And it also seems to be using seprate .properties files. As in, it doesnt show me how it should look inside the .properties file. Another good example is here but again, it doesnt do it in the .properties file.
Let's say I have the following inside my .properties file:
#liquibase.properties file content
url: jdbc:oracle:thin:#xxxxxxxx.str3.xxxxx.xxxxx:1511/xxxxxx.xxxxx.xxxxx.xxxxx
username: SEPRATE_1PEGA_BASEDA
password: XXXXXXXXXXXXX
referenceUrl: jdbc:oracle:thin:#xxxxxxxx.str2.xxxxx.xxxxx:1511/xxxxxx.xxxxx.xxxxx.xxxxx
referenceUsername: SEPRATE_1PEGA_BASEDB
referencePassword: YYYYYYYYYYYYYY
changeLogFile: diff_change.txt
diffTypes: tables, views, columns, indexes, foreignkeys, primarykeys, uniqueconstraints
And I am wanting to send a paramater to replace the "str2" and "SEPRATE_1PEGA_BASEDB" varibles currently hard-coded inside the .properties file. So I write the .properties file like so:
#liquibase.properties file content
url: jdbc:oracle:thin:#xxxxxxxx.str3.xxxxx.xxxxx:1511/xxxxxx.xxxxx.xxxxx.xxxxx
username: SEPRATE_1PEGA_BASEDA
password: XXXXXXXXXXXXX
referenceUrl: jdbc:oracle:thin:#xxxxxxxx.${liquibase.properties.str}.xxxxx.xxxxx:1511/xxxxxx.xxxxx.xxxxx.xxxxx
referenceUsername: ${liquibase.properties.un}
referencePassword: YYYYYYYYYYYYYY
changeLogFile: diff_change.txt
diffTypes: tables, views, columns, indexes, foreignkeys, primarykeys, uniqueconstraints
So would the CLI for this look like:
liquibase --str=str5 --un=BobBarker diff
My liquibase version is Liquibase-3.6.2.

So if I understand your need properly, you would like to replace placeholders inside your property files.
There are 2 things you should think of:
changelog parameter substitutions - I didn't tested that but if you define property parameter.testproperty=originalvalue and then you put on commandline -Dtestproperty=overridenvalue it could probably replace your value and you will be able to use it in changelogs
liquibase configuration parameters substitution - Liquibase is not doing this, because it's reading properties just like they are in file (method parsePropertiesFile last if/else) and it's trying to fill fields in java with that values. So for this you will need to replace placeholders before calling liquibase command or use different property file.

Related

Create a dynamic path inside a yaml file

I have a yml file that contains a path for data source. Something like this:
data_source: s3://bucket/file.csv
I want to change the job to grab only the file with yesterday appended to the title, for example:
file-2021-10-21.csv
so basically, something like this:
data_source: s3://bucket/file-{yesterday}.csv
How can i define it in the yaml file? thanks!
You can have something like below, where all files defined under 'files' variable
data_source: s3://bucket/file-
files:
- 2021-10-21.csv
- 2021-10-21.csv

How do I connect .env file to config.yml?

I'm currently building a Shopify store and would like to use env variables in Themekit's config.yml file. What I'm confused about is how to connect the .env file to the yml file, since I don't think you can just require dotenv. I have my .env file, and the code below in the config.yml. Thanks!
password: ${DEV_PASSWD}
theme_id: ${DEV_THEMEID}
store: ${DEV_SHOP}
You can't include .env file inside a YAML one. However, you can interpolate variables into your config.yml file using the ${} notation.
To help you interpolate variables, there are special files that can be used to automatically to load environment variables for Theme Kit. The following table lists the file paths for each operating system:
macOs: ${HOME}/Library/Application Support/Shopify/Themekit/variables
Linux/BSD: ${XDG_CONFIG_HOME}/Shopify/Themekit/variables
Windows: %APPDATA%\Shopify\Themekit\variables
Even more, you can use the --vars flag in any command to provide a path to a file for loading variables. The variables file has the same format as most .env type files. But note, the .env file is not interpolated by YAML itself and it cannot be connected using standard YAML include directives. All magic is provided exclusively by shopify and its --vars flag.

Logstash variables definition in config file

In my Logstash config file I have many jdbc inputs and all of them use the same database and the same credentials. Each time I want to change for example connection string I have to loop through all jdbc inputs manually. Can I somehow define variables once and then use them in the config file for example like that?
CONNECTION_STRING_VARIABLE => "MY_CONNECTION_STRING"
jdbc {
...
jdbc_connection_string => CONNECTION_STRING_VARIABLE
...
}
I don't want to use environmental variables because of user and password fields and I want to store variables in one place.
For passwords, you may use property jdbc_password_filepath.
I read that you don't want to set environment variables. I am giving a way here, so that, it will not be available in environment of every shell but will get loaded for logstash only.
You may create a script that exports all variables for logstash and call it in logstash service or logstash command line.
For example, create a file - exportVariablesForLogstash.sh
export jdbc_url="jdbc:mysql://example.local:3306/sampledb"
export jdbc_username=mysqluser
Add following in starting of logstash service or logstash commandline script. Note dot at starting.
. exportVariablesForLogstash.sh
And then you may use these variables as documented here. https://www.elastic.co/guide/en/logstash/current/environment-variables.html. I believe you already know this one.

Use multiple env files

I'm wondering if there's a way in Laravel to specify a set of env files to load. My exact problem is I want to add something like a suffix to all my .js and .css resources. Ideally I'd have a suffix like the release date because it would be ok for these files to be cached within the same release but I would like the caches to be invalidated on the next release. However I want to avoid reading, modifying and saving the .env file if possible and would instead prefer to create a new file e.g. .env.rdate which would be generated via a script, e.g.
echo APP_RELEASE_DATE=`date +%s` > env.rdate
or something like this. Is this at all possible or do I have to read/update/write the .env file instead?
Create your .env.rdate file next to .env file.
Put this to your AppServiceProvider boot method:
$dotenv = new \Dotenv\Dotenv(base_path(),'.env.rdate');
$dotenv->overload();
After you can use in your project:
ENV('APP_RELEASE_DATE')

It is possible to reference another SQL file from SQL script

Basically I want to execute an SQL file from an SQL file in Postgres.
Similar question for mysql: is it possible to call a sql script from a stored procedure in another sql script?
Why?
Because I have 2 data files in a project and I want to have one line that can be commented/un-commented that loads the second file.
Clarification:
I want to call B.SQL from A.SQL
Clarification2:
This is for a Spring Project that uses hibernate to create the database from the initial SQL file (A.SQL).
On further reflection it seems I may have to handle this from java/string/hibernate.
Below is the configuration file:
spring.datasource.url=jdbc:postgresql://localhost:5432/dbname
spring.datasource.username=postgres
spring.datasource.password=root
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.data=classpath:db/migration/postgres/data.sql
spring.jpa.hibernate.ddl-auto=create
Import of other files is not supported in Sql, but if you execute the script with psql can you use the \i syntax:
SELECT * FROM table_1;
\i other_script.sql
SELECT * FROM table_2;
This will probably not work if you execute the sql with other clients than psql.
Hibernate is just:
reading all your SQL files line per line
strip any comment (lines starting with --, // or /*)
removes any ; at the end
executes the result as a single statement
(see SchemaExport.importScript and SingleLineSqlCommandExtractor)
There is no support for an include here.
What you can do:
Define your own ImportSqlCommandExtractor which knows how to include a file - you can set that extractor with hibernate.hbm2ddl.import_files_sql_extractor=(fully qualified class name)
Define your optional file as additional import file with hibernate.hbm2ddl.import_files=prefix.sql,optional.sql,postfix.sql, you can either add and remove the file reference as you like, or you can even exclude the file from your artifact - a missing file will only create a debug message.
Create an Integrator which sets the hibernate.hbm2ddl.import_files property dynamically - depending on some environment property

Resources