I have a yml file that contains a path for data source. Something like this:
data_source: s3://bucket/file.csv
I want to change the job to grab only the file with yesterday appended to the title, for example:
file-2021-10-21.csv
so basically, something like this:
data_source: s3://bucket/file-{yesterday}.csv
How can i define it in the yaml file? thanks!
You can have something like below, where all files defined under 'files' variable
data_source: s3://bucket/file-
files:
- 2021-10-21.csv
- 2021-10-21.csv
Related
I try to write a map in y yaml file and want to use it in yaml file itself with a key. for example ->
in my yaml file I wrote -
pod_env_map:
pod1: "env1"
pod2: "env2"
pod3: "env3"
and in this yaml file itself I'm getting a var $pod_name.
So now i want to write stages with some command for all pod and correspond env.
So instead of putting multiple checks of $pod_name == 'pod1'....
I want to use above dict somehow in code like pod_env_map[$pod_name], please help me to know what should be the syntax for this? I tried to find but didn't get relavent info anywhere.
I am experimenting with the flow editor in PowerAutomate to merge a bunch of CSVs from OneDrive that I am syncing via rclone.
The structure is:
OneDrive(Root)/Folder/Subfolder/*.csv
I would like to merge them into a dataset (master CSV) that I can use with PowerBI.
Because this dataset is updated daily, new csvs will get added to the folder, thus my triggering event is "When a file is created"
The automation looks like this:
When a file is created >
Initialize a String Variable >
Find files in folder >
Search Query: *
Folder: Same as #1
FileSearch Mode: OneDriveSearch
Apply to each >
Get File content
Append to string variable
Compose (string variable) >
Create file >
File Path: whatever/path/
File Name: whatever.csv
File Contents: Outputs
The automation runs fine, and creates my master csv.
Except it's blank!
What's going on?
When a File is Created:
Initialize Variable:
Find Files in Folder:
Apply to Each:
Append String to Variable:
Compose:
Create a File:
At the end of the day, I get a CSV with some data written to it. *Previously, I thought it was blank, but it does, indeed have data.. but it appears truncated.
Something to note here... it looks like it retains the headers..
It looks like it grabbed the first 31 files, but there are 229 files in that folder.
Thanks in advance
Hey all I am wondering if its possible to place different .properties inside just one .properties file so that I do not have to make a seprate .praperties file for each of my database?
This site here is doing what I would like to do but it doesnt explain how to go about sending those contexts to the .properties file. And it also seems to be using seprate .properties files. As in, it doesnt show me how it should look inside the .properties file. Another good example is here but again, it doesnt do it in the .properties file.
Let's say I have the following inside my .properties file:
#liquibase.properties file content
url: jdbc:oracle:thin:#xxxxxxxx.str3.xxxxx.xxxxx:1511/xxxxxx.xxxxx.xxxxx.xxxxx
username: SEPRATE_1PEGA_BASEDA
password: XXXXXXXXXXXXX
referenceUrl: jdbc:oracle:thin:#xxxxxxxx.str2.xxxxx.xxxxx:1511/xxxxxx.xxxxx.xxxxx.xxxxx
referenceUsername: SEPRATE_1PEGA_BASEDB
referencePassword: YYYYYYYYYYYYYY
changeLogFile: diff_change.txt
diffTypes: tables, views, columns, indexes, foreignkeys, primarykeys, uniqueconstraints
And I am wanting to send a paramater to replace the "str2" and "SEPRATE_1PEGA_BASEDB" varibles currently hard-coded inside the .properties file. So I write the .properties file like so:
#liquibase.properties file content
url: jdbc:oracle:thin:#xxxxxxxx.str3.xxxxx.xxxxx:1511/xxxxxx.xxxxx.xxxxx.xxxxx
username: SEPRATE_1PEGA_BASEDA
password: XXXXXXXXXXXXX
referenceUrl: jdbc:oracle:thin:#xxxxxxxx.${liquibase.properties.str}.xxxxx.xxxxx:1511/xxxxxx.xxxxx.xxxxx.xxxxx
referenceUsername: ${liquibase.properties.un}
referencePassword: YYYYYYYYYYYYYY
changeLogFile: diff_change.txt
diffTypes: tables, views, columns, indexes, foreignkeys, primarykeys, uniqueconstraints
So would the CLI for this look like:
liquibase --str=str5 --un=BobBarker diff
My liquibase version is Liquibase-3.6.2.
So if I understand your need properly, you would like to replace placeholders inside your property files.
There are 2 things you should think of:
changelog parameter substitutions - I didn't tested that but if you define property parameter.testproperty=originalvalue and then you put on commandline -Dtestproperty=overridenvalue it could probably replace your value and you will be able to use it in changelogs
liquibase configuration parameters substitution - Liquibase is not doing this, because it's reading properties just like they are in file (method parsePropertiesFile last if/else) and it's trying to fill fields in java with that values. So for this you will need to replace placeholders before calling liquibase command or use different property file.
In one of my Laravel based application, I want to include a JSON key file that is currently located in public/key/store.json
Now I want to write a constant in Laravel .env file so that I can access that file anytime I want. So, I write the following line of code:
KEY_FILE='/public/key/store.json'
But it show file path does not exist.
Can anyone tell me what's wrong in my declaration?
you should push your path in your app/config.php not in your .env
I've found the following .travis.yml template.
I've noticed this:
repo: {GITHUB_USER}/{PROJECT_NAME}
Is this a special .yml variable syntax I'm not familiar with? Where can I set these values (GITHUB_USER, PROJECT_NAME)?
I know I can use environment variables, like so:
repo: $GITHUB_USER/$PROJECT_NAME
but this syntax looks different.
That is not a valid YAML file. After the first } the YAML parser will expect a block style continuation. This means either a key that aligns with repo or outdenting. Instead it finds a / and any YAML parser should throw an error on that.
This looks like a template for a YAML file, e.g. using something like the following in Python after loading the contents of the file in string templ:
templ.format(**dict(GITHUB_USER="Janez", PROJECT_NAME="test"))
On the other hand the recommended extension for YAML files has been .yaml for many more years than Travis exists, so maybe that is why they used the .yml extension.