I often have a task a bit like this: insert a large number of users onto to the users table with similar properties. Not always that simple, but in general, list of strings -> list of corresponding sql statements.
my usual solution is this with the list of usernames in excel use a formula to generate a load of insert statements
=concatenate("insert into users values(username .......'",A1,"'.....
and then I fill down the formula to get all the insert rows.
This works but sometimes the statement is long, sometimes including a few different steps for each, and cramming it all into an excel formula and getting all the wrapping quotes right is a pain.
I'm wondering if there is a better way. What I really want is to be able to have a template file template txt:
insert into users
([username],
[company] ...
)
values('<template tag1>...
and then using some magic command line tool, to simply be able to type something like
command_line> make_big_file_using_template template.txt /values [username1 username2]
/output: bigfile.txt
and this gives me a big file with the template repeated for each username value with the tag replaced with the username.
So does such a command exist, or are my expectations of command line tools too high? Any freely available windows tool will do. I could whip up a c# program to do this in not too much time but I feel like there must be an easy to use tool out there already.
This is trivial using a Powershell script. PS allows inline variables in strings, so you could do something like:
$Tag1 = 'blah'
$Tag2 = 'foo'
$SQLHS = #"
INSERT INTO users
([username],
[company],...)
VALUES
('$tag1', '$tag2'...)
"#
set-content 'C:\Mynewfile.txt' -value $SQLHS
The #"...."# is a here-string, which makes it very easy to write readable code without escaping quotes and such.
The above could be very easily modified to accept parameters for the various tags and another for the output file, or to run for a set of values located in another .txt or .csv file as inputs.
EDIT:
To modify it to accept parameters, you can just add a param() block at top:
param($outfile, $tab1, $tab2, $tab3)
Then use those $variables in your script:
set-content "$outfile" -value $SQLHS
Related
I have several processes with almost same flow like "Get some parameters, extract data from database according to them and upload them to target". The parameters vary slightly across processes as well as targets but only a bit. Most of the process is the same. I would like to extract those differences to parameter-context and dynamically load them. My idea is to have parameters defined following way and then using them.
So core of question is:
How to dynamically choose which parameter group load and use?
Having several parameter contexts with same-named/different-valued parameters and dynamically switching them would be probably the best, but it is not possible as far as I know.
Also duplicating flows is out-of-the-table. Any error correction would be spread out over several places and maintenance would be a nightmare.
Moreover, I know I can do it like "In GenetrateFlowFile for process A set value1=#{A_value1} and in GenetrateFlowFile for process B set value1=#{B_value1}. But this is tedious, error-prone and scales kinda bad. Not speaking of situation when I can have dozens of parameters and several processes. Also it is a kind of hardcoding, not configuring...
I was hoping for something like defining group=A and then using it like value1=#{ ${ group:append('_value1') } } but this does not work - it is evaluated as parameter literally named ${ group:append('_value1') }.
TL;DR: Use evaluateELString().
The actual solution is to set in GenetrateFlowFile processor group=A and in next UpdateAttribute processor set the following:
value1=${ group:prepend('hash{ '):append('_value1 }'):replace('hash', '#'):evaluateELString() }
The magic being done here is "Take value of group slap around it #{ and _value1 } to make it valid NiFi Expression Language statement and then evaluate it." (Notice - the word hash and function replace is there since I didn´t manage to escape the # char right before {.)
If you would like to have your value1 at the beginning of the statement then you can use following code. The result is same, it is easier to use (often-changed value value1 is at the beginning of the statement) and is less readable "what is really going on?"-wise.
value1=${ literal('value1'):prepend('_'):prepend(${ group }):prepend('hash{ '):append(' }'):replace('hash', '#'):evaluateELString() }
I study genetic data from 288 fish samples (Fish_one, Fish_two ...)
I have four files per fish, each with a different suffix.
eg. for sample_name Fish_one:
file 1 = "Fish_one.1.fq.gz"
file 2 = "Fish_one.2.fq.gz"
file 3 = "Fish_one.rem.1.fq.gz"
file 4 = "Fish_one.rem.2.fq.gz"
I would like to apply the following concatenate instructions to all my samples, using maybe a text file containing a list of all the sample_name, that would be provided to a loop?
cp sample_name.1.fq.gz sample_name.fq.gz
cat sample_name.2.fq.gz >> sample_name.fq.gz
cat sample_name.rem.1.fq.gz >> sample_name.fq.gz
cat sample_name.rem.2.fq.gz >> sample_name.fq.gz
In the end, I would have only one file per sample, ideally in a different folder.
I would be very grateful to receive a bit of help on this one, even though I'm sure the answer is quite simple for a non-novice!
Many thanks,
Noé
I would like to apply the following concatenate instructions to all my
samples, using maybe a text file containing a list of all the
sample_name, that would be provided to a loop?
In the first place, the name of the cat command is mnemonic for "concatentate". It accepts multiple command-line arguments naming sources to concatenate together to the standard output, which is exactly what you want to do. It is poor form to use a cp and three cats where a single cat would do.
In the second place, although you certainly could use a file of name stems to drive the operation you describe, it's likely that you don't need to go to the trouble to create or maintain such a file. Globbing will probably do the job satisfactorily. As long as there aren't any name stems that need to be excluded, then, I'd probably go with something like this:
for f in *.rem.1.fq.gz; do
stem=${f%.rem.1.fq.gz}
cat "$stem".{1,2,rem.1,rem.2}.fq.gz > "${other_dir}/${stem}.fq.gz"
done
That recognizes the groups present in the current working directory by the members whose names end with .rem.1.fq.gz. It extracts the common name stem from that member's name, then concatenates the four members to the correspondingly-named output file in the directory identified by ${other_dir}. It relies on brace expansion to form the arguments to cat, so as to minimize code and (IMO) improve clarity.
I'm trying to get Word to fill in cells in a table. The script works when run as a macro from within Word, but fails when saved as a .vbs file and double-clicked, or run with wscript. This is a part of it.
set obj = GetObject(,"Word.Application)
With obj
With .Selection
MsgBox .text
If (.Information(wdWithInTable) = True) Then
.Collapse Direction:=wdCollapseStart
tCols = .Tables(1).Columns.Count
tRow = .Information(wdStartOfRangeRowNumber)
tCol = .Information(wdStartOfRangeColumnNumber)
For I = 2 To 5
.Tables(1).Cell(tRow, I).Range.Text = "fred" & Str(I)
Next
` now make new row
For I = 1 To tCols - tCol + 1
.MoveRight unit:=wdCell
Next
End If
End With
End With
I have three problems. First, it won't compile unless I comment out the .Collapse and .MoveRight lines. Second, although the MsgBox .text displays the selected text, I get "out of range" errors if I try to access any .Information property.
I'm sure I'm missing something very simple: I usually write software for Macs, and I'd do this using AppleScript. This is my first attempt at getting anything done under Windows.
VBScript and VBA are different languages.
They are a bit similar, but not very. Moreover, VBScript is not like AppleScript; it doesn't let you easily interface with running programs.
The interfaces you'll get from VBScript can behave subtly differently in VBA and VBScript. However, I think you've got two problems here:
:= is invalid syntax in VBScript; you'll need to find an alternative way of calling the function. Try just using positional arguments.
You've no guarantee that this will open the expected file; there could be another instance of Word that it's interacting with instead.
Since your code is not running within the Word environment it would require a reference to the Word object library in order to use enumeration constants (those things that start with wd).
VBScript, however, cannot work with references, which means the only possibility is to use the long value equivalents of the enumerations. You'll find these in the Word Language References. Simplest to use is probably the Object Browser in Word's VBA Editor. (In Word: Alt+F11 to open the VBA Editor; F2 to start the Object Browser; type in the term in the "Search" box, click on the term, then look in the bottom bar.)
The code in the question uses, for example:
wdWithInTable
wdCollapseStart
wdStartOfRangeRowNumber
wdStartOfRangeColumnNumber
wdCell
The reason you get various kinds of errors depends on where these are used.
Also, VBScript can't used named parameters such as Unit:=. Any parameters must be passed in comma-delimited format, if there's more than one, in the order specified by the method or property. If there are optional parameters you don't want to use these should be left "blank":
MethodName parameter, parameter, , , parameter
My knowledge on SQL and Oracle does not go beyond querying I'm afraid, but what I'm looking to do is run some kind of script which will produce multiple reports from one execution. I have a whole bunch of code and at the moment I'm rerunning it about 30 times, each time replacing the following in the WHERE clause with a different "SUBJECT_ID":
SELECT ...
FROM ...
WHERE ...
AND (v.SUBJECT_ID LIKE 'B%')
...
I'm thinking I should be able to run some kind of loop script and output all the 30 reports in one go...? I am hoping to get some kind of training on this quite soon, but any help would be greatly appreciated!
Assuming by
each time replacing the following in the WHERE clause with a different "SUBJECT_ID"
you mean v.subject_id like '<some value>', where is different for each loop round the code, I would do the following if I were you:
replace the v.subject_id like '<some value>' with v.subject_id like '&&subj_id.' (the . is necessary to say that's the end of the parameter name) and create that as it's own script, eg. report.sql.
Then I'd create a new script and do:
define subj_id set <some value>;
##report.sql
define subj_id set <some other value>;
##report.sql
...
I created a GUI and used uiimport to import a dataset into matlab workspace, I would like to pass this imported data to another function in matlab...How do I pass this imported dataset into another function....I tried doing diz...but it couldnt pick diz....it doesnt pick the data on the matlab workspace....any ideas??
[file_input, pathname] = uigetfile( ...
{'*.txt', 'Text (*.txt)'; ...
'*.xls', 'Excel (*.xls)'; ...
'*.*', 'All Files (*.*)'}, ...
'Select files');
uiimport(file_input);
M = dlmread(file_input);
X = freed(M);
I think that you need to assign the result of this statement:
uiimport(file_input);
to a variable, like this
dataset = uiimport(file_input);
and then pass that to your next function:
M = dlmread(dataset);
This is a very basic feature of Matlab, which suggests to me that you would find it valuable to read some of the on-line help and some of the documentation for Matlab. When you've done that you'll probably find neater and quicker ways of doing this.
EDIT: Well, #Tim, if all else fails RTFM. So I did, and my previous answer is incorrect. What you need to pass to dlmread is the name of the file to read. So, you either use uiimport or dlmread to read the file, but not both. Which one you use depends on what you are trying to do and on the format of the input file. So, go RTFM and I'll do the same. If you are still having trouble, update your question and provide details of the contents of the file.
In your script you have three ways to read the file. Choose one on them depending on your file format. But first I would combine file name with the path:
file_input = fullfile(pathname,file_input);
I wouldn't use UIIMPORT in a script, since user can change way to read the data, and variable name depends on file name and user.
With DLMREAD you can only read numerical data from the file. You can also skip some number of rows or columns with
M = dlmread(file_input,'\t',1,1);
skipping the first row and one column on the left.
Or you can define a range in kind of Excel style. See the DLMREAD documentation for more details.
The filename you pass to DLMREAD must be a string. Don't pass a file handle or any data. You will get "Filename must be a string", if it's not a string. Easy.
FREAD reads data from a binary file. See the documentation if you really have to do it.
There are many other functions to read the data from file. If you still have problems, show us an example of your file format, so we can suggest the best way to read it.