Unreachable command line option in MAIN - syntax

zef search includes :$update as a named argument:
multi MAIN('search', Int :$wrap = False, :$update, *#terms ($, *#))
However, it's not recognized as such:
% zef search --update
Usage:
/home/jmerelo/.rakudobrew/bin/../moar-2019.03.1/install/share/perl6/site/bin/zef [--wrap=<Int>] search [<terms> ...] -- Get a list of possible distribution candidates for the given terms
/home/jmerelo/.rakudobrew/bin/../moar-2019.03.1/install/share/perl6/site/bin/zef [--version] -- Detailed version information
/home/jmerelo/.rakudobrew/bin/../moar-2019.03.1/install/share/perl6/site/bin/zef [-h|--help]
What am I missing here? How could I assign a value to this $update?

If I run this MAIN candidate by itself, it works:
$ perl6 -e 'multi MAIN("search", Int :$wrap = False, :$update, *#terms ($, *#)) { say "foo" }' search --update
foo
so it looks to me there is more than one candidate that matches, and that causes the USAGE feedback message to appear.
Named parameters are used as tie-breakers only, unless they are made mandatory (which makes them effectively a part of the dispatch process). So maybe the fix is to make two candidates:
multi MAIN('search', Int :$wrap = False, :$update!, *#terms ($, *#)) { ... }
multi MAIN('search', Int :$wrap = False, *#terms ($, *#)) { ... }
?

Related

Snakemake, how to change output filename when using wildcards

I think I have a simple problem but I don't how to solve it.
My input folder contains files like this:
AAAAA_S1_R1_001.fastq
AAAAA_S1_R2_001.fastq
BBBBB_S2_R1_001.fastq
BBBBB_S2_R2_001.fastq
My snakemake code:
import glob
samples = [os.path.basename(x) for x in sorted(glob.glob("input/*.fastq"))]
name = []
for x in samples:
if "_R1_" in x:
name.append(x.split("_R1_")[0])
NAME = name
rule all:
input:
expand("output/{sp}_mapped.bam", sp=NAME),
rule bwa:
input:
R1 = "input/{sample}_R1_001.fastq",
R2 = "input/{sample}_R2_001.fastq"
output:
mapped = "output/{sample}_mapped.bam"
params:
ref = "refs/AF086833.fa"
run:
shell("bwa mem {params.ref} {input.R1} {input.R2} | samtools sort > {output.mapped}")
The output file names are:
AAAAA_S1_mapped.bam
BBBBB_S2_mapped.bam
I want the output file to be:
AAAAA_mapped.bam
BBBBB_mapped.bam
How can I or change the outputname or rename the files before or after the bwa rule.
Try this:
import pathlib
indir = pathlib.Path("input")
paths = indir.glob("*_S?_R?_001.fastq")
samples = set([x.stem.split("_")[0] for x in paths])
rule all:
input:
expand("output/{sample}_mapped.bam", sample=samples)
def find_fastqs(wildcards):
fastqs = [str(x) for x in indir.glob(f"{wildcards.sample}_*.fastq")]
return sorted(fastqs)
rule bwa:
input:
fastqs = find_fastqs
output:
mapped = "output/{sample}_mapped.bam"
params:
ref = "refs/AF086833.fa"
shell:
"bwa mem {params.ref} {input.fastqs} | samtools sort > {output.mapped}"
Uses an input function to find the correct samples for rule bwa. There might be a more elegant solution, but I can't see it right now. I think this should work, though.
(Edited to reflect OP's edit.)
Unfortunately, I've also had this problem with filenames with the following logic: {batch}/{seq_run}_{index}_{flowcell}_{lane}_{read_orientation}.fastq.gz.
I think that the core problem is that none of the individual wildcards are unique. Also, not all values for all wildcards can be combined; seq_run1 was run on lane1, not lane2. Therefore, expand() does not work.
After multiple attempts in Snakemake (see below), my solution was to standardize input with mv / sed / rename. Removing {batch}, {flowcell} and {lane} made it possible to use {sample}, a unique combination of {seq_run} and {index}.
What did not work (but it could be worth to try for others in the same situation):
Adding the zip argument to expand()
Renaming output using the following syntax:
output: "_".join(re.split("[/_]", "{full_filename}")[1,2]+".fastq.gz"

terraform: lambda invocation during destroy

I have lambda invocation in our terraform-built environment:
data "aws_lambda_invocation" "this" {
count = var.invocation == "true" ? 1 : 0
function_name = aws_lambda_function.this.function_name
input = <<JSON
{
"Name": "Invocation"
}
JSON
}
The problem: the function is invoked not only during creation ("apply") but deletion ("destroy") too. How to invoke it during creation only? I thought about checking environment variables in the lambda (perhaps TF adds name of the process here or something like that) but I hope there's a better way.
Worth checking if you can use the -var 'lambda_xxx=execute' option while running the terraform command to check if the lambda code needs to be executed or not terraform docs
Using that variable lambda_xxx passed in via the command line while executing the command, you can check in the terraform code whether you want to run the lambda code or not.
Below code creates a waf only if the count is 1
resource "aws_waf_rule" "wafrule" {
depends_on = ["aws_waf_ipset.ipset"]
name = "${var.environment}-WAFRule"
metric_name = "${replace(var.environment, "-", "")}WAFRule"
count = "${var.is_waf_enabled == "true" ? 1 : 0}"
predicates {
data_id = "${aws_waf_ipset.ipset.id}"
negated = false
type = "IPMatch"
}
}
Variable declared in variables.tf file
variable "is_waf_enabled" {
type = "string"
default = "false"
description = "String value to indicate if WAF/API KEY is turned on or off (true/any_value)"
}
When you run the command any value other than true is considered false as we are just checking for string true.
Similarly you can do this for your lambda.
There are better alternative solutions for this problem now, which weren't available at the time the question was asked.
Lambda Invocation Resource in AWS provider: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_invocation
Lambda Based Resource in LambdaBased provider: https://registry.terraform.io/providers/thetradedesk/lambdabased/latest/docs/resources/lambdabased_resource
With the disclaimer that I'm the developer of the latter one: If the underlying problem is managing a resource through lambda functions, the lambda based resource has some good features tailored specifically to accomplish that with the obvious drawback of adding another provider dependency.

yargs .command error: second argument to option must be an object

Suppose I have a file "test.js":
var args = require('yargs')
.command('command', 'command usage here', {alias: "c"} )
.argv;
Then I run:
>node test command
I got this error:
second argument to option must be an object
If I remove the 3rd parameter of .command:
var args = require('yargs')
.command('command', 'command usage here')
.argv;
Everything is fine.
I must make a dumb mistake. But I just cannot figure it out.
Thanks
Your 3rd argument is not required, that's why it works when you remove it. I'm pretty sure the 3rd argument has to be a function call.
var args = require('yargs')
.command('command',
'command explanation',
function(yargs){
//insert yargs.options here
yargs.options({
c:{
demand: true,//if you require the command
alias: 'c',// you will enter -c to use it
description: 'explain what it does'
}
});
})
.argv;
an example of usage might be:
C:\WorkingDirectory>node app.js command -c run
your code could include console.log(args.c);//prints out run

Can't add jvmArgs using add(), why?

Can anyone explain why the first sample working while the second does nothing?
test {
jvmArgs '-Xdebug',
'-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=4000'
}
test {
jvmArgs.add('-Xdebug')
jvmArgs.add('-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=4000')
}
Because in the second example this method is invoked. You get the list, modify it but the changes are not reflected to settings - read only access. In the first example this method is invoked and arguments passed are set.
Here's the explanation, a copy of the list is returned (it's for safety, security reasons - mutable types should be always returned as a copy)
public List<String> getJvmArgs() {
List<String> args = new ArrayList<String>();
for (Object extraJvmArg : extraJvmArgs) {
args.add(extraJvmArg.toString());
}
return args;
}
I found this problematic with the normal command line arguments using Gradle as well -- Even to the extent that Example 1 works and Example 2 will fail to add the extra argument:
runArgs is set in the main build.gradle
As follows:
ext {
runArgs = [ '-server=localhost', '-port=8080' ];
}
Simply appending to the command line appears challenging (see below).
Example 1:
debug.doFirst(){
// ... <snip> ...
// command line arguments
//
println " debug args (a): ${args}."
runArgs.add( "-memo=${project.name}:debug" );
args = runArgs;
println " debug args (b): ${args}."
}
Output is correct, shows "-memo" parameter is added but also the passed-in args have been replaced by the script variable using this approach.
debug args (a): [-server, localhost, -port, 8080 ].
debug args (b): [-server=localhost, -port=8080, -memo=Client:debug].
Example 2:
debug.doFirst(){
// ... <snip> ...
// command line arguments
//
println " debug args (a): ${args}."
args.add( "-memo=${project.name}:debug" );
println " debug args (b): ${args}."
}
Output (correctly?) shows there was no add(), per answer: from Opal above.
debug args (a): [-server, localhost, -port, 8080 ].
debug args (b): [-server, localhost, -port, 8080 ].
I've posted this example to show that there may be alternatives to accepting the status quo, i expect the jvmArgs to work in a similar pattern. I didn't find examples for adding extra debug specific arguments (say). So here is one.
I also saw in a couple of places (on-line and books), examples such as:
jvmArgs.add( "-DAPPLICATION_LOCATION=City" );
jvmArgs.add( "-DSERVER_HOST=localhost" );
Which as we now understand, do not work.
The use-case I set-out to implement is for the sub-project build.gradle Script to supply missing arguments and/or script specific parameters (e.g. as in the debug run example). It is clear to me that if you want to do this, the script will need to either replace the command line or analyse the args passed-in and then wrinkle-out the defaults by some mechanism.
Hopefully the example will give others more insight.
Actually this hack is working:
def jvmArgsCopy = jvmArgs
jvmArgsCopy.add("-XX:MaxDirectMemorySize=2g")
jvmArgs = jvmArgsCopy
And to add to this here it is in Kotlin gradle
val jvmArgsCopy: ArrayList<String> = jvmArgs as ArrayList<String>
jvmArgsCopy.add("-Xdebug")
jvmArgsCopy.add("-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=4000")
jvmArgs = jvmArgsCopy

Forcing a package's function to use user-provided function

I'm running into a problem with the MNP package which I've traced to an unfortunate call to deparse (whose maximum width is limited to 500 characters).
Background (easily skippable if you're bored)
Because mnp uses a somewhat idiosyncratic syntax to allow for varying choice sets (you include cbind(choiceA,choiceB,...) in the formula definition), the left hand side of my formula call is 1700 characters or so when model.matrix.default calls deparse on it. Since deparse supports a maximum width.cutoff of 500 characters, the sapply(attr(t, "variables"), deparse, width.cutoff = 500)[-1L] line in model.matrix.default has as its first element:
[1] "cbind(plan1, plan2, plan3, plan4, plan5, plan6, plan7, plan8, plan9, plan10, plan11, plan12, plan13, plan14, plan15, plan16, plan17, plan18, plan19, plan20, plan21, plan22, plan23, plan24, plan25, plan26, plan27, plan28, plan29, plan30, plan31, plan32, plan33, plan34, plan35, plan36, plan37, plan38, plan39, plan40, plan41, plan42, plan43, plan44, plan45, plan46, plan47, plan48, plan49, plan50, plan51, plan52, plan53, plan54, plan55, plan56, plan57, plan58, plan59, plan60, plan61, plan62, plan63, "
[2] " plan64, plan65, plan66, plan67, plan68, plan69, plan70, plan71, plan72, plan73, plan74, plan75, plan76, plan77, plan78, plan79, plan80, plan81, plan82, plan83, plan84, plan85, plan86, plan87, plan88, plan89, plan90, plan91, plan92, plan93, plan94, plan95, plan96, plan97, plan98, plan99, plan100, plan101, plan102, plan103, plan104, plan105, plan106, plan107, plan108, plan109, plan110, plan111, plan112, plan113, plan114, plan115, plan116, plan117, plan118, plan119, plan120, plan121, plan122, plan123, "
[3] " plan124, plan125, plan126, plan127, plan128, plan129, plan130, plan131, plan132, plan133, plan134, plan135, plan136, plan137, plan138, plan139, plan140, plan141, plan142, plan143, plan144, plan145, plan146, plan147, plan148, plan149, plan150, plan151, plan152, plan153, plan154, plan155, plan156, plan157, plan158, plan159, plan160, plan161, plan162, plan163, plan164, plan165, plan166, plan167, plan168, plan169, plan170, plan171, plan172, plan173, plan174, plan175, plan176, plan177, plan178, plan179, "
[4] " plan180, plan181, plan182, plan183, plan184, plan185, plan186, plan187, plan188, plan189, plan190, plan191, plan192, plan193, plan194, plan195, plan196, plan197, plan198, plan199, plan200, plan201, plan202, plan203, plan204, plan205, plan206, plan207, plan208, plan209, plan210, plan211, plan212, plan213, plan214, plan215, plan216, plan217, plan218, plan219, plan220, plan221, plan222, plan223, plan224, plan225, plan226, plan227, plan228, plan229, plan230, plan231, plan232, plan233, plan234, plan235, "
[5] " plan236, plan237, plan238, plan239, plan240, plan241, plan242, plan243, plan244, plan245, plan246, plan247, plan248, plan249, plan250, plan251, plan252, plan253, plan254, plan255, plan256, plan257, plan258, plan259, plan260, plan261, plan262, plan263, plan264, plan265, plan266, plan267, plan268, plan269, plan270, plan271, plan272, plan273, plan274, plan275, plan276, plan277, plan278, plan279, plan280, plan281, plan282, plan283, plan284, plan285, plan286, plan287, plan288, plan289, plan290, plan291, "
[6] " plan292, plan293, plan294, plan295, plan296, plan297, plan298, plan299, plan300, plan301, plan302, plan303, plan304, plan305, plan306, plan307, plan308, plan309, plan310, plan311, plan312, plan313)"
When model.matrix.default tests this against the variables in the data.frame, it returns an error.
The problem
To get around this, I've written a new deparse function:
deparse <- function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
}
However, when I run mnp again and step through, it returns the same error for the same reason (base::deparse is being run, not my deparse).
This is somewhat surprising to me, as what I expect is more typified by this example, where the user-defined function temporarily over-writes the base function:
> print <- function() {
+ cat("user-defined print ran\n")
+ }
> print()
user-defined print ran
I realize the right way to solve this problem is to rewrite model.matrix.default, but as a tool for debugging I'm curious how to force it to use my deparse and why the anticipated (by me) behavior is not happening here.
The functions fixInNamespace and assignInNamespace are provided to allow editing of existing functions. You could try ... but I will not since mucking with deparse looks too dangerous:
assignInNamespace("deparse",
function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
} , "base")
There is an indication on the help page that the use of such functions has restrictions and I would not be surprised that such core function might have additional layers of protection. Since it works via side-effect, you should not need to assign the result.
This is how packages with namespaces search for functions, as described in Section 1.6, Package Namespaces of Writing R Extensions
Namespaces are sealed once they are loaded. Sealing means that imports
and exports cannot be changed and that internal variable bindings
cannot be changed. Sealing allows a simpler implementation strategy
for the namespace mechanism. Sealing also allows code analysis and
compilation tools to accurately identify the definition corresponding
to a global variable reference in a function body.
The namespace controls the search strategy for variables used by
functions in the package. If not found locally, R searches the package
namespace first, then the imports, then the base namespace and then
the normal search path.

Resources