I have some older repository variables that I'm looking to turn off. Is there a good way to see where these variables are being used in reports?
Im assuming you are on a Linux platform? You could create a test report using one of those variables (or all of them if you wish), and then run a grep command on the catalog in the database to spit out all reports that have that variable in them. I have done this on columns many times.
For example, I have a column named Reporting Item Description under the - Product Details folder. When anyone uses this column in OBIEE, this is how it is referenced in the catalog:
I would perform the following grep command that would spit out all reports (including their path in the catalog) that have that certain column (or in your case, variable):
find ./ -type f -! -name "*.atr" -exec grep -l -e " - Product Details"."Reporting Item Description"" {} \;
This same scenario should hold true for repository variables (or any variables).
Unfortunately, I think your best bet might be to search the Catalog using the file system. As the catalog objects are all stored as XML you can search them for the repository variables you have.
Related
I'm using the software Bowtie 2 which aligns genome sequences. I have all my indexes in a directory miniReference1.
When I call Bowtie2 with the -x <dir> option I get the error that my index is not a Bowtie 2 index. What am I doing wrong? Below is a screenshot:
The -x argument takes the path to the basename of the index. That means, if the index called miniReference1.{suffix} is inside a folder also called miniReference1 then it must be -x path/to/miniReference1/miniReference1. There is no need to use this variable, just use the plain path.
I had a similar issue running bowtie 2.5.0 against index files built by bowtie 2.4.5. The error message shows readU: No such file or directory even though the files are there.
I rebuilt the indexes using bowtie 2.5.0 and the issue was resolved. So I'd suggest rebuilding the indexes using the same version as the aligner, and then running it again.
I'm quoting the relevant part of the manual:
-x The basename of the index for the reference genome. The basename is the name of any of the index files up to but not including the final .1.bt2 / .rev.1.bt2 / etc. bowtie2 looks for the specified index first in the current directory, then in the directory specified in the BOWTIE2_INDEXES environment variable. [Emphasis by me.]
After reading this, I assume that what was missing is to tell BowTie2 the directory it is supposed to look in by setting the environment variable BOWTIE2_INDEXES. In bash you would do that by issuing the command
export BOWTIE2_INDEXES=miniReference1
If that doesn't help try to provide the absolute path. It is unclear to me whether Bowtie2 expects a Posix path name like /c/Users/raghdad/Documents/project-x/sequences/miniReference1 or a Windows path name like C:\Users\.... (pwd in your terminal gives you the Posix name for the current directory).
The -x file base name argument, miniReference1, is correct but does not name a directory; instead, it tells BowTie2 the set of indexes you want to work with, which is identified by the common "base name" of the files (the part up to the first dot). The reason for this scheme is probably that you could have different sets of indexes in the same directory, and each set would have a distinct and unique base name. Of course your way of putting each set in its own directory appears to be much cleaner, unless there is a reason to assess several index sets in one go.
I have written a makefile which has pretty complicated dependency, and executes with multiple jobs in parallel (make -j100 for example). I am trying to find a way to print all the current running target names. Any idea? Thanks in advance.
If what you want is a kind of command that you can run from time to time while make is running, and that shows all currently executing recipes, you could slightly modify your recipes such that they first create a temporary file with the name of the target, do whatever they are supposed to do and delete the temporary file. Listing these temporary files anytime will then show you the currently executing recipes.
Example if all targets are located under the directory from which make is called (or sub-directories of it):
TAGSDIR := .tags
MKTAG = mkdir -p "$(TAGSDIR)/$(#D)" && touch "$(TAGSDIR)/$#"
RMTAG = rm -f "$(TAGSDIR)/$#"
<target>: <prerequisites>
#$(MKTAG)
<regular recipe>
#$(RMTAG)
And list all files under .tags to get the names of all currently running recipes. Example with find:
find .tags -type f -printf '%P\n'
You could even encapsulate this in an infinite loop and refresh the list e.g. every second:
while true; do clear; find -type f -printf '%P\n'; sleep 1; done
EDIT
Andreas noticed that this works only if the targets are all located under the directory from which make is called. If a target is ../foobar, for instance, the temporary tag file would be .tags/../foobar, which is not what we want.
Andreas suggests to substitute .. with \.\. and / with \/. We could maybe find a way to do something like this under GNU/Linux and macOS (but not exactly, you cannot have a slash in a file name) but there could still be other issues under Windows (C:, backslashes...).
We could also store the name of the target in a text file and use mktemp or an equivalent to generate the text file with a unique name. But we would then need a way to propagate this unique name from MKTAG to RMTAG. This is doable with a shell variable and a one-line recipe (or the .ONESHELL special target) but not very nice.
As you use GNU make we could also use abspath and create temporary files named $(TAGSDIR)/$(abspath $#) but I do not know what abspath does under Windows with drive letters, nor do I know if you can name a file something\c:\something under Windows...
So, if your targets are not all located under the directory from which make is called, the best is to use another solution.
When i executed this command (go get gopkg.in/goracle.v2) in windows command prompt, got this error.
..\..\go\src\gopkg.in\goracle.v2\conn.go:52:17: could not determine kind of name for C.dpiConn
..\..\go\src\gopkg.in\goracle.v2\conn.go:347:49: could not determine kind of name for C.dpiData
..\..\go\src\gopkg.in\goracle.v2\conn.go:342:20: could not determine kind of name for C.dpiNativeTypeNum
..\..\go\src\gopkg.in\goracle.v2\conn.go:341:21: could not determine kind of name for C.dpiObjectType
..\..\go\src\gopkg.in\goracle.v2\conn.go:343:20: could not determine kind of name for C.dpiOracleTypeNum
..\..\go\src\gopkg.in\goracle.v2\conn.go:747:19: could not determine kind of name for C.dpiShutdownMode
..\..\go\src\gopkg.in\goracle.v2\conn.go:724:18: could not determine kind of name for C.dpiStartupMode
..\..\go\src\gopkg.in\goracle.v2\conn.go:303:15: could not determine kind of name for C.dpiStmt
..\..\go\src\gopkg.in\goracle.v2\conn.go:347:37: could not determine kind of name for C.dpiVar
..\..\go\src\gopkg.in\goracle.v2\conn.go:395:9: could not determine kind of name for C.dpiVersionInfo
..\..\go\src\gopkg.in\goracle.v2\conn.go:351:13: could not determine kind of name for C.int
..\..\go\src\gopkg.in\goracle.v2\conn.go:304:47: could not determine kind of name for C.uint32_t```
The package is archived and it's advised to use the github.com/godror/godror [Reference]
To change everything using sed command, but as you're using Windows please do an equivalent thing:
Commands (using sed)
sed -i -e 's,goracle "gopkg.in/goracle.v2",godror "github.com/godror/godror",g; s,gopkg.in/goracle.v2,github.com/godror/godror,g; s/"goracle"/"godror"/g; s/goracle[.]/godror./g' $(find . -type f -name '*.go')
sed -i -e '/goracle.v2/d' go.mod
But in simpler terms I'd say as gopkg.in/goracle.v2 is now archived, try go get github.com/godror/godror.
I have a folder of data folders with the following structure:
sampleName1-randomNumbers/subfolder1/subfolder2/subfolder3/data1.gz
sampleName1-randomNumbers/subfolder1/subfolder2/subfolder3/data2.gz
sampleName2-randomNumbers/subfolder1/subfolder2/subfolder3/data1.gz
I want to modify all the data.gz within each sample folder by appending the sample name but not the random numbers to get:
sampleName1-randomNumbers/subfolder1/subfolder2/subfolder3/sampleName1_data1.gz
sampleName1-randomNumbers/subfolder1/subfolder2/subfolder3/sampleName1_data2.gz
sampleName2-randomNumbers/subfolder1/subfolder2/subfolder3/sampleName2_data1.gz
It seems like this should be a simple mv for loop but I haven't been able to figure out how to pull part of a folder name using basename.
for i in */Data/Intensities/BaseCalls/*.gz; do mv $i "fastq""/"${i%%-*}"."`basename $i`; done
I couldn't figure out how to make the files stay in their original folder but for my purposes it works to have all the files go to a new folder ("fastq")
I suppose the "sampleName" part doesn't include dashes. In that case, use the standard pattern removal expansion: %%. That is, suppose your full path (relative to directory root) is stored in $path, just do ${path%%-*} to extract the "sampleName" part. Search for %% in the Bash Reference Manual for more details. As a simple example:
> path=sampleName1-randomNumbers/subfolder1/subfolder2/subfolder3/data1.gz
> echo ${path%%-*}
sampleName1
Otherwise, you could also use more advanced substring extraction based on regex. See BashFAQ/100 or Manipulating Strings from the TLDP Advanced Bash Scripting Guide.
Update. Here's the full command to perform the job described, and it is entirely native to the shell:
for file in */Data/Intensities/BaseCalls/*.gz; do
mv "$file" "${file%/*}/${file%%-*}_${file##*/}"
done
I have a directory with files. The archive is very big and has 1.5 million pdf files inside.
the directory is stored on an IBM i server with OS V7R1 and the machine is new and very fast.
The files are named like this :
invoice_[custno]_[year']_[invoice_number].pdf
invoice_081500_2013_7534435564.pdf
No I try to find files with the find command using the Shell.
find . -name 'invoice_2013_*.pdf' -type f | ls -l > log.dat
The command took a long time so I aborted the operation with no result.
If I try it with smaller directories all works fine.
Later I want to have a job that runs everey day and finds the files created the last 24 hours but I it aleays runs so slow I can forget this.
That invocation would never work because ls does not read filenames from stdin.
Possible solutions are:
Use the find utility's built-in list option:
find . -name 'invoice_2013_*.pdf' -type f -ls > log.dat
Use the find utility's -exec option to execute ls -l for each matching file:
find . -name 'invoice_2013_*.pdf' -type f -exec ls {} \; > log.dat
Pipe the filenames to the xargs utility and let it execute ls -l with the filenames as parameters:
find . -name 'invoice_2013_*.pdf' -type f | xargs ls -l > log.dat
A pattern search of 1.5 million files in a single directory is going to be inefficient on any filesystem.
For looking only at a list of new entries in the directory, you might consider journaling the directory. You would specify INHERIT(*NO) to prevent journaling all the files in the directory as well. Then you could simply extract the recent journal entries with DSPJRN to find out what objects had been added.
I don't think I'd put more than maybe 15k files in a single directory. Some QShell utilities run into trouble at around 16k files. But I'm not sure I'd store them in a directory in any case, except maybe for ones over 16MB if that's a significant fraction of the total. I'd possibly look to store them in CLOBs/BLOBs in the database first.
Storing as individual streamfile objects brings ownership/authority problems that need to be addressed. Some profile is getting entries into its owned-objects table, and I'd expect that profile to be getting pretty large. Perhaps getting to one or more limits.
By storing in the database, you drop to a single owned object.
Or perhaps a few similar objects... There might be a purging/archiving process that moves rows off to a secondary or tertiary table. Hard to guess how that might need to be structured, if at all.
Saves could also benefit, especially SAVSECDTA and SAV saves. Security data is greatly reduced. And saving a 4GB table is faster than saving a thousand 4MB objects (or whatever the breakdown might be).
Other than determining how the original setup and implementation would go in your environment, the big tricky part could involve volatility. If these are stable objects with relatively few changes and few deletions, it should be okay. But if BLOBs are often modified, it can bring trouble when the table takes at a significant fraction of DASD capacity. It gets particularly rough when it exceeds the size of DASD free space and a re-org is needed. With low volatility, that's much less of a concern.
Typically what is done in such cases is to create subdirectories -- perhaps by using the first letter of each file.. For example, the file
abcsdsjahdjhfdsfds.xyz would be store in
/something/a/abcsdsjahdjhfdsfds.xyz
that would cut down on the size each subdirectory..