I want to embed a file placed one level above the golang file code.
for example:
dir1
file.go
dir2
file.txt
How to embed file.txt inside file.go using go:embed?
The documentation states:
Patterns may not contain ‘.’ or ‘..’ or empty path elements, nor may they begin or end with a slash.
So what you are trying to do is not supported directly. Further information is available in the comments on this issue.
One thing you can do is to put a go file in dir2, embed file.txt in that and then import/use that in dir1/file.go (assuming the folders are in the same package).
This is not supported in the embed package as stated by #Brits (https://pkg.go.dev/embed)
A pattern I like to use is to create an resources.go file in my project's internal package and put all my embedded resources in there eg:
├── cmd\
│ └── cool.go
└── internal\
└── resources\
├── resources.go
├── fonts\
│ └── coolfont.ttf
└── icons\
└── coolicon.ico
resources.go
import _ "embed"
//go:embed fonts/coolfont.fs
var fonts byte[] // embed single file
//go:embed icons/*
var icons embed.FS // embed whole directory
There are libraries that can help with this as well such as those listed here https://github.com/avelino/awesome-go#resource-embedding
But I've not run into a use case where plain old embed wasn't enough for my needs.
Related
When I use golang's embed package, I encounter confusion about path matching. that's what * refers to?
Officially called non-Separator characters
Here: https://pkg.go.dev/path/filepath#Match
'*' matches any sequence of non-Separator characters
At first I understood non-Separator characters meant forward slashes(leading slash /), but the following example shows that this is not the case.
# dir tree
├── stubs
│ ├── hello.txt
│ └── xyz
│ └── zyx.txt
└── hello.go
# source file for hello.go
package main
import "embed"
//go:embed stubs/*
var T1 embed.FS
//go:embed stubs*
var T2 embed.FS
func main() {
s, e := T1.ReadFile("stubs/hello.txt")
fmt.Println(string(s))
fmt.Println(e)
s, e = T2.ReadFile("stubs/xyz/zyx.txt")
fmt.Println(string(s))
fmt.Println(e)
}
Then run go run hello.go will execute successfully.
This case tell me * can match /.
what does non-Separator characters mean?
Thanks for the explanation.
T2.ReadFile("stubs/xyz/zyx.txt") works for a different reason than you think.
The documentation for go:embed explains:
If a pattern names a directory, all files in the subtree rooted at
that directory are embedded (recursively), except that files with
names beginning with ‘.’ or ‘_’ are excluded.
https://pkg.go.dev/embed#hdr-Directives
You have declared T2 as:
//go:embed stubs*
var T2 embed.FS
The pattern stubs* matches the directory stubs, so the whole directory stubs, including all files in it (recursively) are included in the T2 filesystem.
I have multiple directories that in turn contain subdirectories. Example:
company_a/raw/2020/12
The value of the first directory (company_a in the sample above) is variable, but always with a pattern "word_letter"
The value of the second directory raw is immutable
The values of the last two directories (/2020/12 in the sample above) are variable.
My purpose is to extract the size of each leaf subdirectory (given the sample path above, the leaf subdir would be 12/) using a for loop.
Is there some kind of reverse basename utility which would allow me to list the entire path, using company_x/ dir as the root dir? Because if I want to extract directories' size, first I need to figure out how to list the last directories in the path.
A sample tree for reference:
$ tree company_b
tree company_b
└── raw
└── 2020
├── 05
│ └── data.raw
├── 06
│ └── data.raw
├── 07
│ └── data.raw
└── 08
└── data.raw
6 directories, 4 files
The du command does this very well using wildcards.
du -h */raw/*/*
Output:
80K company_b/raw/2021/02
80K company_b/raw/2021/05
80K company_b/raw/2021/04
80K company_b/raw/2021/01
80K company_b/raw/2021/03
I have two directories - A and B - that contain a bunch of photo files. Directory A is where I keep photos long-term, and the files inside are sequentially named "Photo-1.jpg, Photo-2.jpg, etc.".
Directory B is where I upload new photos to from my camera, and the naming convention is whatever the camera names the file. I figured out how to run some operations on Directory B to ensure everything is in .jpg format as needed (imagemagik convert), remove duplicate files (fdupes), etc.
My goal now is to move the files from B to A, and end up with the newly-added files in A sequentially named according to A's naming convention described above.
I know how to move the files into A, and then to batch rename everything in A after the new files have been added (which would theoretically occur every night), but I'm guessing there must be a more efficient way of moving the files from B to A without re-naming all 20,000+ photos every night, just because a few new files were added.
I guess my question is two parts - 1) I found a solution that works (us mv to rename all photos every night), is there any downside to this? and 2) If there is a downside and a more elegant method exists, can anyone help with a script that would look at whatever the highest number that exists in A, then re-name the files, appending onto that number, in B as they are moved over to A?
Thank you!
This bash script will only move and rename the new files from DiretoryB into your DirectoryA path. It also handles file names with spaces and/or any other odd characters in their name in DirectoryB
#!/bin/bash
aPath="./photos-A"
bPath="./photos-B"
aPattern="Photo-"
lNum=$(find $aPath -type f -name "*.jpg" -printf "%f\n" | \
awk -F'[-.]' '{if($2>m)m=$2}END{print m}')
while IFS= read -r -d $'\0' photo; do
mv "$photo" "$aPath/$aPattern$((++lNum)).jpg"
done < <(find $bPath -type f -name "*.jpg" -print0)
Note
The command to find the last numbered photo, aka $lNum will run over all 20K+ files, but it should be fairly quick. If it's not, you can always run this once and store the latest number into a file and read from that file.
Proof of Concept
$ tree photos-A/
photos-A/
├── Photo-1.jpg
├── Photo-2.jpg
├── Photo-3.jpg
├── Photo-5.jpg
├── Photo-6.jpg
├── Photo-7.jpg
└── Photo-8.jpg
0 directories, 7 files
$ tree photos-B/
photos-B/
├── bar.jpg
├── baz\ with\ spaces.jpg
└── foo.jpg
0 directories, 3 files
$ ./mvphoto.sh
$ tree photos-A/
photos-A/
├── Photo-10.jpg
├── Photo-11.jpg
├── Photo-1.jpg
├── Photo-2.jpg
├── Photo-3.jpg
├── Photo-5.jpg
├── Photo-6.jpg
├── Photo-7.jpg
├── Photo-8.jpg
└── Photo-9.jpg
0 directories, 10 files
Given this directory structure:
├── script
│ ├── search.rb
│ └── searchable.txt
└── unsearchable.txt
You can only search file under the script (e.g. searchable.txt). But how do I read unsearchable.txt in Ruby?
(I got this error No such file or directory # rb_sysopen - <filename>.txt)
Just one level up to your current file.
file = File.new('../unsearchable.txt')
Or
file = File.join(File.dirname(__FILE__), '..', 'unsearchable.txt')
__FILE__ is your current file name. .. is the parent directory.
Bash Script:
Each file inside table directories will need be renamed from keyspace to newkyespace_456 when it is copied to destination.
└── Main_folder
├── keyspace
│ ├── tableA-12323/keyspace-tableA-12323-ka-1-Data.db
│ ├── tableB-123425/keyspace-tableA-123425-ka-1-Data.db
│ └── tableC-12342/keyspace-tableA-12342-ka-1-Data.db
└── newkeyspace_456 ( given folder) and sub folders
├── tableA-12523
├── tableB-173425
└── tableC-1242
Example is
keyspace/tableA-12323/keyspace-tableA-12323-ka-1-Data.db
to
newkeyspace_456/tableA-12523/newkeyspace_456-tableA-12523-ka-1-Data.db
Note that same table (Type A , B , C) type can be copied to same table type in other keyspaces (Type A , B , C) . The table name also need changes in file name , please note in example 12323 has been renamed to 12523 when copied to diretory newkeyspace_456/tableA-12523.
Type A table files can be copied from keyspace/tableA-12323 to Type A table files in newkeyspace_456/tableA-12523.
How do I approach this problem?
Thanks
tom
Use parameter expansion with string substitution for changing filename, like this:
for fn in $(find ./keyspace -path '*.db') ; do cp "$fn" "${fn//keyspace/newkeyspace_456}" ; done ;