rebuild directory path from tree output - data-structures

I am attempting to take the output of tree (which was saved into a file and passed to me [I do not have access to the original source to use something else for output]) and reconstruct a full directory path for each filename in the document.
Exert from dataset
¦   +-- User
¦   ¦   +-- File Backups
¦   ¦   +-- Pending
¦   ¦   +-- 2019
¦   ¦   +-- Admin.bak
¦   +-- Archived
¦   ¦   +-- Complete
¦   ¦   ¦   +-- 0903
¦   ¦   ¦   ¦   +-- 24433
¦   ¦   ¦   ¦   ¦   +-- Backups
¦   ¦   ¦   ¦   ¦   +-- Pending
¦   ¦   ¦   ¦   ¦   ¦   +-- 20190306
¦   ¦   ¦   ¦   ¦   ¦   +-- 6100000.bak
¦   ¦   ¦   ¦   ¦   +-- Review
¦   ¦   ¦   ¦   ¦   +-- 15446
¦   ¦   ¦   ¦   ¦   +-- 2100000.bak
¦   ¦   ¦   ¦   +-- 3546.zip
¦   ¦   ¦   +-- SQL Backups
¦   ¦   ¦   +-- 02.11.2021
¦   ¦   ¦   ¦   +-- 7670000.bak
¦   ¦   ¦   ¦   +-- 6430000.bak
¦   ¦   ¦   +-- 4370000.bak
¦   ¦   +-- SIM Backups
¦   ¦   +-- DIFFS SIM02
¦   ¦   +-- Control
¦   ¦   ¦   +-- 6887312.bak
¦   ¦   ¦   +-- 7912007.bak
¦   ¦   ¦   +-- 5496427.bak
¦   ¦   ¦   +-- 2574206.bak
Any way to recreate this to a workable data object would be welcome. Overall it is about 2 million lines and excel failed me in being able to import and do text-to-column based on +-- delimiter

Related

Change file structure using Visual Studio to the common one used by GitHub projects (both solution and project in src folder)

A common structure seen on most projects on GitHub is the following. A good example is Json.Net repository. I want to achieve the same thing using Visual Studio 2019:
MyProject
├── Doc
│   ├── txt/md/... files
├── Src
│   ├── .sln
│   ├── Project1 (eg. netstandard lib)
│ └── Project2 (eg. tests)
├── .gitignore
├── LICENSE.md
└── Readme.md
Similar questions on SO (1, 2) move the "projects" into a folder not the "solution". Note that in this structure, the solution is also inside the "Src" folder.
I've adapted some of the suggestions such as switching to folder view and that way I successfully move the entire .sln and project folders into 'src' folder but the problem is that Visual Studio creates a new solution at root. And the final structure becomes this:
MyProject
├── Doc
│   ├── txt/md/... files
├── Src
│   ├── .sln
│   ├── Project1 (eg. netstandard lib)
│ └── Project2 (eg. tests)
├── .gitignore
├── LICENSE.md
├── Readme.md
└── Extra_solution.sln <-- this is the problem
The red circled solution in below screenshot was created as soon as I opened the moved solution:
So the structure you want is common, as outlined by David Fowler here: https://gist.github.com/davidfowl/ed7564297c61fe9ab814. Note that the .sln file is at the root, and I find this to be far more common.
What I do when creating it in Visual Studio is unload the project (via a right-click on the project), then move it, then re-add the project to the solution.
There's another way, and that is to do everything via the command line:
mkdir MyProject
cd MyProject
dotnet new sln
dotnet new classlib -o src/Project1
dotnet new mstest -o src/Project2
dotnet sln add src/Project1
dotnet sln add src/Project2

copy resources with output path containing part of input path

Say two modules mod1 and mod2 have the following structure:
root
├── mod1/src/main/resources/db-migrations
| ├── v1
| | ├── a.sql
| | └── b.sql
| └── v2
| ├── c.sql
| └── d.sql
|
└── mod2/src/main/resources/db-migrations
├── v1
| ├── e.sql
| └── f.sql
└── v2
├── g.sql
└── h.sql
I want to copy all files from db-migrations into a single top-level directory, but grouped by version first, and by module second. So the output should look like this:
root/all-db-migrations
├── v1
| ├── mod1
| | ├── a.sql
| | └── b.sql
| └── mod2
| ├── e.sql
| └── f.sql
└── v2
├── mod1
| ├── c.sql
| └── d.sql
└── mod2
├── g.sql
└── h.sql
If the directory structure wasn't inversed (module name before version), this is easy with the maven resource plugin by just copying the entire db-migrations directory for each module:
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<executions>
<execution>
<id>copy-database-migrations</id>
<phase>process-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>../all-db-migrations/${project.artifactId}
</outputDirectory>
<resources>
<resource>
<directory>src/main/resources/db-migrations</directory>
<filtering>false</filtering>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
However, I could not find a solution to do such a copy operation as described above.

Golang project directory structure

I have some confusion regarding Golang directory structure.
Base on the book The Way to Go by Ivo Balbaert, project code should be placed into src, and recommends the following directory structure.
├──src/
| ├──main.go
| ├──say/
| | ├──say.go
| | ├──say_test.go
├──bin/
| ├──say
└──pkg/
└──linux_amd64/
└──say.a
but I found that many packages in github.com, have no src directory.
For example:
https://github.com/facebookgo/grace
https://github.com/astaxie/beego
So, I don't know whether src directory is needed.
I have some project, their have inter-dependency.
They are managed in a private GitLab repository.
How can I organized them?
This article by Ben Johnson has guided me on this when I was starting with Go.
It's generally good to start with something like this (assuming you are inside your project directory like $GOPATH/src/myproject:
├──cmd/ -- this is where you compose several packages in to main package
| ├──foo -- an example would be `foo`
| | ├──main.go
├──pkg/ -- this is where put your reusable packages
| ├──pkg1 -- reusable package 1
| ├──pkg2 -- reusable package 2
├──otherpackage1
| ├── ...
├──otherpackage2
| ├── ...
You can have a look at this example from go-kit for this kind of project structure.
Sometimes it will depend on your needs. On our workflow, we are using a hot code reload tool called fresh, so we needed to put the main.go on the project root so that the tool can detect all the file changes and rebuild the source code.
├──app/
| ├──app.go
├──model/ --
| ├──model.go
├──store
| ├──store.go
├──main.go -- this is where the app starts
├──...
On the app.go package, I have something like func Run() error which starts the application. On the main.go, I am just calling the function:
...
func main(){
log.Fatal(app.Run())
}
Now there are new ways to organize Go projects.
The GitHub golang-standards repository says:
This is a basic layout for Go application projects. It represents the
most common directory structure with a number of small enhancements
along with several supporting directories common to any real world
application.
This project layout is intentionally generic and it doesn't try to
impose a specific Go package structure.
Or you can follow this slides:
$GOPATH/
src/
github.com/user/repo/
mypkg/
mysrc1.go
mysrc2.go
cmd/mycmd/
main.go
bin/
mycmd
Here is another project layout sample Simple Go project layout with modules
├── LICENSE
├── README.md
├── config.go
├── go.mod
├── go.sum
├── clientlib
│ ├── lib.go
│ └── lib_test.go
├── cmd
│ ├── modlib-client
│ │ └── main.go
│ └── modlib-server
│ └── main.go
├── internal
│ └── auth
│ ├── auth.go
│ └── auth_test.go
└── serverlib
└── lib.go
Some answer points out the go standard layout, however, there is one issue this is not a standard Go project layout by Russ Cox
There are two problems with this GitHub repo:
it claims to host Go standards and does not, in the sense that these are in no way official standards
the project-layout standard it puts forth is far too complex and not a standard
Regarding "why not tell us the standard Go project layout and we'll update the doc?", that only addresses point 2. If there really were standards, they would be in the main Go project doc tree. The standard for project layout would also be a lot shorter. I appreciate your trying to provide a useful resource, but calling it 'golang-standards' is claiming more than it is.
But for the record, the minimal standard layout for an importable Go repo is really:
Put a LICENSE file in your root
Put a go.mod file in your root
Put Go code in your repo, in the root or organized into a directory tree as you see fit
That's it. That's the "standard".
Update at 11/30/2021
Here are a summary from How to structure Go code
Before we start
doc.go file puts the general description of the package
Readme file a general overview of this project
When you have more documentation to provide put them into the docs folder
For linting use golangci-lint. Enable all linters that seem to be reasonable for your project
The flat structure (single package)
courses/
main.go
server.go
user_profile.go
lesson.go
course.go
When to create a new package?
When you have more than one way of starting your application
When you want to extract more detailed implementation
When you started to add a common prefix to closely related things
Modularisation
Organising by kind
.
├── handlers
│ ├── course.go
│ ├── lecture.go
│ ├── profile.go
│ └── user.go
├── main.go
├── models
│ ├── course.go
│ ├── lecture.go
│ └── user.go
├── repositories
│ ├── course.go
│ ├── lecture.go
│ └── user.go
├── services
│ ├── course.go
│ └── user.go
└── utils
└── stings.go
Organising by components
.
├── course
│ ├── httphandler.go
│ ├── model.go
│ ├── repository.go
│ └── service.go
├── main.go
└── profile
├── httphandler.go
├── model.go
├── repository.go
└── service.go
Clean Architecture
You have 4 layers of your application or module (depending on how big your codebase is): Domain, Application, Ports, Adapters. In some sources, names may differ.
The src directory is not needed and in fact a lot of public repositories do not use this structure.
There is a few different way of organizing your project. If you plan on having your project used by an other repository, like lib. I would recommend using a cmd struct something like this. This would also be the recommended way of doing it if there would be more then one way of starting the application. (multipliable main.go files)
├──cmd/
| ├──(application name)
| | ├──main.go
└──say/
├──say.go
└──say_test.go
Otherwise for example if it is a standalone application. You can place the main.go in the root of the repository.
bin and pkg you can keep in the root and add this to .gitignore. (assuming you are using git)
The book describes the directory structure after checkout. It would have been helpful if the book included the .git directory.
$GOPATH/src is required for imports to work.
├──src/
| ├──.git
| | ├──...
| ├──main.go
| ├──say/
| | ├──say.go
| | ├──say_test.go
├──bin/
| ├──say
└──pkg/
└──linux_amd64/
└──say.a
In practice, main.go would actually be in a path that reflects the remote git repository, for instance
.
├── bin
│   └── say
├── pkg
│   └── linux_amd64
│   └── github.com
│      └── pschultz
│      └── hello-world
│   └── say.a
└── src
   └── github.com
   └── pschultz
   └── hello-world
   ├── .git
   │ └── ...
   ├── main.go
   └── say
   ├── say.go
   └── say_test.go

How can I change the characters used to display trees under npm ls?

I'm trying to diff the package versions that a coworker is using. When running npm ls, mine look like this:
+-- grunt-contrib-concat#1.0.1
| +-- chalk#1.1.3
| | +-- ansi-styles#2.2.1
| | +-- escape-string-regexp#1.0.5
| | +-- has-ansi#2.0.0
| | | `-- ansi-regex#2.0.0
but his look like this:
├─┬ grunt-contrib-concat#1.0.1
│ ├─┬ chalk#1.1.3
│ │ ├── ansi-styles#2.2.1
│ │ ├── escape-string-regexp#1.0.5
│ │ ├─┬ has-ansi#2.0.0
│ │ │ └── ansi-regex#2.0.0
We're both using Windows. I tried to explicitly define my encoding as UTF-8 but nothing changed.
Is this configurable somewhere?
I discovered that my co-worker is using the --unicode flag.
When set to true, npm uses unicode characters in the tree output. When false, it uses ascii characters to draw trees.
He must have also configured it in his .npmrc file.

What is the logic behind the changes in directory structure of Symfony2 --> Symfony3?

I'm still learning my way around Symfony2 and wondered if anyone could explain the logic behind the changes to the directory structure in Symfony3? It would be interesting to hear the justification for the changes and would hopefully give some clarify on the 'magic' of inheritance/overwriting classes.
Below are the standard directory structures for both versions of Symfony:
Symfony 2:
blog/
├─ app/
│ ├─ console
│ ├─ cache/
│ ├─ config/
│ ├─ logs/
│ └─ Resources/
├─ src/
│ └─ AppBundle/
├─ vendor/
└─ web/
Symfony 3:
blog-symfony3/
├─ app/
│ ├─ config/
│ └─ Resources/
├─ bin/
│ └─ console
├─ src/
├─ var/
│ ├─ cache/
│ └─ logs/
├─ vendor/
└─ web/
As commented, this has already been outlined in my answer here.
A google search for symfony 3 directory structure also returns it as the first result (at the time of writing).
Here is a quick summary:
The cache, logs and bootstrap.php.cache have been moved to the
/var directory in the root of the project. This makes it easier to
handle permissions, you should make the entire var directory
writable. (similar to a unix like OS). You could also write your own application specific files here.
All binary executables (console, symfony_requirements, doctrine) are all now located within the bin directory (similar to a unix like OS), you can also add your own executables here.
Tip: Update your PATH environment variable with the following to automatically include all files in the bin directory (in the current working directory) in your path. PATH="./bin:$PATH"
phpunit can now be run from the project root without specifying the configuration file via the -c switch.
<3.0 - phpunit -c app/phpunit.xml
>3.0 - phpunit
You can read a more detailed answer here:
What is the new Symfony 3 directory structure?

Resources