I have many branches in git with different set of packages in composer.json
After each git checkout I need to do composer install and composer starts to download missing packages. In that moment, composer removes packages that are needed for other branch. And when I will checkout to other branch, I will need to download that packages again. When it comes to packages such as PHPUnit, Codeception or other frameworks, it takes a very long time.
Is it possible to disable the removal of unused packages in composer?
(I have met this feature in bower or npm.)
Thank you.
Right now this is not supported, as install just performs the actions needed to comply to the project requirements. As technically in your case the requirements change, its behavior is correct. While the feature could be implemented in Composer it's not trivial, as it's 'unnatural' behavior that is quite low-level to hack.
However I think the real issue here is that your workflow is not correct. If different branches in Git have wildly different dependencies it is first of all doubtful that they should really be branches, and not entirely different repositories as they're really different projects then.
If that is not the case the easiest solution is just to clone the repository multiple times, and keep the different clones at their respective branches. That solves all your problems immediately and lets Composer do its work like it was intended. This is also a very common workflow in bigger projects, as in-place branch switching is really only practical for short-lived branches like PR's and feature branches.
Related
I am using go modules on go1.12 to handle my Go dependencies. Is it best practice to also commit the vendor/ directory into version control?
This is somewhat related to Is it best-practice to commit the `vendor` directory? which asks this question in the case of using dep. With dep, commiting vendor/ is the only way to get truly reproducible builds. What about for go modules?
I'd like to give some arguments in favour of committing vendor, go.mod and go.sum.
I agree with the accepted answer's arguments that it's technically unnecessary and bloats the repo.
But here is a list of contra-arguments:
Building the project doesn't depend on some code being available on Github/Gitlab/... or the Go proxy servers. Open source projects may disappear because of censorship, authors incentives, licensing changes or some other reasons I can't currently think of, which did happen on npm, the JavaScript package manager, and broke many projects. Not in your repo, not your code.
We may have used internal or 3rd party Go modules (private) which may also disappear or become inaccessible, but if they are committed in vendor, they are part of our project. Nothing breaks unexpectedly.
Private Go modules may not follow semantic versioning, which means the Go tools will rely on the latest commit hash when fetching them on-the-fly. Repo history may be rewritten (e.g. rebase) and you, a colleague or your CI job may end up with different code for the dependencies they use.
Committing vendor can improve your code review process. Typically we always commit dependency changes in a separate commit, so they can be easily viewed if you're curious.
Here's an interesting observation related to bloating the repo. If I make code review and a team member has included a new dependency with 300 files (or updated one with 300 files changed), I would be pretty curious to deep dive into that and start a discussion about code quality, the need for this change or alternative Go modules. This may lead to actually decrease your binary size and overall complexity.
If I just see a single new line in go.mod in a new Merge Request, chances are I won't even think about it.
CI/CD jobs which perform compilation and build steps need not waste time and network to download the dependencies every time the CI job is executed. All needed dependencies are local and present (go build -mod vendor)
These are on top of my head, if I remember something else, I'll add it here.
Unless you need to modify the vendored packages, you shouldn't. Go modules already gives you reproducible builds as the go.mod file records the exact versions and commit hashes of your dependencies, which the go tool will respect and follow.
The vendor directory can be recreated by running the go mod vendor command, and it's even ignored by default by go build unless you ask it to use it with the -mod=vendor flag.
Read more details:
Go wiki: How do I use vendoring with modules? Is vendoring going away?
Command go: Modules and vendoring
Command go: Make vendored copies of dependencies
Currently, we have all vendored libraries in src/vendor which makes docker-compose build quite fast. Although adding vendored libraries to source control has the disavantage of libraries not being updated and also heavily polluting the diff of pull requests.
Is there a way in between, maybe with caching?
Is there a way in between, maybe with caching?
Yes, several. But don't fight the system/preferred method.
Use $GOPATH/src/MyProject/vendor like you are already doing.
adding vendored libraries to source control has the disavantage of libraries not being updated...
That all depends on your team's management of your repo. If everyone ignores the vendor, ya it will get stale.
Personally I make it a "1st of the month" habit of going through and refreshing all dependencies, running our test suites, and if no errors update for QA integration testing on the dev server and keep an eye on the error logs after release. Tools like godep and gostatus greatly help keep your GOPATH in chrcn with latest, that you can update your vendor folder(s) with quickly.
Just make sure it is a dedicated commit, so it can be reverted in a hurry if an issue creeps up.
also heavily polluting the diff of pull requests
First of all, that's just a process task. I enforce rebasing on all pull requests and reject all merges in all repos. This keeps a very clean git history; but, more to the point, rebasing moves your local commits until after the vendor updates. Shouldn't ever get a conflict unless someone added the same package. Which at that point is easy, just take the latest one and be done.
Sound like there are process issues to work out than worrying about /vendor management.
I am currently using atlassian bamboo build server (cloud based, using aws) and have an initial task that simply does a composer install.
this single task can take quite a bit of time which can be a pain when developers have committed multiple times giving the build server 4 builds all downloading dependencies (these are not parallel).
I wish to speed this process up but canot figure out a way in which to save the dependancies to a common location for use across multiple builds which still allowing the application to run as intended (laravel)
Answer
Remove composer.lock from your .gitignore
Explanation
When you run compose install for the first time, composer has to check all of your dependencies (and their dependencies etc.) or compatibility. Running through the whole dependency tree is quite expensive, which is why it takes so long.
After figuring out all of your dependencies, composer then writes the exact versions it uses into the composer.lock file so that subsequent composer install commands will not have to spend that much time running through the whole graph.
If you commit your composer.lock file it'll come along to your bamboo server. The composer install command will be waaaayy faster.
Committing composer.lock is a best practice regardless. To quote the docs:
Commit your application's composer.lock (along with composer.json) into version control.
This is important because the install command checks if a lock file is present, and if it is, it downloads the versions specified there (regardless of what composer.json says).
This means that anyone who sets up the project will download the exact same version of the dependencies. Your CI server, production machines, other developers in your team, everything and everyone runs on the same dependencies, which mitigates the potential for bugs affecting only some parts of the deployments.
I've started working on a new Perl module and I've decided that I want to make a Ruby version of it as well (once I finish the Perl version). Do people tend to make separate repositories for each language? Or put them in the same repository?
I can easily see how the two sets of code are different enough to be treated as separate projects. But at the same time it's the same functionality written in two languages, so from that perspective it seems like a single project with two language ports.
What's considered best practice in this situation?
FWIW, I'm using git.
EDIT: I should be more clear here. These aren't modules in the sense of git submodules. They're modules that will be submitted to CPAN and RubyGems. Users of this project will likely be installing it via cpan or gem and then using/requiring it in the normal fashion.
In the course of my group's research, we have a couple repos, some with different technologies in each. We divide the repos by research question and checkout only the projects we are working on, with the repos having a uniform hierarchal directory structure that is the same for all projects. Since we already know the repo directory structural, running scripts and finding data becomes much easier.
I would recommend taking the same approach. The higher the division between the two technologies, the easier it will be to contribute to one of them without being confused by the presence of the other.
In the end ask yourself this: If I were to add another language, would I still keep it in one repo? If the answer is yes, keep doing what you're doing. If not, keep these libraries in two separate repos and manage the projects and contributers distinctly.
my experience in this kind of case is to have 2 smaller git repos for each of the modules and either cloning one branch into the consumer projects repo makes it quite simple. another way is to create a naked clone from the module's repo into the consumer projects repo, then just keep updating it as each module's development progresses. the consumer project should ignore the injected repos.
once other dev clones module A, and/or B, then he/she can just push to consumer project, as permissions allow. this is either a pro or a con depends.
If I have a C# solution with multiple projects in it, what would be better, to have the Git repo created in the solution folder, or in each individual project folder? Multiple developers will be working on the projects. What are your experiences with this?
I use several (sometimes overlapping) solutions to contain a collection of related independent applications and shared libraries. As others have mentioned, you really don't want to have a single Git repository containing the source for multiple, independent projects as it makes it much too difficult to track isolated changes.
So, if your solution is structured as mine is then you will definitely want individual Git repositories for each project. This has worked well for me for ten to twelve applications and doesn't create as much maintenance overhead as you might think.
If your solution is truly monolithic (and you're sure you want it that way forever and ever), then it probably makes sense to only have a single repository.
It depends. git repositories are most suited to containing a single configuration item with its own independent lifecycle. If your projects have their own release cycle and are shared between multiple solutions then it might make sense to have them in their own repositories. Usually, though, it is the solution that represents a configuration item with all the constituent projects forming part of the same build. In this case a single git repository at the solution level makes more sense.
git submodule is probably worth consideration here. Each project gets its own repo, the solution gets a repo, and the projects are submodules.
I assume that your solution represents some kind of a product while the projects are just a part of the product.
In this situation I would create the repository on the solution level. This way it is a lot easier to build the whole product at once, especially if the projects depend on each other.
Some though and 3 solutions on the subject can be read on that blog:
https://www.atlassian.com/blog/git/git-and-project-dependencies
package management tool, i.e. nuget in VS, so using reference to a package/compiled module
git submodule (only with command line in VS?)
other build and cross-stack dependency tools
Another solution is just to add a project from the other repo and let it out of the current repo, and latter use the Team Explorer to commit its changes.