How to remove wildcard dependencies from yarn.lock?
In project we want to upgrade Cypress to v10, but in other PR someone introduced wildcard
cypress#*:
version "9.3.1"
// ------- more stuff
which overwrites everything. Wild (pun intended) enough - even after removing Cypress from project it still exists there. I can't rebuild yarn.lock (remove yarn.lock and install all packages) because it's breaking other parts of repo. How I can just remove that wildcard, to not break anything else?
Related
Hi I am not sure if I am doing things correctly. Whenever I rebase an hit an issue with package.json and the lock file. I fix the package.json, but then delete the lockfile and just run pnpm i again.
But recently I noticed an issue where the lockfile in the master branch and a newly generated lockfile were different.
My team members were like, you should never try to generate a new lockfile. Just keep running pnpm i.
So am I doing this wrong, or do you think we actually have botched locking file in our master branch?
When there are conflicts, you may run pnpm install. pnpm will automatically resolve the conflicts and create a new lockfile that you may commit.
IMO, removing the lockfile and running pnpm install is also fine. However, some teams prefer to update dependencies as rarely as possible. If you remove the lockfile, newer versions of dependencies might be installed.
You may also try the resolution-mode=time-based setting. With this setting dependencies will be rarely updated. Even if you remove the lockfile.
TL;DR: Is there any way I can forcefully prevent go get from altering the go.mod file?
When I do a go get of certain packages, e.g.:
$ go get github.com/AsynkronIT/protoactor-go/protobuf/protoc-gen-gograin
It will printout that it has updated dependencies (which are defined in my go.mod file):
go get: upgraded github.com/AsynkronIT/protoactor-go v0.0.0-20200815184336-b225d28383f2 => v0.0.0-20210405044454-10bc19881ad6
# (...) Note, this happens for other packages, not just `AsynkronIT/protoactor-go`.
This causes the go.mod file to change during a CI build, and affects subsequent build stages where, while building something, it’ll try to use an updated version of the dependency, which might introduce breaking changes, instead of the version defined on the go.mod file initially.
I’ve tried using -mod=readonly or making sure the -u flag is not used but it will still update the go.mod file, e.g.:
$ GOFLAGS=-mod=readonly go get github.com/AsynkronIT/protoactor-go/protobuf/protoc-gen-gograin
go get: upgraded github.com/AsynkronIT/protoactor-go v0.0.0-20200815184336-b225d28383f2 => v0.0.0-20210405044454-10bc19881ad6
# (...)
I've also tried finding similar issues, like this one, or this other one, but haven't yet find an alternative to prevent go get commands from altering the go.mod.
The current workaround I'm using to stop this behavior is to do a git checkout -- go.mod right after certain go get … steps to reset any changes done by go get, and hence, avoiding breaking changes with certain dependencies newer versions.
I'm using go version 1.16.3.
For Go 1.16 and after, you can use go install to install binaries without affecting go.mod
go install github.com/AsynkronIT/protoactor-go/protobuf/protoc-gen-gograin
I've just come across the https://github.com/FriendsOfPHP/security-advisories tool which looks a great way to automatically scan for the vulnerabilities that are in that community-contributed database.
It scans a composer.lock file for packages with vulnerabilities. However, it's made me realise that my understanding of Composer is not what it should be!
I have a project that has a composer.json file that requires a single demo/package. That demo package also has requirements, like demo/dep.
The result of running composer install --no-dev is that I have a composer.lock file which includes:
demo/package version 1.0
demo/dep version 1.2
All good so far, and running symfony security:check /path/to/my/project/composer.lock gives me a green light, no vulnerabilities.
However on close inspection of the files now in my vendor dir, I can see there's a vendor/demo/package/composer.lock file, which contains references to demo/dep at version 1.1 - which has a security vulnerability against it.
As I understand, I have the safer 1.2 version installed - so says my project's composer.lock file, but why is a composer.lock file included with the vendor's package?
Does that mean that the dodgy code is installed somewhere, too? Or can I just simply ignore the composer.lock files if there's a composer.lock file in a dir above it or such? composer show does not list the versions in the nested lock file. Or maybe I should ignore composer.lock files if there's no sibling ./vendor/ dir?
Why not simply inspect your folders to find a vulnerable version? If there was any, you should find a vendor folder within that package, that's where that package could have installed stuff from it's own composer.lock
Usually, only the composer.json of a package is evaluated to install dependencies. If there is a lock file within one package's folder, you should ask the maintainer of that package why this is the case, but for installing dependencies on your system, this does not matter.
Side note: writing "usually" refers to the standard model of installations. I've seen some crude stuff where Composer plugins put other rules in place, but this cannot be said for your project without knowing more about the structure.
I use pnpm to manage npm project, and I modified the content of an installed package by accident, say, I cleared the content of node_modules/jquery/dist/jquery.js.
The problem is no matter how i reinstall jquery(pnpm install jquery), the content of this file is always empty. I even tried to delete jquery from pnpm store ~/.pnpm-store/, but that doesn't work(maybe I deleted wrong package)
Finally, I have to delete all the files in ~/.pnpm-store, to download everything, it fixes my problem, but I want to know if there is any easier way to do it.
{ My answer will cover pnpm v2.16.2 }
Short answer: run pnpm install --force. (pnpm update might work as well)
Long answer. When you just run pnpm install, pnpm compares the wanted shrinkwrap file (project/shrinkwrap.yaml) to the current one (project/node_modules/.shrinkwrap.yaml). They equal in your case, so node_modules is not touched.
When --force is used, packages are reverified and relinked from the store. Reverification means that its integrity is checked. You removed a file from jquery, so verification will fail and the package will be reunpacked to the store and relinked to node_modules.
Alternatively, you could remove your project's node_modules and run pnpm install. That would also check the integrity of jquery before linking it to the store.
That being said, I think pnpm install jquery should also probably verify the integrity of jquery. We'll create an issue for this in the pnpm repo.
And maybe we can add some additional commands for reverifying every package in node_modules and re-unpacking all modified dependencies.
A related command currently available is pnpm store status which prints a list of mutated packages
Yarn creates a yarn.lock file after you perform a yarn install.
Should this be committed to the repository or ignored? What is it for?
Yes, you should check it in, see Migrating from npm
What is it for?
The npm client installs dependencies into the node_modules directory non-deterministically. This means that based on the order dependencies are installed, the structure of a node_modules directory could be different from one person to another. These differences can cause works on my machine bugs that take a long time to hunt down.
Yarn resolves these issues around versioning and non-determinism by using lock files and an install algorithm that is deterministic and reliable. These lock files lock the installed dependencies to a specific version and ensure that every install results in the exact same file structure in node_modules across all machines.
Depends on what your project is:
Is your project an application? Then: Yes
Is your project a library? If so: No
A more elaborate description of this can be found in this GitHub issue where one of the creators of Yarn eg. says:
The package.json describes the intended versions desired by the original author, while yarn.lock describes the last-known-good configuration for a given application.
Only the yarn.lock-file of the top level project will be used. So unless ones project will be used standalone and not be installed into another project, then there's no use in committing any yarn.lock-file – instead it will always be up to the package.json-file to convey what versions of dependencies the project expects then.
I see these are two separate questions in one. Let me answer both.
Should you commit the file into repo?
Yes. As mentioned in ckuijjer's answer it is recommended in Migration Guide to include this file into repo. Read on to understand why you need to do it.
What is yarn.lock?
It is a file that stores the exact dependency versions for your project together with checksums for each package. This is yarn's way to provide consistency for your dependencies.
To understand why this file is needed you first need to understand what was the problem behind original NPM's package.json. When you install the package, NPM will store the range of allowed revisions of a dependency instead of a specific revision (semver). NPM will try to fetch update the dependency latest version of dependency within the specified range (i.e. non-breaking patch updates). There are two problems with this approach.
Dependency authors might release patch version updates while in fact introducing a breaking change that will affect your project.
Two developers running npm install at different times may get the different set of dependencies. Which may cause a bug to be not reproducible on two exactly same environments. This will might cause build stability issues for CI servers for example.
Yarn on the other hand takes the route of maximum predictability. It creates yarn.lock file to save the exact dependency versions. Having that file in place yarn will use versions stored in yarn.lock instead of resolving versions from package.json. This strategy guarantees that none of the issues described above happen.
yarn.lock is similar to npm-shrinkwrap.json that can be created by npm shrinkwrap command. Check this answer explaining the differences between these two files.
You should:
add it to the repository and commit it
use yarn install --frozen-lockfile and NOT yarn install as a default both locally and on CI build servers.
(I opened a ticket on yarn's issue tracker to make a case to make frozen-lockfile default behavior, see #4147).
Beware to NOT set the frozen-lockfile flag in the .yarnrc file as that would prevent you from being able to sync the package.json and yarn.lock file. See the related yarn issue on github
yarn install may mutate your yarn.lock unexpectedly, making yarn claims of repeatable builds null and void. You should only use yarn install to initialize a yarn.lock and to update it.
Also, esp. in larger teams, you may have a lot of noise around changes in the yarn lock only because a developer was setting up their local project.
For further information, read upon my answer about npm's package-lock.json as that applies here as well.
This was also recently made clear in the docs for yarn install:
yarn install
Install all the dependencies listed within package.json
in the local node_modules folder.
The yarn.lock file is utilized as follows:
If yarn.lock is present and is enough to satisfy all the dependencies
listed in package.json, the exact versions recorded in yarn.lock are
installed, and yarn.lock will be unchanged. Yarn will not check for
newer versions.
If yarn.lock is absent, or is not enough to satisfy
all the dependencies listed in package.json (for example, if you
manually add a dependency to package.json), Yarn looks for the newest
versions available that satisfy the constraints in package.json. The
results are written to yarn.lock.
If you want to ensure yarn.lock is not updated, use --frozen-lockfile.
From My experience I would say yes we should commit yarn.lock file. It will ensure that, when other people use your project they will get the same dependencies as your project expected.
From the Doc
When you run either yarn or yarn add , Yarn will generate a yarn.lock file within the root directory of your package. You don’t need to read or understand this file - just check it into source control. When other people start using Yarn instead of npm, the yarn.lock file will ensure that they get precisely the same dependencies as you have.
One argue could be, that we can achieve it by replacing ^ with --. Yes we can, but in general, we have seen that majority of npm packages comes with ^ notation, and we have to change notation manually for ensuring static dependency version.But if you use yarn.lock it will programatically ensure your correct version.
Also as Eric Elliott said here
Don’t .gitignore yarn.lock. It is there to ensure deterministic dependency resolution to avoid “works on my machine” bugs.
Not to play the devil's advocate, but I have slowly (over the years) come around to the idea that you should NOT commit the lock files.
I know every bit of documentation they have says that you should. But what good can it possibly do?! And the downsides far outweigh the benefits, in my opinion.
Basically, I have spent countless hours debugging issues that have eventually been solved by deleting lock files. For example, the lock files can contain information about which package registry to use, and in an enterprise environment where different users access different registries, it's a recipe for disaster.
Additionally, the lock files can really mess up your dependency tree. Because yarn and npm create a complex tree and keep external modules of different versions in different places (e.g. in the node_modules folder within a module in the top node_modules folder of your app), if you update dependencies frequently, it can create a real mess. Again, I have spent tons of time trying to figure out what an old version of a module was still being used in a dependency wherein the module version had been updated, only to find that deleting the lock file and the node_modules folder solved all the hard-to-diagnose problems.
I even have shell aliases now that delete the lock files (and sometimes node_modules folders as well!) before running yarn or npm.
Just the other side of the coin, I guess, but blindly following this dogma can cost you........
I'd guess yes, since Yarn versions its own yarn.lock file:
https://github.com/yarnpkg/yarn
It's used for deterministic package dependency resolution.
Yes! yarn.lock must be checked in so any developer who installs the dependencies get the exact same output! With npm [that was available in Oct 2016], for instance, you can have a patch version (say 1.2.0) installed locally while a new developer running a fresh install might get a different version (1.2.1).
Yes, You should commit it. For more about yarn.lock file, refer the official docs here