Using yarn 2's new plug n play (pnp) creates a long .pnp.js file. I get a bunch of merge conflicts while pulling, and these are not autofixed (unlike yarn.lock).
How are these conflicts solved? I'd rather not go them through manually as it's not clear what change to accept.
Example conflict
["virtual:844e49f9c8ad85b5809b347eb507fe8bfdc2d527102f53e0b4f78076a2ad5ea2556763170701137a2cafdc51d5a36d82e448010e65742a300748e0bc70028101#npm:11.2.7", {
"packageLocation": "./.yarn/$$virtual/#testing-library-react-virtual-2e67fd5293/0/cache/#testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip/node_modules/#testing-library/react/",
"packageDependencies": [
["#testing-library/react", "virtual:844e49f9c8ad85b5809b347eb507fe8bfdc2d527102f53e0b4f78076a2ad5ea2556763170701137a2cafdc51d5a36d82e448010e65742a300748e0bc70028101#npm:11.2.7"],
["#babel/runtime", "npm:7.13.10"],
<<<<<<< HEAD
["#testing-library/dom", "npm:7.30.4"],
["#types/react", "npm:17.0.3"],
["#types/react-dom", "npm:17.0.3"],
=======
["#testing-library/dom", "npm:7.31.0"],
["#types/react", "npm:17.0.8"],
["#types/react-dom", "npm:17.0.5"],
>>>>>>> d2bb5d9e537f9647e9757656de230e56282e0b15
["react", "npm:17.0.2"],
I would assume you can delete this file containing merge conflicts.
Next, you run yarn install which will generate this file again.
Or just run yarn install which will overwrite the the .pnp.cjs file and fix the merge conflicts (if any) in the yarn.lock file for you.
From the docs:
The generated .pnp.cjs file can be committed to your repository
as part of the Zero-Installs effort, removing the need to run yarn install in the first place.
As you can read, this file can - not must - be committed. However, if you commit it, you can use all your dependencies immediately after cloning the repo, switching branches, ... without need to run yarn install every time.
Note that the same does not count for yarn.lock file which you should never delete.
Related
I'm trying to optimize our poetry build times by storing a tar.gz of the venv on the end of the build in an azure storage blob (an S3) with key generated by doing a md5sum of the poetry.lock and the pyproject.toml. (this works fine)
So, in the beginning of another build, I would hash those files and try to find if that blob exists in storage. If yes, download it and extract its contents to the .venv/ dir of the project.
This worked fine at the beginning but after a while I started getting builds that were throwing this error when trying to run a command through poetry: (any command)
+ poetry run reorder-python-imports --diff-only <SOME_FILES>
FileNotFoundError
[Errno 2] No such file or directory
at /usr/local/lib/python3.8/os.py:591 in _execvpe
587│ argrest = (args,)
588│ env = environ
589│
590│ if path.dirname(file):
→ 591│ exec_func(file, *argrest)
592│ return
593│ saved_exc = None
594│ path_list = get_exec_path(env)
595│ if name != 'nt':
I have confirmed that there is no .venv folder already in the project dir and that the contents are exactly the same as if it were going to install it from the network.
If I just run poetry install on top of the cached venv, it says it has no more dependencies to install, but it throws the error above.
If I delete the venv and install everything again, commands work fine.
I have no more ideas on how to debug and solve this issue. Help would be very appreciated! :)
Ok, I think I've understood that virtualenvs have some hardcoded paths that are not easy to move around.
For what it says in this post Can I move a virtualenv? we should not move venvs as a good practice.
And there it goes my way of speeding up builds...
Using CircleCI - version: 2.1 - for continuous deployment where caching installed dependencies. Based on save_cache documentation:
Generates and stores a cache of a file or directory of files such as dependencies or source code in our object storage. Later jobs can restore this cache.
Current scenario:
See the simplified caching step below in .circleci/config.yml file:
steps:
- node/with-cache:
steps:
- checkout
- run: npm install
- save_cache:
key: dependencies
paths: node_modules
The problem is coming once adding new package to the project thus package.json file is changing. In the same time CircleCI shows the message for Saving Cache step:
Skipping cache generation, cache already exists for key: dependenciesFound one created at 2020-05-23 19:29:29 +0000 UTC
Then once restoring the cache obviously does not find the newly added package in the build step:
./src/index.tsxCannot find module: 'package-name'. Make sure this package is installed.
Questions:
Is there any way to check package.json changes in the pipeline? Ideally I would install the dependencies only in those cases, so the cache can be purged and updated.
Maybe I did not see something in the documentation. Any help is appreciated, thank you!
The problem is the cache key you used is "dependencies", a plain string. This key never changes, so you will always use the same exact cache.
You need to use a cache key that changes, preferably based on package.lock. Please read the section of cache keys in the CircleCI Docs for more information: https://circleci.com/docs/2.0/caching/#using-keys-and-templates
Like most people writing (and reading) the question about whether to keep composer.lock in version-control, we keep ours there.
However, this causes us trouble every time the file is independently updated in different code-branches. Even when the changes are unrelated and affect the sections of the file afar from each other, the "content-hash" line is causing a conflict every time. Worse, neither "side" is correct and whoever is doing the merging must regenerate the file by hand...
Maybe, the line is not really necessary? Before asking, whether (the current version of) composer will work without it, what functionality would be missing? The hash seems to guard against the file itself changing -- but the source-control system is already doing that...
Can I simply remove the line? If it can not be done today, would it be a desirable feature for composer?
Purpose of the content hash
As you can see in Composer\Package\Locker::getContentHash(), the content hash takes into account the following fields of composer.json:
$relevantKeys = array(
'name',
'version',
'require',
'require-dev',
'conflict',
'replace',
'provide',
'minimum-stability',
'prefer-stable',
'repositories',
'extra',
);
The only reason for the content hash to change is a change of one of the values of the corresponding properties in composer.json.
Composer uses the content hash to determine whether relevant fields in composer.json are in sync with composer.lock. You can run
$ composer validate
to find out if they are in sync.
If composer.json and composer.lock are not in sync, a message similar to this will be shown
The lock file is not up to date with the latest changes in composer.json, it is recommended that you run composer update.
For reference, see https://getcomposer.org/doc/03-cli.md#validate:
You should always run the validate command before you commit your composer.json file, and before you tag a release. It will check if your composer.json is valid.
Resolving conflicts in composer.lock
If you have trouble resolving conflicts in composer.lock, maybe this helps:
Step 1: Accept upstream changes
Usually, you will probably attempt to rebase a branch on top of the upstream changes. When already in conflict, use your IDE, or run
$ git checkout --theirs composer.lock
to accept the upstream changes to composer.lock. Since this is a generated file, you really don't want to resolve conflicts in it.
Step 2: Re-apply changes to composer.json and composer.lock
As pointed out earlier, there are a range of the relevant keys in composer.json. Some of them can be modified by corresponding commands, others cannot.
For example, if one of the changes is a newly added or removed package, run
$ composer require foo/bar:^1.2.3
or
$ composer remove foo/bar
to apply the changes.
If the changes cannot be applied by running a command, manually modify composer.json, then run
$ composer update --lock
This will update the content hash.
For reference, see https://getcomposer.org/doc/03-cli.md#update:
--lock: Only updates the lock file hash to suppress warning about the lock file being out of date.
TL;DR version from #localheinz:
Just resolve the conflict and then regenerate the lock file with:
$ composer update --lock
This will update the content hash.
For reference, see https://getcomposer.org/doc/03-cli.md#update:
--lock: Only updates the lock file hash to suppress warning about the lock file being out of date.
I created a repo and made some changes into a file. Then I committed and kept working. A couple of changes and I committed again.
I had to go back to the first version, so I did:
hg up 5f01300
And the files from the first version came back.
I made some changes and committed again.
hg commit -m 'Updated'
A new head was created and then I tried:
hg push
Mercurial said there were "outstanding uncommitted" stuffs.
I tried to commit again:
hg commit -m 'Changes accepted'
Then I typed:
hg merge
So a lot of errors appeared because there were two heads and I couldn't merge them.
Sometimes I try to merge like I did above and it works perfectly. Usually at the second time I try to repeat the process, it breaks and I can't merge anything.
Can somebody teach how to merge properly?
[SOME ERRORS]
Usually, at the second time, this is what happens:
int main(int argc, char **argv)
{
printf("hello, world!\n");
<<<<<<< local
printf("sure am glad I'm not using CVS!\n");
=======
printf("sure am glad I'm using Mercurial!\n");
>>>>>>> other
return 0;
}
And I have to do:
hg resolve -m hello.c
Merging is how you combine divergent versions of your files. Whenever you change a file in two different ways, Mercurial will try to merge it when you run hg merge.
When the changes to the files don't overlap, Mercurial can do the merge automatically. However, if you change the same line in two different ways, then you need to help Mercurial merge the two edits.
Consider a file that starts with hello, world! in the first revision and then changes into sure am glad I'm not using CVS! in the second revision:
"hello, world" --- "sure am glad I'm not using CVS!"
You now update back to the first revision and make a different change to the text:
"hello, world" --- "sure am glad I'm not using CVS!"
\
\
\
"sure am glad I'm using Mercurial!"
You now have a divergent history and when you hg merge, Mercurial will tell you that it has detected a conflict in the file. Mercurial will search for a merge tool and if it cannot find any on your system, it will dump the conflicting changes in the file. The file then looks like this:
<<<<<<< local
sure am glad I'm not using CVS!
=======
sure am glad I'm using Mercurial!
>>>>>>> other
The markers you see are called merge markers and they show you the region of the file that couldn't be automatically merged. The top region marked with local is the version you had in your working copy before you typed hg merge. The bottom region marked other is the version that was on the branch you merged with.
Mercurial knows that the file is unresolved and you can see with with hg resolve --list. You now edit the file to somehow combine the two versions. Here's one combination:
sure am glad I'm using Mercurial and not CVS!
I've deleted the merge markers and left behind a version I like. You now mark the file as resolved and commit the merge:
$ hg resolve --mark hello.c
$ hg commit
I hope that helps!
Another comment: You write
[...] I tried: hg push. Mercurial said there were "outstanding uncommitted" stuffs.
Mercurial never talks about uncommitted changes when you try to push. Pushing and pulling are unrelated to your working copy (where your uncommitted changes live). So it is okay to push and pull with a dirty working copy. The error you're thinking of is probably
$ hg merge
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
$ hg merge
abort: outstanding uncommitted merges
That is, you cannot start a new merge before you commit or abort the current merge.
My editor produces workspace files and backup folders which are of no interest to users of the software I write. In order to avoid having to list my editor-specific ignores in each project, I'm trying to tell npm to ignore them at the user or global level.
Unfortunately, I'm having no luck doing so. Running npm pack inside my project folder, even if I clear the npm cache first, includes both the workspace file and and two megabytes of backup files. (For a project with only ten kilobytes of code!) I've tried the ignore config setting, a per-user .npmignore, and a global npmignore, all to no effect.
Here's my output from npm config ls -l, snipped to relevant sections:
; userconfig C:\Users\benblank\.npmrc
ignore = "__history *.epp"
; builtin config undefined
prefix = "C:\\Users\\benblank\\AppData\\Roaming\\npm"
; default values
globalignorefile = "C:\\Users\\benblank\\AppData\\Roaming\\npm\\etc\\npmignore"
userignorefile = "C:\\Users\\benblank\\.npmignore"
And the (identical) contents of C:\Users\benblank\.npmignore and C:\Users\benblank\AppData\Roaming\npm\etc\npmignore:
__history
*.epp
What am I doing wrong? I'm running Windows 7, node#0.8.9, and npm#1.1.61.
npm has some major outstanding issues related to npmignore files.
Thankfully there is an even better alternative! It is the package.json files property.
Here is an example from my delivr project.
"files": [
"lib",
"index.js"
]
Why is it better?
The internal machinery makes this mechanism more reliable.
It is a whitelist instead of a blacklist. Typically, assuming you use a lib folder (or similar), there are less files/directories you want included vs those you want excluded, so it is more succinct.
It is DRY. If a .npmignore file exists, npm will not consult .gitignore for ignore patterns, which often needs overlapping and repetitive info. This problem does not exist with a whitelist.
One less file in the repo. package.json is already a manifest that defines your package and it makes a ton of sense for this configuration to live there.