The lerna build is very slow, always (+30 minutes) - performance

I'm just joining a new VueJS / Webpack based on a Lerna code architecture :
package.json
lerna.json
packages/
modules/
plugins/
Approximately each page of the application has been set as a separated module which I find strange and although not an expert I'm not sure this is the correct way of setting up a Lerna architecture.
Nevertheless, the package.json defines the following :
"scripts": {
"bootstrap": "npm install && npm run lerna && npm run app-build",
"lerna": "lerna bootstrap --hoist --nohoist=axios --nohoist=vue-chartist --nohoist=chardist",
"publish": "lerna publish",
"clean": "lerna clean",
"test": "lerna run test --parallel",
"start": "lerna run start --stream --scope=main-module",
"app-build": "lerna run build --stream --scope=main-module",
"doc": "good-doc"}
And the app, although of medium size I would say :
Size of the application with node_modules
Is always very slow (+30 minutes) to build. At each build. The builds are executed like this :
cross-env BACK_URL=back_url npm run bootstrap --hoist
Is there any good pratices to have a quicker build ? Any ideas of what could have been set wrong in my project ? Or maybe this is just normal...

I moved from --hoist to use yarn workspaces (https://yarnpkg.com/blog/2017/08/02/introducing-workspaces/).
My problem was not regarding performance but about having the possibility to use the nohoist option (https://yarnpkg.com/blog/2018/02/15/nohoist/). I had some error with a really simple setup because of some react-scripts dependency, so I needed to exclude to modules from hoisting.
Here's my base config:
--> lerna.json
{
"version": "0.0.0",
"packages": [
"packages/*",
],
"npmClient": "yarn",
"useWorkspaces": true
}
---> package.json
{
"name": "root",
"private": true,
"workspaces": {
"packages": ["packages/*""],
"nohoist": ["**/babel-jest", "**/eslint", "**/jest"]
},
"devDependencies": {
"lerna": "^3.4.3"
}
}

The slow build was due to my computer + a lot of files to build together I guess. We had lerna implemented as each page of the app was a separated package, which was not really was lerna is made for.
We removed lerna from the infrastructure and we're better off now.

I'd say to set "--concurrency 1" to decrease memory usage.
I got better performance with it.
;)

Related

Duplicate "graphql" modules cannot be used at the same time

Good morning/afternoon stackoverflow. I'm using an npm package called #graphql-codegen/cli to generate type definitions/utilites for my GraphQL schema. Recently, I've been encountering an error each time I try to run graphql-codegen command/script. This is the error that I get
"Duplicate "graphql" modules cannot be used at the same time since different
versions may have different capabilities and behavior. The data from one
version used in the function from another could produce confusing and
spurious results."
...
"Ensure that there is only one instance of "graphql" in the node_modules
directory. If different versions of "graphql" are the dependencies of other
relied on modules, use "resolutions" to ensure only one version is installed."
I'm fairly certain this issue is with my environment, not my project. I attempted to create a brand new project from scratch and still received the same error. Here are the things I've tried
Reinstalling node_modules
Using the resolutions property in package.json
Using different versions of the graphql/graphql-cli packages
Completely wiping out my global npm packages
I've spent the last couple of days attempting to resolve this error, but I'm all out of ideas. Any thoughts or recommendations are much appreciated. Also, below is a link to a codesandbox that contains the relevant files
https://codesandbox.io/s/graphql-codegen-cli-example-qq5cj
I had the same problem with codegen.
src/generated/graphql.tsx
Error: Cannot use GraphQLObjectType "FieldError" from another module or realm.
Ensure that there is only one instance of "graphql" in the node_modules
directory. If different versions of "graphql" are the dependencies of other
relied on modules, use "resolutions" to ensure only one version is installed.
https://yarnpkg.com/en/docs/selective-version-resolutions
Duplicate "graphql" modules cannot be used at the same time since different
versions may have different capabilities and behavior. The data from one
version used in the function from another could produce confusing and
spurious results.
running codegen.yml
overwrite: true
schema: "http://localhost:4001/graphql"
documents: "src/graphql/**/*.graphql"
generates:
src/generated/graphql.tsx:
plugins:
- "typescript"
- "typescript-operations"
- "typescript-urql"
I think i had conflicting packages. Removing "urql", which i think caused the problem, and reinstalling it solved the error.
yarn run v1.22.17
$ graphql-codegen --config codegen.yml
√ Parse configuration
√ Generate outputs
with package.json
{
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"gen": "graphql-codegen --config codegen.yml"
},
"dependencies": {
"#chakra-ui/icons": "^1.0.0",
"#chakra-ui/react": "^1.8.5",
"#emotion/react": "^11.0.0",
"#emotion/styled": "^11.0.0",
"formik": "^2.2.9",
"framer-motion": "^4.0.3",
"next": "latest",
"react": "^17.0.2",
"react-dom": "^17.0.2",
"graphql": "^16.3.0",
"urql": "^2.2.0"
},
"devDependencies": {
"#graphql-codegen/cli": "^2.6.2",
"#graphql-codegen/typescript": "2.4.5",
"#graphql-codegen/typescript-operations": "2.3.2",
"#graphql-codegen/typescript-urql": "^3.5.3",
"#graphql-codegen/urql-introspection": "^2.1.1",
"#types/node": "^17.0.21",
"graphql-tag": "^2.12.6",
"typescript": "^4.5.5"
}
}
You can edit your package.json as needed. Delete the file 'yarn.lock', 'package-lock.json' and the folder 'node_modules' to clear your dependencies. And run 'npm install' / 'yarn install' to reinstall your dependencies.
Have fun!

How do I enable SASS Compiling in Cookiecutter Django?

in the documentation of cookiecutter-django relating to SASS Compiling and Live Reload it's stated that I only have to enter "npm start" (after installing npm of course) in the main project folder to enable Live-Reload and SASS compiling. I was wondering how this should be possible without a package.json file but tried it nevertheless, maybe some hidden thing I did not know about. But npm init of course told me that a package.json was missing. I initialized a new project with bootstrap compilation and gulp enabled, same outcome. Still no package.json. Am I missing some key-point? Or is the documentation in this case maybe incomplete? Where do I get the required package.json from? :)
In the project generation at the beginning, you probably chose "None" on the "js_task_runner" step. If you chose "Gulp" here, it will generate gulpfile.js and package.json file, after which you'll be able to use npm install.
As mentioned in other answers, when you start your project and then choose none for task runner options, it will not work. See js_task_runner on cookiecutter-django documentation.
May I ask where you tried to run npm install? The package.json file is located in the root of your app, so you have to run commands from there.
It should be possible to add the needed files afterwards manually. Create a package.json file in the root directory of your app and add following code. Important: Change name to the name of your app.
{
"name": "CHANGE_TO_NAME_OF_YOUR_APP",
"version": "0.1.0",
"dependencies": {},
"devDependencies": {
"bootstrap": "4.1.1",
"gulp-concat": "^2.6.1",
"jquery": "3.3.1",
"popper.js": "1.14.3",
"autoprefixer": "^9.4.7",
"browser-sync": "^2.14.0",
"cssnano": "^4.1.10",
"gulp": "^4.0.0",
"gulp-imagemin": "^5.0.3",
"gulp-plumber": "^1.2.1",
"gulp-postcss": "^8.0.0",
"gulp-rename": "^1.2.2",
"gulp-sass": "^4.0.2",
"gulp-uglify-es": "^1.0.4",
"pixrem": "^5.0.0"
},
"engines": {
"node": ">=8"
},
"browserslist": [
"last 2 versions"
],
"scripts": {
"dev": "gulp"
}
}
Since this project uses gulp.js as task runner you need additionally to the package.json also a gulpfile.js file. Code from this example should work. Check also if you have a scss file in name_of_your_app/static/sass/project.scss.
Now you should be able to run npm install in a first step and compile scss with npm run dev in a second step (see documentation).

Monorepo publishing with lerna and a deploy bot

Here's the scenario:
We want a monorepo for several components and would like to use lerna with yarn workspaces for it.
To ensure no problems happen with semantic versioning, it would be nice to have code reviews for the version numbers as well.
So the package.json defines a version-bump script that shall only be used to increment package versions.
After the tests running and CR being OK, we'd like a deploy bot to publish the packages to our custom registry for us.
For this it would be nice to use lerna publish --skip-git, so that lerna would publish the changed packages only.
The problem here is that lerna publish won't just publish the packages, but asks for their version increments again.
It would be nice to know of an option or workaround to publish without incrementing the version.
Our current workaround is to use lerna exec npm publish, but this will try to publish already published packages again. We also cannot use lerna exec yarn publish because in that case yarn asks for version increments.
The setup looks like this:
lerna.json:
{
"lerna": "2.5.1",
"version": "independent",
"npmClient": "yarn",
"useWorkspaces": true,
"packages": [
"packages/*"
]
}
package.json
{
"name": "…",
"version": "0.0.0",
"description": "…",
"main": "index.js",
"repository": {
"type": "git",
"url": "…"
},
"workspaces": [
"packages/*"
],
"private": true,
"scripts": {
"version-bump": "./node_modules/lerna/bin/lerna.js publish --skip-npm",
"test": "echo well tested"
},
"devDependencies": {
"lerna": "^2.5.1"
}
}
For anyone that needs this functionality, it looks like they're working on it for v3.0:
Separate "version" and "publish" commands - https://github.com/lerna/lerna/issues/961
I'm in the same boat. The feature doesn't exist. Ideally Lerna would have an argument you could pass to skip bumping the version numbers. You're best making some noise over at the project on Github: https://github.com/lerna/lerna/issues

Are there ways to perform postCSS processing withouth gulp or grunt. Only with help of maven?

I want to perform postprocessing in my project with help of postCSS.
As I'm new in frontend I read only ways to perform it by frontend build system (grunt or gulp).
But maybe the ways to postpocess only with maven?
You can use postcss-cli to run it through the command line.
The command usage is pretty straight-forward.
postcss [options] [-o output-file|-d output-directory] [input-file]
Also, if you are using npm along with a package.json file, I would advise you to add a run-script:
{
"name": "my-app",
"script": {
"css": "postcss your options -go here"
},
"dependencies": {
"postcss":"^4.1.13"
}
}
So you can simply run npm run css from maven / CLI without having to worry about prefixing your command node_modules/bin and having your options in maven.

Running a Script on Build (e.g. Gulp or Grunt)

I have a project.json script for 'prepare' that runs a gulpfile to push my bower stuff into the wwwroot. Works great, except during dev I have to manually run it after I update the bower.json package. Any way to automate this during dev? I'd normally use a post-build script but they are no where to be seen. My project.json scripts are looks like this:
"scripts": {
"prepare": [ "gulp bower" ]
}
What i'd love is:
"scripts": {
"post-build": [ "gulp bower" ]
}
You can use the Task Runner Explorer to automate this. (Use the Quick Launch in the upper right, Ctrl+Alt+\, or View->Other Windows->Task Runner Explorer.)
Find the task you want to add (bower or prepare, depending on the route you want to go), right click, and use the context menu to add the bindings.
My gruntfile.js, for example, got the following line added to the top:
/// <binding BeforeBuild='beforeBuild' AfterBuild='afterBuildMinimal' ProjectOpened='watch' />
I'm not certain if the gulpfile.js uses the exact same conventions, but the Task Runner Explorer is the way to go, either way!
there's postrestore and postbuild you can use on project.json.
I use it like this :
"postrestore": [ "npm install", "bower install" ]
"postbuild": [ "brunch build" ]
in your example, I think you want
"postbuild": [ "gulp bower" ]

Resources