Monorepo publishing with lerna and a deploy bot - continuous-integration

Here's the scenario:
We want a monorepo for several components and would like to use lerna with yarn workspaces for it.
To ensure no problems happen with semantic versioning, it would be nice to have code reviews for the version numbers as well.
So the package.json defines a version-bump script that shall only be used to increment package versions.
After the tests running and CR being OK, we'd like a deploy bot to publish the packages to our custom registry for us.
For this it would be nice to use lerna publish --skip-git, so that lerna would publish the changed packages only.
The problem here is that lerna publish won't just publish the packages, but asks for their version increments again.
It would be nice to know of an option or workaround to publish without incrementing the version.
Our current workaround is to use lerna exec npm publish, but this will try to publish already published packages again. We also cannot use lerna exec yarn publish because in that case yarn asks for version increments.
The setup looks like this:
lerna.json:
{
"lerna": "2.5.1",
"version": "independent",
"npmClient": "yarn",
"useWorkspaces": true,
"packages": [
"packages/*"
]
}
package.json
{
"name": "…",
"version": "0.0.0",
"description": "…",
"main": "index.js",
"repository": {
"type": "git",
"url": "…"
},
"workspaces": [
"packages/*"
],
"private": true,
"scripts": {
"version-bump": "./node_modules/lerna/bin/lerna.js publish --skip-npm",
"test": "echo well tested"
},
"devDependencies": {
"lerna": "^2.5.1"
}
}

For anyone that needs this functionality, it looks like they're working on it for v3.0:
Separate "version" and "publish" commands - https://github.com/lerna/lerna/issues/961

I'm in the same boat. The feature doesn't exist. Ideally Lerna would have an argument you could pass to skip bumping the version numbers. You're best making some noise over at the project on Github: https://github.com/lerna/lerna/issues

Related

Duplicate "graphql" modules cannot be used at the same time

Good morning/afternoon stackoverflow. I'm using an npm package called #graphql-codegen/cli to generate type definitions/utilites for my GraphQL schema. Recently, I've been encountering an error each time I try to run graphql-codegen command/script. This is the error that I get
"Duplicate "graphql" modules cannot be used at the same time since different
versions may have different capabilities and behavior. The data from one
version used in the function from another could produce confusing and
spurious results."
...
"Ensure that there is only one instance of "graphql" in the node_modules
directory. If different versions of "graphql" are the dependencies of other
relied on modules, use "resolutions" to ensure only one version is installed."
I'm fairly certain this issue is with my environment, not my project. I attempted to create a brand new project from scratch and still received the same error. Here are the things I've tried
Reinstalling node_modules
Using the resolutions property in package.json
Using different versions of the graphql/graphql-cli packages
Completely wiping out my global npm packages
I've spent the last couple of days attempting to resolve this error, but I'm all out of ideas. Any thoughts or recommendations are much appreciated. Also, below is a link to a codesandbox that contains the relevant files
https://codesandbox.io/s/graphql-codegen-cli-example-qq5cj
I had the same problem with codegen.
src/generated/graphql.tsx
Error: Cannot use GraphQLObjectType "FieldError" from another module or realm.
Ensure that there is only one instance of "graphql" in the node_modules
directory. If different versions of "graphql" are the dependencies of other
relied on modules, use "resolutions" to ensure only one version is installed.
https://yarnpkg.com/en/docs/selective-version-resolutions
Duplicate "graphql" modules cannot be used at the same time since different
versions may have different capabilities and behavior. The data from one
version used in the function from another could produce confusing and
spurious results.
running codegen.yml
overwrite: true
schema: "http://localhost:4001/graphql"
documents: "src/graphql/**/*.graphql"
generates:
src/generated/graphql.tsx:
plugins:
- "typescript"
- "typescript-operations"
- "typescript-urql"
I think i had conflicting packages. Removing "urql", which i think caused the problem, and reinstalling it solved the error.
yarn run v1.22.17
$ graphql-codegen --config codegen.yml
√ Parse configuration
√ Generate outputs
with package.json
{
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"gen": "graphql-codegen --config codegen.yml"
},
"dependencies": {
"#chakra-ui/icons": "^1.0.0",
"#chakra-ui/react": "^1.8.5",
"#emotion/react": "^11.0.0",
"#emotion/styled": "^11.0.0",
"formik": "^2.2.9",
"framer-motion": "^4.0.3",
"next": "latest",
"react": "^17.0.2",
"react-dom": "^17.0.2",
"graphql": "^16.3.0",
"urql": "^2.2.0"
},
"devDependencies": {
"#graphql-codegen/cli": "^2.6.2",
"#graphql-codegen/typescript": "2.4.5",
"#graphql-codegen/typescript-operations": "2.3.2",
"#graphql-codegen/typescript-urql": "^3.5.3",
"#graphql-codegen/urql-introspection": "^2.1.1",
"#types/node": "^17.0.21",
"graphql-tag": "^2.12.6",
"typescript": "^4.5.5"
}
}
You can edit your package.json as needed. Delete the file 'yarn.lock', 'package-lock.json' and the folder 'node_modules' to clear your dependencies. And run 'npm install' / 'yarn install' to reinstall your dependencies.
Have fun!

The lerna build is very slow, always (+30 minutes)

I'm just joining a new VueJS / Webpack based on a Lerna code architecture :
package.json
lerna.json
packages/
modules/
plugins/
Approximately each page of the application has been set as a separated module which I find strange and although not an expert I'm not sure this is the correct way of setting up a Lerna architecture.
Nevertheless, the package.json defines the following :
"scripts": {
"bootstrap": "npm install && npm run lerna && npm run app-build",
"lerna": "lerna bootstrap --hoist --nohoist=axios --nohoist=vue-chartist --nohoist=chardist",
"publish": "lerna publish",
"clean": "lerna clean",
"test": "lerna run test --parallel",
"start": "lerna run start --stream --scope=main-module",
"app-build": "lerna run build --stream --scope=main-module",
"doc": "good-doc"}
And the app, although of medium size I would say :
Size of the application with node_modules
Is always very slow (+30 minutes) to build. At each build. The builds are executed like this :
cross-env BACK_URL=back_url npm run bootstrap --hoist
Is there any good pratices to have a quicker build ? Any ideas of what could have been set wrong in my project ? Or maybe this is just normal...
I moved from --hoist to use yarn workspaces (https://yarnpkg.com/blog/2017/08/02/introducing-workspaces/).
My problem was not regarding performance but about having the possibility to use the nohoist option (https://yarnpkg.com/blog/2018/02/15/nohoist/). I had some error with a really simple setup because of some react-scripts dependency, so I needed to exclude to modules from hoisting.
Here's my base config:
--> lerna.json
{
"version": "0.0.0",
"packages": [
"packages/*",
],
"npmClient": "yarn",
"useWorkspaces": true
}
---> package.json
{
"name": "root",
"private": true,
"workspaces": {
"packages": ["packages/*""],
"nohoist": ["**/babel-jest", "**/eslint", "**/jest"]
},
"devDependencies": {
"lerna": "^3.4.3"
}
}
The slow build was due to my computer + a lot of files to build together I guess. We had lerna implemented as each page of the app was a separated package, which was not really was lerna is made for.
We removed lerna from the infrastructure and we're better off now.
I'd say to set "--concurrency 1" to decrease memory usage.
I got better performance with it.
;)

Displaying Spectron-Webdriverio test results in TeamCity

I would like to display Spectron test results in TeamCity. I have followed the instructions at the Webdriverio TeamCity Reporter page, which are:
npm install wdio-teamcity-reporter --save-dev
and creating a wdio.conf.js file:
exports.config = {
reporters: ['teamcity'],
}
I have placed this file at the top of the project. It has no other entries; I've never needed it before.
I have also tried the additional configuration suggested at wdio-teamcity-reporter npm page.
This is the Jest object in package.json:
"jest": {
"moduleFileExtensions": [
"ts",
"tsx",
"js"
],
"transform": {
"\\.(ts|tsx)$": "<rootDir>/node_modules/ts-jest/preprocessor.js"
},
"roots": [
"<rootDir>/__tests__/",
"<rootDir>/components/"
],
"modulePaths": [
"<rootDir>/__tests__/",
"<rootDir>/components/"
],
"testMatch": [
"**/?(*.)(spec|test).(ts)?(x)"
]
}
And this is the relevant command (that TeamCity calls) in package.json:
"scripts": {
// ...
"test": "jest --maxWorkers=1 --forceExit",
// ...
},
This testing project is built with Typescript and Jest, and only comprises the e2e Spectron tests for an Electron app. The build artifact for that app is a TeamCity dependency for my test 'build'. In my build, TeamCity installs the app, runs the Spectron tests (which are passing), and then uninstalls the app.
All I can see at the moment is the Jest console output within the build log. While there are some hidden artifacts, I see no normal artifacts. I was thinking that the reporting package should have produced an html artifact. How do I go about displaying a test tab, or some other useful set of results?
It turns out that Jest can collect all the Webdriver results. Try using https://www.npmjs.com/package/jest-teamcity.
In jest.config.js use:
"testResultsProcessor": "jest-teamcity"

Global.json and running web application

I am trying to create a new web application using OS X and VS Code from scratch without using any scaffolding tool. My starting point is Scott Allen's tutorial on pluralsight:
https://app.pluralsight.com/library/courses/aspdotnet-core-1-0-fundamentals/table-of-contents
My project structure is:
The global.json file contains
{
"projects": [ "src" ],
"sdk": {
"version": "1.0.0-rc1-update2"
}
}
And the project.json currently contains
{
"version": "1.0.0-*",
"compilationOptions": {
"emitEntryPoint": false
},
"dependencies": {
"Microsoft.AspNet.Server.Kestrel": "1.0.0-rc1-final",
"Microsoft.AspNet.Hosting": "1.0.0-rc1-final"
},
"frameworks": {
"dnx451": {},
"dnxcore50": {}
},
"commands": {
"web": "Microsoft.AspNet.Hosting --server Microsoft.AspNet.Server.Kestrel --server.urls http://localhost:5000"
},
"exclude": [
"wwwroot",
"node_modules"
]
}
I have run dnu restore to get the packages and now I would like to run the web. I need to go to the web app folder and run dnx web in order to do so and the app starts
Is it possible to run the application directly from the root folder, not from the web app folder? Is the global.json file needed in such setup? And how do I change the hosting environment? I have gone through the documentation, but the hosting environment is only clear when using VS 2015.
You cannot simply run from the root because there could be multiple projects that are "executable". But you can pass the project to dnx using the --project/-p argument.
The environment is set using the ASPNET_ENVIRONMENT environment variable.
The global.json file is useful for two things:
The sdk section is only by VS.
The projects section is used all the time and it's useful if you have the projects in multiple folders (for example src and test). If everything is in a single folder, you don't need it.
So, the bare minimum in order to run an web application is:
A folder for your project
A project.json file
A startup file

Running a Script on Build (e.g. Gulp or Grunt)

I have a project.json script for 'prepare' that runs a gulpfile to push my bower stuff into the wwwroot. Works great, except during dev I have to manually run it after I update the bower.json package. Any way to automate this during dev? I'd normally use a post-build script but they are no where to be seen. My project.json scripts are looks like this:
"scripts": {
"prepare": [ "gulp bower" ]
}
What i'd love is:
"scripts": {
"post-build": [ "gulp bower" ]
}
You can use the Task Runner Explorer to automate this. (Use the Quick Launch in the upper right, Ctrl+Alt+\, or View->Other Windows->Task Runner Explorer.)
Find the task you want to add (bower or prepare, depending on the route you want to go), right click, and use the context menu to add the bindings.
My gruntfile.js, for example, got the following line added to the top:
/// <binding BeforeBuild='beforeBuild' AfterBuild='afterBuildMinimal' ProjectOpened='watch' />
I'm not certain if the gulpfile.js uses the exact same conventions, but the Task Runner Explorer is the way to go, either way!
there's postrestore and postbuild you can use on project.json.
I use it like this :
"postrestore": [ "npm install", "bower install" ]
"postbuild": [ "brunch build" ]
in your example, I think you want
"postbuild": [ "gulp bower" ]

Resources