Jenkinsfile: monorepo with package dependencies - jenkins-pipeline

I am creating a monorepo which builds and publishes multiple npm packages to a local Artifactory. All within this monorepo, several npm packages have dependencies with another npm package create.
The layout for this monorepo is as follows, and npm package #ds/cy-uncaught-exceptions-task has a dependency on npm package #ds/cy-log-task.
├── Jenkinsfile
├── packages
│   ├── cy-browser-launch-task
│   ├── cy-ds-thycotic-task
│   └── cy-log-task
└── renovate.json
Jenkins pipeline fails: continuous-integration/jenkins/pr-merge — This commit cannot be built
Jenkins build fails during the publishing of npm package cy-uncaught-exceptions-task:
Jenkins Publishing Error:
This is the Jenkinsfile that performs this publish:
def packages = [
'#ds/cy-ds-thycotic-task': [
projectDir: 'packages/tasks/cy-ds-thycotic-task'
],
'#ds/cy-uncaught-exceptions-task': [
projectDir: 'packages/tasks/cy-uncaught-exceptions-task'
],
'#ds/cy-log-task': [
projectDir: 'packages/tasks/cy-log-task'
]
]
def conditionals = [
[
condition: { env.TARGET_BRANCH == 'master' && env.SCM_ACTION == 'commit' },
overrides: [
nodejsPublish: true,
pushVersionTag: true,
applicationPackages: packages.collectEntries { key, value -> [(key): value + [
bumpPatchVersion: true
]] }
]
],
[
condition: { env.TARGET_BRANCH == 'master' && env.SCM_ACTION == 'pull_request' },
overrides: [
nodejsPublish: true,
applicationPackages: packages.collectEntries { key, value -> [(key): value + [
publishArgs: '--tag pr',
versionSuffixValue: { 'pr.' + env.ARTIFACT_BUILD_NUMBER }
]] }
]
]
]
What do I need to do to Jenkinsfile perform this build?
Thank you, appreciate the feedback.

I resolved it using lerna
Install these npm packages:
lerna
#lerna/bootstrap
#lerna/version
Jenkinsfile remains the same:
def packages = [
'#ds/cy-ds-thycotic-task': [
projectDir: 'packages/tasks/cy-ds-thycotic-task'
],
'#ds/cy-log-task': [
projectDir: 'packages/tasks/cy-log-task'
],
'#ds/cy-uncaught-exceptions-task': [
projectDir: 'packages/tasks/cy-uncaught-exceptions-task'
]
]
def conditionals = [
[
condition: { env.TARGET_BRANCH == 'master' && env.SCM_ACTION == 'commit' },
overrides: [
nodejsPublish: true,
pushVersionTag: true,
applicationPackages: packages.collectEntries { key, value -> [(key): value + [
bumpPatchVersion: true
]] }
]
],
[
condition: { env.TARGET_BRANCH == 'master' && env.SCM_ACTION == 'pull_request' },
overrides: [
nodejsPublish: true,
applicationPackages: packages.collectEntries { key, value -> [(key): value + [
publishArgs: '--tag pr',
versionSuffixValue: { 'pr.' + env.ARTIFACT_BUILD_NUMBER }
]] }
]
]
]
lerna.json is defined:
{
"npmClient": "npm",
"npmClientArgs": [
"--registry",
"https://npm.docusignhq.com"
],
"command": {
"publish": {
"ignoreChanges": [
"**/.spec.*",
"**/*.md",
"**/tsconfig*.*"
],
"registry": "https://npm.docusignhq.com"
}
},
"packages": [
"packages/tasks/*"
],
"version": "independent"
}
The monorepo's base package.json has these scripts:
"prebuild": "lerna run audit:npm --stream && lerna run code:clean --stream",
"build": "lerna run build --stream",
"clean": "lerna clean",
"code:check": "lerna run code:check --stream",
"code:clean": "lerna run code:clean --stream",
"postinstall": "lerna bootstrap",
And each package build package.json has the following scripts:
"audit": "npx npm-run-all audit:*",
"audit:depcheck": "depcheck --specials=bin,eslint,istanbul,jest,mocha --ignore-patterns=node_modules --ignores=#types/*,eslint-*,#typescript-eslint/*,ts-node,husky",
"audit:npm": "npm audit",
"audit:npm-check-updates": "npx npm-check-updates --errorLevel 1",
"prebuild": "npm run audit:npm && npm run code:clean",
"build": "npx npm-run-all build:*",
"postbuild": "npm run audit:depcheck",
"build:only": "tsc",
"code:check": "gts lint",
"code:clean": "gts clean",
audit of dependencies occurs with each build, and if it fails then an audit upgrade is performed, a new install, and a completely new local build.
This assures that builds fail locally if dependencies fail before monorepo is committed to GitHub and it expected Jenkins build.

Related

Error in Opening Electron App in Mac:"file:///Applications/../Contents/Resources/app.asar/dist/index.html"

I have build the electron app for mac but when I ran it , it is giving the below error
"Not allowed to load local resource: file:///Applications/e-admin.app/Contents/Resources/app.asar/dist/index.html".
Here are the steps I did.
1) electron-builder build --mac
It created the following files are dist folder.
a) e-admin-0.0.0.dmg , e-admin-0.0.0-mac.zip,index.html and a mac folder
b) in mac folder I could see mac/e-admin.app/Contents/Resources/app.asar file( not the folder)
2) I double clicked and installed the e-admin-0.0.0.dmg and moved to application folder.
3) Opened the app.
Do I have to do any thing with app.asar file?(unpack or some thing?) or any etc procedure to make it work?
my package.json
{
"name": "e-admin",
"version": "0.0.0",
"scripts": {
"postinstall": "electron-builder install-app-deps",
"ng": "ng",
"start": "npm-run-all -p electron:serve ng:serve",
"build": "npm run electron:serve-tsc && ng build",
"build:dev": "npm run build -- -c dev",
"build:prod": "npm run build -- -c production",
"ng:serve": "ng serve",
"ng:serve:web": "ng serve -c web -o",
"electron:serve-tsc": "tsc -p tsconfig-serve.json",
"electron:serve": "wait-on http-get://localhost:4200/ && npm run electron:serve-tsc && electron . --serve",
"electron:local": "npm run build:prod && electron .",
"electron:linux": "npm run build:prod && electron-builder build --linux",
"electron:windows": "npm run build:prod && electron-builder build --windows",
"electron:mac": "npm run build:prod && electron-builder build --mac",
"test": "ng test",
"e2e": "npm run build:prod && cross-env TS_NODE_PROJECT='e2e/tsconfig.e2e.json' mocha --timeout 300000 --require ts-node/register e2e/**/*.e2e.ts",
"version": "conventional-changelog -i CHANGELOG.md -s -r 0 && git add CHANGELOG.md",
"lint": "ng lint"
},
"main": "main.js",
"private": true,
"dependencies": {...}
and angular.josn file
"projects": {
"eAdmin": {
"root": "",
"sourceRoot": "src",
"projectType": "application",
"prefix": "app",
"schematics": {},
"architect": {
"build": {
"builder": "#angular-devkit/build-angular:browser",
"options": {
"outputPath": "dist",
"index": "src/index.html",
"main": "src/main.ts",
....
main.ts contains
if (serve) {
require('electron-reload')(__dirname, {
electron: require(`${__dirname}/node_modules/electron`)
});
win.loadURL('http://localhost:4200');
} else {
win.loadURL(url.format({
pathname: path.join(__dirname, 'dist/index.html'),
protocol: 'file:',
slashes: true
}));
}
After muiltple trial and error, The following changes worked for me
1) index.html
`<base href="/">`
to
<base href="./">
2) in main.ts, change the directory name dist to something else
from :
win.loadURL(url.format({
pathname: path.join(__dirname, 'dist/index.html'),
protocol: 'file:',
slashes: true
}));
To:
win.loadURL(url.format({
pathname: path.join(__dirname, 'angular_build/index.html'),
protocol: 'file:',
slashes: true
}));
3) Angular.js
"outputPath": "dist/",
to
"outputPath": "angular_build/",
Although I have seen suggesions to change step 2 & 3 , I was quite skepitcal.
Looks like dist directlory some how does not work and have to change to some thing like as mentioned above

Heroku CI with Postgres postgis extension

I'm using a postgis extension to my database on Heroku. In production this works just fine; I'm mapping like a boss.
But, I can't get my app to build on HerokuCI because the database addon in the test environment doesn't have GDAL installed (part of the postgis extension):
django.core.exceptions.ImproperlyConfigured: Could not find the GDAL library (tried "gdal", "GDAL", "gdal2.3.0", "gdal2.2.0", "gdal2.1.0", "gdal2.0.0", "gdal1.11.0"). Is GDAL installed? If it is, try setting GDAL_LIBRARY_PATH in your settings.
-----> test command `python manage.py migrate --noinput && python manage.py test` failed with exit status 1
I can't find out how to create extensions in the heroku CI environment. Heroku's app json schema say nothing about how to create extensions on the test databases.
I know that I can't use in-dyno database for CI, as postgis is not an available extension for that, but I'm not using an in-dyno one, I'm using a plan...
Here's my app schema:
{
"name": "mymegaapp",
"scripts": {
"heroku-prebuild": "create extension postgis",
"heroku-postbuild": "echo heroku-postbuild script runs here."
},
"env": {
},
"formation": {
},
"addons": [
],
"buildpacks": [
{
"url": "heroku/python"
},
{
"url": "heroku/nodejs"
}
],
"environments": {
"test": {
"scripts": {
"test": "python manage.py migrate --noinput && python manage.py test"
},
"env": {
"DJANGO_DEBUG": "1"
},
"addons":[
{
"plan": "heroku-postgresql",
"options": {
"version": "11"
}
},
"heroku-redis:hobby-dev"
]
}
}
}
Note that I've tried to create the postgis extension in the prebuild script, which doesn't make a difference.
I've also tried invoking the creation before the test script:
"scripts": {
"test": "heroku create extension postgis && python manage.py migrate --noinput && python manage.py test"
},
which doesn't work because the heroku cli isn't installed in the dyno.
The key questions:
How can I create extensions in HerokuCI?
Is postgis an allowable extension at all in HerokuCI?
You've almost got it. There's no first class support for creating pg extensions directly from the CLI. However, you can pass a statement into psql.
Try this in your postdeploy or test setup script:
psql -c "create extension postgis" $DATABASE_URL As you're aware, you'll need to do this with a real database, not in-dyno. If you've attached the database as something other than DATABASE_URL, you will need to update that in the script.
[Edit by OP to add useful info] In this case, GDAL libraries will also be needed, which can be installed by setting the BUILD_WITH_GEO_LIBRARIES environment variable to 1. The final working solution in app.json will be:
"environments": {
"test": {
"scripts": {
"postdeploy": "psql -c \"create extension postgis\" $DATABASE_URL",
"test": "python manage.py migrate --noinput && python manage.py test"
},
"env": {
"DJANGO_DEBUG": "1",
"BUILD_WITH_GEO_LIBRARIES": "1"
},
"addons":[
{
"plan": "heroku-postgresql",
"options": {
"version": "11"
}
}
]
}
}
Here a 2021 update as the setting BUILD_WITH_GEO_LIBRARIES is now deprecated and unsupported by Heroku:
This app.json worked for my CI setup with Django / Heroku:
{
"buildpacks": [
{
"url": "https://github.com/heroku/heroku-geo-buildpack.git"
},
{
"url": "heroku/python"
}
],
"environments": {
"test": {
"scripts": {
"test": "python3 manage.py migrate --noinput && python3 manage.py test --keepdb"
},
"env": {
"DJANGO_DEBUG": "1"
}
}
}
}

Istanbul test coverage report show that i'm covering only one file

This is the first time that I'm using Istanbul 2.0, but the reports shows that I'm covering only one file, and this file doesn't matter enough.
In my package.json I put the following settings
"nyc": {
"check-coverage": true,
"include": [
"src/**/*.js"
],
"exclude": [
"src/config/*.js",
"src/adapter/*.js"
]
},
"scripts": {
"start": "npm install && nodemon src/server",
"test": "nyc --reporter=html mocha --recursive"
}
UPDATE
When I change de package JSON puting the reporter in lcov, it works.
"test": "nyc --reporter=lcov mocha --recursive"
VS Code
Istanbul HTML Report

How to insert image files in Electron + React.js + Webpack desktop app package for Mac OS

I am building a desktop application for Mac OS with Electron + React + Webpack mode.
When I build a Electron desktop app as production version, I would like to insert some image files in the setup file(Test.app and Test.dmg) and use the image as following:
let Logo = process.resourcesPath + '/logo.png';
class Loading extends Component {
render() {
return (
<div className={styles.container}>
<img className={styles.logo} src={Logo} />
...
</div>
);
}
}
But this app can't get the image because the logo.png file didn't be include in Test.app/Content/Resources/ directory when I run 'npm run package' command and build a Test.app for Mac OS.
Here, the issue is as following:
GET file:///Applications/Test.app/Contents/Resources/logo.png
net::ERR_FILE_NOT_FOUND
I would like to insert this image file in my Test.app package for Mac OS.
Please help me how to insert some image files into Test.app package for a Mac OS desktop application.
webpack.config.base.js
import path from 'path';
import webpack from 'webpack';
import { dependencies as externals } from './app/package.json';
export default {
externals: Object.keys(externals || {}),
module: {
rules: [{
test: /\.jsx?$/,
exclude: /node_modules/,
use: {
loader: 'babel-loader',
options: {
cacheDirectory: true
}
}
}]
},
output: {
path: path.join(__dirname, 'app'),
filename: 'bundle.js',
// https://github.com/webpack/webpack/issues/1114
libraryTarget: 'commonjs2'
},
resolve: {
extensions: ['.js', '.jsx', '.json'],
modules: [
path.join(__dirname, 'app'),
'node_modules',
],
},
plugins: [
new webpack.NamedModulesPlugin(),
],
};
webpack.config.production.js
import path from 'path';
import webpack from 'webpack';
import validate from 'webpack-validator';
import ExtractTextPlugin from 'extract-text-webpack-plugin';
import merge from 'webpack-merge';
import HtmlWebpackPlugin from 'html-webpack-plugin';
import BabiliPlugin from 'babili-webpack-plugin';
import baseConfig from './webpack.config.base';
export default validate(merge(baseConfig, {
devtool: 'cheap-module-source-map',
entry: './app/index',
output: {
publicPath: '../dist/'
},
module: {
loaders: [
// Extract all .global.css to style.css as is
{
test: /\.global\.css$/,
loader: ExtractTextPlugin.extract(
'style-loader',
'css-loader'
)
},
// Pipe other styles through css modules and append to style.css
{
test: /^((?!\.global).)*\.css$/,
loader: ExtractTextPlugin.extract(
'style-loader',
'css-loader?modules&importLoaders=1&localIdentName=[name]__[local]___[hash:base64:5]'
)
},
// Fonts
{ test: /\.woff(\?v=\d+\.\d+\.\d+)?$/, loader: 'url-loader?name=fonts/[hash].[ext]&limit=50000&mimetype=application/font-woff' },
{ test: /\.woff2(\?v=\d+\.\d+\.\d+)?$/, loader: 'url-loader?name=fonts/[hash].[ext]&limit=50000&mimetype=application/font-woff' },
{ test: /\.ttf(\?v=\d+\.\d+\.\d+)?$/, loader: 'file-loader?name=fonts/[hash].[ext]' },
{ test: /\.eot(\?v=\d+\.\d+\.\d+)?$/, loader: 'file-loader?name=fonts/[hash].[ext]' },
{ test: /\.svg(\?v=\d+\.\d+\.\d+)?$/, loader: 'file-loader?name=fonts/[hash].[ext]' },
// Images
{
test: /\.(?:ico|gif|png|jpg|jpeg|webp)$/,
loader: 'url-loader'
}
]
},
plugins: [
new webpack.optimize.OccurrenceOrderPlugin(),
new webpack.DefinePlugin({
'process.env.NODE_ENV': JSON.stringify('production')
}),
new BabiliPlugin({
// Disable deadcode until https://github.com/babel/babili/issues/385 fixed
deadcode: false,
}),
new webpack.optimize.UglifyJsPlugin({
compressor: {
screw_ie8: true,
warnings: false
}
}),
new ExtractTextPlugin('style.css', { allChunks: true }),
new HtmlWebpackPlugin({
filename: 'app.html',
template: 'app/app.html',
inject: false
})
],
target: 'electron-renderer'
}));
webpack.config.electron.js
/**
* Build config for electron 'Main Process' file
*/
import webpack from 'webpack';
import validate from 'webpack-validator';
import merge from 'webpack-merge';
import BabiliPlugin from 'babili-webpack-plugin';
import baseConfig from './webpack.config.base';
export default validate(merge(baseConfig, {
devtool: 'source-map',
entry: ['babel-polyfill', './main.development'],
// 'main.js' in root
output: {
path: __dirname,
filename: './main.js'
},
plugins: [
deadcode: false,
}),
new webpack.DefinePlugin({
'process.env': {
NODE_ENV: JSON.stringify('production')
}
})
],
target: 'electron-main',
node: {
__dirname: false,
__filename: false
}
}));
scripts section of package.json
"scripts": {
"test": "cross-env NODE_ENV=test BABEL_DISABLE_CACHE=1 node --trace-warnings ./test/runTests.js",
"test-all": "npm run lint && npm run flow && npm run test && npm run build && npm run test-e2e",
"test-watch": "npm test -- --watch",
"test-e2e": "cross-env NODE_ENV=test BABEL_DISABLE_CACHE=1 node --trace-warnings ./test/runTests.js e2e",
"lint": "eslint --cache --format=node_modules/eslint-formatter-pretty .",
"lint-fix": "npm run lint -- --fix",
"lint-styles": "stylelint app/*.css app/components/*.css --syntax scss",
"hot-updates-server": "cross-env NODE_ENV=development node --trace-warnings -r babel-register ./node_modules/webpack-dev-server/bin/webpack-dev-server --config webpack.config.renderer.dev.js",
"build": "concurrently \"npm run copy-images\" \"npm run build-main\" \"npm run build-renderer\"",
"build-dll": "cross-env NODE_ENV=development node --trace-warnings -r babel-register ./node_modules/webpack/bin/webpack --config webpack.config.renderer.dev.dll.js --progress --profile --colors",
"build-main": "cross-env NODE_ENV=production node --trace-warnings -r babel-register ./node_modules/webpack/bin/webpack --config webpack.config.main.prod.js --progress --profile --colors",
"build-renderer": "cross-env NODE_ENV=production node --trace-warnings -r babel-register ./node_modules/webpack/bin/webpack --config webpack.config.renderer.prod.js --progress --profile --colors",
"start": "cross-env NODE_ENV=production electron ./app/",
"prestart": "npm run build",
"flow": "flow",
"flow-typed": "rm -rf flow-typed && flow-typed install --overwrite || true",
"start-hot-renderer": "cross-env HOT=1 NODE_ENV=development electron -r babel-register -r babel-polyfill ./app/main.development",
"postinstall": "concurrently \"npm run flow-typed\" \"npm run build-dll\" \"install-app-deps\" \"node node_modules/fbjs-scripts/node/check-dev-engines.js package.json\"",
"dev": "cross-env START_HOT=1 npm run hot-updates-server",
"package": "npm run build && build --publish never",
"package-win": "npm run build && build --win --x64",
"package-linux": "npm run build && build --linux",
"package-all": "npm run build && build -mwl",
"cleanup": "mop -v",
"copy-images": "cp -Rf app/shared/images/ app/dist/images/"
},
"browserslist": "electron 1.4",
"build": {
"productName": "Test",
"appId": "org.Test",
"files": [
"dist/",
"node_modules/",
"app.html",
"main.js",
"main.js.map",
"package.json"
],
"dmg": {
"contents": [
{
"x": 130,
"y": 220
},
{
"x": 410,
"y": 220,
"type": "link",
"path": "/Applications"
}
]
},
"win": {
"target": [
"nsis"
]
},
"linux": {
"target": [
"deb",
"AppImage"
]
},
"directories": {
"buildResources": "resources",
"output": "release"
}
},
after electron app installed, it put all the source code and resource files into a .asar package. so to insert any image, u have to use a directory outside of app.
Generally this directory can be a Document folder.
const { remote } = require('electron');
let tempFolder =path.join( remote.app.getPath("documents"), 'AppFiles');
use the tempFolder directory to save image.

How to set a .rc file conditionally

I am using babel and have a .babelrc file for its configuration:
{
"stage": 0,
"ignore": [
"node_modules",
"bower_components",
"testing",
"test"
]
}
However, when I'm developing locally, having this .babelrc file disallows me from running Babel's CLI babel-node in the testing folder (see: babel-node no longer working in different directory )
That said, when I push to Heroku, I need this configuration because I need to make sure the testing folder isn't compiled.
How can I conditionally set a .babelrc file that doesn't involve me having to remember to switch it back to the production version everytime I want to push to Heroku?
You can set conditional things using the env option in your .babelrc
from their docs:
{
"stage": 0,
"env": {
"development": {
"ignore": [
"node_modules",
"bower_components",
"testing",
"test"
]
}
}
}
Then, in your package.json
"scripts": {
"start": "NODE_ENV=production node index.js",
"dev": "NODE_ENV=development node index.js"
}
it checks BABEL_ENV, then NODE_ENV

Resources