Heroku node_modules not cached - heroku

I have toplevel package.json where I set cacheDirectories as per https://devcenter.heroku.com/articles/nodejs-support#custom-caching
"scripts": {
"start": "npm --prefix frontend start && npm --prefix mockserver start",
},
"dependencies": {
},
"devDependencies": {
},
"cacheDirectories": ["frontend/node_modules", "mockserver/node_modules"],
...
So no dependencies in toplevel, but I have frontend/package.json and mockserver/package.json with a lot of dependencies that get installed in their respective node_modules. This is how the respective package.jsons look like (for example mockserver):
"scripts": {
"start": "npm install && npm run build && node dist/server.js",
},
dependencies...
devdependencies...
This works fine and deploys, but it does not cache the the two node_modules defined to be cached.
Here is the output from heroku build:
-----> Creating runtime environment
NPM_CONFIG_LOGLEVEL=error
NPM_CONFIG_PRODUCTION=true
NODE_VERBOSE=false
NODE_ENV=demo
NODE_MODULES_CACHE=true
npm scripts will see NODE_ENV=production (not 'demo')
https://docs.npmjs.com/misc/config#production
-----> Installing binaries
engines.node (package.json): 7.10.0
engines.npm (package.json): unspecified (use default)
Downloading and installing node 7.10.0...
Using default npm version: 4.2.0
-----> Restoring cache
Loading 2 from cacheDirectories (package.json):
- frontend/node_modules (not cached - skipping)
- mockserver/node_modules (not cached - skipping)
-----> Building dependencies
Installing node modules (package.json)
-----> Caching build
Clearing previous node cache
Saving 2 cacheDirectories (package.json):
- frontend/node_modules (nothing to cache)
- mockserver/node_modules (nothing to cache)
-----> Build succeeded!
Just to note that the two node_modules were installed in the previous build but it says (not cached - skipping).
Thanks in advance!

I got the answer here: https://github.com/heroku/heroku-buildpack-nodejs/issues/435
In summary, I should have used heroku-postbuild instead of start script to install custom dependencies. And use start only as application starting command.
Top level package.json
"scripts": {
"heroku-postbuild": "npm --prefix frontend run installAndCompile && npm --prefix mockserver run installAndCompile",
"start": "npm --prefix mockserver start"
},
Subproject's package.json
"scripts": {
"installAndCompile": "npm install && npm run compile"
},

Related

Heroku not using cache for Yarn v3

Trying to get Heroku to use cached modules for Yarn v3 using workspaces. I have the following:
package.json:
"engines": {
"node": "16.x",
"yarn": "3.x"
},
"cacheDirectories": [
"node_modules",
"packages/components/node_modules",
"packages/lib/node_modules",
"packages/schema/node_modules",
"packages/web/node_modules",
"packages/web/.next/cache"
]
heroku-buildpack-features:
cache-native-yarn-cache=true
Heroku output:
-----> Creating runtime environment
NPM_CONFIG_LOGLEVEL=error
USE_YARN_CACHE=true
NODE_VERBOSE=false
NODE_ENV=production
NODE_MODULES_CACHE=true
-----> Installing binaries
! You don't need to specify Yarn engine. Heroku will install the latest Yarn 1.x, so that per project version can be used. More information here: https://yarnpkg.com/getting-started/install#global-install
https://devcenter.heroku.com/articles/nodejs-support
engines.node (package.json): 16.x
engines.npm (package.json): unspecified (use default)
engines.yarn (package.json): unspecified (use default)
Resolving node version 16.x...
Downloading and installing node 16.7.0...
Using default npm version: 7.20.3
Resolving yarn version 1.22.x...
Downloading and installing yarn (1.22.11)
Using yarn 3.0.0
-----> Installing binaries
! You don't need to specify Yarn engine. Heroku will install the latest Yarn 1.x, so that per project version can be used. More information here: https://yarnpkg.com/getting-started/install#global-install
https://devcenter.heroku.com/articles/nodejs-support
engines.node (package.json): 16.x
engines.npm (package.json): unspecified (use default)
engines.yarn (package.json): unspecified (use default)
Resolving node version 16.x...
Downloading and installing node 16.7.0...
Using default npm version: 7.20.3
Resolving yarn version 1.22.x...
Downloading and installing yarn (1.22.11)
Using yarn 3.0.0
-----> Restoring cache
Loading 6 from cacheDirectories (package.json):
- node_modules
- packages/components/node_modules (not cached - skipping)
- packages/lib/node_modules
- packages/schema/node_modules
- packages/web/node_modules
- packages/web/.next/cache
<a bunch of resolve stuff>
➤ YN0000: ┌ Fetch step
➤ YN0013: │ #apollo/client#npm:3.4.7 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ #apollo/protobufjs#npm:1.2.2 can't be found in the cache and will be fetched from the remote registry
<and so on>
I've tried adding ".yarn/cache" to cacheDirectories in package.json, but it always says it's empty.
Not really sure where to go from here.
As of 9/2/2021, Heroku's nodeJS buildback doesn't fully support yarn 2. They started adding support in v173, but caching hasn't been fully addressed yet.
The underlying issue is that the buildpack doesn't properly detect that $BUILD_DIR/.yarn/cache exists and sets the environment variable YARN_CACHE_FOLDER to something other than .yarn/cache.(see
https://github.com/heroku/heroku-buildpack-nodejs/blob/752f1d5a139a800920cf5bb1bf70aad1a2954525/bin/compile#L140)
When the buildpack gets to the cache step the .yarn/cache directory doesn't actually exist.
The solution:
I was able to work around this by manually setting YARN_CACHE_FOLDER to .yarn/cache in the config vars. You can do this via the CLI by running heroku config:set YARN_CACHE_FOLDER=.yarn/cache.
Ensure you also have .yarn/cache listed in cacheDirectories.
You will know it works when the build logs show:
-----> Caching build
Saving 1 cacheDirectories (package.json):
- .yarn/cache
If it did not work there would be a (nothing to cache) note next to .yarn/cache.

npm run dev can not install webpack cli

I have a laravel application with the below config:
laravel version 7.30.4
npm version 6.14.6
node version 12.18.3
When I want run npm run dev, this question is asked:
CLI for webpack must be installed.
webpack-cli (https://github.com/webpack/webpack-cli)
Do you want to install 'webpack-cli' (yes/no):
After typing yes, this error is showed:
Error: Cannot find module 'webpack-cli/package.json'
Require stack:
- C:\Users\SibCo\AppData\Roaming\npm-cache\_npx\5852\node_modules\webpack\bin\webpack.js
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:965:15)
at Function.resolve (internal/modules/cjs/helpers.js:78:19)
at runCli (C:\Users\SibCo\AppData\Roaming\npm-cache\_npx\5852\node_modules\webpack\bin\webpack.js:50:26)
at C:\Users\SibCo\AppData\Roaming\npm-cache\_npx\5852\node_modules\webpack\bin\webpack.js:139:5
at processTicksAndRejections (internal/process/task_queues.js:97:5) {
code: 'MODULE_NOT_FOUND',
requireStack: [
'C:\\Users\\SibCo\\AppData\\Roaming\\npm-cache\\_npx\\5852\\node_modules\\webpack\\bin\\webpack.js'
]
}
Try installing webpack globally:
npm install -g webpack webpack-cli
If you're using webpack-cli 4 or webpack 5, change webpack-dev-server to webpack serve.
Example:
"serve": "webpack serve --config config/webpack.dev.js --progress"
You might want also to check this comment on GitHub:
NPM package.json scripts are a convenient and useful means to run locally installed binaries without having to be concerned about their full paths. Simply define a script as such:
For webpack-cli 3.x:
"scripts": { "start:dev": "webpack-dev-server" }
For webpack-cli 4.x:
"scripts": { "start:dev": "webpack serve" }
I updated my npm version to v7.11.2 and delete node_modules and ran :
npm install
again, and it worked for me.

Heroku deployment is failing even though app is running

I have deployed my app from git to Heroku successfully and I enabled automatic deployment that worked fine.
Somewhere in the past 2 weeks the deployment has started failing each time for the same reason, even though there's no problem with the code in the git - it's running fine.
You can see in the log file that it mentions each time:
" Creating an optimized production build...
Failed to compile.
./src/App.js
Cannot find file './pages/MainPage' in './src'.
But the MainPage file is right where it should be and the app is compiling and running locally.
I've tried disconnecting and reconnecting from git and heroku,
I've deleted and re-installed the node modules and the package-lock,
and I cache clean --force.
I don't know what went wrong, any suggestions?
Thank you!
-----> React.js (create-react-app) multi app detected
=====> Downloading Buildpack: https://github.com/heroku/heroku-buildpack-multi.git
=====> Detected Framework: Multipack
=====> Downloading Buildpack: https://github.com/heroku/heroku-buildpack-nodejs.git
=====> Detected Framework: Node.js
-----> Creating runtime environment
NPM_CONFIG_LOGLEVEL=error
NODE_ENV=production
NODE_MODULES_CACHE=true
NODE_VERBOSE=false
-----> Installing binaries
engines.node (package.json): unspecified
engines.npm (package.json): unspecified (use default)
Resolving node version 12.x...
Downloading and installing node 12.20.0...
Using default npm version: 6.14.8
-----> Restoring cache
Cached directories were not restored due to a change in version of node, npm, yarn or stack
Module installation may take longer for this build
-----> Installing dependencies
Installing node modules
> core-js#2.6.12 postinstall /tmp/build_826a0ec4_/node_modules/babel-runtime/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"
> fsevents#1.2.13 install /tmp/build_826a0ec4_/node_modules/webpack-dev-server/node_modules/fsevents
> node install.js
Skipping 'fsevents' build as platform linux is not supported
> fsevents#1.2.13 install /tmp/build_826a0ec4_/node_modules/watchpack-chokidar2/node_modules/fsevents
> node install.js
Skipping 'fsevents' build as platform linux is not supported
> core-js#3.8.0 postinstall /tmp/build_826a0ec4_/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"
> fsevents#1.2.13 install /tmp/build_826a0ec4_/node_modules/jest-haste-map/node_modules/fsevents
> node install.js
Skipping 'fsevents' build as platform linux is not supported
> core-js-pure#3.8.0 postinstall /tmp/build_826a0ec4_/node_modules/core-js-pure
> node -e "try{require('./postinstall')}catch(e){}"
added 1695 packages in 29.379s
-----> Build
Running build
> rosenflix#0.1.0 build /tmp/build_826a0ec4_
> react-scripts build
Creating an optimized production build...
Failed to compile.
./src/App.js
Cannot find file './pages/MainPage' in './src'.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! rosenflix#0.1.0 build: `react-scripts build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the rosenflix#0.1.0 build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /tmp/npmcache.Zqszt/_logs/2020-12-02T21_16_51_485Z-debug.log
-----> Build failed
We're sorry this build is failing! You can troubleshoot common issues here:
https://devcenter.heroku.com/articles/troubleshooting-node-deploys
Some possible problems:
- Node version not specified in package.json
https://devcenter.heroku.com/articles/nodejs-support#specifying-a-node-js-version
Love,
Heroku
! Push rejected, failed to compile React.js (create-react-app) multi app.
! Push failed
Your production build may be referencing a relative import that for some reason does not exist. Reconfigure that import using ${__dirname}/path_to_file and try again.
Here is a link of how to use it.
Hope this works for you.

Hi. I created Quasar App - Website. Trying to push it to Heroku and it says Push fail. Providing console with error

I'm trying to push my git to Heroku but it gives an error right at the end. I had a look on console of deploying a project as I understand warnings are not a big deal for crashing the whole app. As I went through console I notice that critical error is
quasar: not found
error Command failed with exit code 127.
As far as I think is something wrong with npm or yarn package but don't really know what exactly. I specified in json file same version of node what I have and same version of npm what I have on my computer. I don't use yarn, could that be an issue ?
Here is my json
{
"name": "portfolio",
"version": "0.0.1",
"description": "A Quasar Framework app",
"productName": "Eldar Tailov",
"author": "'Eldar' <'tailov47#gmail.com'>",
"private": true,
"scripts": {
"build": "quasar build",
"start": "node server.js",
"heroku-postbuild": "yarn && yarn build"
},
"dependencies": {
"#quasar/extras": "^1.0.0",
"axios": "^0.20.0",
"connect-history-api-fallback": "^1.6.0",
"core-js": "^3.6.5",
"express": "^4.17.1",
"quasar": "^1.0.0",
"serve-static": "^1.14.1",
"vue2-google-maps": "^0.10.7"
},
"devDependencies": {
"#quasar/app": "^2.0.0",
"babel-eslint": "^10.0.1",
"eslint": "^6.8.0",
"eslint-config-prettier": "^6.9.0",
"eslint-loader": "^3.0.3",
"eslint-plugin-vue": "^6.1.2",
"workbox-webpack-plugin": "^5.1.4"
},
"browserslist": [
"last 10 Chrome versions",
"last 10 Firefox versions",
"last 4 Edge versions",
"last 7 Safari versions",
"last 8 Android versions",
"last 8 ChromeAndroid versions",
"last 8 FirefoxAndroid versions",
"last 10 iOS versions",
"last 5 Opera versions"
],
"engines": {
"node": "~12.18.2",
"npm": "~6.14.5",
"yarn": ">= 1.21.1"
}
}
and here is a log of heroku
-----> Node.js app detected
-----> Creating runtime environment
NPM_CONFIG_LOGLEVEL=error
NODE_ENV=production
NODE_MODULES_CACHE=true
NODE_VERBOSE=false
-----> Installing binaries
engines.node (package.json): ~12.18.2
engines.npm (package.json): ~6.14.5
engines.yarn (package.json): >= 1.21.1
Resolving node version ~12.18.2...
Downloading and installing node 12.18.4...
Bootstrapping npm ~6.14.5 (replacing 6.14.6)...
npm ~6.14.5 installed
Resolving yarn version >= 1.21.1...
Downloading and installing yarn (1.22.5)
Installed yarn 1.22.5
-----> Installing dependencies
Installing node modules
> core-js#3.6.5 postinstall /tmp/build_bc8b4d0e_/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"
> fsevents#1.2.13 install /tmp/build_bc8b4d0e_/node_modules/webpack-dev-server/node_modules/fsevents
> node install.js
Skipping 'fsevents' build as platform linux is not supported
> ejs#2.7.4 postinstall /tmp/build_bc8b4d0e_/node_modules/ejs
> node ./postinstall.js
> fsevents#1.2.13 install /tmp/build_bc8b4d0e_/node_modules/watchpack-chokidar2/node_modules/fsevents
> node install.js
Skipping 'fsevents' build as platform linux is not supported
> node-sass#4.14.1 install /tmp/build_bc8b4d0e_/node_modules/node-sass
> node scripts/install.js
Downloading binary from https://github.com/sass/node-sass/releases/download/v4.14.1/linux-x64-72_binding.node
Download complete
Binary saved to /tmp/build_bc8b4d0e_/node_modules/node-sass/vendor/linux-x64-72/binding.node
Caching binary to /tmp/npmcache.XZGoT/_cacache/node-sass/4.14.1/linux-x64-72_binding.node
> node-sass#4.14.1 postinstall /tmp/build_bc8b4d0e_/node_modules/node-sass
> node scripts/build.js
Binary found at /tmp/build_bc8b4d0e_/node_modules/node-sass/vendor/linux-x64-72/binding.node
Testing binary
Binary is fine
> core-js#2.6.11 postinstall /tmp/build_bc8b4d0e_/node_modules/babel-runtime/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"
added 1590 packages in 23.801s
-----> Build
Detected both "build" and "heroku-postbuild" scripts
Running heroku-postbuild
> portfolio#0.0.1 heroku-postbuild /tmp/build_bc8b4d0e_
> yarn && yarn build
yarn install v1.22.5
info No lockfile found.
warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
[1/5] Validating package.json...
[2/5] Resolving packages...
warning vue2-google-maps > babel-runtime > core-js#2.6.11: core-js#<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js#3.
warning #quasar/app > webpack-dev-server > chokidar#2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
warning #quasar/app > webpack-dev-server > chokidar > fsevents#1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
warning #quasar/app > node-sass > request#2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
warning #quasar/app > node-sass > node-gyp > request#2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
warning #quasar/app > node-sass > request > har-validator#5.1.5: this library is no longer supported
warning #quasar/app > webpack > watchpack > watchpack-chokidar2 > chokidar#2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
warning #quasar/app > stylus > css-parse > css > urix#0.1.0: Please see https://github.com/lydell/urix#deprecated
warning #quasar/app > webpack-dev-server > chokidar > braces > snapdragon > source-map-resolve > urix#0.1.0: Please see https://github.com/lydell/urix#deprecated
warning #quasar/app > webpack-dev-server > chokidar > braces > snapdragon > source-map-resolve > resolve-url#0.2.1: https://github.com/lydell/resolve-url#deprecated
warning eslint-loader#3.0.4: This loader has been deprecated. Please use eslint-webpack-plugin
warning workbox-webpack-plugin > workbox-build > rollup-plugin-babel#4.4.0: This package has been deprecated and is no longer maintained. Please use #rollup/plugin-babel.
warning workbox-webpack-plugin > workbox-build > #hapi/joi#15.1.1: joi is leaving the #hapi organization and moving back to 'joi' (https://github.com/sideway/joi/issues/2411)
warning workbox-webpack-plugin > workbox-build > #hapi/joi > #hapi/address#2.1.4: This version has been deprecated and is no longer supported or maintained
warning workbox-webpack-plugin > workbox-build > #hapi/joi > #hapi/bourne#1.3.2: This version has been deprecated and is no longer supported or maintained
warning workbox-webpack-plugin > workbox-build > #hapi/joi > #hapi/topo#3.1.6: This version has been deprecated and is no longer supported or maintained
warning workbox-webpack-plugin > workbox-build > #hapi/joi > #hapi/hoek#8.5.1: This version has been deprecated and is no longer supported or maintained
warning workbox-webpack-plugin > workbox-build > #hapi/joi > #hapi/topo > #hapi/hoek#8.5.1: This version has been deprecated and is no longer supported or maintained
[3/5] Fetching packages...
info fsevents#2.1.3: The platform "linux" is incompatible with this module.
info "fsevents#2.1.3" is an optional dependency and failed compatibility check. Excluding it from installation.
info fsevents#1.2.13: The platform "linux" is incompatible with this module.
info "fsevents#1.2.13" is an optional dependency and failed compatibility check. Excluding it from installation.
[4/5] Linking dependencies...
warning " > eslint-loader#3.0.4" has unmet peer dependency "webpack#^4.0.0 || ^5.0.0".
warning " > workbox-webpack-plugin#5.1.4" has unmet peer dependency "webpack#^4.0.0".
[5/5] Building fresh packages...
success Saved lockfile.
Done in 45.19s.
yarn run v1.22.5
$ quasar build
/bin/sh: 1: quasar: not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
npm ERR! code ELIFECYCLE
npm ERR! syscall spawn
npm ERR! file sh
npm ERR! errno ENOENT
npm ERR! portfolio#0.0.1 heroku-postbuild: `yarn && yarn build`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the portfolio#0.0.1 heroku-postbuild script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /tmp/npmcache.XZGoT/_logs/2020-09-28T01_01_55_709Z-debug.log
-----> Build failed
We're sorry this build is failing! You can troubleshoot common issues here:
https://devcenter.heroku.com/articles/troubleshooting-node-deploys
If you're stuck, please submit a ticket so we can help:
https://help.heroku.com/
Love,
Heroku
! Push rejected, failed to compile Node.js app.
! Push failed
I faced exactly the same issue as you did. I was able to fix it from the suggestions in this link. I think it's because the #quasar/app was listed in the dev dependencies.
https://devcenter.heroku.com/articles/nodejs-support#heroku-specific-build-steps
Basically, I updated the engine part of package.json to match my local env
"engines": {
"node": ">= 12.18.3",
"npm": ">= 6.14.11",
"yarn": ">= 1.22.5"
}
and then set the YARN_PRODUCTION env variable on heroku to false
heroku config:set YARN_PRODUCTION=false
I hope you'll find this useful (P.S. Of course don't set a static value in the engine part, use wildcard instead)
Quasar is probably not installed globally on heroku,
try to run it using npx:
"scripts": {
...
"build": "npx quasar build -m ssr",
"start": "cd dist/ssr; npm i; npm start"
}

Problem with deplyoing application on Heroku

I'm new to Heroku
I have been trying to deploy my app to Heroku but I'm having an issue which I can't resolve
My build passes and the app gets deployed but there is an error which I can't find nor resolve
I'm not really sure where to look nor what to look for since I'm literally too new to Heroku
Here is my build log
-----> Node.js app detected
-----> Creating runtime environment
NPM_CONFIG_LOGLEVEL=error
NODE_ENV=production
NODE_MODULES_CACHE=true
NODE_VERBOSE=false
-----> Installing binaries
engines.node (package.json): 12.x
engines.npm (package.json): 6.x
Resolving node version 12.x...
Downloading and installing node 12.18.3
Bootstrapping npm 6.x (replacing 6.14.6)...
npm 6.x installed
-----> Restoring cache
- node_modules
-----> Installing dependencies
Installing node modules
added 39 packages in 1.246s
-----> Build
-----> Caching build
- node_modules
-----> Pruning devDependencies
audited 39 packages in 0.584s
1 package is looking for funding
run `npm fund` for details
found 0 vulnerabilities
-----> Build succeeded!
-----> Discovering process types
Procfile declares types -> (none)
Default types for buildpack -> web
-----> Compressing...
Done: 23.4M
-----> Launching...
Released v5
https://ferit-virtual-classroom.herokuapp.com/ deployed to Heroku
And here is my package.json
{
"name": "ferit-bot",
"version": "1.0.0",
"description": "",
"publishConfig": { "registry": "https://npm.pkg.github.com/" },
"author": "JaVolimKatarinu",
"license": "ISC",
"dependencies": {
"discord.js": "^12.1.1",
"mysql2": "^2.1.0",
"node-fetch-npm": "^2.0.4"
},
"engines":{
"node": "12.x",
"npm": "6.x"
},
"scripts": {
"start": "main.js"
}
}
I have searched for a potential answer on Heroku support but without any luck
Thank you for reading and I would be really glad for a helpful answer
EDIT:All good, I have managed to deploy the app and its working
Thanks for help
The scripts in package-json file is not properly setup.
There must always be a start command which can be anything to be run in the terminal, but for yours, the script should be
"scripts": {
"start": "node main.js"
}
node keyword is ignored, thats why?
did you add a Procfile ?
web: node main.js

Resources