Heroku not using cache for Yarn v3 - heroku

Question

Trying to get Heroku to use cached modules for Yarn v3 using workspaces. I have the following:
package.json:
"engines": {
"node": "16.x",
"yarn": "3.x"
},
"cacheDirectories": [
"node_modules",
"packages/components/node_modules",
"packages/lib/node_modules",
"packages/schema/node_modules",
"packages/web/node_modules",
"packages/web/.next/cache"
]
heroku-buildpack-features:
cache-native-yarn-cache=true
Heroku output:
-----> Creating runtime environment
NPM_CONFIG_LOGLEVEL=error
USE_YARN_CACHE=true
NODE_VERBOSE=false
NODE_ENV=production
NODE_MODULES_CACHE=true
-----> Installing binaries
! You don't need to specify Yarn engine. Heroku will install the latest Yarn 1.x, so that per project version can be used. More information here: https://yarnpkg.com/getting-started/install#global-install
https://devcenter.heroku.com/articles/nodejs-support
engines.node (package.json): 16.x
engines.npm (package.json): unspecified (use default)
engines.yarn (package.json): unspecified (use default)
Resolving node version 16.x...
Downloading and installing node 16.7.0...
Using default npm version: 7.20.3
Resolving yarn version 1.22.x...
Downloading and installing yarn (1.22.11)
Using yarn 3.0.0
-----> Installing binaries
! You don't need to specify Yarn engine. Heroku will install the latest Yarn 1.x, so that per project version can be used. More information here: https://yarnpkg.com/getting-started/install#global-install
https://devcenter.heroku.com/articles/nodejs-support
engines.node (package.json): 16.x
engines.npm (package.json): unspecified (use default)
engines.yarn (package.json): unspecified (use default)
Resolving node version 16.x...
Downloading and installing node 16.7.0...
Using default npm version: 7.20.3
Resolving yarn version 1.22.x...
Downloading and installing yarn (1.22.11)
Using yarn 3.0.0
-----> Restoring cache
Loading 6 from cacheDirectories (package.json):
- node_modules
- packages/components/node_modules (not cached - skipping)
- packages/lib/node_modules
- packages/schema/node_modules
- packages/web/node_modules
- packages/web/.next/cache
<a bunch of resolve stuff>
➤ YN0000: ┌ Fetch step
➤ YN0013: │ #apollo/client#npm:3.4.7 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ #apollo/protobufjs#npm:1.2.2 can't be found in the cache and will be fetched from the remote registry
<and so on>
I've tried adding ".yarn/cache" to cacheDirectories in package.json, but it always says it's empty.
Not really sure where to go from here.

Answer 1

As of 9/2/2021, Heroku's nodeJS buildback doesn't fully support yarn 2. They started adding support in v173, but caching hasn't been fully addressed yet.
The underlying issue is that the buildpack doesn't properly detect that $BUILD_DIR/.yarn/cache exists and sets the environment variable YARN_CACHE_FOLDER to something other than .yarn/cache.(see
https://github.com/heroku/heroku-buildpack-nodejs/blob/752f1d5a139a800920cf5bb1bf70aad1a2954525/bin/compile#L140)
When the buildpack gets to the cache step the .yarn/cache directory doesn't actually exist.
The solution:
I was able to work around this by manually setting YARN_CACHE_FOLDER to .yarn/cache in the config vars. You can do this via the CLI by running heroku config:set YARN_CACHE_FOLDER=.yarn/cache.
Ensure you also have .yarn/cache listed in cacheDirectories.
You will know it works when the build logs show:
-----> Caching build
Saving 1 cacheDirectories (package.json):
- .yarn/cache
If it did not work there would be a (nothing to cache) note next to .yarn/cache.