derby db upgrade file permission issue - derby

Question

I'm currently running a derby DB instance created from version 10.13.1.1
I connect via the network mode (startNetworkServer) running on a redhat server.
I'm now wanting to upgrade to version 10.14.2.0
However, when trying to connect to the upgraded database, I receive an access denied "java.io.FilePermission" error.
Details:
I went and downloaded both version 10.13.1.1 and 10.14.2.0 onto my windows desktop.
A backup of the database is created using the following command: SYSCS_UTIL.SYSCS_BACKUP_DATABASE
I copied this backup to both the 10.13 and 10.14 folders.
Starting with my current version (13) i start the network server, and then use ij to connect to the database. This works fine, and i can see the tables. This validates my backup is fine.
connect 'jdbc:derby://localhost:1527/c:\Temp\13\database;create=false';
I then start my 14 versions network server, and then go to 14's ij. When I try to connect to the backup:
connect 'jdbc:derby://localhost:1527/c:\Temp\14\database;create=false';
I get the filePermission error:
ERROR XJ001: DERBY SQL error: ERRORCODE: 0, SQLSTATE: XJ001, SQLERRMC: java.security.AccessControlException
access denied ("java.io.FilePermission" "C:\Temp\updating_derby\threatadvisor" "read")
XJ001.U
Fair enough, I assume this is because i'm trying to connect to an older version, without having run the upgrade=true parameter. When I remove the create parameter, and add the upgrade parameter, it still fails with the same issue.
Ok, so perhaps I can't upgrade a DB via the network server, and I have to directly connect to the DB. From within my app, I use the following connection string:
jdbc:derby:C:/Temp/14/database;upgrade=true;
The app has the version 14 jar on the classpath, so should use it and upgrade. Which it does, the app starts normally and I see all the data. How do I know it upgraded? Because I tried to connect to this 14 database using 13 network server and ij, and it fails (as expected due to version).
So i'm done right? No, I once more try to connect to this now upgraded database via the network server, using ij and i once again get the java.io.FilePermission issue.
I went in and ensured the actual OS permissions on the folders and files inside the "database" folder are not just read-only. None are. Yet still it errors.
I've even tried running 14 network server on the redhat box (on a different port), and trying to connect to this db via ij and even there i get the file permission issue.
I'm really at a loss as to what to do next. Please help!
FYI, the full issue from the derby.log file:
Tue Jun 11 12:04:15 AEST 2019 : Apache Derby Network Server - 10.14.2.0 - (1828579) started and ready to accept connections on port 1527
Tue Jun 11 12:04:28 AEST 2019 Thread[DRDAConnThread_2,5,main] Cleanup action starting
java.security.AccessControlException: access denied ("java.io.FilePermission" "C:\Temp\14\database" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.security.AccessController.checkPermission(AccessController.java:884)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
at java.io.File.exists(File.java:814)
at java.io.WinNTFileSystem.canonicalize(WinNTFileSystem.java:434)
at java.io.File.getCanonicalPath(File.java:618)
at org.apache.derby.impl.io.DirStorageFactory.doInit(Unknown Source)
at org.apache.derby.impl.io.BaseStorageFactory.init(Unknown Source)
at org.apache.derby.impl.io.DirStorageFactory.init(Unknown Source)
at org.apache.derby.impl.services.monitor.StorageFactoryService.privGetStorageFactoryInstance(Unknown Source)
at org.apache.derby.impl.services.monitor.StorageFactoryService.access$400(Unknown Source)
at org.apache.derby.impl.services.monitor.StorageFactoryService$12.run(Unknown Source)
at org.apache.derby.impl.services.monitor.StorageFactoryService$12.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.derby.impl.services.monitor.StorageFactoryService.getCanonicalServiceName(Unknown Source)
at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.(Unknown Source)
at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.EmbeddedDriver.connect(Unknown Source)
at org.apache.derby.impl.drda.Database.makeConnection(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.getConnFromDatabaseName(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.verifyUserIdPassword(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseSECCHK(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
Cleanup action completed
EDIT 1
Now trying to setup the security.policy file as per this guide. However, after creating a new policy file based off the template in the demo directory, we can't even get derby to pick up our file.
When we try to run:
java -classpath "C:\Temp\14\lib\derby.jar;C:\Temp\14\lib\derbynet.jar;C:\Temp\14\lib\derbyclient.jar;C:\Temp\14\lib\derbytools.jar;C:\Temp\14\lib\derbyoptionaltools.jar" -Djava.security.manager -Djava.security.policy=C:\Temp\14\server.policy org.apache.derby.drda.NetworkServerControl start
We get the following error:
java.security.AccessControlException: access denied org.apache.derby.security.SystemPermission( "engine", "usederbyinternals" )
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.security.AccessController.checkPermission(AccessController.java:884)
at org.apache.derby.iapi.security.SecurityUtil.checkDerbyInternalsPrivilege(Unknown Source)
at org.apache.derby.iapi.services.monitor.Monitor.getMonitorLite(Unknown Source)
at org.apache.derby.iapi.services.property.PropertyUtil$2.run(Unknown Source)
at org.apache.derby.iapi.services.property.PropertyUtil$2.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.derby.iapi.services.property.PropertyUtil.getMonitorLite(Unknown Source)
at org.apache.derby.iapi.services.property.PropertyUtil.getSystemProperty(Unknown Source)
at org.apache.derby.iapi.services.property.PropertyUtil.getSystemProperty(Unknown Source)
at org.apache.derby.impl.drda.NetworkServerControlImpl.init(Unknown Source)
at org.apache.derby.impl.drda.NetworkServerControlImpl.(Unknown Source)
at org.apache.derby.drda.NetworkServerControl.main(Unknown Source)
I know this line is in the policy file (and uncommented):
permission org.apache.derby.security.SystemPermission "engine", "usederbyinternals";
However, I don't think it is even picking up our policy file, as if we change our reference to a non-existing policy file, we still get the same error.

Answer 1

Thanks to #BryanPendleton for pointing me in the right direction. For the initial issue, it was indeed because we needed the server.policy file. His link was helpful:
db.apache.org/derby/docs/10.14/security/csecjavasecurity.html
The second issue which we were having was resolved by using the server.policy file template located here:
https://builds.apache.org/job/Derby-docs/lastSuccessfulBuild/artifact/trunk/out/security/rsecbasicserver.html
Instead of the one provided in the download (the one in the derby download didn't have as many jars mentioned in it). More to the point, the way we referenced the jars had to be tweaked. You will see all the examples were for unix format, whereas we were developing on a test windows PC. Therefore instead of something like (unix):
grant codeBase "file:///home/someone/derby/lib/derby.jar"
We needed to do:
grant codeBase "file:///C:/Temp/14/lib/derby.jar"
Note the additional '/' after 'file' - we had assumed it was merely "file://C:...."

Answer 2

There is another solution to the problem which is to use this code:
https://github.com/apache/hive/blob/master/core/src/test/java/org/apache/hive/hcatalog/DerbyPolicy.java
and use this:
Policy.setPolicy(new DerbyPolicy());
To get a policy set programmatically.