Cannot restart Jenkins on Windows Server 2012 (port in use) - windows

I'm running Jenkins on a Windows Server 2012 which runs ok, but I can't restart Jenkins e.g. after updating plugins.
The error I get in the jenkins.err.log is
SEVERE winstone.Logger#logInternal: Container startup failed
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
Caused: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8080
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:308)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:396)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at winstone.Launcher.<init>(Launcher.java:187)
Caused: java.io.IOException: Failed to start Jetty
at winstone.Launcher.<init>(Launcher.java:189)
at winstone.Launcher.main(Launcher.java:362)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at Main._main(Main.java:375)
at Main.main(Main.java:151)
So apparently the port that Jenkins was running on (8080) is still in use after the restart.
The server console gives me this:
Child process [5424 - C:\Program Files (x86)\Jenkins\jre\bin\java -Xrs -Xmx512m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -Dhudson.model.DirectoryBrowserSupport.CSP="sandbox; default-src 'none'; img-src 'self'; style-src 'self' 'unsafe-inline';" -jar "C:\Program Files (x86)\Jenkins\jenkins.war" --httpPort=8080 --ajp13Port=-1 --webroot="C:\Program Files (x86)\Jenkins\war"] finished with 1
I have checked which process is using port 8080 using netstat, but can't find anything.
I know I can change the port when starting Jenkins using the "httpPort"-flag. Doing so (e.g. changing the port to 8081) will work - until I restart Jenkins again, this time 8081 will be in use (and also 8080 and all other ports I've been using before). This can only be resolved by restarting the server, then 8080 (and all other ports) will be available again.
Both "solutions" (1. keep changing the port, 2. restarting the server) are just tedious workarounds for me as I have to inform all colleagues which port is currently in use all the time or let administration restart the server over and over again.
The flag "--ajp13Port=-1" is an attempt to get this fixed, I also tried disabling the "RunawayProcessKiller"-extension as suggested in another post, but both didn't change anything.
Any hint to get this fixed is much appreciated.

Related

derby db upgrade file permission issue

I'm currently running a derby DB instance created from version 10.13.1.1
I connect via the network mode (startNetworkServer) running on a redhat server.
I'm now wanting to upgrade to version 10.14.2.0
However, when trying to connect to the upgraded database, I receive an access denied "java.io.FilePermission" error.
Details:
I went and downloaded both version 10.13.1.1 and 10.14.2.0 onto my windows desktop.
A backup of the database is created using the following command: SYSCS_UTIL.SYSCS_BACKUP_DATABASE
I copied this backup to both the 10.13 and 10.14 folders.
Starting with my current version (13) i start the network server, and then use ij to connect to the database. This works fine, and i can see the tables. This validates my backup is fine.
connect 'jdbc:derby://localhost:1527/c:\Temp\13\database;create=false';
I then start my 14 versions network server, and then go to 14's ij. When I try to connect to the backup:
connect 'jdbc:derby://localhost:1527/c:\Temp\14\database;create=false';
I get the filePermission error:
ERROR XJ001: DERBY SQL error: ERRORCODE: 0, SQLSTATE: XJ001, SQLERRMC: java.security.AccessControlException
access denied ("java.io.FilePermission" "C:\Temp\updating_derby\threatadvisor" "read")
XJ001.U
Fair enough, I assume this is because i'm trying to connect to an older version, without having run the upgrade=true parameter. When I remove the create parameter, and add the upgrade parameter, it still fails with the same issue.
Ok, so perhaps I can't upgrade a DB via the network server, and I have to directly connect to the DB. From within my app, I use the following connection string:
jdbc:derby:C:/Temp/14/database;upgrade=true;
The app has the version 14 jar on the classpath, so should use it and upgrade. Which it does, the app starts normally and I see all the data. How do I know it upgraded? Because I tried to connect to this 14 database using 13 network server and ij, and it fails (as expected due to version).
So i'm done right? No, I once more try to connect to this now upgraded database via the network server, using ij and i once again get the java.io.FilePermission issue.
I went in and ensured the actual OS permissions on the folders and files inside the "database" folder are not just read-only. None are. Yet still it errors.
I've even tried running 14 network server on the redhat box (on a different port), and trying to connect to this db via ij and even there i get the file permission issue.
I'm really at a loss as to what to do next. Please help!
FYI, the full issue from the derby.log file:
Tue Jun 11 12:04:15 AEST 2019 : Apache Derby Network Server - 10.14.2.0 - (1828579) started and ready to accept connections on port 1527
Tue Jun 11 12:04:28 AEST 2019 Thread[DRDAConnThread_2,5,main] Cleanup action starting
java.security.AccessControlException: access denied ("java.io.FilePermission" "C:\Temp\14\database" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.security.AccessController.checkPermission(AccessController.java:884)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
at java.io.File.exists(File.java:814)
at java.io.WinNTFileSystem.canonicalize(WinNTFileSystem.java:434)
at java.io.File.getCanonicalPath(File.java:618)
at org.apache.derby.impl.io.DirStorageFactory.doInit(Unknown Source)
at org.apache.derby.impl.io.BaseStorageFactory.init(Unknown Source)
at org.apache.derby.impl.io.DirStorageFactory.init(Unknown Source)
at org.apache.derby.impl.services.monitor.StorageFactoryService.privGetStorageFactoryInstance(Unknown Source)
at org.apache.derby.impl.services.monitor.StorageFactoryService.access$400(Unknown Source)
at org.apache.derby.impl.services.monitor.StorageFactoryService$12.run(Unknown Source)
at org.apache.derby.impl.services.monitor.StorageFactoryService$12.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.derby.impl.services.monitor.StorageFactoryService.getCanonicalServiceName(Unknown Source)
at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.(Unknown Source)
at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.EmbeddedDriver.connect(Unknown Source)
at org.apache.derby.impl.drda.Database.makeConnection(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.getConnFromDatabaseName(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.verifyUserIdPassword(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseSECCHK(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
Cleanup action completed
EDIT 1
Now trying to setup the security.policy file as per this guide. However, after creating a new policy file based off the template in the demo directory, we can't even get derby to pick up our file.
When we try to run:
java -classpath "C:\Temp\14\lib\derby.jar;C:\Temp\14\lib\derbynet.jar;C:\Temp\14\lib\derbyclient.jar;C:\Temp\14\lib\derbytools.jar;C:\Temp\14\lib\derbyoptionaltools.jar" -Djava.security.manager -Djava.security.policy=C:\Temp\14\server.policy org.apache.derby.drda.NetworkServerControl start
We get the following error:
java.security.AccessControlException: access denied org.apache.derby.security.SystemPermission( "engine", "usederbyinternals" )
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.security.AccessController.checkPermission(AccessController.java:884)
at org.apache.derby.iapi.security.SecurityUtil.checkDerbyInternalsPrivilege(Unknown Source)
at org.apache.derby.iapi.services.monitor.Monitor.getMonitorLite(Unknown Source)
at org.apache.derby.iapi.services.property.PropertyUtil$2.run(Unknown Source)
at org.apache.derby.iapi.services.property.PropertyUtil$2.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.derby.iapi.services.property.PropertyUtil.getMonitorLite(Unknown Source)
at org.apache.derby.iapi.services.property.PropertyUtil.getSystemProperty(Unknown Source)
at org.apache.derby.iapi.services.property.PropertyUtil.getSystemProperty(Unknown Source)
at org.apache.derby.impl.drda.NetworkServerControlImpl.init(Unknown Source)
at org.apache.derby.impl.drda.NetworkServerControlImpl.(Unknown Source)
at org.apache.derby.drda.NetworkServerControl.main(Unknown Source)
I know this line is in the policy file (and uncommented):
permission org.apache.derby.security.SystemPermission "engine", "usederbyinternals";
However, I don't think it is even picking up our policy file, as if we change our reference to a non-existing policy file, we still get the same error.
Thanks to #BryanPendleton for pointing me in the right direction. For the initial issue, it was indeed because we needed the server.policy file. His link was helpful:
db.apache.org/derby/docs/10.14/security/csecjavasecurity.html
The second issue which we were having was resolved by using the server.policy file template located here:
https://builds.apache.org/job/Derby-docs/lastSuccessfulBuild/artifact/trunk/out/security/rsecbasicserver.html
Instead of the one provided in the download (the one in the derby download didn't have as many jars mentioned in it). More to the point, the way we referenced the jars had to be tweaked. You will see all the examples were for unix format, whereas we were developing on a test windows PC. Therefore instead of something like (unix):
grant codeBase "file:///home/someone/derby/lib/derby.jar"
We needed to do:
grant codeBase "file:///C:/Temp/14/lib/derby.jar"
Note the additional '/' after 'file' - we had assumed it was merely "file://C:...."
There is another solution to the problem which is to use this code:
https://github.com/apache/hive/blob/master/core/src/test/java/org/apache/hive/hcatalog/DerbyPolicy.java
and use this:
Policy.setPolicy(new DerbyPolicy());
To get a policy set programmatically.

Debug Jenkins Slave Setup?

I've got some desktops running Windows 7 that I'm trying to setup as Jenkins slaves. The slaves work if I connect using the command line provided on the slave status page, such as java -jar slave.jar -jnlpUrl http://jenkins-master.mycompany.com:8080/computer/slave1/slave-agent.jnlp -secret 42abcdef123456abef222ccc11. But if I connect using the java web start button on the status page it fails with an exception:
java.lang.NullPointerException
at com.sun.javaws.Launcher.prepareToLaunch(Unknown Source)
at com.sun.javaws.Launcher.prepareToLaunch(Unknown Source)
at com.sun.javaws.Launcher.launch(Unknown Source)
at com.sun.javaws.Main.launchApp(Unknown Source)
at com.sun.javaws.Main.continueInSecureThread(Unknown Source)
at com.sun.javaws.Main.access$000(Unknown Source)
at com.sun.javaws.Main$1.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
I get the same error on 3 different slave machines. In all three cases I'm launching from a browser running on the slave machine.
I'd like to have the slaves running as a Windows service. I could set it up so the service issues the java command line that does work, but I'd prefer to use the service facilities provided by jenkins itself. Plus I'd simply like to understand why the web start method isn't working. How do I go about debugging this?

Eclipse- Cannot assign requested address: JVM_Bind

I was trying to configure tomcat6 in eclipse. After configure tomcat in eclipse i have started server. It shows below error messages,
java.net.BindException: Cannot assign requested address: JVM_Bind
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383)
at java.net.ServerSocket.bind(ServerSocket.java:328)
at java.net.ServerSocket.<init>(ServerSocket.java:194)
at org.apache.catalina.core.StandardServer.await(StandardServer.java:373)
at org.apache.catalina.startup.Catalina.await(Catalina.java:662)
at org.apache.catalina.startup.Catalina.start(Catalina.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
After searching Google i checked with ports in command prompt. but none of the applications running in this port as below,
C:\>netstat -an | find "9000"
C:\>netstat -an | find "8005"
C:\>netstat -an | find "8009"
C:\>
Anything am i missed? If anything wrong please correct me.
It means those ports are not available at that time. There might be chance of having tomcat server in your PC, So try to Change the port numbers by double clicking on your apache server in servers tab. then your apache tomcat will run.
If you have changed you ip recently, the entry for localhost in the hosts file is invalid. Update the ip in your hosts file.

Error: Tomcat 7 Configuration as Windows Service via tomcat7.exe

I am attempting to run tomcat as a service. I am able to run it succesfully when I specify just a startpath or just jvmoptions. However, if I have both I am receiving the following error:
2013-04-25 11:34:01 Commons Daemon procrun stderr initialized
java.lang.ClassNotFoundException: org.apache.catalina.startup.Catalina
at java.net.URLClassLoader$1.run(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at org.apache.catalina.startup.Bootstrap.init(Bootstrap.java:236)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:426)
The commands I am running are as follows:
service.bat install WebServices
tomcat7 //US//WebServices --Startup=auto --DisplayName="Web Services"
tomcat7 //US//WebServices --StartPath="C:\Program Files\engine"
tomcat7 //US//WebServices --JvmOptions="-Xmx4g;-XX:PermSize=2048m;-XX:MaxPermSize=2048m"
As I stated, the service starts fine if I don't run the JvmOptions line, and it successfully writes log files to my startpath directory. Then if I don't include startpath it will also run correctly, and it just won't output to the startpath. The error only occurs when both variables are set.
The issue was that when I was changing the start path and assigning variables it was losing track of catalina home. So I added
-Dcatalina.home=apacheInstallLoc;-Dcatalina.base=apacheInstallLoc
to my JvmOptions and everything worked.

Deploying to tc Server in Idea

I'm trying to run my application in Idea using tc Server Idea plugin and receive the following log
Connected to server
wrapper | Starting the VMware vFabric tc Runtime instance - tcruntime-C-STS-vfabric-tc-server-developer-2.8.2.RELEASE-insight service...
wrapper | VMware vFabric tc Runtime instance - tcruntime-C-STS-vfabric-tc-server-developer-2.8.2.RELEASE-insight started.
[11:53:08.709] Not allowed to connect. Check role and password.
[11:53:08.709] There is an application at /bps. Starting redeploy...
[11:53:08.714] Not allowed to connect. Check role and password.
[11:53:08.715] Starting undeployment of /bps ...
[11:53:08.720] Not allowed to connect. Check role and password.
[11:53:08.721] Starting deployment of 'bps:war exploded' to /bps ...
[11:53:08.725] Not allowed to connect. Check role and password.
Can anybody describe what's the problem? I cann't fully understand how to config users and roles in tc Server.
UPDATE
The above problem is solved by enabling jmx. But now there is another problem
16-Apr-2013 14:36:36.023 SEVERE [RMI TCP Connection(4)-127.0.0.1] org.apache.tomcat.util.modeler.BaseModelMBean.invoke Exception invoking method deployApplication
java.lang.RuntimeException: Unable to find host
at com.springsource.tcserver.serviceability.deploy.TcContainerDeployer.deployApplication(TcContainerDeployer.java:409)
at com.springsource.tcserver.serviceability.deploy.TcContainerDeployer.deployApplication(TcContainerDeployer.java:399)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:301)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:792)
at com.sun.jmx.remote.security.MBeanServerAccessController.invoke(MBeanServerAccessController.java:468)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1486)
at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:96)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1327)
at java.security.AccessController.doPrivileged(Native Method)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1426)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:847)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Rewriten CrazyCoders comment as answer:
I needed to check Run/Debug Configuration|Server tab|JMX authentification enabled.
Password and role could be found in: CATALINA_BASE/conf/jmxremote.password
I have a suggestion what host it's all about.
Part of tc Server deployment API is notion of service and host. There are some services, and each service contain some number of hosts. IDEA plugin provides by default Catalina service and localhost host. They are configured in Run/Debug Configuration | Deployment tab | Server service name and Server host name(you need to have something to deploy to see these settings).
Now how to get proper values. They are available via JMX. Connect with jconsole.exe from your JDK to service:jmx:rmi:///jndi/rmi://<your host>:<your port>/jmxrmi. It may need login and password you use for server's JMX. Then open tc server | Deployer | Operations in the tree in left panel. Operation getServices will list you available services, and getHosts will list you all hosts for selected service.

Resources