Running WSO2 Enterprise Integrator 6.5.0. on RHEL 7. We are in the proces of building flows to read files from an sftp server. But setting up the sftp connection towards a Windows SFTP server fails. We can access this Windows SFTP server correctly with Windows clients like FileZilla/WinSCP.
With netstat we see a connection is build towards the Windows SFTP server but the flow isn't moving - no files are being read. On the point of stopping the server the error as shown below is printed in the wso2carbon.log.
When setting up the connection towards a Linux sftp server ( Plain RHEL 7 box with SSHD ) we don't face any issues. We have the matching private key place under .ssh/id_rsa in the home dir of the user running WSO2 EI.
Searching for the error message ( see snippet below ) we should get it resolved by adding the transport.vfs.AvoidPermissionCheck=true parameter to the VFS URL but unfortunately this doesn't solve our issue.
This is the VFS URL we are using.
sftp://SFTPUSER#SERVER.ACMECORP.ORG/inputdir?transport.vfs.AvoidPermissionCheck=true;vfs.passive=true
Is this a configuration that should work and are we missing a configuration option? Or is this a bug in the WSO2 software?
These URL's mention the issue we are facing.
VFS2 Error cannot delete file and could not get the groups id of the current user (error code: -1)
https://issues.apache.org/jira/browse/VFS-617
https://github.com/wso2/product-ei/issues/3725
[2019-12-06 13:48:59,724] [-1] [] [vfs-Worker-2] ERROR {org.apache.synapse.transport.vfs.VFSTransportListener} - Error checking for existence and readability : sftp://SFTPUSER#SERVER.ACMECORP.ORG/inputdir?transport.vfs.AvoidPermissionCheck=true;vfs.passive=true
org.apache.commons.vfs2.FileSystemException: Could not determine if file "sftp://SFTPUSER#SERVER.ACMECORP.ORG/inputdir?transport.vfs.AvoidPermissionCheck=true;vfs.passive=true" is readable.
at org.apache.commons.vfs2.provider.AbstractFileObject.isReadable(AbstractFileObject.java:1494)
at org.apache.synapse.transport.vfs.VFSTransportListener.scanFileOrDirectory(VFSTransportListener.java:295)
at org.apache.synapse.transport.vfs.VFSTransportListener.poll(VFSTransportListener.java:188)
at org.apache.synapse.transport.vfs.VFSTransportListener.poll(VFSTransportListener.java:134)
at org.apache.axis2.transport.base.AbstractPollingTransportListener$1$1.run(AbstractPollingTransportListener.java:67)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.jcraft.jsch.JSchException: Could not get the groups id of the current user (error code: -1)
at org.apache.commons.vfs2.provider.sftp.SftpFileSystem.getGroupsIds(SftpFileSystem.java:219)
at org.apache.commons.vfs2.provider.sftp.SftpFileObject.getPermissions(SftpFileObject.java:250)
at org.apache.commons.vfs2.provider.sftp.SftpFileObject.doIsReadable(SftpFileObject.java:264)
at org.apache.commons.vfs2.provider.AbstractFileObject.isReadable(AbstractFileObject.java:1492)
... 8 more
UPDATE
Using the same URL but then setting up the WSO2 flow to write a file towards the SFTP server works.
Got this resolved with support from WSO2.
The correct VFS url to use is.
sftp://SFTPUSER#SERVER.ACMECORP.ORG/inputdir?transport.vfs.AvoidPermissionCheck=true&vfs.passive=true So a '&' seperator instead of a ';'.
The documentation of WSO2 just is very fuzzy about the correct syntax to use.
They give different examples across their documentation.
https://docs.wso2.com/display/EI650/VFS+Transport
https://docs.wso2.com/display/EI650/File+Inbound+Protocol
https://docs.wso2.com/display/EI650/Configuring+File+Inbound+Protocol+for+FTP%2C+SFTP+and+FILE+Connections
My objective, is to configure the Jetty in such a way that it enables HTTPS request for Geoserver.
To be a bit specific, I am downloading the Geoserver Windows Installer.
As tested both version 2.9-RC1 and 2.10.2, they throw the warning
"Module not found [ssl]" as soon as it is started.
Jetty version: 9.2.13.v20150730
I am referring to the info here to try setup the simplest test (self-signed) for the jetty, but still no luck.
What I did:
Use Keytool create a self-signed keystore
Replace the keystore generated from 1) to directory etc/keystore
Edit the jetty-ssl.xml, replace the password (attribute default) with mine
Edit the file start.ini, add --module=https, jetty.secure.port=8443
Launch the batch file.
What did i miss?
P.S.: Geoserver works perfectly in HTTP. This post is asking about configuration for HTTPS.
Here are the steps to enable jetty to run geoserver on https, port:8443
• Configuring Jetty for SSL
follow this link to create ssl certificate and keystore. https://www.eclipse.org/jetty/documentation/9.1.5.v20140505/configuring-ssl.html
Now open jetty-ssl.xml file present in Geoserver/etc folder and replace the KeyStorePassword, KeyManagerPassword, TrustStorePassword with the new password which is created while creating keystore.
Open & check jetty.xml file in same folder, on which port https will run.
• Add new Keystore in Geoserver
Copy the created keystore file and replace with the existing available in etc folder of geoserver ex: C:\GeoServer\etc.
Now check if ssl.mod file is present in modules folder or not, ex: C:\GeoServer\modules.
If present, then open the file and replace the password in jetty.keystore.password, jetty.keymanager.password and jetty.truststore.password with the new password. Otherwise download it from “http://central.maven.org/maven2/org/eclipse/jetty/jetty-distribution/9.2.13.v20150730/jetty-distribution-9.2.13.v20150730.tar.gz.
NOTE : Please make sure password entered should be same as in jetty-ssl.xml.
Open start.ini file present in C:\GeoServer folder add --module =ssl, --module=https and jetty.secure.port=8443 (if https is running on Port 8443.
Run geoserver, enter url https://localhost:8443/geoserver (assuming geoserver on local system).
The https module has a dependency on the ssl module. The error is stating it cannot find ssl.mod in the ${jetty.home}/modules (or, depending on your config, ${jetty.base}/modules) directory.
The ssl module comes standard as part of the Jetty distribution so it has either been deleted, moved or renamed as part of your implementation.
For Geoserver 2.12.1, I manually installed "ssl.mod" from jetty-distribution-9.2.13.v20150730.tar.gz in Geoserver's etc/modules directory. Then the warning message didn't occur again in the log.
remember to add --module=ssl to start.ini
complete password and keystore/truststore location in jetty-ssl
port in jetty-https default 443
port in jetty xml can stay at default 8443
open ssl mod in a text editor and comment out the keystore under the [files] tag
ini-template tag --jetty secure port can stay at 8443
Comment out anything that refers to jetty keystore as these are set in jetty-ssl
open https mod and change https port to 443, restart geoserver.
Geoserver should now run over jetty https:// with no port number.
I was trying to download a file form FTP server(Remote machine) using pentaho kettle (Get a file with FTP), I can able to do that in my local machine. But when i try to deploy the app in jboss web server it fails to download resulting in a error "Error getting files from FTP : Login incorrect."
But everything seems to be correct regarding the login details.
Did i have configure any where else in the server? Please help.
There are several things you could check in this case:
Check that the security settings of the remote FTP server allows for the machine that runs the job to establish an FTP connection. If you have access to the server, try the following command to ensure that the server has access:
telnet <your-remote-ftp-server-host> 21
# Or try:
telnet <your-remote-ftp-server-host> 22
Verify that the remote FTP server respects FTP connection requests, otherwise try SFTP (Get a file with FTP & Get a file with SFTP are two entirely different job steps in Kettle)
Wherever you are fetching your FTP credentials from (ideally a configuration file), test that the credentials are properly read by the job in the relevant scope - use the Write To Log step for that matter
The application log in the event viewer shows
Unable to load SiteMinder host configuration object or host configuration file
for Siteminder 12.51 on IIS 7.5 (64bit) OS Windows 2008 (64bit).
When do you get the error? Is it when you're configuring the Webagent?
Anyway, verify the following.
Verify your environment variables are set correctly, if must have references to the Webagent files, you may need to export the envvars in the webagent or policy server folder (nete_wa_env...) file.
if the host config object you're using exists? Verify this using the
Admin UI
If the hostname is configured in the policy server as a trusted host
Verify if the settings are correct in the corresponding SmHost.conf file in the webagent folder
Verify and eliminate any duplicate or conflicting lines in your IIS config files which refer to siteminder.
Verify the host config object and the agent config object settings.
make sure WebAgent.conf is pointing to the correct SmHost.conf and SmHost.conf has the correct HostConfigObject defined (with the exact case that is used in the Policy Store).
If the HCO in the Policy Store is named "DefaultHostSettings" and SmHost.conf contains HostConfigObject=defaulthostsettings you will get this type of error.
Try re-registering the web agent with the policy server using the smreghost command.
I need to recover/reset the admin password for JetBrain's TeamCity.
I have full RDP access to the server so no problems there. It's just been 2 months since we used it so now I have forgotten my login - my usual ones don't work.
It is setup without a database at the moment, so was hoping the usernames would just be in a file somewhere, but no luck finding it so far.
From TeamCity 8 you can log in as a super user and change the password that way. You just need to use an empty username and last occurrence of the "super user authentication token" found in the logs\teamcity-server.log file as your password.
Please see the following for more information:
TeamCity 8 - http://confluence.jetbrains.com/display/TCD8/Super+User
TeamCity 9 - http://confluence.jetbrains.com/display/TCD9/Super+User
TeamCity 10 - https://confluence.jetbrains.com/display/TCD10/Super+User
In case none of those works, see http://sebastienlachance.com/post/Resetting-TeamCity-Password.aspx.
Open a command prompt and go to \webapps\ROOT\WEB-INF\lib folder. Now type the following :
..\..\..\..\jre\bin\java.exe -cp server.jar;common-api.jar;commons-codec-1.3.jar;util.jar;hsqldb.jar ChangePassword username newpassword
For TeamCity 6.5.4
From a command prompt in the [TeamCity install folder]\webapps\ROOT\WEB-INF\lib:
..\..\..\..\jre\bin\java -cp server.jar;common-api.jar;commons-codec-1.3.jar;util.jar;hsqldb.jar ChangePassword admin NewPassword
My username was 'admin' in my case (I think I set it during installation but I can't be sure).
I ommitted the path to TeamCity argument, it's smart enough to use the correct path (mine was c:\users\administrator.BuildServer)
When I provided the (wrong) path to TeamCity as an argument I received this message:
Using TeamCity configuration directory path: c:/TeamCity/.BuildServer
Exception in thread "main" java.sql.SQLException: Table not found in statement [UPDATE users SET PASSWORD = ? WHERE USERNAME = ? AND REALM IS NULL]
at org.hsqldb.jdbc.Util.throwError(Util.java:58)
at org.hsqldb.jdbc.jdbcPreparedStatement.<init>(jdbcPreparedStatement.java:1833)
at org.hsqldb.jdbc.jdbcConnection.prepareStatement(jdbcConnection.java:580)
at ChangePassword.main(ChangePassword.java:14)
In case this confuses other people too.
Super user direct URL :
http://servername:port/login.html?super=1
Open TeamCity log folder (Example C: drive: C:\TeamCity\logs) : teamcity-server.log, find the key: “Super user authentication”
[2019-03-04 12:14:30,770] INFO - jetbrains.buildServer.SERVER - Super user authentication token: `8347518935696887114` (use empty username with the token as the password to access the server)
You could try to reset the installation of TeamCity, by removing TeamCity data directory ($/.BuildServer directory by default)
Try the following:
First stop the TeamCity service (would also stop the build agent if installed).
Next open up a console, go to your java directory and run the following command from there:
java.exe -cp server.jar; hsqldb.jar ChangePassword USERNAME PASSWORD "PATH_TO_YOUR_TEAMCITY_INSTALLATION".BuildServer
I've just had to go through this pain with v5 EAP.
I managed to reset the password successfully by running:
C:\TeamCity\webapps\ROOT\WEB-INF\lib>..\..\..\..\jre\bin\java -cp server.jar;common-api.jar;commons-codec-1.3.jar;util.jar;hsqldb.jar ChangePassword admin password c:\TeamCity\.BuildServer
Although you'll need to substitute C:\TeamCity with wherever your installation is located.
In case this helps someone else, whoever installed TeamCity on my server placed the build directory under the Administrator's Profile, not in C:\TeamCity.
Specifically for TeamCity running on Tomcat on Windows it will be C:\ProgramData\JetBrains\TeamCity
The directory specified as the last parameter needs to be your Data Directory (find in /logs/teamcity-server.log)
You'll get the 'Table not found' error if you don't have this correct.
You'll get a 'The database is already in use' error if you've got TeamCity running.
you can also search you /logs/teamcity-server.log to see whether you created admin, administrator, or some other admin user name.
For everyone that may arrive at this article years after the original answer like I just did, there's a built-in super user account, and the password is regenerated every time team city is started, and the password is in the log. You can use this super user to login and reset any passwords. It's super easy.
https://confluence.jetbrains.com/display/TCD9/Super+User
TeamCity always uses a database - if you haven't explicitly configured one, it uses a HSQLDB database to store data internally.
When using an external database, user information is stored within that database, so it seems pretty likely that the user information in your case will be stored within the HSQLDB system.
You might be able to gain access to the system by futzing around with the database - but I'd suggest taking a backup first.
Second suggestion - drop the support guys at JetBrains an email. Even before my workplace splashed out on a TeamCity Enterprise license, their support was superb - fast, accurate and helpful.
With TeamCity 5 using MySQL (probably other versions and RDBMs as well, but untested), it's possible to update the password directly via SQL:
mysql> update users set password = md5("mypass123") where username = "bob";
Nevertheless, I'd stick with the CLI versions already mentioned by others if there isn't a good reason not to do so.
First point is if you logout the login screen has the username 'TCAdmin' already filled in, when it should be 'administrator'. TCAdmin is the full name of (I think) the default version 5 admin user. Changing that to administrator and then using the password I thought it was solved my issue.
For resetting...
In case it helps someone else on Windows XP on version 5 of TeamCity, my .BuildServer config info was also under my current logged in user's documents and settings folder. Also I was tripped up by a space in the list of jar files in Sebastien's good answer above.
So I changed to this directory in a command prompt:
c:\teamcity\webapps\ROOT\WEB-INF\lib
and then this command line (to set password: Password1) worked for me:
C:\TeamCity\webapps\ROOT\WEB-INF\lib>..\..\..\..\jre\bin\java.exe -cp server.jar;commonapi.jar;commons-codec-1.3.jar;util.jar;hsqldb.jar ChangePassword administrator Password1
Which gave output:
Using TeamCity configuration directory path: C:/Documents and Settings/tamw/.BuildServer
Password changed successfuly
Stop teamcity
You should pass path to your buildserver
e.g. if you installed build server to dir "c:\.BuildServer"
........\jre\bin\java.exe -cp server.jar;common-api.jar;commons-codec-1.3.jar;util.jar;hsqldb.jar ChangePassword username newpassword c:\.BuildServer
To change user password:
Shutdown server
Switch to the /webapps/ROOT/WEB-INF/lib directory
Invoke the following command:
Windows platform:
java -cp server.jar;common-api.jar;commons-codec-1.3.jar;util.jar;hsqldb.jar ChangePassword
Unix platform:
java -cp server.jar:common-api.jar:commons-codec-1.3.jar:util.jar:hsqldb.jar ChangePassword
You can skip the option, if you are using default path for TeamCity data files: /.BuildServer
[Ref: http://confluence.jetbrains.com/display/TCD7/Changing+user+password+with+default+authentication+scheme]
Here's what worked for me.
Shut down server services
> cd c:\TeamCity\webapps\ROOT\WEB-INF\lib>
then
> ..\..\..\..\jre\bin\java.exe -cp server.jar ;common-api.jar;commons-codec-1.3.jar;util.jar;hsqldb.jar ChangePassword admin password1 C:\ProgramData\JetBrains\TeamCity\
Without the path at the end, it would fail with:
Exception in thread "main" java.sql.SQLException: Table not found in statement [
UPDATE users SET PASSWORD = ? WHERE USERNAME = ? AND REALM IS NULL]
at org.hsqldb.jdbc.Util.throwError(Util.java:58)
at org.hsqldb.jdbc.jdbcPreparedStatement.<init>(jdbcPreparedStatement.ja
va:1833)
at org.hsqldb.jdbc.jdbcConnection.prepareStatement(jdbcConnection.java:5
80)
at ChangePassword.main(ChangePassword.java:14)
We are using Teamcity 7 with MS SQL Server as the RDBMS.
To reset your password you can use the following query:
UPDATE users SET password = LOWER(SUBSTRING(sys.fn_sqlvarbasetostr(HASHBYTES('md5','your_new_password')),3,32))
where username = "your_user_name";
Alternatively, You could use the TeamCity Server Log and retrieve the Super User token.
Using the token, go to the URL : http://(server):(port)/login.html?super=1
ie: http://localhost:92/login.html?super=1
Once you logged in, you could always create a new user or reset the password for the account in question.
I passed in the same situation and did login with super user, follow steps below:
1 - Get Token in the "teamcity-server.log" in the path "XX:\TeamCity\logs";
2 - Access and login using token at url: "/login.html?super=1";
More about it:
https://confluence.jetbrains.com/display/TCD18/Super+User