I have experience UnsatisfiedLinkError when I tried to use
flink-s3-fs-hadoop to sink to s3 in my local Windows machine.
I googled and tried several solutions like download hadoop.dll and
winutils.exe, set up HADOOP_HOME and PATH environment variables, copy hadoop.dll to C:\Windows\System32, but none of them worked.
I also tried to load the hadoop library myself in the code by using
System.loadlibrary("hadoop"), it succeed, but the error still happen,
anything additional step I am missing?
Here is my code (the 2nd and 3rd line is just trying to debug and it will succeed on the 2nd line but throw the UnSatisfiedLink exception in 3rd line):
public static void main(String[] args) throws Exception {
System.loadLibrary("hadoop");
NativeIO.Windows.access(null, NativeIO.Windows.AccessRight.ACCESS_READ);
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1);
DataStream<String> ds = env.readTextFile("s3://fts-test/test/input.csv");
ds.print();
ds.writeAsText("s3://fts-test/test/output.csv");
env.execute();
}
By the way I am using Window 7 64 bit, flink 1.5.
Related
Our J2EE based application is run on Websphere Applicaiton Server- AIX server with 2 versions of DB2 Client installed (Db2v9.5, Db2v9.7).
I have a db SAMPLE which is remotely cataloged in both the two DB2 client versions with the same alias name SAMPLE.
If I uncatalog the DB from DB2v9.5 , the application goes down. However if I drop it from DB2v9.7 client the application is not impacted.
From this above test , we understand that some how DB2v9.5 is being used by default.
My objective now is to make the SAMPLE be pointed to only DB2v9.7 and the App must work without the sample DB in DB2v9.5.
Any suggestions on how to do it ?
The JDBC string used is "jdbc:db2:sample" (Note: there is no port for DB2 Client)
I have already tried to point $LIBPATH & $LD_LIBRARY_PATH to DB2v9.7 native lib32 path,
and also pointed the AIX user ID's .profile to DB2v9.7/sqllib/db2profile ,But no luck.
Regards,
Chandru
You have to check the default environment where Java is running, you will detect which Client is using. You could do that by executing "env" via Runtime, and see the defaults.
For example, with a small app like this:
import java.lang.*;
import java.io.*;
public class Test {
public static void main (String[] args) throws Exception {
Process p = Runtime.getRuntime().exec("env");
String line;
BufferedReader input = new BufferedReader(new InputStreamReader(p.getInputStream()));
while ((line = input.readLine()) != null) {
System.out.println(line);
}
input.close();
System.out.println("library " + System.getProperty("java.library.path"));
}
}
You call it with the same parameters you call your other application (JDBC driver included), and then check the output.
java <your params, classpath to db2java.zip> Test
Check the parameters like DB2INSTANCE, PATH, CLASSPATH.
More information in http://www.ibm.com/developerworks/data/library/techarticle/dm-0512kokkat/
List the files in /usr/lib and /usr/include. Do you see any file that points to DB2?
ls -l /usr/lib
ls -l /usr/include
Probably, you executed a db2ln, and that created links to a specific DB2 Client (9.5). Each time you execute the Java application, the LIBPATH takes precedence over /usr/lib, than the specified DB2 version
LIBPATH=/usr/lib:/lib:/opt/IBM/db2/V9.7/lib32
You can run the db2rmln command to remove the links.
Take a look at this page: http://publib.boulder.ibm.com/infocenter/db2luw/v9r5/topic/com.ibm.db2.luw.qb.server.doc/doc/t0006747.html
This issue was resolved by completely killing all the JVM that is running from the same version and path of java and starting them back. (In our case an AIX JVM pConsole was running behind even after putting Dmgr,Node & CL down. Both pConsole and WAS uses the same java)
That was the reason that the updated shared native libraries didn't reflect in the WAS. But after killing&starting back each instance of java, including the ones apart from WAS the native libraries got reflected in the logs .
Sorry about updating the answer after so long.
I just installed the latest JDK 7 Update 21 and wrote the following one-liner (on Windows 7):
public static void main(String[] args) {
System.out.println("java.home = " + System.getProperty("java.home"));
}
the output is (surprisingly):
java.home = D:\Java\jdk1.7.0_21\jre
I believe I fixed all the common causes:
JAVA_HOME is set to "D:\Java\jdk1.7.0_21"
I have set "D:\Java\jdk1.7.0_21\bin" as the first path in the system PATH setting.
There is no java.exe in Windows\System32
I am using a "fresh" command-line
I have searched this and other sites extensively
My registry does not contain a "HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Java Runtime Environment" key. The value of "HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Java Development Kit\1.7" contains a correct "JavaHome" value.
Adding "-server" option to the java.exe call does not change the output
Here is some more output for your information
D:\Temp>echo %PATH%
d:\Java\jdk1.7.0_21\bin
D:\Temp>echo %JAVA_HOME%
D:\Java\jdk1.7.0_21
D:\Temp>type SystemInfo.java
public class SystemInfo {
public static void main(String[] args) {
System.out.println("java.home = " + System.getProperty("java.home"));
}
}
D:\Temp>java -verbose SystemInfo
[Opened d:\Java\jdk1.7.0_21\jre\lib\rt.jar]
[Loaded java.lang.Object from d:\Java\jdk1.7.0_21\jre\lib\rt.jar]
[Loaded java.io.Serializable from d:\Java\jdk1.7.0_21\jre\lib\rt.jar]
...
[Loaded java.lang.Void from d:\Java\jdk1.7.0_21\jre\lib\rt.jar]
java.home = d:\Java\jdk1.7.0_21\jre
[Loaded java.lang.Shutdown from d:\Java\jdk1.7.0_21\jre\lib\rt.jar]
[Loaded java.lang.Shutdown$Lock from d:\Java\jdk1.7.0_21\jre\lib\rt.jar]
(Updated:) Basically I need to know, how to run my java application so that it has access to the java-compiler (found in the JDK's tools.jar) without copying that JAR to some other place.
BTW the main problem behind this is that my jetty can't compile a JSP since the java-compiler is missing. I know I can add "tools.jar" to JRE\lib\ext but thats not a long-term solution.
This is by default -- specified in the C++ code of the Java HotSpot interpreter. Also, it is what Sun originally wanted the System Property to set when the environment's Java home points to a JDK.
See the Java(TM) Tutorials for System Properties where it describes the java.home System Property as the "Installation directory for Java Runtime Environment (JRE)." Also, note that Oracle's Java 7 Javadoc for Class System is WRONG (aghast!) where it describes the java.home System Property as the "Java installation directory."
The answer lies on line 309 of the actual JVM code!
It's perfectly fine what you see as output - it's your default Java execution environment.
In order to configure jetty to use your JDK, maybe this helps.
I have this really simple Hello World example i wrote on a linux host using Mono and gtk#. It simply shows a windows with a button. Now i tried to get the binary running on windows but failed. i installed the gtk# with the standalone installer from the mono homepage. when i start the application it failes due to:
System.DllNotFoundException was unhandled: Unable to load DLL
'libglib-2.0-0.dll': The specified module could not be found.
(Exception from HRESULT: 0x8007007E) Source=glib-sharp
Any suggestions?
BR
You need either to run your app via mono.exe or just add the mono.exe location to the PATH environment variable value. You can do that in run-time like that:
[STAThread]
public static void Main(string[] args)
{
var dllDirectory = #"C:\Program Files (x86)\Mono\bin";
Environment.SetEnvironmentVariable("PATH", Environment.GetEnvironmentVariable("PATH") + ";" + dllDirectory);
Run();
}
private static void Run()
{
Gtk.Application.Init();
Forms.Init();
var app = new App();
var window = new FormsWindow();
window.LoadApplication(app);
window.SetApplicationTitle("Game of Life");
window.Show();
Gtk.Application.Run();
}
You will have to register the DLL 'libglib-2.0-0.dll' in Windows using regsvr32 libglib-2.0-0.dll from the command prompt. An alternative is to just package all the dependencies with your executable.
Hope this was helpful.
-Dave
I have spent quite some time reading previous posts, but none of them seem to help me. I'm trying to run a simple Java app from NetBeans, here the simplest code:
final String DB_URL = "jdbc:derby:CoffeeDB;create=true";
try
{
// Create a connection to the database.
Connection conn = DriverManager.getConnection(DB_URL);
// If the DB already exists, drop the tables.
dropTables(conn);
// Build the Coffee table.
buildCoffeeTable(conn);
// Close the connection.
conn.close();
}
catch (Exception ex)
{
System.out.println("ERROR: " + ex.getMessage());
}
Usually is the environmental variables causing trouble, but I have checked that and they seem to be set up correctly:
CLASSPATH:
C:\Program Files\Java\jdk1.7.0_17\db\lib\derby.jar;C:\Program Files\Java\jdk1.7.0_17\db\lib\derbytools.jar;
DERBY_HOME
C:\Program Files\Java\jdk1.7.0_17\db
PATH: (only the part concerning jdbc )
C:\Program Files\Java\jdk1.7.0_17\db\bin
I already created a table using ij from the command prompt and it worked, is when I try to run it through Netbeans when I get this error. Something interesting though, is when I run the sysinfo from the command prompt, it comes up like this:
------------------ Java Information ------------------
Java Version: 1.7.0_17
Java Vendor: Oracle Corporation
Java home: C:\Program Files\Java\jre7
Java classpath: C:\Program Files\Java\jdk1.7.0_17\db\lib\derby.jar;C:\Program Files\Java\jdk1.7.0_17\db\lib\derbytools.jar;C:\Program Files\Java\jdk1.7.0_17\db/lib/derby.jar;C:\Program Files\Java\jdk1.7.0_17\db/lib/derbynet.jar;C:\Program Files\Java\jdk1.7.0_17\db/lib/derbyclient.jar;C:\Program Files\Java\jdk1.7.0_17\db/lib/derbytools.jar
OS name: Windows Vista
OS architecture: amd64
OS version: 6.0
---> As you can see, somehow more information is added to the previous classpath, and the derbytools.jar and derby.jar appear twice?? combined with inverted slashes??
When debugging through NetBeans, the exception comes from this line in the DriverManager.java file:
Connection con = aDriver.driver.connect(url, info); <--------------- here
if (con != null) {
// Success!
println("getConnection returning " + aDriver.driver.getClass().getName());
return (con);
}
the url is correct: jdbc:derby:CoffeeDB;create=true but the connection comes back null and throws the SQLexception. Any ideas?
The exception message "No suitable driver found for ..." means that none of the loaded JDBC drivers accepted the URL. This makes it very likely that the JDBC driver simply isn't included in the classpath of the application
The environment variable CLASSPATH is almost always ignored when a java application is run from an IDE, and always if run from an executable jar. Make sure that the build-path in the IDE actually contains derby.jar, or if it is an executable jar, that it is listed in the META-INF/manifest.mf in the Class-Path attribute.
you can right click the project and do the following steps...Build -> Configure Build Path -> Libraries -> Add External Jar... now you have to specify the path to the *.jar file.
simple example https://github.com/mraab89/Tutorial_Java_Database_Connectivity_JDBC
I know this was asked elsewhere a year ago but I'm looking for any updates please.
I have a program in Java 7 that uses WatchService to monitor a directory for new files being created there. If I register a directory on my local machine (e.g. c:\NewFiles) then all is fine. However when I try to get it to listen to a folder on a network drive it compiles and runs but exits straight away because it doesn't seem to get notified of any events.. here is just a fragment of my code...
private final WatchService watcher;
private final Map<WatchKey, Path> keys;
this.watcher = FileSystems.getDefault().newWatchService();
this.keys = new HashMap<>();
I register the directory...
String srcDirPath = "G:\\NewFiles";
Path dir = Paths.get(srcDirPath);
WatchKey key = dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
keys.put(key, dir);
then at the end of the loop that checks for events...
if (keys.isEmpty()) {
System.out.println("Drive is inaccessible");
break;
}
Here I always get the "Drive is inaccessible" warning. I tried copying my Java files to this network directory and running them from there but I get the same problem.
Is it possible to run Watchservice like this from a local machine to listen to changes to a directory on a network drive? Am I getting a permission or firewall problem here? I am using Windows 7 and most likely the network drive is also on a Windows 7 machine.