I am using the following technologies
Java 1.8
Spring Boot 1.5
PostgreSQL 9.5
I am facing a problem while backing up a large database using pg_dump, however, it is working fine for small database. Following is the code that I am using at present.
ArrayList<String> commands = new ArrayList<>();
commands.add("C:\\Program Files\\PostgreSQL\\9.5\\bin\\pg_dump");
commands.add("-h");
commands.add(ipaddress);
commands.add("-p");
commands.add(port);
commands.add("-U");
commands.add(username);
commands.add("-F");
commands.add("c");
commands.add("-b");
commands.add("-f");
commands.add("backup.dump");
commands.add("-d");
commands.add(database);
ProcessBuilder pb = new ProcessBuilder(commands);
pb.environment().put("PGPASSWORD", password);
Process process = pb.start();
try (BufferedReader buf = new BufferedReader(new InputStreamReader(process.getErrorStream()))) {}
process.waitFor();
process.destroy();
What appears to be happening is that the service for backup is being called twice, one after another, and quitting without properly backing up the database. It also doesn't return call back to the EJB. Am I missing something here, or should I change the way I am doing things?
Related
I want to read multiple files, name*.txt and process them.
For that I am using MultiResourceItemReader.
It is reading all files and process and write at one time only. I want to read multiple files seperately, process and write to them.
The code:
#Bean
public MultiResourceItemReader<POJO> multiResourceItemReader() {
MultiResourceItemReader<POJO> resourceItemReader = new MultiResourceItemReader<POJO>();
ClassLoader cl = this.getClass().getClassLoader();
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver(cl);
Resource[] resources = resolver.getResources("file:" + filePath );
resourceItemReader.setResources(resources);
resourceItemReader.setDelegate(reader());
return resourceItemReader;
}
That's how the MultiResourceItemReader is designed to work. In your case, you can create a job instance per file.
There are many advantages of making one thing do one thing and do it well, one of them in your use case is restartability: If one of the jobs fail, you only restart the failed one.
It is very easy and well documentented how to run H2 in server mode. Just code:
Server.createTcpServer().start();
Very easy. But I'm unable to find an answer on:
how to give the created database an other name than 'test', because this is the default name.
how to persist the data. As it seems, this way the server starts an in-memory DB. But what I'm looking for is something that persists the data permanently.
Any idea?
As described in Connecting to a Database using JDBC, the database file name may be specified in the database URL, for example
jdbc:h2:tcp://localhost/~/src/java/MyDatabase;IFEXISTS=TRUE
Data will be stored in the file MyDatabase,h2.db.
See Opening a Database Only if it Already Exists for the effect of specifying ;IFEXISTS=TRUE. Compare the URL with these embedded mode examples.
Addendum: Where will you give this URL?
Once the server is started, you can connect to the running database by passing the URL to your preferred client, as suggested here, or programmatically via java.sql.Connection, as shown here.
After an other day and switch to derby I find a solution for this problem. I don't know why this is not handled by the H2 documention, but ok.
First at all. For me the connection url and the physical location of the DB-Files are different things. And I don't like this file things, because I want to take a look in the db on the fly.
So, whats to is this
private Server startDb() {
Server retVal = null;
try {
final String userDir = System.getProperty("user.dir");
// System.setProperty("h2.baseDir", userDir + "/data/jumpstart");
retVal = Server.createTcpServer("-baseDir", userDir + "/data/jumpstart");
retVal.start();
Connection conn = null;
try {
Class.forName("org.h2.Driver");
conn = DriverManager.getConnection("jdbc:h2:tcp://localhost/jumpstart", "sa", "sa");
} finally {
if (conn != null)
conn.close();
}
} catch (final Exception ex) {
}
return retVal;
}
As you can see here, the files of the db will be stored in the directoy /data/jumpstart.
The URL of JDBC connect hide this detail. It only address the the DB with name jumpstart.
That it is.
HTH
Andreas
got the following problem:
I tried to create a simpel file upload functionality in grails. I just created a domain class with a
byte[] rawFile
as property. Grails did most of the rest for me. It worked fine for the standard hsqldb in the dev environment.
Then I deployed it to the server with an oracle db configured (thin driver). Everything but the file upload works fine with the oracle db. For the file upload I get a (as far as I can remember)
SQLException: ORA-01461: can bind a LONG value only for insert into a LONG
I tried several ways to fix it (including some columnmappings to blobs and using java.sql.blob instead of byte[]) but nothing really worked and I went in a direction where my code wouldn't be db independent anymore.
Google didn't really help me and my grails books don't help too.
Saving the file to the disk isn't a good solution im my opinion.
So here's my question:
how do I create a file upload in grails which works with oracle?
Update: got some additional infos. Managed to reproduce the problem with the XE-Edition of Oracle:
Hibernate creates a VARBINARY(255) column for the rawFile. So I tried to upload a 4 bytes file and it worked.
I then changed the type of the column manually to 'blob' and it worked with bigger files.
I then added
static mapping = {
columns {
rawFile type:'blob'
}
}
to my domain class and it stopped working:
ERROR errors.GrailsExceptionResolver - [B cannot be cast to java.sql.Blob
java.lang.ClassCastException: [B cannot be cast to java.sql.Blob
:-(
Instead of setting the type to blob try to increase the maxSize constraint:
static constraints = {
rawFile(maxSize: 20 * 1024 * 1024) // 20 MBs
// ...
}
If you want to use a Blob field in Oracle then set your domain property to byte[] and set the type to org.hibernate.type.MaterializedBlobType. The MaterializedBlobType handles conversion back and forth between Oracle (presumably other databases, but I've only done this on Oracle) and byte[].
byte[] blobFile
static mapping = {
blobFile type: org.hibernate.type.MaterializedBlobType
}
I'm not sure what you are doing in your controller, try doing it manually to see what happens:
request.fileMap.each { name, file ->
if (!file.empty) {
model.rawFile = file.bytes
}
}
model.save()
Try setting the sqlType.
Using a domain field with type byte[] with sqlType set to "blob" in the mapping block works for me using Grails 2.3.1 and Oracle 11g. Grails handles the type conversion automatically.
class Image {
byte[] image
static mapping = {
image(sqlType: "blob")
}
}
Even though I've been a stalker here for ages, this is the first post I'm making. Hopefully, it won't end here and more optimistically future posts might actually be me trying to give a hand to someone else, I do owe this community that much and more.
Now, what I'm trying to do is simple and most probably the reason behind it not working is my own stupidity. However, I'm stumped here.
I'm working on an ASP.Net website that interacts with an SQL Server 2008 R2 database. So far everything has been going okay but updating a row (or more) just won't work. I even tried copying and pasting code from this site and others but it's always the same thing.
In short: No exception or errors are shown when the update command executes (it even gives the correct count of affected rows) but no changes are actually made on the database.
Here's a simplified version of my code (the original had more commands and tons of parameters each, but even when it's like this it doesn't work):
protected void btSubmit_Click(object sender, EventArgs e)
{
using (SqlConnection connection =
new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["ApplicationServices"].ConnectionString))
{
string commandString = "UPDATE [impoundLotAlpha].[dbo].[Vehicle]" +
"SET [VehicleMake] = #VehicleMake" +
" WHERE [ComplaintID] = #ComplaintID";
using (SqlCommand command = new SqlCommand(commandString, connection))
{
SqlTransaction transaction = null;
try
{
command.Connection.Open();
transaction = connection.BeginTransaction(IsolationLevel.Serializable);
command.Transaction = transaction;
SqlParameter complaintID = new SqlParameter("#complaintID", SqlDbType.Int);
complaintID.Value = HttpContext.Current.Request.QueryString["complaintID"];
command.Parameters.Add(complaintID);
SqlParameter VehicleMake = new SqlParameter("#VehicleMake", SqlDbType.VarChar, 20);
VehicleMake.Value = tbVehicleMake.Text;
command.Parameters.Add(VehicleMake);
command.ExecuteNonQuery();
transaction.Commit();
}
catch
{
transaction.Rollback();
throw;
}
finally
{
connection.Close();
}
}
}
}
I've tried this with the "SqlTransaction" stuff and without it and nothing changes.
Also, since I'm doing multiple updates at once, I want to have them act as a single transaction. I've found that it can be either done like this or by use of the classes included in the System.Transactions namespace (CommittableTransaction, TransactionScope...).
I tried all I could find but didn't get any different results.
The connection string in web.config is as follows:
<connectionStrings>
<add name="ApplicationServices"
connectionString="Data Source=localhost;Initial Catalog=ImpoundLotAlpha;Integrated Security=True"
providerName="System.Data.SqlClient"/>
</connectionStrings>
So, tldr; version:
What is the mistake that I did with that record update attempt? (Figured it out, check below if you're having a similar issue.)
What is the best method to gather multiple update commands as a single transaction?
Thanks in advance for any kind of help and/or suggestions!
Edit:
It seems that I was lacking some sleep yesterday cause this time it only took me 5 minutes to figure out my mistake.
Apparently the update was working properly but I failed to notice that the textbox values were being overwritten in Page_Load. For some reason I had this part commented:
if (IsPostBack)
return;
The second part of the question still stands. But should I post this as an answer to my own question or keep it like this?
Have you tried running the query against the database directly (i.e. SQL Management Studio itself)? I'm not sure how you'd implement the "START TRANSACTION... COMMIT TRANSACTION" commands from ASP... for what it's worth we do all our database operations from within stored procedures.
I've set up system DSNs, which I can use from other ODBC apps (e.g. iQueryODBC), but in mono, I get "Data source name not found and n" (sic).
I am using "DSN=myodbc" for the connection string, via the connection string builder.
OSX 10.4
Latest Mono packages - 2.4.2.3.
Anyone ever got ODBC working on Mono/ OSX?
(Oh - for what it is worth - and I am fairly certain it is not relevant - the DSN is for MySql 5 driver.)
Full code:
public static void Main (string[] args)
{
OdbcConnectionStringBuilder csb = new OdbcConnectionStringBuilder();
csb.Dsn = args[0];
DataSet d = GetDataSet(csb.ConnectionString , "SELECT * FROM tbl");
Console.WriteLine (d.Tables.Count);
}
public static DataSet GetDataSet(string connectionString, string queryString)
{
Console.WriteLine("GetDataSetFromAdapter(" + connectionString + ")");
DataSet dataSet = new DataSet();
using (OdbcConnection connection = new OdbcConnection(connectionString))
{
OdbcDataAdapter adapter = new OdbcDataAdapter(queryString, connection);
// Open the connection and fill the DataSet.
try
{
connection.Open();
adapter.Fill(dataSet);
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
return dataSet;
}
In the mono website are a example may can help you.
ODBC-Mono.Net
I think I have seen your post on the Mono mailing list about this but was too busy to respond at the time.
I have used both the ODBC interface and the native connector to connect to MySQL databases for Mono on Mac OS X, Linux, Windows and Solaris (and only ever had issues with an old build of Mono under Solaris).
Because of a problem I encountered with a production environment I switched to using the native MySQL connector some time ago though (I downloaded and built it using Mono Develop without issue).
This is an example of how I used the ODBC interface:
string connectionString = "DRIVER={MySQL ODBC 3.51 Driver};"
+ "SERVER=localhost;"
+ "DATABASE=myDatabase;"
+ "UID=root;"
+ "PASSWORD=p4ssw0rd;";
// Connect to database using ODBC Driver
OdbcConnection dbConnection = new OdbcConnection(connectionString);
dbConnection.Open();
// Execute SQL using the ODBC interface
OdbcCommand dbCommand = dbConnection.CreateCommand();
dbCommand.CommandText = sql
OdbcDataReader dbReader = dbCommand.ExecuteReader();
// Get the result and put it into a hashtable (as an example)
ArrayList arrayList = new ArrayList();
int i = 0;
while(dbReader.Read()) {
arrayList.Add(new Hashtable());
Hashtable hashtable = (Hashtable) arrayList[i];
for (int j = 0; j < dbReader.FieldCount; j++) {
hashtable.Add(dbReader.GetName( j ).ToString(), dbReader.GetValue( j ));
}
i++;
}
// Free up resources
dbReader.Close();
dbReader = null;
dbCommand.Dispose();
dbCommand = null;
// Close DB Connection
dbConnection.Close();
If the example does not work (for some non-obvious reason) comment and let me know and I will test it and/or provide a tested example of using the native MySQL provider if that would be helpful.
Thanks for the advice, but 'fraid it does't work - Connection.Open fails, exactly as before, with "Data source name not found..." It thinks the DSN doesn't exist, but... it does, because other apps (non Mono) can use it. I've dived into the Mono source code, for OdbcConnection.Open, but it is just a wrapper for some native code, which I don't have the source for.
Note that I am using totally standard, generic code - a classic 'example' so it is not a question (I'm fairly certain) of not knowing HOW to do it - I am doing it correctly, but it just doesn't work as per the examples elsewhere on the net, because the Odbc connection is asking for a named DSN, and is being told by the native code that the DSN doesn't exist.
Note also that using a DSN-less connection produces EXACTLY the same error - the native code provider still looks for a matching DSN, which (obviously, in this case) it doesn't find because there isn't a DSN called "DATABASE=myDatabase...".
I'm guessing this is OSX 10.4 related, so... I'm maybe not going to find an answer.
I also tried the native MySql connector, but it failed to build. Again, may be 10.4 related. Maybe this is the excuse I need to upgrade... which would (of course!) require a new MacBook... mmmm... shiny!!
Thanks again for the advice, chaps.
For the benefit of anyone else driving by this, you CAN compile the native MySql connector on 10.4. The "sln" which you download from MySql.com contains a lot of other stuff which isn't required, and doesn't compile! But don't be put off - just keep removing projects from the solution until it compiles, and then grab MySql.Data.DLL to add to your project. (You don't have to add it to the GAC - just put it in the "bin" directory of your project, and reference it by file.)
#Iain - thanks for your advice again. I have explained to the Mrs that "a nice man on the internet" said that the only solution is a new MacBook Pro and she is thinking about it!