I hava a JPA query defined in a JpaRepository:
List<Object[]> getDoc ();
one of the columns if a BLOB type
but when I do:
System.out.println("Content Type -> " + obj[1].getClass());
System.out.println("Content -> " + obj[1]);
I got:
Content Type -> class [B
Content -> [B#73e73b8e
and I don't know how to convert it to String
What this means is that your obj[1] is an array of bytes, which is exactly what it should be for a blob.
To convert it into a string, you need to specify the encoding. Assuming the data is encoded according to UTF-8, that would be
String s = new String((byte[]) obj[1], StandardCharsets.UTF_8)
Related
I've got an error
Caused by: org.h2.jdbc.JdbcSQLDataException: Hexadecimal string contains non-hex character: "SWEDISH_MARKET"; SQL statement:
when I pass an enum as parameter to query
Entity has this enum as field.
Field has bellow annotations
#Column(name = "market", nullable = false)
#Enumerated(EnumType.STRING)
private Market type;
query:
#Query(value = "SELECT count(*) " +
"FROM Statements statements WHERE " +
"market = :market",
nativeQuery = true)
long countStatements(#Param("licenseMarket") Market market);
Previously everything was parsed to String and casted in postgres market = (cast :market as text)
Invoking countStatements(Market.SWEDISH_MARKET) throws the exception
You have to use JPQL, then it parse enums automatia
The problem is enum was parsed automatically to the number, regardless column type is text.
Solutions are:
invoke on enum method .name() and pass a param as String,
use SpEL :#{#market?.name()},
List<CustomerDetails> customerDetailsList = repo.getCustomerDetails();
Set<String> combinedNamesList = new HashSet<>();
customerDetailsList.forEach(i -> {
combinedNamesList .add((i.getFirstName() != null ? i.getFirstName().toLowerCase(): "") + (i.getLastName() != null ? i.getLastName().toLowerCase(): ""));
});
I would like to create the combinedNamesList in one operation using streams. Each CustomerDetails object has properties for a firstName and LastName. I would like to combine the two properties into a single String in an array such as:
{BobSmith, RachelSnow, DavidJohnson}
Stream the list and filter all customer objetcs having valid firstname and lastname, and then combine the name using String.format
List<String> combinedNamesList = repo.getCustomerDetails()
.stream()
.filter(cust->cust.getFirstName()!=null && cust.getLastName()!=null)
.map(cust->String.format("%s%s",cust.getFirstName(),cust.getLastName()))
.collect(Collectors.toList());
Adding to Deadpool's answer, thinking this might help someone too.
Person p = new Person("Mohamed", "Anees");
Person p1 = new Person("Hello", "World");
Person p2 = new Person("Hello", "France");
System.out.println(
Stream.of(p, p1, p2)
.map(person -> String.join(" ", person.getFirstName(), person.getLastName()))
.collect(Collectors.toSet()));
Here String.join() is used to concatenate names. And, this also produces a more sensible output than the one you are expecting
[Mohamed Anees, Hello World, Hello France]
If you really need names without space, you can replace " " in String.join() delimiter to ""
You can add filter() in the Stream for null checks before converting to lowercase.
This code:
Array a=rs.getArray("curAccs");
ResultSet rs1=a.getResultSet();
int rs1size=rs1.getFetchSize();
return empty rs1, but a is ARRAY[221,222]
I also cant convert sql array to java
(Integer[])a2.getArray()
printing
java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to [Ljava.lang.Integer;
How to fix it?
The array contents can be retrieved with these calls:
java.sql.Array a = rs.getArray("curAccs");
Object[] values = (Object[]) a.getArray();
Integer intValA = (Integer) values[0];
Integer intValB = (Integer) values[1];
The objective I'm trying to achieve
Obtain the data from source big JSON files ( employee-sample.json)
A simple spark application to read it as text file and store in parquet ( simple-loader.java ). I'm not aware of what is in JSON file so I cannot put any schema , so I want schema on read, and not schema on write. A parquet file with one column named "value" containing the JSON string in created
Create an HIVE external table on parquet files, and when I do "select * from table", I see one-column coming out with JSON data.
What I really need is creating a HIVE table which could read the JSON-data in "value" column and apply schema and emit columns, so that I can create variety of tables on my RAW-data, depending on the need.
I have created hive tables on JSON file, and extracted the columns , but this extract column from parquet and apply JSON schema is tricking me
employee-sample.json
{"name":"Dave", "age" : 30 , "DOB":"1987-01-01"}
{"name":"Steve", "age" : 31 , "DOB":"1986-01-01"}
{"name":"Kumar", "age" : 32 , "DOB":"1985-01-01"}
Simple Spark code to convert JSON to parquet
simple-loader.java
public static void main(String[] args) {
SparkSession sparkSession = SparkSession.builder()
.appName(JsonToParquet.class.getName())
.master("local[*]").getOrCreate();
Dataset<String> eventsDataSet = sparkSession.read().textFile("D:\\dev\\employee-sample.json");
eventsDataSet.createOrReplaceTempView("rawView");
sparkSession.sqlContext().sql("select string(value) as value from rawView")
.write()
.parquet("D:\\dev\\" + UUID.randomUUID().toString());
sparkSession.close();
}
hive table on parquet files
CREATE EXTERNAL TABLE EVENTS_RAW (
VALUE STRING)
STORED AS PARQUET
LOCATION 'hdfs://XXXXXX:8020/employee/data_raw';
I tried by setting JSON serde, but it works only if data is stored in JSON foram, ROW FORMAT SERDE 'com.proofpoint.hive.serde.JsonSerde'
EXPECTED FORMAT
CREATE EXTERNAL TABLE EVENTS_DATA (
NAME STRING,
AGE STRING,
DOB STRING)
??????????????????????????????
Create hive external table example:
public static final String CREATE_EXTERNAL = "CREATE EXTERNAL TABLE %s" +
" (%s) " +
" PARTITIONED BY(%s) " +
" STORED AS %s" +
" LOCATION '%s'";
/**
* Will create an external table and recover the partitions
*/
public void createExternalTable(SparkSession sparkSession, StructType schema, String tableName, SparkFormat format, List<StructField> partitions, String tablePath){
String createQuery = createTableString(schema, tableName, format, partitions, tablePath);
logger.info("Going to create External table with the following query:\n " + createQuery);
sparkSession.sql(createQuery);
logger.debug("Finish to create External table with the following query:\n " + createQuery);
recoverPartitions(sparkSession, tableName);
}
public String createTableString(StructType schema, String tableName, SparkFormat format, List<StructField> partitions, String tablePath){
Set<String> partitionNames = partitions.stream().map(struct -> struct.name()).collect(Collectors.toSet());
String columns = Arrays.stream(schema.fields())
//Filter the partitions
.filter(field -> !partitionNames.contains(field.name()))
//
.map(HiveTableHelper::fieldToStringBuilder)
.collect(Collectors.joining(", "));
String partitionsString = partitions.stream().map(HiveTableHelper::fieldToStringBuilder).collect(Collectors.joining(", "));
return String.format(CREATE_EXTERNAL, tableName, columns, partitionsString, format.name(), tablePath);
}
/**
*
* #param sparkSession
* #param table
*/
public void recoverPartitions(SparkSession sparkSession, String table){
String query = "ALTER TABLE " + table + " RECOVER PARTITIONS";
logger.debug("Start: " + query);
sparkSession.sql(query);
sparkSession.catalog().refreshTable(table);
logger.debug("Finish: " + query);
}
public static StringBuilder fieldToStringBuilder(StructField field){
StringBuilder sb = new StringBuilder();
sb.append(field.name()).append( " ").append(field.dataType().simpleString());
return sb;
}
I have a List of Objects called md. Each of this objects has an activityName, a startTime and an endTime(for the activity).
I want to iterate over this list and for each activity, get the startTime and endTime.
Map<String,Long> m1 = new HashMap<String,Long>();
m1 = md
.stream()
.map(s->s.activityName)
.collect(HashMap<String,Long>::new,
(map,string)->{
String d1 = md.get(md.indexOf(string)).startTime;
String d2 = md.get(md.indexOf(string)).endTime;
.
.
.
},HashMap<String,Long>::putAll);
It gives me java.lang.ArrayIndexOutOfBoundsException: -1 when I try to get the index of string String d1 = md.get(md.indexOf(string)).startTime;
Is there any other way to simplify the code using Lambda expressions?
What if I have two activities with the same name (Drinking for ex).Will it only return the index of the first Drinking activity it finds?
It seems that you are missing that fact that once you do:
md.stream().map(s -> s.activityName)
your Stream has become Stream<String>; while your md is still List<YourObject>
And in the map operation you are trying to find a String inside md, this obviously does not exist, thus a -1.
So you need a Map<String, Long> that is activitaName -> duration it takes(could be Date/Long)
md.stream()
.collect(Collectors.toMap(s -> s.activityName, x -> {
Date start = // parse s.startTime
Date end = // parse s.endTime
return end.minus(start);
}));
Now the parsing depends on the dates you use.