Save an object with image ( save both object data and image too) inside mongoDB using Java - spring

I want to know specifically about saving an object with an image inside it. What I want to do is saving an entire object with image inside it, Image must be saved. I tried this but it saves only File instance with file path. Its not saving the image. Any help would be appreciated. Thank you. Here is my code for saving an object but its saving a file instance instead of an image.
import java.io.File;
import org.springframework.data.mongodb.core.mapping.Document;
import com.discusit.model.Artwork;
#Document(collection="Artwork")
public class ArtworkImpl implements Artwork {
private String artworkName;
private String artworkVersion;
private String fileName;
private File file;
public ArtworkImpl() {
}
public ArtworkImpl(String name, String version, String fileName, File file) {
this.artworkName = name;
this.artworkVersion = version;
this.fileName = fileName;
this.file = file;
}
public String getArtworkName() {
return artworkName;
}
public void setArtworkName(String artworkName) {
this.artworkName = artworkName;
}
public String getArtworkVersion() {
return artworkVersion;
}
public void setArtworkVersion(String artworkVersion) {
this.artworkVersion = artworkVersion;
}
public String getFileName() {
return fileName;
}
public void setFileName(String fileName) {
this.fileName = fileName;
}
public File getFile() {
return file;
}
public void setFile(File file) {
this.file = file;
}
}
Here is my main method :-
NOTE : Main method works fine, but not saving image, instead saving file instance.
public class MainApplication {
public static void main(String[] args) {
ApplicationContext ctx =
new AnnotationConfigApplicationContext(SpringMongoConfig.class);
GridFsOperations gridOperations =
(GridFsOperations) ctx.getBean("gridFsTemplate");
DBObject metaData = new BasicDBObject();
metaData.put("extra1", "anything 1");
metaData.put("extra2", "anything 2");
InputStream inputStream = null;
try {
inputStream = new FileInputStream("/home/discusit/Downloads/birds.jpg");
gridOperations.store(inputStream, "birds.jpg", "image/jpg", metaData);
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
if (inputStream != null) {
try {
inputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
System.out.println("Done");
}
}
I want to save object with image.
UPDATE : I did this but by converting an image to byte array and fetching byte array and converting back to Image, just want to know is there any other way by which I can save an image directly in mongoDB but without converting it to byte array ????

You need to clarify the following about mongoDB:
1. MongoDB is a document oriented database, in which the documents are stored in a format called BSON and limited to a maximun of 16MB. "Think of BSON as a binary representation of JSON (JavaScript Object Notation) documents"[1].
The BSON format support a BinData type, in which you can store the binary representation of a file as long as the 16MB limit will not be exceded.
2. MongoDB provides a way to store files GridFS "GridFS is a specification for storing and retrieving files that exceed the BSON-document size limit of 16MB"[2].
GridFS divide the files in chunks of 256K and use two collections to store the files, one for metadata and one for the file chunks, this collections are called fs.files and fs.chunks respectively.
A file stored within these collections, looks like this:
>db.fs.files.find()
{
"_id": ObjectId("51a0541d03643c8cf4122760"),
"chunkSize": NumberLong("262144"),
"length": NumberLong("3145782"),
"md5": "c5dda7f15156783c53ffa42b422235b2",
"filename": "test.png",
"contentType": "image/bmp",
"uploadDate": ISODate("2013-05-25T06:03:09.611Z"),
"aliases": null,
"metadata": {
"extra1": "anything 1",
"extra2": "anything 2"
}
}
>db.fs.chunks.find()
{
"_id": ObjectId("51a0541e03643c8cf412276c"),
"files_id": ObjectId("51a0541d03643c8cf4122760"),
"n": 11,
"data": BinData(0, "BINARY_DATA_WILL_BE_STORED_HERE")
}
.
.
Note how the ObjectId in the files collections match the files_id in the chunks collections.
After this clarification the short answer to your question is:
Yes, you can store the files directly in mongoDB using GridFS.
In the following link you can find a GridFS working example using Spring Data:
http://www.mkyong.com/mongodb/spring-data-mongodb-save-binary-file-gridfs-example/
[1] http://docs.mongodb.org/manual/reference/glossary/#term-bson
[2] http://docs.mongodb.org/manual/core/gridfs/

Related

How to add file in Solr?

I use Apache Solr so that I can work with files, I can add regular text fields via Spring, but I don’t know how to add TXT / pdf
#SolrDocument(solrCoreName = "accounting")
public class Accounting {
#Id
#Field
private String id;
#Field
private File txtFile;
#Field
private String docType;
#Field
private String docTitle;
public Accounting() {
}
public Accounting(String id, String docType, String docTitle) {
this.id = id;
this.docTitle = docTitle;
this.docType = docType;
}
here is the problem with the txtFile field
<field name="docTitle" type="strings"/>
<field name="docType" type="strings"/>
These fields that I manually added to schema.xml, I can not figure out how to add a field here that will be responsible for the file, for example, I will add here a txt file, how to do it? Thank you very much. And do I correctly declare the field private File txtFile; in the entity for the file?
Solr will not store the actual file anywhere. Depending on your config it can store the binary content though. Using the extract request handler Apache Solr which relies on Apache Tika to extract the content from the document.
You can try something like below code. The current code is not using anything from the springboot. Here the content is read from the pdf document and then the data is indexed into solr along with id and filename. I have used the tika apis to extract the content of the pdf.
public static void main(final String[] args) throws IOException, TikaException, SAXException {
String urlString = "http://localhost:8983/solr/TestCore1";
SolrClient solr = new HttpSolrClient.Builder(urlString).build();
BodyContentHandler handler = new BodyContentHandler();
Metadata metadata = new Metadata();
File file = new File("C://Users//abhijitb//Desktop//TestDocument.pdf");
FileInputStream inputstream = new FileInputStream(file);
ParseContext pcontext = new ParseContext();
// parsing the document using PDF parser
PDFParser pdfparser = new PDFParser();
pdfparser.parse(inputstream, handler, metadata, pcontext);
// getting the content of the document
//System.out.println("Contents of the PDF :" + handler.toString());
try {
String fileName = file.getName();
SolrInputDocument document = new SolrInputDocument();
document.addField("id", "123456");
document.addField("title", fileName);
document.addField("text", handler.toString());
solr.add(document);
solr.commit();
} catch (SolrServerException | IOException e) {
e.printStackTrace();
}
}
Once you index the data, it can be verified on the solr admin page by querying for it.
Please find the image for your reference.

List Files from Templates Directory in Spring Boot

I would like to generate a blog posts overview. For that I want to read the html files from a folder inside the templates folder in the resources folder where Spring Boot stores its templates.
I tried that but it doesnt return an error but also list no files.
What is the way to go here?
Thanks
#Controller
public class Route {
#Autowired
private ResourceLoader resourceLoader;
#RequestMapping("/")
public String home() throws IOException {
final String path = "templates/blog";
final Resource res = resourceLoader.getResource("templates/blog");
try (final BufferedReader reader = new BufferedReader(new InputStreamReader(res.getInputStream()))) {
reader.lines().forEachOrdered(System.out::println);
}
return "blog/a";
}
}
#Controller
public class Route {
#Value("classpath:templates/blog/*")
private Resource[] resources;
#RequestMapping("/")
public String home() throws IOException {
for (final Resource res : resources) {
System.out.println(res.getFilename());
}
return "blog/a";
}
}
did the trick to me.
You should be able to achieve this using NIO2.
In order for NIO2 to work, it requires the concept of FileSystem, and one can be created from the jar URI. Then this file system can be used with Files/Paths.
The code below contains two branches - the first handles loading the files from inside Jar, the second branch - when the code runs from IDE or via "mvn spring-boot:run".
All streams are being used via try-with-resources so they will be auto-closed.
The find function starts from the top of the file system and recursively searches for html files.
public static void readFile(String location) throws URISyntaxException {
URI uri = Objects.requireNonNull(ReadFromJar.class.getClassLoader().getResource(location)).toURI();
if (uri.getScheme().equals("jar")) { //inside jar
try (FileSystem fs = FileSystems.newFileSystem(uri, Collections.emptyMap())) { //build a new FS that represents the jar's contents
Files.find(fs.getPath("/"), 10, (path, fileAttr) -> // control the search depth (e.g. 10)
fileAttr.isRegularFile() //match only files
&& path.toString().contains("blog") //match only files in paths containing "blog"
&& path.getFileName().toString().matches(".*\\.html")) // match only html files
.forEach(ReadFromJar::printFileContent);
} catch (IOException ex) {
ex.printStackTrace();
}
}
else { //from IDE or spring-boot:run
final Path path = Paths.get(uri);
try (DirectoryStream<Path> dirStream = Files.newDirectoryStream(path)) {
dirStream.forEach(ReadFromJar::printFileContent);
} catch (IOException e) {
e.printStackTrace();
}
}
}
private static void printFileContent(final Path file) {
try {
System.out.println("Full path: " + file.toAbsolutePath().toString());
Files.lines(file).forEach(System.out::println);
} catch (IOException e) {
e.printStackTrace();
}
}

How to upload an image or a video to a persistant folder in class-path with Spring-Boot?

I am new in Spring-Boot...
I want to upload images or videos, and store them in a persistant folder "upload-storage" in the class-path of my project in the server. I don't want to store them in the database (20 Mo).
Spring-Boot store them in target/upload-storage.
That functions : I can show the videos on the view with the controller and Thymeleaf. I can close tomcat, close the browser, and open them again : that functions.
But the day after, upload-storage is disapeared !
I think that I don't use the good process.
But I found how to upload an image : ok. I found how to show images from a folder in class-path : ok. I found how to upload images to database. But nothing to store the uploaded images in a persistant folder.
Can you help me ? Can you tell me the good process ?
Some details :
I have an entity "video" to store name, extension, length,... of the video.
I have "VideoRepository" and "VideoService" to manage the requests with "Video".
I have a "StorageService" and "StorageServiceImpl" to manage the upload of video and images : It as to upload the video and save it in a folder called "upload-storage" : I will come back on it farther.
I have a videoForm.html first with a form to select a file and send it to "UploadController", then an other form to show the video, the datas extracted from the video, modify the name or add precisions, and send this form to a "VideoController" who save the entity.
A part of the code of "UploadController" :
`
#Controller
public class UploadController extends BaseController {
private final StorageService storageServiceImpl;
#Autowired
public UploadController(StorageService storageServiceImpl) {
this.storageServiceImpl = storageServiceImpl;
}
#PostMapping("/upload")
public String recupereUpload(#RequestParam("file") MultipartFile file,Model model){
String filename ="";
try {
final long limit = 200 * 1024 * 1024;
if (file.getSize() > limit) {
model.addAttribute("message", "Taille du fichier trop grand (>200MB)");
model.addAttribute("ok", false );
}
filename = storageServiceImpl.store(file);
model.addAttribute("filename", filename);
model.addAttribute("message", "Le téléchargement de " + filename+" est réussi !");
} catch (Exception e) {
model.addAttribute("message", "FAIL to upload " + filename + "!");
model.addAttribute("ok", false );
}
Video video = new Video();
model.addAttribute("ok", true );
model.addAttribute("video", video);
String baseName = storageServiceImpl.getBaseName(filename);
String ext = storageServiceImpl.getExtension(filename);
model.addAttribute("nom", baseName);
model.addAttribute("ext", ext);
model.addAttribute("nomorigin", filename);
model.addAttribute("size", Math.round(file.getSize()/1024));
String typExt = storageServiceImpl.getType(ext);
model.addAttribute("typExt", typExt);
return "elementVideo/videoForm";
}
`
"StorageServiceImpl" has different methods :
getExtension(String filename){...}
getType(String ext){...}
getType(String ext){...}
getBaseName(String filename){...}
The main method is store(MultipartFile file) {...} :
#Service
public class StorageServiceImpl implements StorageService {
private final Path storageLocation = Paths.get("upload-storage");
#Override
public String store(MultipartFile file) {
try {
// Vérification de l'existence :
if (file.isEmpty()) {
throw new Exception("Failed to store empty file " + file.getOriginalFilename() );
}
// Vérification de la nature et traitement du fichier uploadé :
String ext = getExtension(file.getOriginalFilename());
String[] extAutorise = {"mp4", "avi","ogg","ogv","jpg","jpeg","png","gif"};
String fileNameTarget ="";
if ( ArrayUtils.contains( extAutorise, ext)) {
//Définir le fichier destination :
fileNameTarget = file.getOriginalFilename();
fileNameTarget = fileNameTarget.replaceAll(" ", "_");
File dir = storageLocation.toFile();
String serverFile = dir.getAbsolutePath() + File.separator + fileNameTarget ;
try {
try (InputStream is = file.getInputStream();
BufferedOutputStream stream = new BufferedOutputStream(new FileOutputStream(serverFile))
) {
int i;
while ((i = is.read()) != -1) {
stream.write(i);
}
stream.flush();
}
} catch (IOException e) {
System.out.println("error : " + e.getMessage());
}
}
return fileNameTarget;
} catch (Exception e) {
throw new RuntimeException("FAIL!");
}
}
`
With this code, a folder "upload-storage" is created at the root of the project.
The video is uploaded in this folder...
But in "videoForm.html", the code
<video id="video" th:src="'/upload-storage/'+${filename}" height="60"
autoplay="autoplay"></video>
shows nothing.
I have an other solution.
In StorageServiceImpl, I use the code :
private final String storageLocation = this.getClass().getResource("/static/").getPath();
at place of :
private final Path storageLocation = Paths.get("upload-storage");
then :
File dir = new File(storageLocation + File.separator + "upload-storage");
at place of :
File dir = storageLocation.toFile();
then :
File serverFile = new File(dir.getAbsolutePath() + File.separator + fileNameTarget);
at place of :
String serverFile = dir.getAbsolutePath() + File.separator + fileNameTarget ;
With this solution, upload-storage is created in target folder.
I use an other controller BaseController :
public class BaseController {
public static final String PARAM_BASE_URL = "baseURL";
public String getBaseURL(HttpServletRequest request){
return request.getScheme() + "://" + request.getServerName() + ":" + request.getServerPort() + request.getContextPath();
}
}
`
UploadController extends this BaseController.
I add HttpServletRequest request in recupereUpload() :
#PostMapping("/upload")
public String recupereUpload(#RequestParam("file") MultipartFile file,
Model model, HttpServletRequest request ){
I add in the model sent by recupereUpload :
model.addAttribute(PARAM_BASE_URL, getBaseURL(request));
And at last, I can see my video in videoForm.html with the code :
<video id="video" th:src="${baseURL}+'/upload-storage/'+${filename}" height="60" autoplay="autoplay"></video>
I can close Tomcat, close Eclipse, close the machine, and open all again : all is preserved and I can see the video.
But some time later : all is disappeared.
There must be a better solution.
Can you help me ?
Why dont you use Spring Content for the video content portion of your solution? That way you won't need to implement any of the video content handling. Spring Content will provide this for you. To add Spring Content to your project:
Add Spring Content to your classpath.
pom.xml
<dependency>
<groupId>com.github.paulcwarren</groupId>
<artifactId>spring-content-rest-boot-starter</artifactId>
<version>0.0.10</version>
</dependency>
<dependency>
<groupId>com.github.paulcwarren</groupId>
<artifactId>content-fs-spring-boot-starter</artifactId>
<version>0.0.10</version>
</dependency>
Associate content with your Video entity.
Video.java
#Entity
public class Video {
...
#ContentId
private String contentId;
#ContentLength
private Long contetLen;
#MimeType
private String mimeType;
...
Set up a "persistent folder" as the root of your video store. This is where uploaded videos will be stored/streamed from. Also create a VideoStore interface to describe to SC how you want to associate your content.
SpringBootApplication.java
#SpringBootApplication
public class YourSpringBootApplication {
public static void main(String[] args) {
SpringApplication.run(YourSpringBootApplication.class, args);
}
#Configuration
#EnableFilesystemStores
public static class StoreConfig {
File filesystemRoot() {
return new File("/path/to/your/videos");
}
#Bean
public FileSystemResourceLoader fsResourceLoader() throws Exception {
return new FileSystemResourceLoader(filesystemRoot().getAbsolutePath());
}
}
#StoreRestResource(path="videos")
public interface VideoStore extends ContentStore<Video,String> {
//
}
}
This is all you need to create a REST-based video service at /videos. Essentially, when your application starts, Spring Content will look at your dependencies (seeing Spring Content FS/REST), look at your VideoStore interface and inject an implementation of that interface based on the filesystem. It will also inject a controller that forwards http requests to that implementation as well. This saves you having to implement any of this yourself.
So...
POST /videos/{video-entity-id}
with a multipart/form-data request will store the video in /path/to/your/videos and associate it with the video entity whose id is video-entity-id.
GET /videos/{video-entity-id}
will fetch it again. This supports partial content requests or byte ranges; i.e. video streaming too.
and so on...support full CRUD.
There are a couple of getting started guides here. The reference guide is here. And there is a tutorial video here. The coding bit starts about 1/2 way through.
HTH
Did you enable the upload by adding the following property in the application.properties file?
## MULTIPART (MultipartProperties)
# Enable multipart uploads
spring.servlet.multipart.enabled=true
I have written an article about how to upload a multipart file in spring boot using thymeleaf. Here is the service used for the upload.
package com.uploadMultipartfile.storage;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.util.StringUtils;
import org.springframework.web.multipart.MultipartFile;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
#Service
public class FileSystemStorageService implements StorageService
{
private final Path rootLocation;
#Autowired
public FileSystemStorageService(StorageProperties properties) {
this.rootLocation = Paths.get(properties.getUploadDir()).toAbsolutePath().normalize();
try {
Files.createDirectories(this.rootLocation);
} catch (Exception ex) {
throw new StorageException("Could not create the directory where the uploaded files will be stored.", ex);
}
}
#Override
public String store(MultipartFile file)
{
// Normalize file name
String fileName = StringUtils.cleanPath(file.getOriginalFilename());
try
{
if (file.isEmpty())
{
throw new StorageException("Failed to store empty file " + file.getOriginalFilename());
}
// Copy file to the target location (Replacing existing file with the same name)
Path targetLocation = this.rootLocation.resolve(fileName);
Files.copy(file.getInputStream(), targetLocation, StandardCopyOption.REPLACE_EXISTING);
return fileName;
}
catch (IOException e)
{
throw new StorageException("Failed to store file " + file.getOriginalFilename(), e);
}
}
#Override
public void init()
{
try
{
Files.createDirectory(rootLocation);
}
catch (IOException e)
{
throw new StorageException("Could not initialize storage", e);
}
}
}
Here is a link to get the code of the application. http://mkaroune.e-monsite.com/pages/spring/spring-boot-multipart-file-upload.html

How to externalize the queries to xml files using spring

I am using spring and their JDBC template to do read/write operations to the database. I am facing a problem in my reporting module that i have to frequently change the query sqls to cater to frequent changes.
Though using spring jdbc ORM, is there a way to externalize my query parameters such that i just change it in the XML & restart and there is no need to rebuild my source again for deployment. Any approach ORM (preferred) or simple Sql will do.
As of now i have to change the query again and again ,rebuild the source and deploy.
I am not sure if Spring provides some out of the box solutions to implement what you want. But here is one way to get it done, which i had implemented ones. So i will try to reduce some hardwork for you.
You might need to implement a utility to load from resources xml file. Something like this.
public final class LoadFromResourceFileUtils {
public static String loadQuery(final String libraryPath,
final String queryName) {
final InputStream is = StreamUtils
.streamFromClasspathResource(libraryPath);
if (is == null) {
throw new RuntimeException(String.format(
"The SQL Libary %s could not be found.", libraryPath));
}
final Document doc = XMLParseUtils.parse(is);
final Element qryElem = (Element) doc.selectSingleNode(String.format(
"SQLQueries/SQLQuery[#name='%s']", queryName));
final String ret = qryElem == null ? null : qryElem.getText();
return ret;
}
}
You would need to store your queries in an XML say queries.xml and keep it in your classpath, for e.g
<?xml version="1.0" encoding="UTF-8"?>
<SQLQueries>
<SQLQuery name="myQuery">
<![CDATA[
your query
]]>
</SQLQuery>
</SQLQueries>
And in your DAO you can do this to get the query
String query = LoadFromResourceFileUtils.loadQuery(
"queries.xml", "myQuery");
XMLParseUtils and StreamUtils for your reference
public final class XMLParseUtils {
public static Document parse(final InputStream inStream) {
Document ret = null;
try {
if (inStream == null) {
throw new RuntimeException(
"XML Input Stream for parsing is null");
}
final SAXReader saxReader = new SAXReader();
ret = saxReader.read(inStream);
} catch (final DocumentException exc) {
throw new RuntimeException("XML Parsing error", exc);
}
return ret;
}
}
public final class StreamUtils {
public static InputStream streamFromClasspathResource(
final String resourceClassPath) {
final Class<StreamUtils> clazz = StreamUtils.class;
final ClassLoader clLoader = clazz.getClassLoader();
final InputStream inStream = clLoader
.getResourceAsStream(resourceClassPath);
if (inStream == null) {
if(LOGGER.isDebugEnabled()){
LOGGER.debug(String.format("Resource %s NOT FOUND.",
resourceClassPath));
}
}
return inStream;
}
}

How to Convert gson to LinkedHashMap<String, List<String>>?

i'm new to gson and i wonder how convert json data to LinkedHashMap<String, List<String>>
my json data is show like below:
{ "data":
{
"data1": ["asdf", "qwer"],
"data2": ["xczv", "aweqrfds123", "sfdgq234"],
"data3": ["dsafasd", "xcvr123", "sdfa324123"]
}
}
field names of json data of data are dynamic, so i want to convert json data of data to LinkedHashMap<String, List<String>>
how can i do that ?
You can use TypeToken to convert it into expected type with Gson#fromJson(Reader,Type)
As per JSON string it is LinkedHashMap<String,LinkedHashMap<String,ArrayList<String>>>
Sample code:
BufferedReader reader = new BufferedReader(new FileReader(new File("json.txt")));
Type type = new TypeToken<LinkedHashMap<String,LinkedHashMap<String,ArrayList<String>>>>() {}.getType();
LinkedHashMap<String,LinkedHashMap<String,ArrayList<String>>> data = new Gson().fromJson(reader, type);
LinkedHashMap<String,ArrayList<String>> innerMap = data.get("data");
System.out.println(new GsonBuilder().setPrettyPrinting().create().toJson(innerMap));
This is not how it works in Gson world - you can't convert JSON to any Java class you want, unless you want to do all of that manually. The common approach works as described below:
Create a Java class, which matches your JSON format, e.g. you can use a Java class generator described here: http://jsongen.byingtondesign.com/
Use GsonBuilder to read your Json from a file and to import it to the generated class
I've used that approach and the Java file that has been generated (after I've fixed a minor syntax error in your initial JSON) looks like this:
package com.json;
import java.util.List;
public class Data{
private List data1;
private List data2;
private List data3;
public List getData1(){
return this.data1;
}
public void setData1(List data1){
this.data1 = data1;
}
public List getData2(){
return this.data2;
}
public void setData2(List data2){
this.data2 = data2;
}
public List getData3(){
return this.data3;
}
public void setData3(List data3){
this.data3 = data3;
}
}
To start working with the newly created class you can use the template below:
is = new InputStreamReader(new FileInputStream(new File('<path-to-json>')), "UTF-8")/;
Gson gson = new GsonBuilder().create();
Data d = gson.fromJson(is, Data.class);
// Start using your d instance here

Resources