I Wanted to deploy an web application on Heroku which has to be served using staticfilesfolder in WebSocketServer.ja
port(getHerokuAssignedPort());
webSocketIdleTimeoutMillis(1000000000);
staticFileLocation("/public");
webSocket("/em/", WebSocketHandler.class);
init();
and websocketClient.js
var webSocket = new WebSocket("ws://" + location.hostname + ":" + location.port + "/em/");
But I was unable to server the *.html, *.css, *.js file from /public folder.It is working fine with my local system not working on heroku site.It shows application error.
public class Bootstrap {
private static final int DEFAULT_PORT = 4567;
public static void main(String[] args) throws URISyntaxException {
port(getHerokuAssignedPort());
externalStaticFileLocation(getPublicFolderPath());
}
private static int getHerokuAssignedPort() {
ProcessBuilder processBuilder = new ProcessBuilder();
if (processBuilder.environment().get("PORT") != null) {
return Integer.parseInt(processBuilder.environment().get("PORT"));
}
return DEFAULT_PORT;
}
private static String getPublicFolderPath() throws URISyntaxException{
CodeSource codeSource = Bootstrap.class.getProtectionDomain().getCodeSource();
File jarFile = new File(codeSource.getLocation().toURI().getPath());
String jarDir = jarFile.getParentFile().getPath();
return jarDir + File.separator + "public";
}}
/public directory should be in the same folder as build jar file.
Related
I am kind of running into this interesting problem. I wanted to delete the jetty working directory before jetty restarts next time. The application has 2 war files and this is implemented using embed jetty. Name of the working folder looks like below:
jetty-0_0_0_0-10000-oaxui_war-_ui_oax-any-4214704288653178451
jetty-0_0_0_0-10000-oaxservice_war-_api_oax-any-1938823993160354573
Options 1: We have run.sh file which actually starts JettyServer.So I thought to place the below code in the file just before that.
echo "now deleting"
DELTEMP="rm -rf /tmp/jetty*"
exec $DELTEMP
echo "deleted....."
Result: It actually deletes the jetty working directory but does not let it create one working directory also.
Option 2: Create a temp directory in the location provided by us as below and then later delete this folder using the run.sh and above command but the path will be custom. Unfortunately, this also didn't help as the working directory was not being created in the first place.
private HandlerCollection getWebAppHandlers() throws SQLException, NamingException{
//Setting the war and context path for the service layer: oaxservice
File tempDir1 = new File("/faw/service/appshell/tempdir/");
File tempDir2 = new File("/faw/service/appshell/tempdir/");
WebAppContext serviceWebapp = new WebAppContext();
tempDir1.mkdirs();
serviceWebapp.setTempDirectory(tempDir1);
serviceWebapp.setWar(APPSHELL_API_WAR_FILE_PATH);
serviceWebapp.setContextPath(APPSHELL_API_CONTEXT_PATH);
//setting the war and context path for the UI layer: oaxui
WebAppContext uiWebapp = new WebAppContext();
tempDir2.mkdirs();
uiWebapp.setTempDirectory(tempDir2);
uiWebapp.setWar(APPSHELL_UI_WAR_FILE_PATH);
uiWebapp.setContextPath(APPSHELL_UI_CONTEXT_PATH);
uiWebapp.setAllowNullPathInfo(true);
uiWebapp.setInitParameter("org.eclipse.jetty.servlet.Default.dirAllowed", "false");
//set error page handler for the UI context
uiWebapp.setErrorHandler(new CustomErrorHandler());
//handling the multiple war files using HandlerCollection.
HandlerCollection handlerCollection = new HandlerCollection();
handlerCollection.setHandlers(new Handler[]{serviceWebapp, uiWebapp});
return handlerCollection;
}
Below is the complete JettyServer.java file
public class JettyServer {
private final static Logger logger = Logger.getLogger(JettyServer.class.getName());
private static final int JETTY_PORT = 10000;
private static final String JETTY_REALM_PROPERTIES_FILE_NAME = "realm.properties";
private static final String JETTY_REALM_NAME = "myrealm";
private static final String APPSHELL_WAR_FOLDER = "/faw/service/appshell/target/";
private static final String APPSHELL_UI_WAR_FILE_PATH = APPSHELL_WAR_FOLDER+"oaxui.war";
private static final String APPSHELL_API_WAR_FILE_PATH = APPSHELL_WAR_FOLDER+"oaxservice.war";
private static final String APPSHELL_API_CONTEXT_PATH = "/api/oax";
private static final String APPSHELL_UI_CONTEXT_PATH = "/ui/oax";
private static final String JETTY_CONFIG_FOLDER = "/faw/service/appshell/config/";
private static final String JETTY_CONFIG_FILE = JETTY_CONFIG_FOLDER+"datasource.properties";
private static final String JETTY_CONFIG_DATASOURCE_URL = "datasource.url";
private static final String JETTY_CONFIG_APPSHELL_SCHEMA_NAME = "appshell.datasource.username";
private static final String JETTY_CONFIG_APPSHELL_SCHEMA_PWD = "appshell.datasource.password";
private static final String JETTY_CONFIG_APPSHELL_DS_INITIAL_POOL_SIZE = "appshell.datasource.initialPoolSize";
private static final String JETTY_CONFIG_APPSHELL_DS_MAX_POOL_SIZE = "appshell.datasource.maxPoolSize";
private static final String JETTY_CONFIG_FAWCOMMON_SCHEMA_NAME = "fawcommon.datasource.username";
private static final String JETTY_CONFIG_FAWCOMMON_SCHEMA_PWD = "fawcommon.datasource.password";
private static final String JETTY_CONFIG_FAWCOMMON_DS_INITIAL_POOL_SIZE = "fawcommon.datasource.initialPoolSize";
private static final String JETTY_CONFIG_FAWCOMMON_DS_MAX_POOL_SIZE = "fawcommon.datasource.maxPoolSize";
private static final int INITIAL_POOL_SIZE_DEFAULT = 0;
private static final int MAX_POOL_SIZE_DEFAULT = 50;
private static final String JNDI_NAME_FAWAPPSHELL = "jdbc/CXOMetadataDatasource";
private static final String JNDI_NAME_FAWCOMMON = "jdbc/FawCommonDatasource";
private static final String JETTY_REQUEST_LOG_FILE_NAME = "/faw/logs/appshell/applogs/jetty-request.yyyy_mm_dd.log";
private static final String JETTY_REQUEST_LOG_FILE_NAME_DATE_FORMAT = "yyyy_MM_dd";
private static final boolean JETTY_REQUEST_LOG_FILE_APPEND = true;
private static final int JETTY_REQUEST_LOG_FILE_RETAIN_DAYS = 31;
private static final String JETTY_STDOUT_LOG_FILE_NAME = "/faw/logs/appshell/applogs/jetty-out.yyyy_mm_dd.log";
private static final int MAX_REQUEST_HEADER_SIZE = 65535;
private static final String SSL_SERVER_KEY_STROKE_PATH="/faw/tmp/customscript/certs/server.jks";
private static final String SSL_TRUST_KEY_STROKE_PATH="/faw/tmp/customscript/certs/trust.jks";
private static final String SSL_KEY_STROKE_KEY="changeit";
public static QueuedThreadPool threadPool;
public JettyServer() {
try {
//Redirect system out and system error to our print stream.
RolloverFileOutputStream os = new RolloverFileOutputStream(JETTY_STDOUT_LOG_FILE_NAME, true);
PrintStream logStream = new PrintStream(os);
System.setOut(logStream);
System.setErr(logStream);
Server server = new Server(JETTY_PORT);
server.addBean(getLoginService());
//Set SSL context
if (isIDCSEnvironment) {
try {
logger.info("Configuring Jetty SSL..");
HttpConfiguration http_config = new HttpConfiguration();
http_config.setSecureScheme("https");
http_config.setSecurePort(JETTY_PORT);
SslContextFactory sslContextFactory = new SslContextFactory.Server();
sslContextFactory.setKeyStorePath(SSL_SERVER_KEY_STROKE_PATH);
sslContextFactory.setCertAlias("server");
sslContextFactory.setKeyStorePassword(SSL_KEY_STROKE_KEY);
sslContextFactory.setTrustStorePath(SSL_TRUST_KEY_STROKE_PATH);
sslContextFactory.setTrustStorePassword(SSL_KEY_STROKE_KEY);
HttpConfiguration https_config = new HttpConfiguration(http_config);
https_config.addCustomizer(new SecureRequestCustomizer());
ServerConnector https = new ServerConnector(server, new SslConnectionFactory(sslContextFactory, HttpVersion.HTTP_1_1.asString()), new HttpConnectionFactory(https_config));
https.setPort(JETTY_PORT);
server.setConnectors(new Connector[]{https});
logger.info("Jetty SSL successfully configured..");
} catch (Exception e){
logger.severe("Error configuring Jetty SSL.."+e);
throw e;
}
}
Configuration.ClassList classlist = Configuration.ClassList.setServerDefault(server);
classlist.addAfter("org.eclipse.jetty.webapp.FragmentConfiguration",
"org.eclipse.jetty.plus.webapp.EnvConfiguration",
"org.eclipse.jetty.plus.webapp.PlusConfiguration");
//register oaxui and oaxservice web apps
HandlerCollection webAppHandlers = getWebAppHandlers();
for (Connector c : server.getConnectors()) {
c.getConnectionFactory(HttpConnectionFactory.class).getHttpConfiguration().setRequestHeaderSize(MAX_REQUEST_HEADER_SIZE);
c.getConnectionFactory(HttpConnectionFactory.class).getHttpConfiguration().setSendServerVersion(false);
}
threadPool = (QueuedThreadPool) server.getThreadPool();
// request logs
RequestLogHandler requestLogHandler = new RequestLogHandler();
AsyncRequestLogWriter asyncRequestLogWriter = new AsyncRequestLogWriter(JETTY_REQUEST_LOG_FILE_NAME);
asyncRequestLogWriter.setFilenameDateFormat(JETTY_REQUEST_LOG_FILE_NAME_DATE_FORMAT);
asyncRequestLogWriter.setAppend(JETTY_REQUEST_LOG_FILE_APPEND);
asyncRequestLogWriter.setRetainDays(JETTY_REQUEST_LOG_FILE_RETAIN_DAYS);
asyncRequestLogWriter.setTimeZone(TimeZone.getDefault().getID());
requestLogHandler.setRequestLog(new AppShellCustomRequestLog(asyncRequestLogWriter));
webAppHandlers.addHandler(requestLogHandler);
StatisticsHandler statisticsHandler = new StatisticsHandler();
statisticsHandler.setHandler(new AppshellStatisticsHandler());
webAppHandlers.addHandler(statisticsHandler);
// set handler
server.setHandler(webAppHandlers);
//start jettyMetricsPsr
JettyMetricStatistics.logJettyMetrics();
// set error handler
server.addBean(new CustomErrorHandler());
// GZip Handler
GzipHandler gzip = new GzipHandler();
server.setHandler(gzip);
gzip.setHandler(webAppHandlers);
//setting server attribute for datasources
server.setAttribute("fawappshellDS", new Resource(JNDI_NAME_FAWAPPSHELL, DatasourceUtil.getFawAppshellDatasource()));
server.setAttribute("fawcommonDS", new Resource(JNDI_NAME_FAWCOMMON, DatasourceUtil.getCommonDatasource()));
//new Resource(server, JNDI_NAME_FAWAPPSHELL, getFawAppshellDatasource());
//new Resource(server, JNDI_NAME_FAWCOMMON, getFawCommonDatasource());
Map<String, String> configDetails;
if (isIDCSEnvironment) {
configDetails = DatasourceUtil.getConfigMap();
} else {
configDetails = DatasourceUtil.configMapInternal;
}
if (isIDCSEnvironment && configDetails.containsKey(PODDB_ATP_ENABLED) && BooleanUtils.toBoolean(configDetails.get(PODDB_ATP_ENABLED))){
configDetails.put(DatabagProperties.LCM_MASTER_PROPERTY,"true");
try {
DBUtils.migrateDBATP(configDetails, DatasourceUtil.getFawAppshellDatasourceATP());
configDetails.remove(DatabagProperties.LCM_MASTER_PROPERTY, "true");
} catch(SQLException e){
logger.info("Exception while executing DBUtils.migrateDBATP.");
if(LCMUtils.isATPDBDown(e)) {
logger.info("Redis flow starts....Skipping consumption from Redis for now");
}
else{
logger.info("This is not eligible for redis flow. Actual Exception : "+ e);
throw e;
}
}
} else if (isIDCSEnvironment && configDetails.containsKey(PODDB_ATP_ENABLED) && !BooleanUtils.toBoolean(configDetails.get(PODDB_ATP_ENABLED))){
try{
DBUtils.migrateDB();
} catch (FawLCMPluginException e) {
logger.info(" This is not eligible for Redis consumption and exception is : " + e);
}
}
//For Dev env..
if (!isIDCSEnvironment && configDetails.containsKey(PODDB_ATP_ENABLED) && BooleanUtils.toBoolean(configDetails.get(PODDB_ATP_ENABLED))){
configDetails.put(DatabagProperties.LCM_MASTER_PROPERTY,"true");
try{
DBUtils.migrateDBATP(configDetails,DatasourceUtil.getDevFawAppshellDatasourceATP());
configDetails.remove(DatabagProperties.LCM_MASTER_PROPERTY,"true");
}
catch(SQLException e){
logger.info("Exception while executing DBUtils.migrateDBATP for dev environment.");
if(LCMUtils.isATPDBDown(e)) {
logger.info("Redis flow starts....Skipping consumption from Redis for now");
} else{
logger.info("This is not eligible for redis flow. Actual Exception for dev: "+ e);
}
}
} else if (!isIDCSEnvironment && configDetails.containsKey(PODDB_ATP_ENABLED) && !BooleanUtils.toBoolean(configDetails.get(PODDB_ATP_ENABLED))){
try {
DBUtils.migrateDB();
}catch (FawLCMPluginException e) {
logger.info(" This is not eligible for Redis consumption and exception in dev is : " + e);
}
}
server.start();
server.join();
} catch (Exception e) {
e.printStackTrace();
}
}
private HandlerCollection getWebAppHandlers() throws SQLException, NamingException{
//Setting the war and context path for the service layer: oaxservice
File tempDir1 = new File("/faw/service/appshell/tempdir/");
File tempDir2 = new File("/faw/service/appshell/tempdir/");
WebAppContext serviceWebapp = new WebAppContext();
tempDir1.mkdirs();
serviceWebapp.setTempDirectory(tempDir1);
serviceWebapp.setWar(APPSHELL_API_WAR_FILE_PATH);
serviceWebapp.setContextPath(APPSHELL_API_CONTEXT_PATH);
//setting the war and context path for the UI layer: oaxui
WebAppContext uiWebapp = new WebAppContext();
tempDir2.mkdirs();
uiWebapp.setTempDirectory(tempDir2);
uiWebapp.setWar(APPSHELL_UI_WAR_FILE_PATH);
uiWebapp.setContextPath(APPSHELL_UI_CONTEXT_PATH);
uiWebapp.setAllowNullPathInfo(true);
uiWebapp.setInitParameter("org.eclipse.jetty.servlet.Default.dirAllowed", "false");
//set error page handler for the UI context
uiWebapp.setErrorHandler(new CustomErrorHandler());
//handling the multiple war files using HandlerCollection.
HandlerCollection handlerCollection = new HandlerCollection();
handlerCollection.setHandlers(new Handler[]{serviceWebapp, uiWebapp});
return handlerCollection;
}
/**
* The name of the LoginService needs to correspond to what is configured a webapp's web.xml which is
* <realm-name>myrealm</realm-name> and since it has a lifecycle of its own, we register it as a bean
* with the Jetty server object so it can be started and stopped according to the lifecycle of the server itself.
*
* #return the login service instance
* #throws FileNotFoundException In case realmProps is null
*/
public LoginService getLoginService() throws IOException {
URL realmProps = JettyServer.class.getClassLoader().getResource(JETTY_REALM_PROPERTIES_FILE_NAME);
if (realmProps == null)
throw new FileNotFoundException("Unable to find " + JETTY_REALM_PROPERTIES_FILE_NAME);
return new HashLoginService(JETTY_REALM_NAME, realmProps.toExternalForm());
}
public static void main(String[] args) {
new JettyServer();
}
}
Run.sh file :
#!/bin/bash
set -eu # Exit on error
set -o pipefail # Fail a pipe if any sub-command fails.
VERSION=1.0
cd "$(dirname "$0")"
RED='\033[0;31m'
ORANGE='\033[0;33m'
GREEN='\033[0;32m'
NC='\033[0m' # No Color
function die() {
printf "${RED}%s\n${NC}" "$1" >&2
exit 1
}
function warn() {
printf "${ORANGE}%s\n${NC}" "$1"
}
function info() {
printf "${GREEN}%s\n${NC}" "$1"
}
CURR_PID=$$
mem_args=8192
info "mem args "$mem_args
export MAX_MEM=$mem_args
#put the managed server specific variable setting below, then export the variables
USER_MEM_ARGS="-Xms4096m -Xmx"$MAX_MEM"m -XX:MetaspaceSize=1024M -XX:MaxMetaspaceSize=1024m"
export USER_MEM_ARGS
info "USER_MEM_ARGS= ${USER_MEM_ARGS}"
#FAW_JAVA_OPTIONS="-Djava.util.logging.config.file=/faw/service/appshell/config/ucp_log.properties -Doracle.jdbc.fanEnabled=false $USER_MEM_ARGS -Xloggc:/faw/logs/appshell/applogs/gc.jetty.log -XX:+HeapDumpOnOutOfMemoryError -XshowSettings:vm -XX:+PrintCodeCache -XX:ReservedCodeCacheSize=512M -XX:+UnlockCommercialFeatures -XX:+FlightRecorder -XX:FlightRecorderOptions=maxage=30m,defaultrecording=true,stackdepth=1024,dumponexit=true,dumponexitpath=/faw/logs/jetty/jetty_1618996595.jfr -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:+ExitOnOutOfMemoryError -XX:+DisableExplicitGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=4 -XX:GCLogFileSize=5M -XX:HeapDumpPath=/faw/logs/appshell_hprof-dumps_`date +%x_%r|awk -F" " '{print $1}'|sed 's/\//_/g'`.hprof"
FAW_JAVA_OPTIONS="-Djava.util.logging.config.file=/faw/service/appshell/config/ucp_log.properties -Doracle.jdbc.fanEnabled=false $USER_MEM_ARGS -Xloggc:/faw/logs/appshell/applogs/gc.jetty.log -XX:+HeapDumpOnOutOfMemoryError -XshowSettings:vm -XX:+PrintCodeCache -XX:ReservedCodeCacheSize=512M -XX:+UnlockCommercialFeatures -XX:+FlightRecorder -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:+ExitOnOutOfMemoryError -XX:+DisableExplicitGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=4 -XX:GCLogFileSize=5M -XX:HeapDumpPath=/faw/logs/appshell_hprof-dumps_`date +%x_%r|awk -F" " '{print $1}'|sed 's/\//_/g'`.hprof"
echo "now deleting"
DELTEMP="rm -rf /tmp/jetty*"
exec $DELTEMP
echo "deleted....."
info "FAW_JAVA_OPTIONS= ${FAW_JAVA_OPTIONS}"
if [[ -e "/faw/environment" ]]; then
sh /faw/tmp/customscript/setUpTLSCerts.sh
fi
CMD="java ${FAW_JAVA_OPTIONS} -cp /faw/service/appshell/target:/faw/service/appshell/libs/* oracle.biapps.cxo.docker.jetty.server.JettyServer --lib /faw/service/appshell/libs"
ulimit -c 0
# Now execute it
info "Executing: $CMD"
exec $CMD
Any help is appreciated. Thank you.
Thank you "Joakim Erdfelt" for the input. Got it. The below solution works fine for me in embedded jetty. I will be creating a custom delete method for deleting the directory first and then creating the jetty working directory.
public void cleanJettyWorkingDirectory(){
final File folder = new File(JETTY_TEMP_WORKING_DIRECTORY);
final File[] files = folder.listFiles((dir, name) -> name.matches( "oax.*" ));
if (files!=null && files.length > 0) {
for ( final File file : files ) {
try {
FileUtils.deleteDirectory(file);
logger.info("Cleaning of the directory is completed.");
} catch (IOException e) {
logger.info("Unable to delete the Jetty working directory");
}
}
}else{
logger.info("No working directory file is present starting with oax for deletion.");
}
}
Also while setting the war , we will do below to create directories:
try {
cleanJettyWorkingDirectory();
logger.info("starting to create temp working directory for oax service ");
java.nio.file.Path oaxServicePath = Files.createTempDirectory("oaxservice");
String oaxServiceTempPath = oaxServicePath.toString();
logger.info("starting to create temp working directory for oax ui ");
java.nio.file.Path uiPath = Files.createTempDirectory("oaxui");
String oaxUIServiceTempPath = uiPath.toString();
File oaxServiceTempDir = new File(oaxServiceTempPath);
File oaxUITempDir = new File(oaxUIServiceTempPath);
serviceWebapp.setTempDirectory(oaxServiceTempDir);
uiWebapp.setTempDirectory(oaxUITempDir);
logger.info("The process of creating working directory is completed.");
} catch (IOException e) {
logger.log(Level.SEVERE, "Exception while creating directories.",e);
}
I did simple repackaging and that caused Access-Control-Allow-Origin issue with my s3 cloud.
I have a local S3 compatible server to store videos , using Spring , am streaming the videos directly from my local cloud .
Everything is working as expected until I tried to repackage my classes .
I had one package com.example.video with the following classes:
S3Config.java this contain the AmazonS3Client
User.java A model class
VideoController.java Simple controller
VideoStreamingServiceApplication.java application class
When I created new package com.example.s3 package and moved both User.java and S3config.java , i had auto wired issue and that was fixed by using component scan as this answer suggested .
Even after the autowired issue fixed am getting an error when I try to stream .
Access to XMLHttpRequest at "http://localhost/9999/recordins/a.m3u8" fom origin 'null' has been block by CORS policy: No 'Access-Control-Allow-Origin' header is present on the request resource .
Although I do have the header mentioned in my request , Here is my VideoController.java
#RestController
#RequestMapping("/cloud")
#ConfigurationProperties(prefix = "amazon.credentials")
public class VideoCotroller {
#Autowired
private S3Config s3Client;
private String bucketName= "recordings";
Logger log = LoggerFactory.getLogger(VideoCotroller.class);
#Autowired
User userData;
#GetMapping(value = "/recordings/{fileName}", produces = { MediaType.APPLICATION_OCTET_STREAM_VALUE })
public ResponseEntity<StreamingResponseBody> streamVideo(HttpServletRequest request, #PathVariable String fileName) {
try {
long rangeStart = 0;
long rangeEnd;
AmazonS3 s3client = s3Client.getAmazonS3Client();
String uri = request.getRequestURI();
System.out.println("Fetching " + uri);
S3Object object = s3client.getObject("recordings", fileName);
long size = object.getObjectMetadata().getContentLength();
S3ObjectInputStream finalObject = object.getObjectContent();
final StreamingResponseBody body = outputStream -> {
int numberOfBytesToWrite = 0;
byte[] data = new byte[(int) size];
while ((numberOfBytesToWrite = finalObject.read(data, 0, data.length)) != -1) {
outputStream.write(data, 0, numberOfBytesToWrite);
}
finalObject.close();
};
rangeEnd = size - 1;
return ResponseEntity.status(HttpStatus.OK)
.header("Content-Type", "application/vnd.apple.mpegurl")
.header("Accept-Ranges", "bytes")
// HERE IS THE ACCESS CONTROL ALLOW ORIGIN
.header("Access-Control-Allow-Origin", "*")
.header("Content-Length", String.valueOf(size))
.header("display", "staticcontent_sol, staticcontent_sol")
.header("Content-Range", "bytes" + " " + rangeStart + "-" + rangeEnd + "/" + size)
.body(body);
//return new ResponseEntity<StreamingResponseBody>(body, HttpStatus.OK);
} catch (Exception e) {
System.err.println("Error "+ e.getMessage());
return new ResponseEntity<StreamingResponseBody>(HttpStatus.BAD_REQUEST);
}}
If I restore the packages to one , as it was before everything is working fine .
MY QUESTION : Why repackaging caused this issue , any idea how to fix this ?
I would like to generate a blog posts overview. For that I want to read the html files from a folder inside the templates folder in the resources folder where Spring Boot stores its templates.
I tried that but it doesnt return an error but also list no files.
What is the way to go here?
Thanks
#Controller
public class Route {
#Autowired
private ResourceLoader resourceLoader;
#RequestMapping("/")
public String home() throws IOException {
final String path = "templates/blog";
final Resource res = resourceLoader.getResource("templates/blog");
try (final BufferedReader reader = new BufferedReader(new InputStreamReader(res.getInputStream()))) {
reader.lines().forEachOrdered(System.out::println);
}
return "blog/a";
}
}
#Controller
public class Route {
#Value("classpath:templates/blog/*")
private Resource[] resources;
#RequestMapping("/")
public String home() throws IOException {
for (final Resource res : resources) {
System.out.println(res.getFilename());
}
return "blog/a";
}
}
did the trick to me.
You should be able to achieve this using NIO2.
In order for NIO2 to work, it requires the concept of FileSystem, and one can be created from the jar URI. Then this file system can be used with Files/Paths.
The code below contains two branches - the first handles loading the files from inside Jar, the second branch - when the code runs from IDE or via "mvn spring-boot:run".
All streams are being used via try-with-resources so they will be auto-closed.
The find function starts from the top of the file system and recursively searches for html files.
public static void readFile(String location) throws URISyntaxException {
URI uri = Objects.requireNonNull(ReadFromJar.class.getClassLoader().getResource(location)).toURI();
if (uri.getScheme().equals("jar")) { //inside jar
try (FileSystem fs = FileSystems.newFileSystem(uri, Collections.emptyMap())) { //build a new FS that represents the jar's contents
Files.find(fs.getPath("/"), 10, (path, fileAttr) -> // control the search depth (e.g. 10)
fileAttr.isRegularFile() //match only files
&& path.toString().contains("blog") //match only files in paths containing "blog"
&& path.getFileName().toString().matches(".*\\.html")) // match only html files
.forEach(ReadFromJar::printFileContent);
} catch (IOException ex) {
ex.printStackTrace();
}
}
else { //from IDE or spring-boot:run
final Path path = Paths.get(uri);
try (DirectoryStream<Path> dirStream = Files.newDirectoryStream(path)) {
dirStream.forEach(ReadFromJar::printFileContent);
} catch (IOException e) {
e.printStackTrace();
}
}
}
private static void printFileContent(final Path file) {
try {
System.out.println("Full path: " + file.toAbsolutePath().toString());
Files.lines(file).forEach(System.out::println);
} catch (IOException e) {
e.printStackTrace();
}
}
I am new in Spring-Boot...
I want to upload images or videos, and store them in a persistant folder "upload-storage" in the class-path of my project in the server. I don't want to store them in the database (20 Mo).
Spring-Boot store them in target/upload-storage.
That functions : I can show the videos on the view with the controller and Thymeleaf. I can close tomcat, close the browser, and open them again : that functions.
But the day after, upload-storage is disapeared !
I think that I don't use the good process.
But I found how to upload an image : ok. I found how to show images from a folder in class-path : ok. I found how to upload images to database. But nothing to store the uploaded images in a persistant folder.
Can you help me ? Can you tell me the good process ?
Some details :
I have an entity "video" to store name, extension, length,... of the video.
I have "VideoRepository" and "VideoService" to manage the requests with "Video".
I have a "StorageService" and "StorageServiceImpl" to manage the upload of video and images : It as to upload the video and save it in a folder called "upload-storage" : I will come back on it farther.
I have a videoForm.html first with a form to select a file and send it to "UploadController", then an other form to show the video, the datas extracted from the video, modify the name or add precisions, and send this form to a "VideoController" who save the entity.
A part of the code of "UploadController" :
`
#Controller
public class UploadController extends BaseController {
private final StorageService storageServiceImpl;
#Autowired
public UploadController(StorageService storageServiceImpl) {
this.storageServiceImpl = storageServiceImpl;
}
#PostMapping("/upload")
public String recupereUpload(#RequestParam("file") MultipartFile file,Model model){
String filename ="";
try {
final long limit = 200 * 1024 * 1024;
if (file.getSize() > limit) {
model.addAttribute("message", "Taille du fichier trop grand (>200MB)");
model.addAttribute("ok", false );
}
filename = storageServiceImpl.store(file);
model.addAttribute("filename", filename);
model.addAttribute("message", "Le téléchargement de " + filename+" est réussi !");
} catch (Exception e) {
model.addAttribute("message", "FAIL to upload " + filename + "!");
model.addAttribute("ok", false );
}
Video video = new Video();
model.addAttribute("ok", true );
model.addAttribute("video", video);
String baseName = storageServiceImpl.getBaseName(filename);
String ext = storageServiceImpl.getExtension(filename);
model.addAttribute("nom", baseName);
model.addAttribute("ext", ext);
model.addAttribute("nomorigin", filename);
model.addAttribute("size", Math.round(file.getSize()/1024));
String typExt = storageServiceImpl.getType(ext);
model.addAttribute("typExt", typExt);
return "elementVideo/videoForm";
}
`
"StorageServiceImpl" has different methods :
getExtension(String filename){...}
getType(String ext){...}
getType(String ext){...}
getBaseName(String filename){...}
The main method is store(MultipartFile file) {...} :
#Service
public class StorageServiceImpl implements StorageService {
private final Path storageLocation = Paths.get("upload-storage");
#Override
public String store(MultipartFile file) {
try {
// Vérification de l'existence :
if (file.isEmpty()) {
throw new Exception("Failed to store empty file " + file.getOriginalFilename() );
}
// Vérification de la nature et traitement du fichier uploadé :
String ext = getExtension(file.getOriginalFilename());
String[] extAutorise = {"mp4", "avi","ogg","ogv","jpg","jpeg","png","gif"};
String fileNameTarget ="";
if ( ArrayUtils.contains( extAutorise, ext)) {
//Définir le fichier destination :
fileNameTarget = file.getOriginalFilename();
fileNameTarget = fileNameTarget.replaceAll(" ", "_");
File dir = storageLocation.toFile();
String serverFile = dir.getAbsolutePath() + File.separator + fileNameTarget ;
try {
try (InputStream is = file.getInputStream();
BufferedOutputStream stream = new BufferedOutputStream(new FileOutputStream(serverFile))
) {
int i;
while ((i = is.read()) != -1) {
stream.write(i);
}
stream.flush();
}
} catch (IOException e) {
System.out.println("error : " + e.getMessage());
}
}
return fileNameTarget;
} catch (Exception e) {
throw new RuntimeException("FAIL!");
}
}
`
With this code, a folder "upload-storage" is created at the root of the project.
The video is uploaded in this folder...
But in "videoForm.html", the code
<video id="video" th:src="'/upload-storage/'+${filename}" height="60"
autoplay="autoplay"></video>
shows nothing.
I have an other solution.
In StorageServiceImpl, I use the code :
private final String storageLocation = this.getClass().getResource("/static/").getPath();
at place of :
private final Path storageLocation = Paths.get("upload-storage");
then :
File dir = new File(storageLocation + File.separator + "upload-storage");
at place of :
File dir = storageLocation.toFile();
then :
File serverFile = new File(dir.getAbsolutePath() + File.separator + fileNameTarget);
at place of :
String serverFile = dir.getAbsolutePath() + File.separator + fileNameTarget ;
With this solution, upload-storage is created in target folder.
I use an other controller BaseController :
public class BaseController {
public static final String PARAM_BASE_URL = "baseURL";
public String getBaseURL(HttpServletRequest request){
return request.getScheme() + "://" + request.getServerName() + ":" + request.getServerPort() + request.getContextPath();
}
}
`
UploadController extends this BaseController.
I add HttpServletRequest request in recupereUpload() :
#PostMapping("/upload")
public String recupereUpload(#RequestParam("file") MultipartFile file,
Model model, HttpServletRequest request ){
I add in the model sent by recupereUpload :
model.addAttribute(PARAM_BASE_URL, getBaseURL(request));
And at last, I can see my video in videoForm.html with the code :
<video id="video" th:src="${baseURL}+'/upload-storage/'+${filename}" height="60" autoplay="autoplay"></video>
I can close Tomcat, close Eclipse, close the machine, and open all again : all is preserved and I can see the video.
But some time later : all is disappeared.
There must be a better solution.
Can you help me ?
Why dont you use Spring Content for the video content portion of your solution? That way you won't need to implement any of the video content handling. Spring Content will provide this for you. To add Spring Content to your project:
Add Spring Content to your classpath.
pom.xml
<dependency>
<groupId>com.github.paulcwarren</groupId>
<artifactId>spring-content-rest-boot-starter</artifactId>
<version>0.0.10</version>
</dependency>
<dependency>
<groupId>com.github.paulcwarren</groupId>
<artifactId>content-fs-spring-boot-starter</artifactId>
<version>0.0.10</version>
</dependency>
Associate content with your Video entity.
Video.java
#Entity
public class Video {
...
#ContentId
private String contentId;
#ContentLength
private Long contetLen;
#MimeType
private String mimeType;
...
Set up a "persistent folder" as the root of your video store. This is where uploaded videos will be stored/streamed from. Also create a VideoStore interface to describe to SC how you want to associate your content.
SpringBootApplication.java
#SpringBootApplication
public class YourSpringBootApplication {
public static void main(String[] args) {
SpringApplication.run(YourSpringBootApplication.class, args);
}
#Configuration
#EnableFilesystemStores
public static class StoreConfig {
File filesystemRoot() {
return new File("/path/to/your/videos");
}
#Bean
public FileSystemResourceLoader fsResourceLoader() throws Exception {
return new FileSystemResourceLoader(filesystemRoot().getAbsolutePath());
}
}
#StoreRestResource(path="videos")
public interface VideoStore extends ContentStore<Video,String> {
//
}
}
This is all you need to create a REST-based video service at /videos. Essentially, when your application starts, Spring Content will look at your dependencies (seeing Spring Content FS/REST), look at your VideoStore interface and inject an implementation of that interface based on the filesystem. It will also inject a controller that forwards http requests to that implementation as well. This saves you having to implement any of this yourself.
So...
POST /videos/{video-entity-id}
with a multipart/form-data request will store the video in /path/to/your/videos and associate it with the video entity whose id is video-entity-id.
GET /videos/{video-entity-id}
will fetch it again. This supports partial content requests or byte ranges; i.e. video streaming too.
and so on...support full CRUD.
There are a couple of getting started guides here. The reference guide is here. And there is a tutorial video here. The coding bit starts about 1/2 way through.
HTH
Did you enable the upload by adding the following property in the application.properties file?
## MULTIPART (MultipartProperties)
# Enable multipart uploads
spring.servlet.multipart.enabled=true
I have written an article about how to upload a multipart file in spring boot using thymeleaf. Here is the service used for the upload.
package com.uploadMultipartfile.storage;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.util.StringUtils;
import org.springframework.web.multipart.MultipartFile;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
#Service
public class FileSystemStorageService implements StorageService
{
private final Path rootLocation;
#Autowired
public FileSystemStorageService(StorageProperties properties) {
this.rootLocation = Paths.get(properties.getUploadDir()).toAbsolutePath().normalize();
try {
Files.createDirectories(this.rootLocation);
} catch (Exception ex) {
throw new StorageException("Could not create the directory where the uploaded files will be stored.", ex);
}
}
#Override
public String store(MultipartFile file)
{
// Normalize file name
String fileName = StringUtils.cleanPath(file.getOriginalFilename());
try
{
if (file.isEmpty())
{
throw new StorageException("Failed to store empty file " + file.getOriginalFilename());
}
// Copy file to the target location (Replacing existing file with the same name)
Path targetLocation = this.rootLocation.resolve(fileName);
Files.copy(file.getInputStream(), targetLocation, StandardCopyOption.REPLACE_EXISTING);
return fileName;
}
catch (IOException e)
{
throw new StorageException("Failed to store file " + file.getOriginalFilename(), e);
}
}
#Override
public void init()
{
try
{
Files.createDirectory(rootLocation);
}
catch (IOException e)
{
throw new StorageException("Could not initialize storage", e);
}
}
}
Here is a link to get the code of the application. http://mkaroune.e-monsite.com/pages/spring/spring-boot-multipart-file-upload.html
I'm using Embedded Jetty server to run my tests and Maven for build.
Following is the code used to start Jetty before tests.
System.out.println("Initializing Jetty Server...");
jettyServer = new Server(0);
WebAppContext webapp = new WebAppContext("src/main/webapp", "/testApp");
jettyServer.addHandler(webapp);
jettyServer.start();
int actualPort = jettyServer.getConnectors()[0].getLocalPort();
String baseUrl = "http://localhost:" + actualPort + "/testApp";
All the tests passes if I run it with 'Run as Junit Test'. No problems here.
But if I run it as Maven Test or Maven Install The tests fails withe following cause
Caused by: com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException: 500 /WEB-INF/pages/test1.jsp(3,62) PWC6188: The absolute uri: http://java.sun.com/jsp/jstl/core cannot be resolved in either web.xml or the jar files deployed with this application
I have the JSTL dependancy added in POM.xml. Please help to resolve this..
You didn't initialize and setup the environment in a way that is suitable for JSP use.
It requires a bunch of extra work.
You'll need to manipulate classloaders, setup some initializers, declare the javac implementation behavior, and even declare the jsp servlet handling. (missing any one of these and you'll be subject to the environment that you executed under, which is different in your 3 examples)
For a complete example maven project see https://github.com/jetty-project/embedded-jetty-jsp
public class Main
{
// Resource path pointing to where the WEBROOT is
private static final String WEBROOT_INDEX = "/webroot/";
public static void main(String[] args) throws Exception
{
int port = 8080;
LoggingUtil.config();
Log.setLog(new JavaUtilLog());
Main main = new Main(port);
main.start();
main.waitForInterrupt();
}
private static final Logger LOG = Logger.getLogger(Main.class.getName());
private int port;
private Server server;
private URI serverURI;
public Main(int port)
{
this.port = port;
}
public URI getServerURI()
{
return serverURI;
}
public void start() throws Exception
{
server = new Server();
ServerConnector connector = new ServerConnector(server);
connector.setPort(port);
server.addConnector(connector);
URL indexUri = this.getClass().getResource(WEBROOT_INDEX);
if (indexUri == null)
{
throw new FileNotFoundException("Unable to find resource " + WEBROOT_INDEX);
}
// Points to wherever /webroot/ (the resource) is
URI baseUri = indexUri.toURI();
// Establish Scratch directory for the servlet context (used by JSP compilation)
File tempDir = new File(System.getProperty("java.io.tmpdir"));
File scratchDir = new File(tempDir.toString(),"embedded-jetty-jsp");
if (!scratchDir.exists())
{
if (!scratchDir.mkdirs())
{
throw new IOException("Unable to create scratch directory: " + scratchDir);
}
}
// Set JSP to use Standard JavaC always
System.setProperty("org.apache.jasper.compiler.disablejsr199","false");
// Setup the basic application "context" for this application at "/"
// This is also known as the handler tree (in jetty speak)
WebAppContext context = new WebAppContext();
context.setContextPath("/");
context.setAttribute("javax.servlet.context.tempdir",scratchDir);
context.setResourceBase(baseUri.toASCIIString());
context.setAttribute(InstanceManager.class.getName(), new SimpleInstanceManager());
server.setHandler(context);
// Add Application Servlets
context.addServlet(DateServlet.class,"/date/");
//Ensure the jsp engine is initialized correctly
JettyJasperInitializer sci = new JettyJasperInitializer();
ServletContainerInitializersStarter sciStarter = new ServletContainerInitializersStarter(context);
ContainerInitializer initializer = new ContainerInitializer(sci, null);
List<ContainerInitializer> initializers = new ArrayList<ContainerInitializer>();
initializers.add(initializer);
context.setAttribute("org.eclipse.jetty.containerInitializers", initializers);
context.addBean(sciStarter, true);
// Set Classloader of Context to be sane (needed for JSTL)
// JSP requires a non-System classloader, this simply wraps the
// embedded System classloader in a way that makes it suitable
// for JSP to use
ClassLoader jspClassLoader = new URLClassLoader(new URL[0], this.getClass().getClassLoader());
context.setClassLoader(jspClassLoader);
// Add JSP Servlet (must be named "jsp")
ServletHolder holderJsp = new ServletHolder("jsp",JspServlet.class);
holderJsp.setInitOrder(0);
holderJsp.setInitParameter("logVerbosityLevel","DEBUG");
holderJsp.setInitParameter("fork","false");
holderJsp.setInitParameter("xpoweredBy","false");
holderJsp.setInitParameter("compilerTargetVM","1.7");
holderJsp.setInitParameter("compilerSourceVM","1.7");
holderJsp.setInitParameter("keepgenerated","true");
context.addServlet(holderJsp,"*.jsp");
//context.addServlet(holderJsp,"*.jspf");
//context.addServlet(holderJsp,"*.jspx");
// Add Example of mapping jsp to path spec
ServletHolder holderAltMapping = new ServletHolder("foo.jsp", JspServlet.class);
holderAltMapping.setForcedPath("/test/foo/foo.jsp");
context.addServlet(holderAltMapping,"/test/foo/");
// Add Default Servlet (must be named "default")
ServletHolder holderDefault = new ServletHolder("default",DefaultServlet.class);
LOG.info("Base URI: " + baseUri);
holderDefault.setInitParameter("resourceBase",baseUri.toASCIIString());
holderDefault.setInitParameter("dirAllowed","true");
context.addServlet(holderDefault,"/");
// Start Server
server.start();
// Show server state
if (LOG.isLoggable(Level.FINE))
{
LOG.fine(server.dump());
}
// Establish the Server URI
String scheme = "http";
for (ConnectionFactory connectFactory : connector.getConnectionFactories())
{
if (connectFactory.getProtocol().equals("SSL-http"))
{
scheme = "https";
}
}
String host = connector.getHost();
if (host == null)
{
host = "localhost";
}
int port = connector.getLocalPort();
serverURI = new URI(String.format("%s://%s:%d/",scheme,host,port));
LOG.info("Server URI: " + serverURI);
}
public void stop() throws Exception
{
server.stop();
}
/**
* Cause server to keep running until it receives a Interrupt.
* <p>
* Interrupt Signal, or SIGINT (Unix Signal), is typically seen as a result of a kill -TERM {pid} or Ctrl+C
*/
public void waitForInterrupt() throws InterruptedException
{
server.join();
}
}
In the following repo: https://github.com/ericminio/learning-jetty
You can find:
The JstlTest that demoes Jetty serving jsp containing a c:forEach tag
The pom needed to remove the error message about resolving http://java.sun.com/jsp/jstl/core
Hope it helps