Spring boot with flyway - use a dynamic db password - spring-boot

I want to integrate flyway in my spring boot project, however i cannot write the password to the database in the properties file.
with my normal datasource i use the following code
#Configuration
#Slf4j
public class SwatDataBaseConfig {
#Value("${swat.decrypt.location}")
private String fileLocation;
#Value("${swat.datasource.url}")
private String dbURL;
#Value("${swat.datasource.driver-class-name}")
private String driverName;
#Value("${swat.datasource.username}")
private String userName;
#Value("${swat.datasource.password}")
private String hashedPassword;
#Bean
public DataSource primaryDataSource() {
// String password = encryptor.decrypt(hashedPassword);
// log.debug("password is: " + password);
String password = null;;
Process process = null;
try {
process = new ProcessBuilder(fileLocation, hashedPassword).start();
} catch (IOException e) {
log.error("Could not read the file", e);
return null;
}
InputStream is = process.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line;
try {
while ((line = br.readLine()) != null) {
password = line;
try {
password = line.split(" ")[1].trim();
}catch(Exception ex) {
log.error("Error while sanitating the password", ex);
}
}
} catch (IOException e) {
log.error("Could not process the file output", e);
}
PoolProperties poolProperties = new PoolProperties();
poolProperties.setUrl(dbURL);
poolProperties.setUsername(userName);
poolProperties.setPassword(password);
poolProperties.setDriverClassName(driverName);
poolProperties.setTestOnBorrow(true);
poolProperties.setValidationQuery("SELECT 1");
poolProperties.setValidationInterval(0);
DataSource ds = new org.apache.tomcat.jdbc.pool.DataSource(poolProperties);
process.destroy();
return ds;
// return DataSourceBuilder.create().url(dbURL).driverClassName(driverName).username(userName).password(password).build();
}
this takes my encrypted password from the properties, decrypts it and than passes it to the database,
In all the flyway examples i've seen , the password is written as plain text in the flyway properties file
How can i configure it the way i'm using the primary datasource

In this case you can use environment variables to define your password.
To do that you can set the property as :
swat.datasource.password=${SWAT_PASSWORD}
and when you will deploy or run your application you just need to set the SWAT_PASSWORD variable in your environment.
you can also use a default value that the system will use it if the variable is not set
swat.datasource.password=${SWAT_PASSWORD:default_password}

You can override the Flyway Bean by adding your own data source and configuration to Flyway where you can pass your own decrypted password. You will now have to manually configure Flyway in this Bean as auto-configuration will no longer work in order to use other Flyway properties.
#Configuration
public class FlywayConfiguration {
#Value("${spring.flyway.url}")
private String flywayUrl;
#Value("${spring.flyway.user}")
private String flywayUser;
#Value("${spring.flyway.password}")
private String flywayPassword;
// can still use spring.flyway.enabled
#ConditionalOnProperty(prefix = "spring.flyway", name = "enabled", matchIfMissing = true)
#ConfigurationProperties(prefix = "spring.flyway")
#Bean(initMethod = "migrate")
public Flyway flyway() {
DataSource dataSource = new DataSource();
try {
PoolProperties prop = new PoolProperties();
prop.setUrl(flywayUrl);
prop.setUsername(flywayUser);
prop.setPassword(flywayPassword); // Add custom code to decrypt password
dataSource = new DataSource(prop);
} catch (Exception e) {
log.error(e);
log.error(e.getMessage());
}
// Configure Flyway with other properties if necessary
return Flyway.configure().dataSource(dataSource).load();
}
}
More information on the Flyway Java API:
https://flywaydb.org/documentation/api/
Execute Flyway Database Migrations on Startup

Related

Loading value from json upon start up application

I want to load the values from json file upon the Spring Boot Application is started.
My code for the Configuration File is like the below:
#Configuration
#Getter
public class FedexAPIConfig {
private final static String JSON_FILE = "/static/config/fedex-api-credentials.json";
private final boolean IS_PRODUCTION = false;
private FedexAPICred apiCredentials;
public FedexAPIConfig() {
try (InputStream in = getClass().getResourceAsStream(JSON_FILE);
BufferedReader reader = new BufferedReader(new InputStreamReader(in, StandardCharsets.UTF_8))) {
JSONObject json = new JSONObject();
// this.apiCredentials = new JSONObject(new JSONTokener(reader));
if (IS_PRODUCTION) {
json = new JSONObject(new JSONTokener(reader)).getJSONObject("production");
} else {
json = new JSONObject(new JSONTokener(reader)).getJSONObject("test");
}
System.out.println(json.toString());
this.apiCredentials = FedexAPICred.builder()
.url(json.optString("url"))
.apiKey(json.optString("api_key"))
.secretKey(json.optString("secret_key"))
.build();
} catch (FileNotFoundException fnfe) {
fnfe.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
}
and with this, when the application is in progress of startup, values are successfully printed on the console.Startup console log
When I tried to call this value from other ordinary class, like the below:, it brings nothing but just throws NullPointerException... What are my faults and what shall I do?
public class FedexOAuthTokenManager extends OAuthToken {
private static final String VALIDATE_TOKEN_URL = "/oauth/token";
private static final String GRANT_TYPE_CLIENT = "client_credentials";
private static final String GRANT_TYPE_CSP = "csp_credentials";
#Autowired
private FedexAPIConfig fedexApiConfig;
#Autowired
private Token token;
#Override
public void validateToken() {
// This is the part where "fedexApiConfig" is null.
FedexAPICred fedexApiCred = fedexApiConfig.getApiCredentials();
Response response = null;
try {
RequestBody body = new FormBody.Builder()
.add("grant_type", GRANT_TYPE_CLIENT)
.add("client_id", fedexApiCred.getApiKey())
.add("client_secret", fedexApiCred.getSecretKey())
.build();
response = new HttpClient().post(fedexApiCred.getUrl() + VALIDATE_TOKEN_URL, body);
if (response.code() == 200) {
JSONObject json = new JSONObject(response.body().string());
token.setAccessToken(json.optString("access_token"));
token.setTokenType(json.optString("token_type"));
token.setExpiredIn(json.optInt("expires_in"));
token.setExpiredDateTime(LocalDateTime.now().plusSeconds(json.optInt("expires_in")));
token.setScope(json.optString("scope"));
}
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
}
fedexApiConfg is null even though I autowired it in prior to call.
And this FedexOAuthTokenManager is called from other #Component class by new FedexOAuthTokenManager()
Did you try like below?
Step 1: Create one Configuration class like below
public class DemoConfig implements ApplicationListener<ApplicationPreparedEvent> {
#Override
public void onApplicationEvent(ApplicationPreparedEvent event) {
//Load the values from the JSON file and populate the application
//properties dynamically
ConfigurableEnvironment environment = event.getApplicationContext().getEnvironment();
Properties props = new Properties();
props.put("spring.datasource.url", "<my value>");
//Add more properties
environment.getPropertySources().addFirst(new PropertiesPropertySource("myProps", props));
}
To listen to a context event, a bean should implement the ApplicationListener interface which has just one method onApplicationEvent().The ApplicationPreparedEvent is invoked very early in the lifecycle of the application
Step 2: Customize in src/main/resources/META-INF/spring.factories
org.springframework.context.ApplicationListener=com.example.demo.DemoConfig
Step 3: #Value in spring boot is commonly used to inject the configuration values into the spring boot application. Access the properties as per your wish.
#Value("${spring.datasource.url}")
private String valueFromJSon;
Try this sample first in your local machine and then modify your changes accordingly.
Refer - https://www.baeldung.com/spring-value-annotation
Refer - https://www.knowledgefactory.net/2021/02/aws-secret-manager-service-as.html

Send emails to multiple email addresses, It feeds email addresses from a csv file

I wanna create an application to send emails to several recipients. It feeds email addresses from a csv file and sends an email to each recipient, and I'm getting some trouble doing this.
Could you help me please?
Here is my CSVHelper.java
#Component
public class CSVHelper
{
public static String TYPE = "text/csv";
static String[] HEADERs = { "id", "email", "dateEcheance"};
//This method is used to filter the csv file and get only the emails
public List<ContactsFile> csvToEmails() throws NumberFormatException, ParseException
{
InputStream is = null;
try (BufferedReader fileReader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
CSVParser csvParser = new CSVParser(fileReader, CSVFormat.DEFAULT.withFirstRecordAsHeader().withIgnoreHeaderCase().withTrim());)
{
List<ContactsFile> emailsList = new ArrayList<>();
Iterable<CSVRecord> csvRecords = csvParser.getRecords();
for (CSVRecord csvRecord : csvRecords)
{
ContactsFile contact = new ContactsFile(csvRecord.get("email"));
emailsList.add(contact);
}
System.out.println(emailsList);
return emailsList;
}
catch (IOException e) { throw new RuntimeException("fail to get emails: " + e.getMessage()); }
}
We call csvToEmails() method in the controller to send the emails
#Autowired
private CSVHelper csvHelper;
#PostMapping("/getdetails")
public #ResponseBody EmailNotification sendMail(#RequestBody EmailNotification details) throws Exception {
MimeMessage message = sender.createMimeMessage();
MimeMessageHelper helper = new MimeMessageHelper(message,
MimeMessageHelper.MULTIPART_MODE_MIXED_RELATED,
StandardCharsets.UTF_8.name());
try {
helper.setTo((InternetAddress) csvHelper.csvToEmails());
helper.setText(details.getMessage(),true);
helper.setSubject("Test Mail");
} catch (javax.mail.MessagingException e) {
e.printStackTrace();
}
sender.send(message);
return details;
this is an example of the csv file:
id,email,dateEcheance
1,address#email.com,10/05/2021
2,address2#email.com,10/02/2021
I'm new to spring boot, and I'm in trouble completing this project.
Assuming that you've mail server configurations in place. First, you need to provide the JavaMailSender implementation. Spring does provide the implementation for the same. You've to create a bean and config the mail server configurations as follows.
#Configuration
public class MailConfig {
#Autowired
private Environment env;
#Bean
public JavaMailSender mailSender() {
try {
JavaMailSenderImpl mailSenderImpl = new JavaMailSenderImpl();
mailSenderImpl.setHost(env.getRequiredProperty("mail.smtp.host"));
mailSenderImpl.setUsername(env.getRequiredProperty("mail.smtp.username"));
mailSenderImpl.setPassword(env.getRequiredProperty("mail.smtp.password"));
mailSenderImpl.setDefaultEncoding(env.getRequiredProperty("mail.smtp.encoding"));
mailSenderImpl.setPort(Integer.parseInt(env.getRequiredProperty("mail.smtp.port")));
mailSenderImpl.setProtocol(env.getRequiredProperty("mail.transport.protocol"));
Properties p = new Properties();
p.setProperty("mail.smtp.starttls.enable", "true");
p.setProperty("mail.smtp.auth", "true");
mailSenderImpl.setJavaMailProperties(p);
return mailSenderImpl;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
Configuration settings in application.properties. Add settings according to the mail server.
# SMTP mail settings
mail.smtp.host=**********
mail.smtp.username=**********
mail.smtp.password=**********
mail.smtp.port=**********
mail.smtp.socketFactory.port=**********
mail.smtp.encoding=utf-8
mail.transport.protocol=smtp
mail.smtp.auth=true
mail.smtp.timeout=10000
mail.smtp.starttls.enable=true
mail.smtp.socketFactory.class=javax.net.ssl.SSLSocketFactory
CSVHelper.java
#Component
public class CSVHelper {
// This method is used to filter the csv file and get only the emails
public List<String> csvToEmails() {
InputStream is = ClassLoader.getSystemClassLoader().getResourceAsStream("csvFile.csv");
try (BufferedReader fileReader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
CSVParser csvParser = new CSVParser(fileReader,
CSVFormat.DEFAULT.withFirstRecordAsHeader().withIgnoreHeaderCase().withTrim());) {
List<String> emails = new ArrayList<>();
Iterable<CSVRecord> csvRecords = csvParser.getRecords();
for (CSVRecord csvRecord : csvRecords) {
String email = csvRecord.get("email");
emails.add(email);
}
return emails;
} catch (IOException e) {
throw new RuntimeException("fail to get emails: " + e.getMessage());
}
}
}
Send mail service
#Service
public class MailSenderService {
#Autowired
private CSVHelper csvHelper;
#Autowired
private JavaMailSender sender;
public void sendMail(EmailNotification details) {
try {
List<String> recipients = csvHelper.csvToEmails();
String[] to = recipients.stream().toArray(String[]::new);
JavaMailSenderImpl jms = (JavaMailSenderImpl) sender;
MimeMessage message = jms.createMimeMessage();
MimeMessageHelper helper = new MimeMessageHelper(message, true);
// add the sender email address below
helper.setFrom("sender#mail.com");
helper.setTo(to);
helper.setText(details.getMessage(), true);
helper.setSubject("Test Mail");
sender.send(message);
} catch (javax.mail.MessagingException | NumberFormatException e) {
// action for exception case
}
}
}

Export entities to database schema through java code

A long time ago, I did that with a code like that:
Configuration config = new Configuration();
Properties props = new Properties();
FileInputStream fos = = new FileInputStream( file_name );
props.load(fos);
fos.close();
config.setProperties(props);
config.addAnnotatedClass(...);
Connection conn = DriverManager.getConnection(url,usuario,senha);
SchemaExport schema = new SchemaExport();
schema.create(true, true);
But now, if I try use this code, I got a compilation error. Seeing the javadoc for SchemaExport, I notice a lot of changes in the methods used on this example.
Hpw I could do that now?
update
based on the suggested link, I implemented the method this way:
public void criarTabelas(String server, String user, String pass) throws Exception {
StandardServiceRegistry standardRegistry = new StandardServiceRegistryBuilder().applySetting("hibernate.hbm2ddl.auto", "create").applySetting("hibernate.dialect", dialect).applySetting("hibernate.id.new_generator_mappings", "true").build();
MetadataSources sources = new MetadataSources(standardRegistry);
for(Class<?> entity : lista_entidades())
sources.addAnnotatedClass(entity);
MetadataImplementor metadata = (MetadataImplementor) sources.getMetadataBuilder().build();
SchemaExport export = new SchemaExport();
export.create(EnumSet.of(TargetType.DATABASE), metadata);
}
private List<Class<?>> lista_entidades() throws Exception {
List<Class<?>> lista = new ArrayList<Class<?>>();
ClassPathScanningCandidateComponentProvider scanner = new ClassPathScanningCandidateComponentProvider(false);
scanner.addIncludeFilter(new AnnotationTypeFilter(Entity.class));
for (BeanDefinition bd : scanner.findCandidateComponents("org.loja.model"))
lista.add(Class.forName(bd.getBeanClassName()));
return lista;
}
Now I need a way to establish a jdbc connection and associate to the SchemaExport.
I solve this issue with this code:
public void criarTabelas(String server, String user, String pass) throws Exception {
Connection conn = DriverManager.getConnection(url_prefix+server+"/"+url_suffix, user, pass);
StandardServiceRegistry standardRegistry = new StandardServiceRegistryBuilder()
.applySetting("hibernate.hbm2ddl.auto", "create")
.applySetting("hibernate.dialect", dialect)
.applySetting("hibernate.id.new_generator_mappings", "true")
.applySetting("javax.persistence.schema-generation-connection", conn)
.build();
MetadataSources sources = new MetadataSources(standardRegistry);
for(Class<?> entity : lista_entidades())
sources.addAnnotatedClass(entity);
MetadataImplementor metadata = (MetadataImplementor) sources.getMetadataBuilder().build();
SchemaExport export = new SchemaExport();
export.create(EnumSet.of(TargetType.DATABASE), metadata);
conn.close();
}
private List<Class<?>> lista_entidades() throws Exception {
List<Class<?>> lista = new ArrayList<Class<?>>();
ClassPathScanningCandidateComponentProvider scanner = new ClassPathScanningCandidateComponentProvider(false);
scanner.addIncludeFilter(new AnnotationTypeFilter(Entity.class));
for (BeanDefinition bd : scanner.findCandidateComponents("org.loja.model"))
lista.add(Class.forName(bd.getBeanClassName()));
System.out.println("lista: "+lista);
return lista;
}

Keycloak.json configuration file import

I have configured my goclient with keycloak.I am reading the keycloak server details from keycloak.json file which is present inside my Go client.
My whole setup is in Kubernetes cluster.Everytime when new keycloak server comes up how should I pass keycloak.json file to the Go client so that it can read the details from json file?
Is there a way I can get the keycloak.json file with all the client configuration details without having to login to keycloak and download it ?
"realm": "SolutionAdmin",
"auth-server-url": "localhost:8081/auth",
"ssl-required": "none",
"resource": "admin",
"credentials": {
"secret": "6ee0f523-c392-4406-bb14-ba315125c6ea"
},
"confidential-port": 0
}
if you want to have a pre-configured embedded keycloak server, you'll need to extend KeycloakApplication and have it configured to trigger in lieu of the base class. in this example, KeycloakProperties is just a representation of all the keycloak properties we used in application.properties. but you can get the gist of it. fair warning: i did not write this, but am figuring out how a coworker did for another project.
public class EmbeddedKeycloakApplication extends KeycloakApplication {
public final KeycloakProperties keycloakProperties;
public EmbeddedKeycloakApplication() {
super();
keycloakProperties = SpringContextAdapter.getBean(KeycloakProperties.class);
System.getProperties().putAll(keycloakProperties.getRealmProperties());
createMasterRealmAdminUser();
createRealm();
}
private void createMasterRealmAdminUser() {
log.debug("Creating administrative user.");
String username = keycloakProperties.getServer(KeycloakProperties.ADMIN_USER_USERNAME);
String password = keycloakProperties.getServer(KeycloakProperties.ADMIN_USER_PASSWORD);
String email = keycloakProperties.getServer(KeycloakProperties.ADMIN_USER_EMAIL);
KeycloakSession session = getSessionFactory().create();
ApplianceBootstrap applianceBootstrap = new ApplianceBootstrap(session);
try {
session.getTransactionManager().begin();
if (!applianceBootstrap.isNoMasterUser()) {
log.debug("Administrative user already exists. No work to do.");
return;
}
applianceBootstrap.createMasterRealmUser(username, password);
RealmModel adminRealm = session.realms().getRealm(Config.getAdminRealm());
UserModel adminUser = session.users().getUserByUsername(username, adminRealm);
adminUser.setEmail(email);
adminUser.setEmailVerified(true);
session.getTransactionManager().commit();
log.info("Created administrative user {}", username);
} catch (Exception ex) {
log.error("Couldn't create keycloak master admin user: {}", ex.getMessage());
session.getTransactionManager().rollback();
}
session.close();
}
private void createRealm() {
String realmImportFilename = keycloakProperties.getRealmImportFile();
KeycloakSession session = getSessionFactory().create();
String realmId = keycloakProperties.getRealm(KeycloakProperties.REALM_ID);
try {
session.getTransactionManager().begin();
RealmManager manager = new RealmManager(session);
if (manager.getRealm(realmId) != null) {
log.debug("{} realm already exists. No work to do.", realmId);
return;
}
Resource realmImportFile = new ClassPathResource(realmImportFilename);
RealmRepresentation rep =
JsonSerialization.readValue(
realmImportFile.getInputStream(), RealmRepresentation.class, true);
manager.importRealm(rep);
log.info("Imported Realm json file {}", realmImportFilename);
session.getTransactionManager().commit();
} catch (Exception e) {
log.error("Failed to import Realm json file {}: {}", realmImportFilename, e.getMessage(), e);
session.getTransactionManager().rollback();
}
session.close();
}
}
#Configuration
public class EmbeddedKeycloakConfig {
#Bean
ServletRegistrationBean<HttpServlet30Dispatcher> keycloakJaxRsApplication(
KeycloakProperties keycloakProperties, DataSource dataSource) throws NamingException {
mockJndiEnvironment(dataSource);
var contextPath = keycloakProperties.getServer(KeycloakProperties.SERVER_CONTEXT_PATH);
ServletRegistrationBean<HttpServlet30Dispatcher> servlet =
new ServletRegistrationBean<>(new HttpServlet30Dispatcher());
servlet.addInitParameter(
"javax.ws.rs.Application", EmbeddedKeycloakApplication.class.getName());
servlet.addInitParameter(
ResteasyContextParameters.RESTEASY_SERVLET_MAPPING_PREFIX, contextPath);
servlet.addInitParameter(ResteasyContextParameters.RESTEASY_USE_CONTAINER_FORM_PARAMS, "true");
servlet.addInitParameter(ResteasyContextParameters.RESTEASY_DISABLE_HTML_SANITIZER, "true");
servlet.addUrlMappings(contextPath + "/*");
servlet.setLoadOnStartup(1);
servlet.setAsyncSupported(true);
return servlet;
}
#Bean
FilterRegistrationBean<EmbeddedKeycloakRequestFilter> keycloakSessionManagement(
KeycloakProperties keycloakProperties) {
FilterRegistrationBean<EmbeddedKeycloakRequestFilter> filter = new FilterRegistrationBean<>();
filter.setName("Keycloak Session Management");
filter.setFilter(new EmbeddedKeycloakRequestFilter());
filter.addUrlPatterns(
keycloakProperties.getServer(KeycloakProperties.SERVER_CONTEXT_PATH) + "/*");
return filter;
}
private void mockJndiEnvironment(DataSource dataSource) throws NamingException {
NamingManager.setInitialContextFactoryBuilder(
env ->
environment ->
new InitialContext() {
#Override
public Object lookup(Name name) {
return lookup(name.toString());
}
#Override
public Object lookup(String name) {
if ("spring/datasource".equals(name)) {
return dataSource;
}
return null;
}
#Override
public NameParser getNameParser(String name) {
return CompositeName::new;
}
#Override
public void close() {
// NOOP
}
});
}
}

Karaf executing commands in java console

The below code executes karaf commands in the console. This is working fine in karaf version 3.0.3. It fails in 4.0.0 or above
#Inject
CommandProcessor commandProcessor;
private class dummyCallable implements Callable{
final ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
final PrintStream printStream = new PrintStream(byteArrayOutputStream);
final CommandSession commandSession = commandProcessor.createSession(System.in, printStream, System.err);
String[] commands;
dummyCallable( String[] commands){
this.commands = commands;
}
public String call() {
try {
for(String command:commands) {
System.err.println(command);
commandSession.execute(command);
}
} catch (Exception e) {
e.printStackTrace(System.err);
}
return byteArrayOutputStream.toString();
}
}
protected String executeCommands(final String ...commands) {
String response;
dummyCallable dd = new dummyCallable(commands);
ExecutorService executor = Executors.newFixedThreadPool(1);
FutureTask<String> futureTask2 = new FutureTask<String>(dd);
try {
executor.submit(futureTask2);
response = futureTask2.get();
} catch (Exception e) {
e.printStackTrace(System.err);
response = "SHELL COMMAND TIMED OUT: ";
}
return response;
}
Can anybody provide me with a code which will work karaf 4.0.0
In Karaf 4, you should inject into your services a SessionFactory and call sessionFactory.createSession(System.in, printStream, System.err)
The methods are the same between Karaf3 and Karaf4, the class and package have changed :
CommandProcessor => SessionFactory
CommandSession => Session

Resources