Activate profile while submitting Apache storm topology of a maven project (Without recompiling) - maven

Question

I have to activate the profile while submitting the jar to storm like we do for a spring/boot project. like below
java -jar project-xyz.jar --spring.profiles.active=dev.
It is a maven project with multiple sub modules. we have the resources structured
Anything that is common to all the profiles will go in the root directory of resources, and anything that is specific to a profile (like DB connections) will go in that particular profile directory.
The root directory file will have the place holders for profile specific properties and will be replaced by the actual properties defined in profile directory.
Ex: ${mysql.host} will be resolved to localhost when local profile is active.
This final file will be placed in the classpath when we build the jar using
mvn clean install -P{profile}
And then the topology is submitted to storm like the following.
storm jar project-xyz.jar path.to.class.containing.main.method
The final properties file generated by maven will be read by a property reader, stored and served whenever requested.
private static final Map<String, Properties> properties = new HashMap<>();
public static String getProperty( String file, String key )
{
try {
if ( !properties.containsKey( file ) ) {
synchronized ( properties ) {
if ( !properties.containsKey( file ) ) {
loadProps( file );
}
}
}
return properties.get( file ).getProperty( key );
} catch ( IOException e ) {
throw new PropertyException( "There was a problem getting " + key + " from " + file + ".properties", e );
}
}
private static void loadProps( String file ) throws IOException
{
InputStream inputStream = PropertyFileHandler.class.getClassLoader().getResourceAsStream( file + ".properties" );
if ( inputStream == null ) {
throw new FileNotFoundException( "Property file " + file + ".properties not found" );
}
Properties prop = new Properties();
prop.load( inputStream );
properties.put( file, prop );
}
I've already gone through this question and its different in a way that I use maven and I have to activate the profile instead of providing specific property.
So, is there a way I can activate the profile while submitting the jar to storm somewhat like the following?
storm jar project-xyz.jar --profiles.active=dev
If not, what can I do to achieve this without re-compiling the source? Does converting it to a spring boot project help?

Answer 1

You won't be able to do this as long as you're using Maven to generate a jar containing a single properties file. That jar can only be used in the environment you generated it for.
Instead, you could include all the possible properties files in the jar, and decide which one to read based on a system or environment property.
A better option is probably that you generate the property files separately to the jar, and then put the jar plus property file into Storm. You can pass the property file path to your Storm submission code as a command line argument to your main method. You would then read the properties into memory, and pass them to your bolts/spouts. You need to do this loading eagerly (i.e. read the entire properties at topology submission time, rather than when you need a specific property), so your lazy loading configuration code should be discarded.
You might want something like Flux for convenience:
To enable property filtering, use the --filter command line option and
specify a .properties file. For example, if you invoked flux like so:
storm jar myTopology-0.1.0-SNAPSHOT.jar org.apache.storm.flux.Flux
--local my_config.yaml --filter dev.properties
With the following dev.properties file:
kafka.zookeeper.hosts: localhost:2181
You would then be able to reference those properties by key in your
.yaml file using ${} syntax:
- id: "zkHosts"
className: "org.apache.storm.kafka.ZkHosts"
constructorArgs:
- "${kafka.zookeeper.hosts}"
See the documentation at https://storm.apache.org/releases/2.1.0/flux.html.