I created a custom Spring Cloud Service Connector by defining the following two classes:
#ServiceLabel("kafka")
public class KafkaServiceInfo extends BaseServiceInfo {
private static Logger logger = Logger.getLogger(KafkaServiceInfo.class.getName());
public static final String BROKERS = "brokers";
public static final String REGISTRY = "schemaregistry";
protected List<String> brokers;
protected String registry;
public KafkaServiceInfo(String id, List<String> brokers, String registry) {
super(id);
this.brokers = brokers;
this.registry = registry;
}
#ServiceProperty
public String getRegistry() {
return registry;
}
#ServiceProperty
public List<String> getBrokers() {
return brokers;
}
}
And this class:
public class KafkaServiceInfoCreator extends CloudFoundryServiceInfoCreator<KafkaServiceInfo> {
private static Logger logger = Logger.getLogger(KafkaServiceInfoCreator.class.getName());
public static final String USER_PROVIDED_SERVICE_NAME = "kafka";
public KafkaServiceInfoCreator() {
super(new Tags(USER_PROVIDED_SERVICE_NAME), null);
}
public KafkaServiceInfo createServiceInfo(Map<String, Object> serviceData) {
String id = getId(serviceData);
Map<String, Object> credentials = getCredentials(serviceData);
List<String> brokers = (List<String>) credentials.get(KafkaServiceInfo.BROKERS);
String registry = (String) credentials.get(KafkaServiceInfo.REGISTRY);
logger.info("KafkaServiceInfo created for Cloud Foundry Service \"" + id + "\"");
logger.info("Kafka Brokers configured for Cloud Foundry Service: " + brokers.toString());
logger.info("Schema Registry configured for Cloud Foundry Service: " + registry);
return new KafkaServiceInfo(id, brokers, registry);
}
#Override
public boolean accept(Map<String, Object> serviceData) {
return getId(serviceData).contains(USER_PROVIDED_SERVICE_NAME);
}
}
On my PCF instance, I created a user-provided service that looks in the VCAPS env variables like this:
"user-provided": [
{
"credentials": {
"brokers": [
"<some-ip-here>:29092"
],
"schemaregistry": "http://<some-ip-here>:8081"
},
"label": "user-provided",
"name": "kafka",
"syslog_drain_url": "",
"tags": [],
"volume_mounts": []
}
]
I also added the service definition file into the META-INF folder.
src/main/resources/META-INF/services/org.springframework.cloud.cloudfoundry.CloudFoundryServiceInfoCreator
with the content:
path.to.my.file.KafkaServiceInfoCreator
I would now expect to see the properties whose getters are annotated with #ServiceProperty in the cloud.services.kafka.* path. However, they do not show up. Instead, I just have the following 2 entries:
"cloud.services.kafka.id": "kafka",
"cloud.services.null.id": "kafka",
I am wondering what is going wrong here and also why I have the second entry with the "null" in between.
Any ideas what could be wrong here? The classes somehow seem to be found since I get the log messages defined in the creator class above.
Regards, Lars
The #ServiceProperty annotation expects to be provided with either a category or name attribute. These values are used to build the key that is placed into the cloud.services map. If neither category or name attribute are provided, the #ServiceProperty annotation does not result in the property appearing in the map.
A typical usage is #ServiceProperty(category="connection"), with the name defaulting to the name of the property using bean naming rules. In your case, adding the category="connection" attribute should result in
"cloud.services.kafka.id": "kafka",
"cloud.services.kafka.connection.registry": "http://<some-ip-here>:8081",
"cloud.services.kafka.connection.brokers": ["<some-ip-here>:29092"],
I'm not sure yet where the "cloud.services.null.id": "kafka" entry is coming from. Let me know if that still appears after adding the attributes to the #ServiceProperty annotation.
After digging quite a bit deeper, I found the "reason" explained here.
The cloud foundry java buildpack includes the auto-reconfiguration library which by itself contains a copy of the org.springframework.cloud namespace. This copy does NOT consider any custom ServiceInfo classes.
An since it is also this auto-reconfiguration that exposes the cloud.services.* properties to the environment, my personal ones are not picked-up and exposed as well. So I will switch this auto-reconfiguration off and configure the required stuff manually.
The spring cloud connector documentation is misleading here as well, since the properties in cloud.service.* are only added by java auto reconfiguration to the environment.
Related
can anyone suggest any tutorial/sample project for Implementing Snapshot in AXON 4.4.2 with springBoot 2.3.3.
i went through the documentation(https://docs.axoniq.io/reference-guide/axon-framework/tuning/event-snapshots#snapshotting) and did below:
The AxonConfig.class
#Configuration
public class AxonConfig {
#Bean
public SnapshotTriggerDefinition app1SnapshotTrigger(Snapshotter snapshotter) {
return new EventCountSnapshotTriggerDefinition(snapshotter, 10);
}
}
The Aggregate
#Aggregate(snapshotTriggerDefinition = "app1SnapshotTrigger")
public class MyAggregate {
#AggregateIdentifier
private String id;
private String name;
#AggregateMember
private List<Address> addresses = new ArrayList<>();
private MyAggregate () {
}
#CommandHandler
private MyAggregate (CreateNameCommand createNameCommand) {
-----
}
#EventSourcingHandler
private void on(NameCreatedEvent nameCreatedEvent) {
----
}
Am i missing something. Will it create a snapshot at the threshold value 10.
Thanks.
unfortunately we have no sample demo ready to show in this case.
From your code snippet looks that all is in place. Maybe there is some other configuration that is taking over your annotation.
To give a try, I applied your configuration to our https://github.com/AxonIQ/giftcard-demo/
First note that can guide is the following
if you declared a Repository as we did in https://github.com/AxonIQ/giftcard-demo/blob/master/src/main/java/io/axoniq/demo/giftcard/command/GcCommandConfiguration.java#L17 this configuration will take over Annotation placed into your aggregate. If you prefer annotation, you can remove this Bean definition.
Here the piece of code, instead, to have this configured as a Bean
#Bean
public Repository<GiftCard> giftCardRepository(EventStore eventStore, SnapshotTriggerDefinition giftCardSnapshotTrigger) {
return EventSourcingRepository.builder(GiftCard.class)
.snapshotTriggerDefinition(giftCardSnapshotTrigger)
.eventStore(eventStore)
.build();
}
#Bean
public SpringAggregateSnapshotterFactoryBean snapshotter() {
var springAggregateSnapshotterFactoryBean = new SpringAggregateSnapshotterFactoryBean();
//Setting async executors
springAggregateSnapshotterFactoryBean.setExecutor(Executors.newSingleThreadExecutor());
return springAggregateSnapshotterFactoryBean;
}
#Bean("giftCardSnapshotTrigger")
public SnapshotTriggerDefinition giftCardSnapshotTriggerDefinition(Snapshotter snapshotter) {
return new EventCountSnapshotTriggerDefinition(snapshotter, 10);
}
You can check that your snapshot is working fine looking at client log : after 10 events on the same agggregateId, you should find this info log entry
o.a.a.c.event.axon.AxonServerEventStore : Snapshot created
To check you can use the REST api to retrieve the events from an aggregate
curl -X GET "http://localhost:8024/v1/events?aggregateId=A01"
This will produce a stream containing events starting from the latest Snapshot: you will have nine events listed until the tenth event will be processed. After that, the endpoint will list events from the snapshot.
You can also check /actuator/health endpoint: it will shows the last snapshot token if the showDetails is enabled (enabled by default in EE, not enabled by default in SE).
Corrado.
Though similar, the specific problem I have is not addressed in Use JASPIC auth module on WebSphere 8.5
I am getting the following error message:
SECJ8027E: The path and name of file where JASPI persistent registrations are stored must be specified using property com.ibm.websphere.jaspi.configuration.
I can set the custom property in the administration to some existing folder but I wanted to make sure that is the right approach or if there is some step I was missing.
Note I am specifically using the "embedded in application" approach rather than a server installed JASPIC module so I have something like this
#WebListener
public class JaspicInitializer implements
ServletContextListener {
#Override
public void contextInitialized(final ServletContextEvent sce) {
final Map<String, String> options = new HashMap<>();
AuthConfigFactory.getFactory()
.registerConfigProvider(AuthModuleConfigProvider.class.getName(), options, "HttpServlet", null, null);
}
}
I had the error on both WebSphere 8.5.5.11 and 9.0.0.3
From #Uux comment, I changed the way I do the registration so it no longer give the error.
#WebListener
public class JaspicInitializer implements
ServletContextListener {
private String registrationID;
#Override
public void contextDestroyed(final ServletContextEvent sce) {
AuthConfigFactory.getFactory().removeRegistration(registrationID);
}
#Override
public void contextInitialized(final ServletContextEvent sce) {
final ServletContext context = sce.getServletContext();
registrationID = AuthConfigFactory.getFactory()
.registerConfigProvider(new AuthModuleConfigProvider(), "HttpServlet",
context.getVirtualServerName() + " " + context.getContextPath(), "JEE Sample");
}
}
Also WebSphere Global Security needs to be configured with
Enable application security
Enable Java Authentication SPI (JASPI)
Our Ops guys want the Spring boot configuration (i.e. all properties) to be dumped to the log file when the app starts. I assume this can be done by injecting the properties with annotation #ConfigurationProperties and printing them.
The questions is whether there is a better or built-in mechanism to achieve this.
Given there does not seem to be a built in solution besides, I was try to cook my own. Here is what I came up with:
#Component
public class ConfigurationDumper {
#Autowired
public void init(Environment env){
log.info("{}",env);
}
}
The challenge with this is that it does not print variables that are in my application.yml. Instead, here is what I get:
StandardServletEnvironment
{
activeProfiles=[],
defaultProfiles=[default],
propertySources=[
servletConfigInitParams,
servletContextInitParams,
systemProperties,
systemEnvironment,
random,
applicationConfig: [classpath: /application.yml]
]
}
How can I fix this so as to have all properties loaded and printed?
If you use actuator , env endpoint will give you all the configuration properties set in ConfigurableEnvironment and configprops will give you the list of #ConfigurationProperties, but not in the log.
Take a look at the source code for this env endpoint, may be it will give you an idea of how you could get all the properties you are interested in.
There is no built-in mechanism and it really depends what you mean by "all properties". Do you want only the actual keys that you wrote or you want all properties (including defaults).
For the former, you could easily listen for ApplicationEnvironmentPreparedEvent and log the property sources you're interested in. For the latter, /configprops is indeed a much better/complete output.
This logs only the properties configured *.properties file.
/**
* maps given property names to its origin
* #return a map where key is property name and value the origin
*/
public Map<String, String> fromWhere() {
final Map<String, String> mapToLog = new HashMap<>();
final MutablePropertySources propertySources = env.getPropertySources();
final Iterator<?> it = propertySources.iterator();
while (it.hasNext()) {
final Object object = it.next();
if (object instanceof MapPropertySource) {
MapPropertySource propertySource = (MapPropertySource) object;
String propertySourceName = propertySource.getName();
if (propertySourceName.contains("properties")) {
Map<String, Object> sourceMap = propertySource.getSource();
for (String key : sourceMap.keySet()) {
final String envValue = env.getProperty(key);
String env2Val = System.getProperty(key);
String source = propertySource.getName().contains("file:") ? "FILE" : "JAR";
if (envValue.equals(env2Val)) {
source = "ENV";
}
mapToLog.putIfAbsent(key, source);
}
}
}
}
return mapToLog;
}
my example output which depicts the property name, value and from where it comes. My property values are describing from where they come.:
myprop: fooFromJar from JAR
aPropFromFile: fromExternalConfFile from FILE
mypropEnv: here from vm arg from ENV
ENV means that I have given it by -D to JVM.
JAR means it is from application.properties inside JAR
FILE means it is from application.properties outside JAR
I've got some code like this to read a value that could be set either with a sling:OsgiConfig node or after being set in the Felix UI...
#Component(immediate = true, metatype = true, label = "Dummy Service")
public class DummyService {
#Property(label = "Dummy Service Value")
public static final String DUMMY_VALUE = "dummyValue";
private static String m_strDummyValue = "default value";
public static String getDummyValue(){
return m_strDummyValue;
}
#Activate
protected void activate(ComponentContext context) {
configure(context.getProperties());
}
#Deactivate
protected void deactivate(ComponentContext context) {
}
#Modified
protected void modified(ComponentContext componentContext) {
configure(componentContext.getProperties());
}
public void updated(Dictionary properties) throws ConfigurationException {
configure(properties);
}
private void configure(Dictionary properties) {
m_strDummyValue = OsgiUtil.toString(properties.get(DUMMY_VALUE), null);
}
}
And could be called in any consuming class as
DummyService.getDummyValue();
This is currently working in our development environment. It's also very similar to some code that another vendor wrote and is currently in production in the client environment, and seems to be working. However, I ran across this post OSGi component configurable via Apache Felix... which recommends against using a static accessor like this. Are there potential problems where getDummyValue() could return an incorrect value, or is the recommendation more about being philosophically consistent with OSGi's patterns?
Generally statics are frowned upon especially in OSGi as it involves a tight code coupling. It would be better to have DummySerivce be an interface and your class implement it with the component being a service. Then others would reference your component's service. Once injected with the service, they can call the service's methods.
You shouldn't do this for one major reason: there is no guarantee that DummyService has been configured when you access the static method - in contrast with a service reference.
Being new to Spring Boot I am wondering on how I can configure connection details for MongoDB.
I have tried the normal examples but none covers the connection details.
I want to specify the database that is going to be used and the url/port of the host that runs MongoDB.
Any hints or tips?
Just to quote Boot Docs:
You can set spring.data.mongodb.uri property to change the url, or alternatively specify a host/port. For example, you might declare the following in your application.properties:
spring.data.mongodb.host=mongoserver
spring.data.mongodb.port=27017
All available options for spring.data.mongodb prefix are fields of MongoProperties:
private String host;
private int port = DBPort.PORT;
private String uri = "mongodb://localhost/test";
private String database;
private String gridFsDatabase;
private String username;
private char[] password;
It's also important to note that MongoDB has the concept of "authentication database", which can be different than the database you are connecting to. For example, if you use the official Docker image for Mongo and specify the environment variables MONGO_INITDB_ROOT_USERNAME and MONGO_INITDB_ROOT_PASSWORD, a user will be created on 'admin' database, which is probably not the database you want to use. In this case, you should specify parameters accordingly on your application.properties file using:
spring.data.mongodb.host=127.0.0.1
spring.data.mongodb.port=27017
spring.data.mongodb.authentication-database=admin
spring.data.mongodb.username=<username specified on MONGO_INITDB_ROOT_USERNAME>
spring.data.mongodb.password=<password specified on MONGO_INITDB_ROOT_PASSWORD>
spring.data.mongodb.database=<the db you want to use>
spring.data.mongodb.host and spring.data.mongodb.port are not supported if you’re using the Mongo 3.0 Java driver. In such cases, spring.data.mongodb.uri should be used to provide all of the configuration, like this:
spring.data.mongodb.uri=mongodb://user:secret#mongo1.example.com:12345
In a maven project create a file src/main/resources/application.yml with the following content:
spring.profiles: integration
# use local or embedded mongodb at localhost:27017
---
spring.profiles: production
spring.data.mongodb.uri: mongodb://<user>:<passwd>#<host>:<port>/<dbname>
Spring Boot will automatically use this file to configure your application. Then you can start your spring boot application either with the integration profile (and use your local MongoDB)
java -jar -Dspring.profiles.active=integration your-app.jar
or with the production profile (and use your production MongoDB)
java -jar -Dspring.profiles.active=production your-app.jar
You can define more details by extending AbstractMongoConfiguration.
#Configuration
#EnableMongoRepositories("demo.mongo.model")
public class SpringMongoConfig extends AbstractMongoConfiguration {
#Value("${spring.profiles.active}")
private String profileActive;
#Value("${spring.application.name}")
private String proAppName;
#Value("${spring.data.mongodb.host}")
private String mongoHost;
#Value("${spring.data.mongodb.port}")
private String mongoPort;
#Value("${spring.data.mongodb.database}")
private String mongoDB;
#Override
public MongoMappingContext mongoMappingContext()
throws ClassNotFoundException {
// TODO Auto-generated method stub
return super.mongoMappingContext();
}
#Override
#Bean
public Mongo mongo() throws Exception {
return new MongoClient(mongoHost + ":" + mongoPort);
}
#Override
protected String getDatabaseName() {
// TODO Auto-generated method stub
return mongoDB;
}
}
In case that somebody is trying to connect to a Atlas MongoDB Cluster in application.properties has to have a config like:
spring.data.mongodb.uri=mongodb+srv://databaseUsername:usernamePassword#cluster0.j4koa.mongodb.net/databaseUsername?retryWrites=true&w=majority
In my case I needed to set up MongoDB for integration tests using Testcontainers. Using properites file was not an option since Mongo port had to be specified during runtime. I wanted to preseve original MongoDB autoconfiguration provided by SpringBoot but override some of the properties. This can be achieved by defining a bean of type MongoClientSettingsBuilderCustomizer which can be used to customize mongo settings :
#Bean
public MongoClientSettingsBuilderCustomizer clientSettingsBuilderCustomizer(final GenericContainer<?> mongoDBContainer) {
String database = environment.getProperty("spring.data.mongodb.database");
ConnectionString connectionString = new ConnectionString(String.format("mongodb://localhost:%s/%s", mongoDBContainer.getFirstMappedPort(), database));
return settings -> settings.applyConnectionString(connectionString);
}
If you simply want to read a MongoDB connection string from an environment variable, one way is to set the following environment variable:
SPRING_DATA_MONGODB_URI=mongodb://localhost:27017/trying-mongo
This doesn't require any changes in the application.properties as the spring data mongo will read the value from the above environment variable by default.
Here is How you can do in Spring Boot 2.0 by creating custom MongoClient adding Providing more control for Connection ,
Please follow github Link for Full Source Code
#Configuration
#EnableMongoRepositories(basePackages = { "com.frugalis.repository" })
#ComponentScan(basePackages = { "com.frugalis.*" })
#PropertySource("classpath:application.properties")
public class MongoJPAConfig extends AbstractMongoConfiguration {
#Value("${com.frugalis.mongo.database}")
private String database;
#Value("${com.frugalis.mongo.server}")
private String host;
#Value("${com.frugalis.mongo.port}")
private String port;
#Value("${com.frugalis.mongo.username}")
private String username;
#Value("${com.frugalis.mongo.password}")
private String password;
#Override
protected String getDatabaseName() {
return database;
}
#Override
protected String getMappingBasePackage() {
return "com.frugalis.entity.mongo";
}
#Bean
public MongoTemplate mongoTemplate() throws Exception {
return new MongoTemplate(mongoClient(), getDatabaseName());
}
#Override
#Bean
public MongoClient mongoClient() {
List<MongoCredential> allCred = new ArrayList<MongoCredential>();
System.out.println("???????????????????"+username+" "+database+" "+password+" "+host+" "+port);
allCred.add(MongoCredential.createCredential(username, database, password.toCharArray()));
MongoClient client = new MongoClient((new ServerAddress(host, Integer.parseInt(port))), allCred);
client.setWriteConcern(WriteConcern.ACKNOWLEDGED);
return client;
}}