Spring amqp delay messaging with rabbitMQ - spring

I am struggling hard to find out the way for scheduled/Delaying messages in Spring AMQP/Rabbit MQ and found solution in here.But i still with a prolem
about Spring AMQP/Rabbit MQ which can not received any message.
My source as the following:
#Configuration
public class AmqpConfig {
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setAddresses("172.16.101.14:5672");
connectionFactory.setUsername("admin");
connectionFactory.setPassword("admin");
connectionFactory.setPublisherConfirms(true);
return connectionFactory;
}
#Bean
#Scope("prototype")
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
return template;
}
#Bean
CustomExchange delayExchange() {
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-delayed-type", "direct");
return new CustomExchange("my-exchange", "x-delayed-message", true, false, args);
}
#Bean
public Queue queue() {
return new Queue("spring-boot-queue", true);
}
#Bean
Binding binding(Queue queue, Exchange delayExchange) {
return BindingBuilder.bind(queue).to(delayExchange).with("spring-boot-queue").noargs();
}
#Bean
public SimpleMessageListenerContainer messageContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(connectionFactory());
container.setQueues(queue());
container.setExposeListenerChannel(true);
container.setMaxConcurrentConsumers(1);
container.setConcurrentConsumers(1);
container.setAcknowledgeMode(AcknowledgeMode.MANUAL);
container.setMessageListener(new ChannelAwareMessageListener() {
public void onMessage(Message message, Channel channel) throws Exception {
byte[] body = message.getBody();
System.err.println("receive msg : " + new String(body));
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false); //确认消息成功消费
}
});
return container;
}
}
#Component
public class Send implements RabbitTemplate.ConfirmCallback{
private RabbitTemplate rabbitTemplate;
#Autowired
public Send(RabbitTemplate rabbitTemplate) {
this.rabbitTemplate = rabbitTemplate;
this.rabbitTemplate.setConfirmCallback(this);
rabbitTemplate.setMandatory(true);
}
public void sendMsg(String content) {
CorrelationData correlationId = new CorrelationData(UUID.randomUUID().toString());
rabbitTemplate.convertAndSend("my-exchange", "", content, new MessagePostProcessor() {
#Override
public Message postProcessMessage(Message message) throws AmqpException {
message.getMessageProperties().setHeader("x-delay", 6000);
return message;
}
},correlationId);
System.err.println("delay message send ................");
}
/**
* 回调
*/
#Override
public void confirm(CorrelationData correlationData, boolean ack, String cause) {
System.err.println(" callback id :" + correlationData);
if (ack) {
System.err.println("ok");
} else {
System.err.println("fail:" + cause);
}
}
}
Is there someone could give a help.
Thanks all.

Delay messaging is nothing to do with Spring amqp, it's a library which will reside with your code, so the library can't hold any message as such. There are two approaches you can try:
Old Approach:
Set the TTL(time to live) header in each message/queue(policy) and then introduce a DLQ to handle it. once the ttl expired your messages will move from DLQ to main queue so that your listener can process it.
Latest Approach:
Recently RabbitMQ came up with RabbitMQ Delayed Message Plugin , using which you can achieve the same and this plugin support available since RabbitMQ-3.5.8.
You can declare an exchange with the type x-delayed-message and then publish messages with the custom header x-delay expressing in milliseconds a delay time for the message. The message will be delivered to the respective queues after x-delay milliseconds
Details:
To use the delayed-messaging feature, declare an exchange with the type x-delayed-message:
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-delayed-type", "direct");
channel.exchangeDeclare("my-exchange", "x-delayed-message", true, false, args);
Note that we pass an extra header called x-delayed-type, more on it under the Routing section.
Once we have the exchange declared we can publish messages providing a header telling the plugin for how long to delay our messages:
byte[] messageBodyBytes = "delayed payload".getBytes("UTF-8");
Map<String, Object> headers = new HashMap<String, Object>();
headers.put("x-delay", 5000);
AMQP.BasicProperties.Builder props = new AMQP.BasicProperties.Builder().headers(headers);
channel.basicPublish("my-exchange", "", props.build(), messageBodyBytes);
byte[] messageBodyBytes2 = "more delayed payload".getBytes("UTF-8");
Map<String, Object> headers2 = new HashMap<String, Object>();
headers2.put("x-delay", 1000);
AMQP.BasicProperties.Builder props2 = new AMQP.BasicProperties.Builder().headers(headers2);
channel.basicPublish("my-exchange", "", props2.build(), messageBodyBytes2);
In the above example we publish two messages, specifying the delay time with the x-delay header. For this example, the plugin will deliver to our queues first the message with the body "more delayed payload" and then the one with the body "delayed payload".
If the x-delay header is not present, then the plugin will proceed to route the message without delay.
More here: git

Related

RabbitMQ Spring "Cannot determine target ConnectionFactory for lookup key" when using Java lambda parallelStream

We have a Spring Java application using RabbitMQ, and here is the scenario:
There is a consumer receiving messages from a queue and sending them to another one. We are using "SimpleRabbitListenerContainerFactory" as the container factory, but when sending the messages to the other queue inside a "parallelStream" we've got an IllegalStateException "Cannot determine target ConnectionFactory for lookup key" Exception
When we remove the "parallelStream" it works flawlessly.
public void sendMessage(final StagingMessage stagingMessage, final Long timestamp, final String country) {
final List<TransformedMessage> messages = processMessageList(stagingMessage);
messages.parallelStream().forEach(message -> {
final TransformedMessage transformedMessage = buildMessage(timestamp, ApiConstants.POST_METHOD, country);
myMessageSender.sendQueue(country, transformedMessage);
});
}
Connectio Facotory, where the lookup key is set:
#Configuration
#EnableRabbit
public class RabbitBaseConfig {
#Autowired
private QueueProperties queueProperties;
#Bean
#Primary
public ConnectionFactory connectionFactory(final ConnectionFactory connectionFactoryA, final ConnectionFactory connectionFactoryB) {
final SimpleRoutingConnectionFactory simpleRoutingConnectionFactory = new SimpleRoutingConnectionFactory();
final Map<Object, ConnectionFactory> map = new HashMap<>();
for (final String queue : queueProperties.getAQueueMap().values()) {
map.put("[" + queue + "]", connectionFactoryA);
}
for (final String queue : queueProperties.getBQueueMap().values()) {
map.put("[" + queue + "]", connectionFactoryB);
}
simpleRoutingConnectionFactory.setTargetConnectionFactories(map);
return simpleRoutingConnectionFactory;
}
#Bean
public Jackson2JsonMessageConverter jackson2JsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
}
Welcome to stack overflow!
You should always show the pertinent code and configuration beans when asking questions like this.
I assume you are using the RoutingConnectionFactory.
It uses a ThreadLocal to store the lookup key so the send has to happen on the same thread that set the key.
You generally should never go asynchronous in a listener anyway; you risk message loss. To increase concurrency, use the concurrency properties on the container.
EDIT
One technique would be to convey the lookup key in a message header:
#Bean
public RabbitTemplate template(ConnectionFactory rcf) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(rcf);
Expression expression = new SpelExpressionParser().parseExpression("messageProperties.headers['cfSelector']");
rabbitTemplate.setSendConnectionFactorySelectorExpression(expression);
return rabbitTemplate;
}
#RabbitListener(queues = "foo")
public void listen1(String in) {
IntStream.range(0, 10)
.parallel()
.mapToObj(i -> in + i)
.forEach(val -> {
this.template.convertAndSend("bar", val.toUpperCase(), msg -> {
msg.getMessageProperties().setHeader("cfSelector", "[bar]");
return msg;
});
});
}

Spring Integration - Dynamic MailReceiver configuration

I'm pretty new to spring-integration anyway I'm using it in order to receive mails and elaborate them.
I used this spring configuration class:
#Configuration
#EnableIntegration
#PropertySource(value = { "classpath:configuration.properties" }, encoding = "UTF-8", ignoreResourceNotFound = false)
public class MailReceiverConfiguration {
private static final Log logger = LogFactory.getLog(MailReceiverConfiguration.class);
#Autowired
private EmailTransformerService emailTransformerService;
// Configurazione AE
#Bean
public MessageChannel inboundChannelAE() {
return new DirectChannel();
}
#Bean(name= {"aeProps"})
public Properties aeProps() {
Properties javaMailPropertiesAE = new Properties();
javaMailPropertiesAE.put("mail.store.protocol", "imap");
javaMailPropertiesAE.put("mail.debug", Boolean.TRUE);
javaMailPropertiesAE.put("mail.auth.debug", Boolean.TRUE);
javaMailPropertiesAE.put("mail.smtp.socketFactory.fallback", "false");
javaMailPropertiesAE.put("mail.imap.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
return javaMailPropertiesAE;
}
#Bean(name="mailReceiverAE")
public MailReceiver mailReceiverAE(#Autowired MailConfigurationBean mcb, #Autowired #Qualifier("aeProps") Properties javaMailPropertiesAE) throws Exception {
return ConfigurationUtil.getMailReceiver("imap://USERNAME:PASSWORD#MAILSERVER:PORT/INBOX", new BigDecimal(2), javaMailPropertiesAE);
}
#Bean
#InboundChannelAdapter( autoStartup = "true",
channel = "inboundChannelAE",
poller = {#Poller(fixedRate = "${fixed.rate.ae}",
maxMessagesPerPoll = "${max.messages.per.poll.ae}") })
public MailReceivingMessageSource pollForEmailAE(#Autowired MailReceiver mailReceiverAE) {
MailReceivingMessageSource mrms = new MailReceivingMessageSource(mailReceiverAE);
return mrms;
}
#Transformer(inputChannel = "inboundChannelAE", outputChannel = "transformerChannelAE")
public MessageBean transformitAE( MimeMessage mailMessage ) throws Exception {
// amministratore email inbox
MessageBean messageBean = emailTransformerService.transformit(mailMessage);
return messageBean;
}
#Splitter(inputChannel = "transformerChannelAE", outputChannel = "nullChannel")
public List<Message<?>> splitIntoMessagesAE(final MessageBean mb) {
final List<Message<?>> messages = new ArrayList<Message<?>>();
for (EmailFragment emailFragment : mb.getEmailFragments()) {
Message<?> message = MessageBuilder.withPayload(emailFragment.getData())
.setHeader(FileHeaders.FILENAME, emailFragment.getFilename())
.setHeader("directory", emailFragment.getDirectory()).build();
messages.add(message);
}
return messages;
}
}
So far so good.... I start my micro-service and there is this component listening on the specified mail server and mails are downloaded.
Now I have this requirement: mail server configuration (I mean the string "imap://USERNAME:PASSWORD#MAILSERVER:PORT/INBOX") must be taken from a database and it can be configurable. In any time a system administrator can change it and the mail receiver must use the new configuration.
As far as I understood I should create a new instance of MailReceiver when a new configuration is present and use it in the InboundChannelAdapter
Is there any best practice in order to do it? I found this solution: ImapMailReceiver NO STORE attempt on READ-ONLY folder (Failure) [THROTTLED];
In this solution I can inject the ThreadPoolTaskScheduler if I define it in my Configuration class; I can also inject the DirectChannel but every-time I should create a new MailReceiver and a new ImapIdleChannelAdapter without considering this WARN message I get when the
ImapIdleChannelAdapter starts:
java.lang.RuntimeException: No beanfactory at org.springframework.integration.expression.ExpressionUtils.createStandardEvaluationContext(ExpressionUtils.java:79) at org.springframework.integration.mail.AbstractMailReceiver.onInit(AbstractMailReceiver.java:403)
Is there a better way to satisfy my scenario?
Thank you
Angelo
The best way to do this is to use the Java DSL and dynamic flow registration.
Documentation here.
That way, you can unregister the old flow and register a new one, each time the configuration changes.
It will automatically handle injecting dependencies such as the bean factory.

Send websocket message to user across dynos

I have a spring boot application running on heroku. I make use of websockets for sending messages to and from client and server for a specific user . I use spring boot's SimpMessagingTemplate.convertAndSendToUser to send and receive messages, which works fine for when a user needs get a message back from the server. I use Heroku session affinity which means that even if I scale up the number of sessions the user and websocket still share the same session.
My problem comes when I need a user to send a message to another user. It works fine if both users are sharing the session, but not if the message will not come through.
Is it possible to send a message from one user to another across different sessions using, SimpMessagingTemple? Or would I need to use a message broker, eg Redis.
I was looking into implementing sending a message using StringRedisTemplate but not sure how to send a message to a particular user.
private SimpMessagingTemplate messagingTemplate;
#Autowired
public MessageController(SimpMessagingTemplate messagingTemplate) {
this.messagingTemplate = messagingTemplate;
}
#MessageMapping("/secured/user-in")
public void sendToDevice(Message msg, #AuthenticationPrincipal User principal) throws Exception {
if (msg.getTo() != null) {
String email = msg.getTo();
Message out = new Message();
out.setMsg(msg.getMsg());
out.setFrom(msg.getFrom());
out.setTo(msg.getTo());
out.setSentTime(new Date());
out.setStatus(msg.getStatus());
messagingTemplate.convertAndSendToUser(email, "/secured/topic", out);
}
}
JS
function connect() {
var socket = new SockJS('/secured/user-in');
ST.stompClient = Stomp.over(socket);
var headers = {};
headers[ST.getHeader()] = ST.getToken();
ST.getStompClient().connect(headers, function (frame) {
retries = 1;
console.log('Connected: ' + frame);
ST.getStompClient().subscribe('/user/secured/topic', function (event){
var msg = JSON.parse(event.body);
showMessage(msg.msg);
});
});
}
UPDATE 1
I am guessing I could do something like this, as done here:
SimpMessageHeaderAccessor headerAccessor = SimpMessageHeaderAccessor
.create(SimpMessageType.MESSAGE);
headerAccessor.setSessionId(sessionId);
headerAccessor.setLeaveMutable(true);
messagingTemplate.convertAndSendToUser(sessionId,"/queue/something", payload,
headerAccessor.getMessageHeaders());
But how could I get the session id of another user, I am using Redis to store session info: #EnableRedisHttpSession
I had my terminology a bit mixed up I was trying to send a message to another user on another dyno rather than session.
Ended up using redis sub/pub.
So when a message is receive by the controller it is published to redis, and the redis MessageListenerAdapter envokes the convertAndSendToUser method.
#MessageMapping("/secured/user-in")
public void sendToDevice(Message msg, #AuthenticationPrincipal User principal) throws Exception {
publishMessageToRedis(msg);
}
private void publishMessageToRedis(Message message) throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper();
String messageString = objectMapper.writeValueAsString(message);
stringRedisTemplate.convertAndSend("message", messageString);
}
redis config
#Bean
RedisMessageListenerContainer container( MessageListenerAdapter chatMessageListenerAdapter) throws URISyntaxException {
RedisMessageListenerContainer container = new RedisMessageListenerContainer();
container.setConnectionFactory(connectionFactory());
container.addMessageListener(chatMessageListenerAdapter, new PatternTopic("message"));
return container;
}
#Bean("chatMessageListenerAdapter")
MessageListenerAdapter chatMessageListenerAdapter(RedisReceiver redisReceiver) {
return new MessageListenerAdapter(redisReceiver, "receiveChatMessage");
}
public class RedisReceiver {
private static final Logger LOG = LogManager.getLogger(RedisReceiver.class);
private final WebSocketMessageService webSocketMessageService;
#Autowired
public RedisReceiver(WebSocketMessageService webSocketMessageService) {
this.webSocketMessageService = webSocketMessageService;
}
// Invoked when message is publish to "chat" channel
public void receiveChatMessage(String messageStr) throws IOException {
ObjectMapper objectMapper = new ObjectMapper();
Message message = objectMapper.readValue(messageStr, Message.class);
webSocketMessageService.sendChatMessage(message);
}
}
#Service
public class WebSocketMessageService {
private final SimpMessagingTemplate template;
private static final Logger LOG = LogManager.getLogger(WebSocketMessageService.class);
public WebSocketMessageService(SimpMessagingTemplate template) {
this.template = template;
}
public void sendChatMessage(Message message) {
template.convertAndSendToUser(message.getTo(), "/secured/topic", message);
}
}
Solution was based off this git repository

Spring-Boot MQTT Configuration

I have a requirement to send payload to a lot of devices whose names are picked from Database. Then, i have to send to different topics, which will be like settings/{put devicename here}.
Below is the configuration i was using which i got from spring-boot reference documents.
MQTTConfiguration.java
#Configuration
#IntegrationComponentScan
public class MQTTConfiguration {
#Autowired
private Settings settings;
#Autowired
private DevMqttMessageListener messageListener;
#Bean
MqttPahoClientFactory mqttClientFactory() {
DefaultMqttPahoClientFactory clientFactory = new DefaultMqttPahoClientFactory();
clientFactory.setServerURIs(settings.getMqttBrokerUrl());
clientFactory.setUserName(settings.getMqttBrokerUser());
clientFactory.setPassword(settings.getMqttBrokerPassword());
return clientFactory;
}
#Bean
MessageChannel mqttOutboundChannel() {
return new DirectChannel();
}
#Bean
#ServiceActivator(inputChannel = "mqttOutboundChannel")
public MessageHandler mqttOutbound() {
MqttPahoMessageHandler messageHandler = new MqttPahoMessageHandler("dev-client-outbound",
mqttClientFactory());
messageHandler.setAsync(true);
messageHandler.setDefaultTopic(settings.getMqttPublishTopic());
return messageHandler;
}
#MessagingGateway(defaultRequestChannel = "mqttOutboundChannel")
public interface DeviceGateway {
void sendToMqtt(String payload);
}
}
Here, i am sending to only 1 topic. So i added the bean like below to send to multiple number of topics;
#Bean
public MqttClient mqttClient() throws MqttException {
MqttClient mqttClient = new MqttClient(settings.getMqttBrokerUrl(), "dev-client-outbound");
MqttConnectOptions connOptions = new MqttConnectOptions();
connOptions.setUserName(settings.getMqttBrokerUser());
connOptions.setPassword(settings.getMqttBrokerPassword().toCharArray());
mqttClient.connect(connOptions);
return mqttClient;
}
and i send using,
try {
mqttClient.publish(settings.getMqttPublishTopic()+device.getName(), mqttMessage);
} catch (MqttException e) {
LOGGER.error("Error While Sending Mqtt Messages", e);
}
Which works.
But my question is, Can i achieve the same, using output channel for better performance? If yes, any help is greatly appreciated. Thank You.
MqttClient is synchronous.
The MqttPahoMessageHandler uses an MqttAsyncClient and can be configured (set async to true) to not wait for the confirmation, but publish the confirmation later as an application event.
If you are using your own code and sending multiple messages in a loop, it will probably be faster to use an async client, and wait for the IMqttDeliveryToken completions later.

Why import AsyncRabbitTemplate in spring-amqp

When processing the reply message with AsyncRabbitTemplate.sendAndReceive() or AsyncRabbitTemplate.convertSendAndReceive() method, since the reply message is returned asynchronously with calling method, we can use message listener for reply queue to receive and process reply message, why spring-amqp framework import AsyncRabbitTemplate and RabbiteMessageFuture to process the reply message? For message listener, we can control the related consumer thread,
but for RabbitMessageFuture, the background thread can not be managed.
-------------------Added on 2017/01/06----------------------------
It's simply your choice.
Replies can come back in a different order to sends.
With the async template, the framework takes care of the correlation
for you the reply will appear in the future returned by the send
method.
When you use your own listener, you will have to take care of the
correlation yourself.
Thank you. I know this difference.But there is still a problem. If I use message listener, I can ack the reply message manually(If my message listener
implements ChannelAwareMessageListener interface and I can get the channel instance).But when I use asyncRabbitTemplate, can I ack the reply message manually? It seems that sendAndReceive method ack the reply message automatically.
I don't understand what you mean; since you can inject the listener
container into the template, you have the same "control" either way.
It seems there is some problem in this mean.
I created a rabbitTemplate instance and simple message listener container. But when I use them to construct an asyncRabbitTemplate instance as following code:
#Bean(name="rabbitTemplate")
public RabbitTemplate getRabbitTemplate()
{
RabbitTemplate rabbitTemplate = new RabbitTemplate(getConnectionFactory());
rabbitTemplate.setUseTemporaryReplyQueues(false);
rabbitTemplate.setReplyAddress("replyQueue");
rabbitTemplate.setReceiveTimeout(60000);
rabbitTemplate.setReplyTimeout(60000);
return rabbitTemplate;
}
#Bean(name="asyncRabbitTemplate")
public AsyncRabbitTemplate getAsyncRabbitTemplate()
{
AsyncRabbitTemplate asyncRabbitTemplate =
new AsyncRabbitTemplate(getRabbitTemplate(), createReplyListenerContainer());
asyncRabbitTemplate.setAutoStartup(true);
asyncRabbitTemplate.setReceiveTimeout(60000);
return asyncRabbitTemplate;
}
#Bean(name="replyMessageListenerContainer")
public SimpleMessageListenerContainer createReplyListenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(getConnectionFactory());
listenerContainer.setQueueNames("replyQueue");
listenerContainer.setMessageListener(getRabbitTemplate());
listenerContainer.setRabbitAdmin(getRabbitAdmin());
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
return listenerContainer;
}
I found I can not send message successfully. The consumer server can not receive the message.
But when I create asyncRabbitTemplate instance with following code, I found the message can be sent and received successfully.
#Bean(name="asyncRabbitTemplate")
public AsyncRabbitTemplate getAsyncRabbitTemplate()
{
AsyncRabbitTemplate asyncRabbitTemplate =
new AsyncRabbitTemplate(getConnectionFactory(),
"sendMessageExchange",
"sendMessageKey",
"replyQueue");
asyncRabbitTemplate.setReceiveTimeout(60000);
asyncRabbitTemplate.setAutoStartup(true);
return asyncRabbitTemplate;
}
If there is something wrong with my source code?
I used the spring-boot-ampq 1.4.3.RELEASE.
It's simply your choice.
Replies can come back in a different order to sends.
With the async template, the framework takes care of the correlation for you - the reply will appear in the future returned by the send method.
When you use your own listener, you will have to take care of the correlation yourself.
For message listener, we can control the related consumer thread, but for RabbitMessageFuture, the background thread can not be managed.
I don't understand what you mean; since you can inject the listener container into the template, you have the same "control" either way.
EDIT
#SpringBootApplication
public class So41481046Application {
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext context = SpringApplication.run(So41481046Application.class, args);
AsyncRabbitTemplate asyncTemplate = context.getBean(AsyncRabbitTemplate.class);
RabbitConverterFuture<String> future = asyncTemplate.convertSendAndReceive("foo");
try {
String out = future.get(10, TimeUnit.SECONDS);
System.out.println(out);
}
finally {
context.close();
}
System.exit(0);
}
#Bean
public AsyncRabbitTemplate asyncTemplate(RabbitTemplate rabbitTemplate, ConnectionFactory connectionFactory) {
rabbitTemplate.setRoutingKey(queue().getName());
rabbitTemplate.setReplyAddress(replyQueue().getName());
return new AsyncRabbitTemplate(rabbitTemplate, replyContainer(connectionFactory));
}
#Bean
public Queue queue() {
return new AnonymousQueue();
}
#Bean
public Queue replyQueue() {
return new AnonymousQueue();
}
#Bean
public SimpleMessageListenerContainer replyContainer(ConnectionFactory connectionFactory) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(connectionFactory);
container.setQueueNames(replyQueue().getName());
return container;
}
#Bean
public SimpleMessageListenerContainer remoteContainer(ConnectionFactory connectionFactory) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(connectionFactory);
container.setQueueNames(queue().getName());
container.setMessageListener(new MessageListenerAdapter(new Object() {
#SuppressWarnings("unused")
public String handleMessage(String in) {
return in.toUpperCase();
}
}));
return container;
}
}

Resources