Redis, SpringBoot and HttpSession: should I encrypt the session data? - session
I'm using Spring Boot 1.3.3 to build a web application. I use Redis for handling the session.
I'll set some "crucial" data into the HttpSession and I'd like to understand how this will work with Redis. Is the information stored server side plus a key on browser side or all the data is in a cookie in the user browser?
I'd like to see a documentation reference for the answer or to get an authoritative answer (e.g. a Pivotal dev).
While I agree with most of what the other answers in here have said, none of the other answers actually answered the question.
I'm going to assume that you are using SpringSession with Redis in Spring Boot.
In order to use SpringSession, you have likely configured (directly or indirectly) a servlet filter that extends SessionRepositoryFilter.
SessionRepositoryFilter uses a SessionRepository. Since you are using Redis, it is likely that your configuration makes use of RedisOperationsSessionRepository.
RedisOperationsSessionRepository implements SessionRepository, as you might have guessed and is ultimately responsible for fetching, storing, and deleting sessions based on a key (in your case, a key that is probably stored as a cookie on the user's browser).
RedisOperationSessionRepository, by default, uses JdkSerializationRedisSerializer, which implements RedisSerializer, to serialize session data prior to handing said data off to Redis.
According to the documentation for RedisOperationSessionRepository, it is possible to set the default serializer that RedisOperationSessionRepository will use, via it's setDefaultSerializer method.
You could theoretically extend JdkSerializationRedisSerializer and perform encryption and decryption there. JdkSerializationRedisSerializer looks like this:
public class JdkSerializationRedisSerializer implements RedisSerializer<Object> {
private Converter<Object, byte[]> serializer = new SerializingConverter();
private Converter<byte[], Object> deserializer = new DeserializingConverter();
public Object deserialize(byte[] bytes) {
if (SerializationUtils.isEmpty(bytes)) {
return null;
}
try {
return deserializer.convert(bytes);
} catch (Exception ex) {
throw new SerializationException("Cannot deserialize", ex);
}
}
public byte[] serialize(Object object) {
if (object == null) {
return SerializationUtils.EMPTY_ARRAY;
}
try {
return serializer.convert(object);
} catch (Exception ex) {
throw new SerializationException("Cannot serialize", ex);
}
}
}
So a potential way to add encryption might look like:
#Component
#Qualifier("springSessionDefaultRedisSerializer") //SB 2.0.0+
public class CrypticRedisSerializer extends JdkSerializationRedisSerializer {
#Override
public Object deserialize(byte[] bytes) {
byte[] decrpyted;
try {
decrpyted = EncryptionUtils.decrypt(bytes);
return super.deserialize(decrpyted);
} catch (NoSuchPaddingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (GeneralSecurityException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// handle expections or allow to propagate, your choice!
return null;
}
#Override
public byte[] serialize(Object object) {
byte[] bytes = super.serialize(object);
try {
return EncryptionUtils.encrypt(bytes);
} catch (NoSuchPaddingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (GeneralSecurityException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// handle expections or allow to propagate, your choice!
return null;
}
}
Where EncrpytionUtils might look like:
public class EncryptionUtils {
private static SecretKeySpec skeySpec;
static {
try {
ClassPathResource res = new ClassPathResource("key.key");
if(res != null){
File file = res.getFile();
FileInputStream input = new FileInputStream(file);
byte[] in = new byte[(int)file.length()];
input.read(in);
skeySpec = new SecretKeySpec(in, "AES");
input.close();
}
}catch (FileNotFoundException e) {
e.printStackTrace();
}catch (IOException e) {
e.printStackTrace();
}
}
public static byte[] encrypt(byte[] input)
throws GeneralSecurityException, NoSuchPaddingException{
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, skeySpec);
return cipher.doFinal(input);
}
public static byte[] decrypt(byte[] input) throws GeneralSecurityException, NoSuchPaddingException{
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.DECRYPT_MODE, skeySpec);
return cipher.doFinal(input);
}
}
All you would need to do to implement this is make sure that you set your custom serializer as the default that RedisOperationSessionRepository users.
Please note:
I have not tested the above code
I am not advocating that the above code is an ideal solution or THE solution, but simply demonstrating a mechanism for introducing encrpytion into SpringSession with Redis.
Obviously, you can use whatever 2-way encrpytion algorithm you want. EncrpytionUtils is just an example.
This will impact performance. How much? Hard to say without testing. Just be aware that there will be some performance impact.
If you are really worried about encrypting session data sent to Redis, then I highly recommend that you also make sure that your servers are secured. Make sure that only the servers that need to access your Redis server can. Place it behind a firewall. If you are using a cloud service like AWS, place your Redis server in a VPN and inside of a private subnet. Check out this article.
Redis does not support connection encryption currently. However, like they suggest, you could use Sniped to ensure your connections are encrypted.
Documentation and reference to check out:
SpringSession
RedisOperationsSessionRepository
SessionRepository
Very good article on redis security from the creator or redis - http://antirez.com/news/96 it's pretty interesting read. Read the comments as well.
One thing I am curious is, does your "crucial" data has to be stored in session? If it's not super critical for performance you can just save it in your DB. I use redis in our product just for storing tokens along with basic user data, I have seen people dumping large data size as a session data, which is fine but i don't think it's a really good idea.
In my opinion you should avoid encrypting data in redis, otherwise it will be a performance overhead. So, you may want to put redis nodes in a protected zone(internal) where only the traffic from your application is allowed to reach. If its not possible then IPSec/Stunnel can be used to secure the communication.
By the way, storing the session data as HTTPSession attribute will be faster than retrieving it from Redis. But I believe you would have chosen redis probably because of the volume of the data.
Related
Why my Spring redis cache doesn't work even with anntations
I follow the instructions on this tutorial (https://www.baeldung.com/spring-boot-redis-cache) #Cacheable(value = "itemCache") public UserInfo getUserInfo(String id) { // without explicit manipulate cache, the value can't be retrieved from Redis UserInfo res = cacheManager.getCache("itemCache").get(id, UserInfo.class); if (res != null) { return res; } try { ... retrieve from database ... res = convertMapToUserInfo(id, userInfoMap); // without explicit manipulate cache, the value can't be stored in Redis cacheManager.getCache("itemCache").put(id, res); return res; } } catch (Exception e) { ... } finally { ... } return null; } The weird thing is, I have to put/get items from Cache manualy, even I use Cacheable annotation. Without explicit manipulate cache, the returned value of getUserInfo can't be cached. The RedisConfiguration contains code #Bean public RedisCacheManagerBuilderCustomizer redisCacheManagerBuilderCustomizer() { return (builder) -> builder .withCacheConfiguration("itemCache", this.cacheConfiguration()); } #Bean public RedisCacheConfiguration cacheConfiguration() { return RedisCacheConfiguration.defaultCacheConfig() .disableCachingNullValues() .serializeValuesWith(RedisSerializationContext.SerializationPair.fromSerializer(new GenericJackson2JsonRedisSerializer())); } And I also add #EnableCaching to my Application class. Could anyone help me figure out why my cache doesn't take effect. Thanks!
I've recently found out that if we use just one entry with the builder it does'nt work. But if you have more than one cache in your project and include all these entries in this method or even if you add a 'dummy' entry, it works. Maybe a bug or a known issue in Spring implementation, I don't know. #Bean public RedisCacheManagerBuilderCustomizer redisCacheManagerBuilderCustomizer() { return (builder) -> builder .withCacheConfiguration("itemCache", this.cacheConfiguration()) .withCacheConfiguration("dummy", this.cacheConfiguration()); } I hope helps.
Spring Beans constructed multiple times?
I am teaching myself how to use REST Assured and Spring right now. I am running into a few problems with the Spring Beans. Below is the System.println output from the REST Assured gets every time a bean is created. As of now, it should actually only be created once per use of the beans (I would imagine or at least that's what I want). Top of Cucumber Steps private BasicApi guideBox; private ApplicationContext context = new AnnotationConfigApplicationContext(BaseApiConfig.class); It is important to note here that the Application Context is used in each scenario to give guideBox what it needs for each scenario. The output below is currently generated for each cucumber scenario indicating that the 3 beans I am currently using are created and initialized before each scenario is run. This output occurs three times meaning the beans are created three times; once before each of the three scenarios. As you can see from the first result giving an error for too many API requests, this is all happening too fast and too often. Is there a way to stop bean creation until that specific bean is needed, or is every bean going to be created every time? Is there a way to create all the beans just one time instead of before every scenario? Output During Each Scenario {"error":"You are sending API requests too quickly. You are limited to 1 API request per second. Please refer to the API docs for more information."} {"results":[{"id":2098,"title":"Arrested Development","alternate_titles":[],"container_show":0,"first_aired":"2003-11-02","imdb_id":"tt0367279","tvdb":72173,"themoviedb":4589,"freebase":"\/m\/02hct1","wikipedia_id":496020,"tvrage":{"tvrage_id":2649,"link":"http:\/\/www.tvrage.com\/shows\/id-2649"},"artwork_208x117":"http:\/\/static-api.guidebox.com\/091414\/thumbnails_small\/2098-4213483650-208x117-show-thumbnail.jpg","artwork_304x171":"http:\/\/static-api.guidebox.com\/091414\/thumbnails_medium\/2098-6882581105-304x171-show-thumbnail.jpg","artwork_448x252":"http:\/\/static-api.guidebox.com\/091414\/thumbnails_large\/2098-7307781619-448x252-show-thumbnail.jpg","artwork_608x342":"http:\/\/static-api.guidebox.com\/091414\/thumbnails_xlarge\/2098-677562375-608x342-show-thumbnail.jpg"}],"total_results":1} {"results":[{"id":1,"region":"US","name":"United States","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/us.png"},{"id":12,"region":"AI","name":"Anguilla","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ai.png"},{"id":11,"region":"AG","name":"Antigua and Barbuda","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ag.png"},{"id":16,"region":"AR","name":"Argentina","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ar.png"},{"id":14,"region":"AM","name":"Armenia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/am.png"},{"id":3,"region":"AU","name":"Australia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/au.png"},{"id":17,"region":"AT","name":"Austria","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/at.png"},{"id":18,"region":"AZ","name":"Azerbaijan","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/az.png"},{"id":28,"region":"BS","name":"Bahamas","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/bs.png"},{"id":23,"region":"BH","name":"Bahrain","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/bh.png"},{"id":31,"region":"BY","name":"Belarus","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/by.png"},{"id":20,"region":"BE","name":"Belgium","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/be.png"},{"id":32,"region":"BZ","name":"Belize","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/bz.png"},{"id":25,"region":"BM","name":"Bermuda","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/bm.png"},{"id":27,"region":"BO","name":"Bolivia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/bo.png"},{"id":30,"region":"BW","name":"Botswana","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/bw.png"},{"id":8,"region":"BR","name":"Brazil","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/br.png"},{"id":108,"region":"VG","name":"British Virgin Islands","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/vg.png"},{"id":26,"region":"BN","name":"Brunei","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/bn.png"},{"id":22,"region":"BG","name":"Bulgaria","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/bg.png"},{"id":75,"region":"KH","name":"Cambodia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/kh.png"},{"id":4,"region":"CA","name":"Canada","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ca.png"},{"id":39,"region":"CV","name":"Cape Verde","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/cv.png"},{"id":79,"region":"KY","name":"Cayman Islands","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ky.png"},{"id":35,"region":"CL","name":"Chile","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/cl.png"},{"id":37,"region":"CO","name":"Colombia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/co.png"},{"id":38,"region":"CR","name":"Costa Rica","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/cr.png"},{"id":40,"region":"CY","name":"Cyprus","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/cy.png"},{"id":41,"region":"CZ","name":"Czech Republic","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/cz.png"},{"id":42,"region":"DK","name":"Denmark","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/dk.png"},{"id":43,"region":"DM","name":"Dominica","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/dm.png"},{"id":44,"region":"DO","name":"Dominican Republic","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/do.png"},{"id":46,"region":"EC","name":"Ecuador","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ec.png"},{"id":48,"region":"EG","name":"Egypt","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/eg.png"},{"id":114,"region":"SV","name":"El Salvador","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/sv.png"},{"id":47,"region":"EE","name":"Estonia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ee.png"},{"id":52,"region":"FM","name":"Federated States Of Micronesia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/fm.png"},{"id":51,"region":"FJ","name":"Fiji","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/fj.png"},{"id":50,"region":"FI","name":"Finland","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/fi.png"},{"id":9,"region":"FR","name":"France","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/fr.png"},{"id":55,"region":"GM","name":"Gambia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/gm.png"},{"id":7,"region":"DE","name":"Germany","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/de.png"},{"id":54,"region":"GH","name":"Ghana","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/gh.png"},{"id":56,"region":"GR","name":"Greece","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/gr.png"},{"id":53,"region":"GD","name":"Grenada","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/gd.png"},{"id":57,"region":"GT","name":"Guatemala","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/gt.png"},{"id":58,"region":"GW","name":"Guinea-Bissau","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/gw.png"},{"id":61,"region":"HN","name":"Honduras","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/hn.png"},{"id":60,"region":"HK","name":"Hong Kong","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/hk.png"},{"id":63,"region":"HU","name":"Hungary","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/hu.png"},{"id":67,"region":"IN","name":"India","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/in.png"},{"id":64,"region":"ID","name":"Indonesia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/id.png"},{"id":65,"region":"IE","name":"Ireland","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ie.png"},{"id":66,"region":"IL","name":"Israel","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/il.png"},{"id":69,"region":"IT","name":"Italy","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/it.png"},{"id":72,"region":"JP","name":"Japan","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/jp.png"},{"id":71,"region":"JO","name":"Jordan","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/jo.png"},{"id":81,"region":"LA","name":"Lao People\u2019s Democratic Republic","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/la.png"},{"id":88,"region":"LV","name":"Latvia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/lv.png"},{"id":82,"region":"LB","name":"Lebanon","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/lb.png"},{"id":86,"region":"LT","name":"Lithuania","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/lt.png"},{"id":87,"region":"LU","name":"Luxembourg","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/lu.png"},{"id":94,"region":"MO","name":"Macau","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/mo.png"},{"id":140,"region":"MY","name":"Malaysia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/my.png"},{"id":97,"region":"MT","name":"Malta","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/mt.png"},{"id":98,"region":"MU","name":"Mauritius","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/mu.png"},{"id":5,"region":"MX","name":"Mexico","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/mx.png"},{"id":93,"region":"MN","name":"Mongolia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/mn.png"},{"id":141,"region":"MZ","name":"Mozambique","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/mz.png"},{"id":142,"region":"NA","name":"Namibia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/na.png"},{"id":146,"region":"NL","name":"Netherlands","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/nl.png"},{"id":6,"region":"NZ","name":"New Zealand","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/nz.png"},{"id":145,"region":"NI","name":"Nicaragua","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ni.png"},{"id":143,"region":"NE","name":"Niger","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ne.png"},{"id":147,"region":"NO","name":"Norway","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/no.png"},{"id":149,"region":"OM","name":"Oman","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/om.png"},{"id":150,"region":"PA","name":"Panama","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/pa.png"},{"id":126,"region":"PY","name":"Paraguay","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/py.png"},{"id":151,"region":"PE","name":"Peru","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/pe.png"},{"id":153,"region":"PH","name":"Philippines","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ph.png"},{"id":155,"region":"PL","name":"Poland","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/pl.png"},{"id":125,"region":"PT","name":"Portugal","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/pt.png"},{"id":127,"region":"QA","name":"Qatar","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/qa.png"},{"id":89,"region":"MD","name":"Republic Of Moldova","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/md.png"},{"id":128,"region":"RO","name":"Romania","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ro.png"},{"id":129,"region":"RU","name":"Russia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ru.png"},{"id":130,"region":"SA","name":"Saudi Arabia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/sa.png"},{"id":134,"region":"SG","name":"Singapore","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/sg.png"},{"id":136,"region":"SK","name":"Slovakia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/sk.png"},{"id":135,"region":"SI","name":"Slovenia","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/si.png"},{"id":111,"region":"ZA","name":"South Africa","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/za.png"},{"id":49,"region":"ES","name":"Spain","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/es.png"},{"id":84,"region":"LK","name":"Sri Lanka","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/lk.png"},{"id":76,"region":"KN","name":"St. Kitts and Nevis","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/kn.png"},{"id":115,"region":"SZ","name":"Swaziland","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/sz.png"},{"id":133,"region":"SE","name":"Sweden","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/se.png"},{"id":34,"region":"CH","name":"Switzerland","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ch.png"},{"id":124,"region":"TW","name":"Taiwan","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/tw.png"},{"id":119,"region":"TJ","name":"Tajikistan","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/tj.png"},{"id":118,"region":"TH","name":"Thailand","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/th.png"},{"id":123,"region":"TT","name":"Trinidad and Tobago","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/tt.png"},{"id":122,"region":"TR","name":"Turkey","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/tr.png"},{"id":120,"region":"TM","name":"Turkmenistan","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/tm.png"},{"id":104,"region":"UG","name":"Uganda","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ug.png"},{"id":103,"region":"UA","name":"Ukraine","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ua.png"},{"id":10,"region":"AE","name":"United Arab Emirates","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ae.png"},{"id":2,"region":"GB","name":"United Kingdom","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/gb.png"},{"id":107,"region":"VE","name":"Venezuela","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/ve.png"},{"id":109,"region":"VN","name":"Vietnam","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/vn.png"},{"id":112,"region":"ZW","name":"Zimbabwe","map_img":"http:\/\/static-api.guidebox.com\/misc\/flags_iso\/48\/zw.png"}]} Java Config File Just in case there is something wrong with how I am configuring my config file, here it is. #Configuration #PropertySources( #PropertySource(value = {"classpath:/properties/guideBox.properties"})) public class BaseApiConfig { #Bean public GuideBoxProperties properties() { return new GuideBoxProperties(); } #Bean public BasicApi provideBasicApi() { return new BasicApi(); } #Bean public BasicApi provideHome() throws Exception { return new BasicApi(properties().getApiFull()); } #Bean public BasicApi provideArrestedDevleopmentSearch() throws Exception { return new BasicApi(properties().getArrestedDevelopmentSearch()); } #Bean public BasicApi provideAllRegions() throws Exception { return new BasicApi(properties().getAllRegions()); } } A Cucumber Scenario #Given("^that the database contains \"([^\"]*)\"$") public void that_the_database_contains(String title) { showTitle = title; try { guideBox = (BasicApi) context.getBean("provideArrestedDevleopmentSearch"); } catch (Exception e) { guideBox.basicHandle("Init Error: ", e); } } #When("^the client requests a database search for Arrested Development$") public void the_client_requests_a_database_search_for_Arrested_Development() { try { guideBox.searchJson("results.title"); } catch (Exception e) { guideBox.basicHandle("Json Search Error: ", e); } } #Then("^the title should be visible in the search results$") public void the_title_should_be_visible_in_the_search_results() { try { Assert.assertThat(guideBox.getJsonResults(), CoreMatchers.containsString(showTitle)); } catch(Exception e) { guideBox.basicHandle("Assertion Error: ", e); } } UPDATE 1 I managed to fix the problem in a general sense. I made the BasicApi parameterized constructor less busy, resulting in a bit more code in the step defs to manually control some requests. This got rid of the too many API request error. That being said, although the program now works as expected, my question does still stand, as I think (or hope) that there is probably a better way to do this than using the ApplicationContext the way I am using it.
Transaction with binded thread connection in Spring
I want to bind a connection to a thread, and use that connection for any JdbcTemplate calls, to finally commit the changes or do a rollback. I'm declaring all sentences from a Groovy script, so I can't control how many SQL query will be call, that's why I have to used this method and not a TransactionalTemplate or something like that. this script will call a helper class that will use that connection and JdbcTemplate, let's call that class SqlHelper. Right now my non-working-as-needed solution is call from groovy script to that SqlHelper to initialize a transaction: initTransaction(ecommerce) which calls public void initTransaction(DataSource dataSource) { DefaultTransactionDefinition transactionDefinition = new DefaultTransactionDefinition(); transactionDefinition.setReadOnly(false); transactionDefinition.setIsolationLevel(TransactionDefinition.ISOLATION_SERIALIZABLE); // dataSourceTransactionManager.setDataSource(dataSource); // dataSourceTransactionManager.setRollbackOnCommitFailure(true); // dataSourceTransactionManager.setTransactionSynchronization(TransactionSynchronization.STATUS_COMMITTED); // dataSourceTransactionManager.setDataSource(dataSource); try { Connection connection = DataSourceUtils.getConnection(dataSource); connection.setAutoCommit(false); DataSourceUtils.prepareConnectionForTransaction(connection, transactionDefinition); } catch (CannotGetJdbcConnectionException e) { throw new RuntimeException(e); } catch (SQLException e) { throw new RuntimeException(e); } } After that the script will call some SQL operations, like sqlUpdate(ecommerce, insertSentence, insertParams) which calls public Integer update(DataSource dataSource, String sql, Map<String, Object> paramMap) { return new NamedParameterJdbcTemplate(dataSource).update(sql, paramMap); } Finally I want to finish the transaction committing the changes using commitTransaction(dataSource) which calls public void commitTransaction(DataSource dataSource) { Connection connection = DataSourceUtils.getConnection(dataSource); try { connection.commit(); } catch (Exception e) { rollbackTransaction(dataSource); } // DataSourceUtils.resetConnectionAfterTransaction(connection, TransactionDefinition.ISOLATION_DEFAULT); // SimpleTransactionStatus transactionStatus = new SimpleTransactionStatus(false); // try { // dataSourceTransactionManager.commit(transactionStatus); // jta.commit(transactionStatus); // } catch (TransactionException e) { // dataSourceTransactionManager.rollback(transactionStatus); // throw new RuntimeException(e); // } } private void rollbackTransaction(DataSource dataSource) { Connection connection = DataSourceUtils.getConnection(dataSource); try { connection.rollback(); } catch (SQLException e) { throw new RuntimeException(e); } DataSourceUtils.resetConnectionAfterTransaction(connection, TransactionDefinition.ISOLATION_DEFAULT); } I left commented blocks of some testing to show you what approaches I tried. I don't know very well how Spring transaction works, so I'm just trying different things and trying to learn how all this stuff works... I will provide you more information if you want ;) Thank you in advance. UPDATE As M. Denium suggested, that's what I have for now: I declared the variable, using the TransactionStatus as ThreadSafe and finally solved as: private DataSourceTransactionManager dataSourceTransactionManager = null; private DefaultTransactionDefinition transactionDefinition = null; private static final ThreadLocal<TransactionStatus> transactionStatus = new ThreadLocal<TransactionStatus>() { #Override protected TransactionStatus initialValue() { return null; } }; And then using the same call from Groovy script, using the helper methods: public void initTransaction(DataSource dataSource) { dataSourceTransactionManager = new DataSourceTransactionManager(); transactionDefinition = new DefaultTransactionDefinition(); transactionDefinition.setReadOnly(false); transactionDefinition.setIsolationLevel(TransactionDefinition.ISOLATION_SERIALIZABLE); dataSourceTransactionManager.setRollbackOnCommitFailure(true); dataSourceTransactionManager.setTransactionSynchronization(TransactionSynchronization.STATUS_UNKNOWN); dataSourceTransactionManager.setDataSource(dataSource); transactionStatus.set(dataSourceTransactionManager.getTransaction(null)); } public void commitTransaction() { try { LOG.info("Finishing transaction..."); dataSourceTransactionManager.commit(transactionStatus.get()); dataSourceTransactionManager.getDataSource().getConnection().close(); LOG.info("Done."); } catch (Throwable e) { dataSourceTransactionManager.rollback(transactionStatus.get()); throw new RuntimeException("An exception during transaction. Rolling back."); } }
You are trying to reimplement the things that are already implemented by the transaction abstraction of Spring. Simply use the proper PlatformTransactionManager (you can probably grab that from an ApplicationContext) keep a reference to the TransactionStatus instead of a DataSource and use that to commit/rollback. public TransactionStatus initTransaction() { return transactionManager.getTransaction(null); } public void commit(TransactionStatus status) { transactionManager.commit(status); } Instead of passing the TransactionStatus around you could also store it in a ThreadLocal and retrieve it in the commit method. This would ease the pain. Another tip you shouldn't be creating JdbcTemplates and NamedParameterJdbcTemplates those are heavy objects to create. Upon construction they consult a connection to determine which database and version this is needed for the exception conversion. So performance wise this isn't a smart thing to do. Create a single instance and reuse, the templates are thread safe so you would only be needing a single instance. However I would strongly argue that you should actually be using Groovy and not to try to work around it. Groovy has the Sql class that can help you. You already have access to the DataSource so doing something like this would be all that is needed. def sql = new Sql(dataSource); sql.withTransaction { sql.execute "INSERT INTO city (name, state, founded_year) VALUES ('Minneapolis', 'Minnesota', 1867)" sql.execute "INSERT INTO city (name, state, founded_year) VALUES ('Orlando', 'Florida', 1875)" sql.execute "INSERT INTO city (name, state, founded_year) VALUES ('Gulfport', 'Mississippi', 1887)" } This is plain Groovy, no need to develop additional classes or to write extensive documentation to get it working. Just Groovy...
Broadcasting using the protocol Zab in ZooKeeper
Good morning, I am new to ZooKeeper and its protocols and I am interested in its broadcast protocol Zab. Could you provide me with a simple java code that uses the Zab protocol of Zookeeper? I have been searching about that but I did not succeed to find a code that shows how can I use Zab. In fact what I need is simple, I have a MapReduce code and I want all the mappers to update a variable (let's say X) whenever they succeed to find a better value of X (i.e. a bigger value). In this case, the leader has to compare the old value and the new value and then to broadcast the actual best value to all mappers. How can I do such a thing in Java? Thanks in advance, Regards
You don't need to use the Zab protocol. Instead you may follow the below steps: You have a Znode say /bigvalue on Zookeeper. All the mappers when starts reads the value stored in it. They also put an watch for data change on the Znode. Whenever a mapper gets a better value, it updates the Znode with the better value. All the mappers will get notification for the data change event and they read the new best value and they re-establish the watch for data changes again. That way they are in sync with the latest best value and may update the latest best value whenever there is a better value. Actually zkclient is a very good library to work with Zookeeper and it hides a lot of complexities ( https://github.com/sgroschupf/zkclient ). Below is an example that demonstrates how you may watch a Znode "/bigvalue" for any data change. package geet.org; import java.io.UnsupportedEncodingException; import org.I0Itec.zkclient.IZkDataListener; import org.I0Itec.zkclient.ZkClient; import org.I0Itec.zkclient.exception.ZkMarshallingError; import org.I0Itec.zkclient.exception.ZkNodeExistsException; import org.I0Itec.zkclient.serialize.ZkSerializer; import org.apache.zookeeper.data.Stat; public class ZkExample implements IZkDataListener, ZkSerializer { public static void main(String[] args) { String znode = "/bigvalue"; ZkExample ins = new ZkExample(); ZkClient cl = new ZkClient("127.0.0.1", 30000, 30000, ins); try { cl.createPersistent(znode); } catch (ZkNodeExistsException e) { System.out.println(e.getMessage()); } // Change the data for fun Stat stat = new Stat(); String data = cl.readData(znode, stat); System.out.println("Current data " + data + "version = " + stat.getVersion()); cl.writeData(znode, "My new data ", stat.getVersion()); cl.subscribeDataChanges(znode, ins); try { Thread.sleep(36000); } catch (InterruptedException e) { e.printStackTrace(); } } #Override public void handleDataChange(String dataPath, Object data) throws Exception { System.out.println("Detected data change"); System.out.println("New data for " + dataPath + " " + (String)data); } #Override public void handleDataDeleted(String dataPath) throws Exception { System.out.println("Data deleted " + dataPath); } #Override public byte[] serialize(Object data) throws ZkMarshallingError { if (data instanceof String){ try { return ((String) data).getBytes("UTF-8"); } catch (UnsupportedEncodingException e) { e.printStackTrace(); } } return null; } #Override public Object deserialize(byte[] bytes) throws ZkMarshallingError { try { return new String(bytes, "UTF-8"); } catch (UnsupportedEncodingException e) { e.printStackTrace(); } return null; } }
Addressing Scalability,Performance and Optimization issues in RMI Application?
my problem is: this design is working fine for one ball but i m unable to get it work for multiple balls, i have basically problem in replacing the "this" keyword in updateClients (). i thought i need to do something like this but i m failed: System.out.println("in ballimpl" + j.size()); for (ICallback aClient : j) { aClient.updateClients(BallImpl[i]); } The current situation of code is : The model remote object, which is iterating client list and calling update method of them, public class BallImpl extends UnicastRemoteObject implements Ball,Runnable { private List<ICallback> clients = new ArrayList<ICallback>(); protected static ServerServices chatServer; static ServerServices si; BallImpl() throws RemoteException { super(); } .... public synchronized void move() throws RemoteException { loc.translate((int) changeInX, (int) changeInY); } public void start() throws RemoteException { if (gameThread.isAlive()==false ) if (run==false){ gameThread.start(); } } /** Start the ball bouncing. */ // Run the game logic in its own thread. public void run() { while (true) { run=true; // Execute one game step try { updateClients(); } catch (RemoteException e) { e.printStackTrace(); } try { Thread.sleep(50); } catch (InterruptedException ex) { } } } public void updateClients() throws RemoteException { si = new ServerServicesImpl(); List<ICallback> j = si.getClientNames(); System.out.println("in messimpl " + j.size()); if (j != null) { System.out.println("in ballimpl" + j.size()); for (ICallback aClient : j) { aClient.updateClients(this); } } else System.err.println("Clientlist is empty"); } } The client which is implementing callback interface and has update method implementation : public final class thenewBallWhatIwant implements Runnable, ICallback { ..... #Override public void updateClients(final Ball ball) throws RemoteException { try { ball.move(); try { Thread.sleep(50); } catch (Exception e) { System.exit(0); } } catch (Exception e) { System.out.println("Exception: " + e); } } ..... } thanks for any feedback. jibbylala
Separate your RMI logic from your Ball logic. You should be able to run your ball simulation without needing any RMI modules. Just to run it locally, to test it. Then you should find a way to wrap that process in RMI so that you can still run it locally to test it without any sort of RMI interface. This block of code is the engine, and it is very important to be able to test it in as atomic a form as possible. Having extra parts integrated with it just increases the complexity of what will undoubtedly be some of the most complex code. Don't let any extra interfaces into your engine. It should be very specific and few the packages required to use your engine. Any new functionality your software needs, implement it appropriately in the engine to support generic design. Wrap that to provide specific functionality outside the core of the engine. This protects the engine design against changes to the environment. It also allows for more complete testing of the engine. We make exceptions sometimes in cases where something will only ever be used in one way. But in this case, testing without RMI would seem to be critical to getting your engine working correctly. If your engine runs faster than the network can keep up due to large numbers of clients connecting, do you want the whole game to slow down, or do you want the clients to lag behind? I say, you want to be able to make that choice.