According to documentation, I can use something like this in exceptionExpression: #Retryable(exceptionExpression="message.contains('this can be retried')")
But I want to get response body and check message inside it (from RestClientResponseException), something similar to this: exceptionExpression = "getResponseBodyAsString().contains('important message')"
I tried like that but it doesn't work. So, is it possible to do something similar and check info from responseBody?
Edit: Adding whole #Retryable annotation parameters with Gary Russell's suggestion:
#Retryable(value = HttpClientErrorException.class, exceptionExpression = "#{#root instanceof T(org.springframework.web.client.HttpClientErrorException) AND responseBodyAsString.contains('important message')}")
I'm using actual RestClientResponseException subclass that I'm catching but is still not triggering retry.
With the current release, the expression incorrectly requires static template markers; they will not be needed in 1.3.
#Retryable(exceptionExpression = "#{responseBodyAsString.contains('foo')}")
However, you can't use this expression if there are include or exclude properties so the expression should check the type:
#Retryable(exceptionExpression =
"#{#root instanceof T(org.springframework.web.client.RestClientResponseException) "
+ "AND responseBodyAsString.contains('foo')}")
EDIT
#SpringBootApplication
#EnableRetry
public class So61488237Application {
public static void main(String[] args) {
SpringApplication.run(So61488237Application.class, args).close();
}
#Bean
public ApplicationRunner runner(Foo foo) {
return args -> {
try {
foo.test(1, "foo.");
}
catch (Exception e) {
}
};
}
}
#Component
class Foo {
#Retryable(exceptionExpression =
"#{#root instanceof T(org.springframework.web.client.RestClientException) "
+ "AND responseBodyAsString.contains('foo')}")
public void test(int val, String str) {
System.out.println(val + ":" + str);
throw new RestClientResponseException("foo", 500, "bar", new HttpHeaders(), "foo".getBytes(),
StandardCharsets.UTF_8);
}
}
1:foo.
1:foo.
1:foo.
I've implemented the following approach, which in my opinion is much more convenient.
#Retryable(value = WebClientException.class,
exceptionExpression = RetryCheckerService.EXPRESSION,
maxAttempts = 5,
backoff = #Backoff(delay = 500))
public List<ResultDto> getSomeResource () {}
Here the RetryCheckerService encapsulates all needed logic.
#Service
public class RetryCheckerService {
public static final String EXPRESSION = "#retryCheckerService.shouldRetry(#root)";
public boolean shouldRetry(WebClientException ex) {
if (ex instanceof WebClientResponseException responseException) {
return responseException.getStatusCode().is5xxServerError()
|| responseException.getStatusCode().equals(HttpStatus.NOT_FOUND);
}
if (ex instanceof WebClientRequestException requestException) {
String message = requestException.getMessage();
if (message == null) {
return false;
}
return message.contains("HttpConnectionOverHTTP");
}
return false;
}
}
Related
I was wondering if anyone can help me with Apache FreeMarker? I'm trying to use a custom model but I can't figure it out.
Imagine I want to dump the result of a query (java ResultSet in a FreeMarker template). What is the best approach?
I have found on Google the class: ResultSetTemplateModel
import java.sql.ResultSet;
import freemarker.template.SimpleScalar;
import freemarker.template.TemplateHashModel;
import freemarker.template.TemplateModel;
import freemarker.template.TemplateModelException;
import freemarker.template.TemplateSequenceModel;
public class ResultSetTemplateModel implements TemplateSequenceModel {
private ResultSet rs = null;
public ResultSetTemplateModel(ResultSet rs) {
this.rs = rs;
}
public TemplateModel get(int i) throws TemplateModelException {
try {
rs.next();
} catch(Exception e) {
throw new TemplateModelException(e.toString());
}
TemplateModel model = new Row(rs);
return model;
}
public int size() throws TemplateModelException {
int size=0;
try {
rs.last();
size = rs.getRow();
rs.beforeFirst();
} catch (Exception e ) {
throw new TemplateModelException( e.toString());
}
return size;
}
class Row implements TemplateHashModel {
private ResultSet rs = null;
public Row(ResultSet rs) {
this.rs = rs;
}
public TemplateModel get(String s) throws TemplateModelException {
TemplateModel model = null;
try {
model = new SimpleScalar( rs.getString(s) );
} catch (Exception e) { e.printStackTrace(); }
return model;
}
public boolean isEmpty() throws TemplateModelException {
boolean isEmpty = false;
if ( rs == null ) { isEmpty = true; }
return isEmpty;
}
}
}
And I have a very simple class (I even made it easier than previous):
public static void main(String[] args) {
try {
Configuration cfg = new Configuration(Configuration.VERSION_2_3_27);
cfg.setTemplateExceptionHandler(TemplateExceptionHandler.RETHROW_HANDLER);
cfg.setClassForTemplateLoading(MyCLASS.class, "/");
StringWriter out = new StringWriter();
Map<String, Object> parameters = new TreeMap<>();
ResultSet rs = getResultSet("Select foo, bar FROM my_table");
parameters.put("hello", "World");
parameters.put("result", rs);
Template temp = cfg.getTemplate("template.txt");
temp.process(parameters, out);
System.out.println("out = " + out);
} catch (IOException | TemplateException e) {
e.printStackTrace();
}
}
My template
Hello ${hello}
<#-- how do I specify ResultSet columns here ?? -->
How can I use the custom template?? Any advice?? I know how to load the template file. But I don't know how to specify that it is a custom model in the template.
THank you guys for the support :)
There are two ways of using ResultSetTemplateModel for wrapping ResultSet-s:
Either extend DefaultObjectWrapper by overriding handleUnknownType, where you return new ResultSetTemplateModel((ResultSet) obj) if obj is a ResultSet, otherwise call super. Then use Configuration.setObjectWrapper to actually use it.
Or, add new ResultSetTemplate(rs) to parameters instead of rs; if something is already a TempalteModel, it will not be wrapped again. Note that if you get a ResultSet from somewhere else in the template, this approach will not work as it avoids your manual wrapping, so extending the DefaultObjectWrapper is what you want generally.
Note that the ResultSetTemplateModel implementation shown is quite limited. The ObjectWrapper should be passed to the constructor as well, and stored in a final field. Then, instead of new SimpleScalar( rs.getString(s) ) it should do objectWrapper.wrap(rs.getObject(s)).
I'm trying to add a new field to request's body, in a Zuul Pre-filter.
I'm using one of the Neflix's Zuul sample projects from here, and my filter's implementation is very similar to UppercaseRequestEntityFilter from this sample.
I was able to apply a transformation such as uppercase, or even to completely modify the request, the only inconvenient is that I'm not able to modify the content of body's request that has a length more than the original length of the body's request.
This is my filter's implementation:
#Component
public class MyRequestEntityFilter extends ZuulFilter {
public String filterType() {
return "pre";
}
public int filterOrder() {
return 10;
}
public boolean shouldFilter() {
RequestContext context = getCurrentContext();
return true;
}
public Object run() {
try {
RequestContext context = getCurrentContext();
InputStream in = (InputStream) context.get("requestEntity");
if (in == null) {
in = context.getRequest().getInputStream();
}
String body = StreamUtils.copyToString(in, Charset.forName("UTF-8"));
body = body.replaceFirst("qqq", "qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq");
// body = body.toUpperCase();
context.set("requestEntity", new ServletInputStreamWrapper(body.getBytes("UTF-8")));
}
catch (IOException e) {
rethrowRuntimeException(e);
}
return null;
}
}
This is the request that I'm doing:
This is the response that I'm receiving:
I was able to obtain what I wanted, using the implementation of PrefixRequestEntityFilter, from sample-zuul-examples:
#Component
public class MyRequestEntityFilter extends ZuulFilter {
public String filterType() {
return "pre";
}
public int filterOrder() {
return 10;
}
public boolean shouldFilter() {
RequestContext context = getCurrentContext();
return true;
}
public Object run() {
try {
RequestContext context = getCurrentContext();
InputStream in = (InputStream) context.get("requestEntity");
if (in == null) {
in = context.getRequest().getInputStream();
}
String body = StreamUtils.copyToString(in, Charset.forName("UTF-8"));
body = body.replaceFirst("qqq", "qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq");
byte[] bytes = body.getBytes("UTF-8");
context.setRequest(new HttpServletRequestWrapper(getCurrentContext().getRequest()) {
#Override
public ServletInputStream getInputStream() throws IOException {
return new ServletInputStreamWrapper(bytes);
}
#Override
public int getContentLength() {
return bytes.length;
}
#Override
public long getContentLengthLong() {
return bytes.length;
}
});
}
catch (IOException e) {
rethrowRuntimeException(e);
}
return null;
}
}
We are working with project reactor and having a huge problem right now. This is how we produce (publish our data):
public Flux<String> getAllFlux() {
return Flux.<String>create(sink -> {
new Thread(){
public void run(){
Iterator<Cache.Entry<String, MyObject>> iterator = getAllIterator();
ObjectMapper mapper = new ObjectMapper();
while(iterator.hasNext()) {
try {
sink.next(mapper.writeValueAsString(iterator.next().getValue()));
} catch (IOException e) {
e.printStackTrace();
}
}
sink.complete();
}
} .start();
});
}
As you can see we are taking data from an iterator and are publishing each item in that iterator as a json string. Our subscriber does the following:
flux.subscribe(new Subscriber<String>() {
private Subscription s;
int amount = 1; // the amount of received flux payload at a time
int onNextAmount;
String completeItem="";
ObjectMapper mapper = new ObjectMapper();
#Override
public void onSubscribe(Subscription s) {
System.out.println("subscribe");
this.s = s;
this.s.request(amount);
}
#Override
public void onNext(String item) {
MyObject myObject = null;
try {
System.out.println(item);
myObject = mapper.readValue(completeItem, MyObject.class);
System.out.println(myObject.toString());
} catch (IOException e) {
System.out.println(item);
System.out.println("failed: " + e.getLocalizedMessage());
}
onNextAmount++;
if (onNextAmount % amount == 0) {
this.s.request(amount);
}
}
#Override
public void onError(Throwable t) {
System.out.println(t.getLocalizedMessage())
}
#Override
public void onComplete() {
System.out.println("completed");
});
}
As you can see we are simply printing the String item which we receive and parsing it into an object using jackson wrapper. The problem we got now is that for most of our items everything works fine:
{"itemId": "someId", "itemDesc", "some description"}
But for some items the String is cut off like this for example:
{"itemId": "some"
And the next item after that would be
"Id", "itemDesc", "some description"}
There is no pattern for those cuts. It is completely random and it is different everytime we run that code. Ofcourse our jackson is gettin an error Unexpected end of Input with that behaviour.
So what is causing such a behaviour and how can we solve it?
Solution:
Send the Object inside the flux instead of the String:
public Flux<ItemIgnite> getAllFlux() {
return Flux.create(sink -> {
new Thread(){
public void run(){
Iterator<Cache.Entry<String, ItemIgnite>> iterator = getAllIterator();
while(iterator.hasNext()) {
sink.next(iterator.next().getValue());
}
}
} .start();
});
}
and use the following produces type:
#RequestMapping(value="/allFlux", method=RequestMethod.GET, produces="application/stream+json")
The key here is to use stream+json and not only json.
We are trying to use google cloud datastore in our project and trying to use objectify as the ORM since google recommends it. I have carefully used and tried everything i could read about and think of but somehow the transactions don't seem to work. Following is my code and setup.
#RunWith(SpringRunner.class)
#EnableAspectJAutoProxy(proxyTargetClass = true)
#ContextConfiguration(classes = { CoreTestConfiguration.class })
public class TestObjectifyTransactionAspect {
private final LocalServiceTestHelper helper = new LocalServiceTestHelper(
// Our tests assume strong consistency
new LocalDatastoreServiceTestConfig().setApplyAllHighRepJobPolicy(),
new LocalMemcacheServiceTestConfig(), new LocalTaskQueueTestConfig());
private Closeable closeableSession;
#Autowired
private DummyService dummyService;
#BeforeClass
public static void setUpBeforeClass() {
// Reset the Factory so that all translators work properly.
ObjectifyService.setFactory(new ObjectifyFactory());
}
/**
* #throws java.lang.Exception
*/
#Before
public void setUp() throws Exception {
System.setProperty("DATASTORE_EMULATOR_HOST", "localhost:8081");
ObjectifyService.register(UserEntity.class);
this.closeableSession = ObjectifyService.begin();
this.helper.setUp();
}
/**
* #throws java.lang.Exception
*/
#After
public void tearDown() throws Exception {
AsyncCacheFilter.complete();
this.closeableSession.close();
this.helper.tearDown();
}
#Test
public void testTransactionMutationRollback() {
// save initial list of users
List<UserEntity> users = new ArrayList<UserEntity>();
for (int i = 1; i <= 10; i++) {
UserEntity user = new UserEntity();
user.setAge(i);
user.setUsername("username_" + i);
users.add(user);
}
ObjectifyService.ofy().save().entities(users).now();
try {
dummyService.mutateDataWithException("username_1", 6L);
} catch (Exception e) {
e.printStackTrace();
}
List<UserEntity> users2 = this.dummyService.findAllUsers();
Assert.assertEquals("Size mismatch on rollback", users2.size(), 10);
boolean foundUserIdSix = false;
for (UserEntity userEntity : users2) {
if (userEntity.getUserId() == 1) {
Assert.assertEquals("Username update failed in transactional context rollback.", "username_1",
userEntity.getUsername());
}
if (userEntity.getUserId() == 6) {
foundUserIdSix = true;
}
}
if (!foundUserIdSix) {
Assert.fail("Deleted user with userId 6 but it is not rolledback.");
}
}
}
Since I am using spring, idea is to use an aspect with a custom annotation to weave objectify.transact around the spring service beans methods that are calling my daos.
But somehow the update due to ObjectifyService.ofy().save().entities(users).now(); is not gettign rollbacked though the exception throws causes Objectify to run its rollback code. I tried printing the ObjectifyImpl instance hashcodes and they are all same but still its not rollbacking.
Can someone help me understand what am i doing wrong? Havent tried the actual web based setup yet...if it cant pass transnational test cases there is no point in actual transaction usage in a web request scenario.
Update: Adding aspect, services, dao as well to make a complete picture. The code uses spring boot.
DAO class. Note i am not using any transactions here because as per code of com.googlecode.objectify.impl.TransactorNo.transactOnce(ObjectifyImpl<O>, Work<R>) a transnational ObjectifyImpl is flushed and committed in this method which i don't want. I want commit to happen once and rest all to join in on that transaction. Basically this is the wrong code in com.googlecode.objectify.impl.TransactorNo ..... i will try to explain my understanding a later in the question.
#Component
public class DummyDaoImpl implements DummyDao {
#Override
public List<UserEntity> loadAll() {
Query<UserEntity> query = ObjectifyService.ofy().transactionless().load().type(UserEntity.class);
return query.list();
}
#Override
public List<UserEntity> findByUserId(Long userId) {
Query<UserEntity> query = ObjectifyService.ofy().transactionless().load().type(UserEntity.class);
//query = query.filterKey(Key.create(UserEntity.class, userId));
return query.list();
}
#Override
public List<UserEntity> findByUsername(String username) {
return ObjectifyService.ofy().transactionless().load().type(UserEntity.class).filter("username", username).list();
}
#Override
public void update(UserEntity userEntity) {
ObjectifyService.ofy().save().entity(userEntity);
}
#Override
public void update(Iterable<UserEntity> userEntities) {
ObjectifyService.ofy().save().entities(userEntities);
}
#Override
public void delete(Long userId) {
ObjectifyService.ofy().delete().key(Key.create(UserEntity.class, userId));
}
}
Below is the Service class
#Service
public class DummyServiceImpl implements DummyService {
private static final Logger LOGGER = LoggerFactory.getLogger(DummyServiceImpl.class);
#Autowired
private DummyDao dummyDao;
public void saveDummydata() {
List<UserEntity> users = new ArrayList<UserEntity>();
for (int i = 1; i <= 10; i++) {
UserEntity user = new UserEntity();
user.setAge(i);
user.setUsername("username_" + i);
users.add(user);
}
this.dummyDao.update(users);
}
/* (non-Javadoc)
* #see com.bbb.core.objectify.test.services.DummyService#mutateDataWithException(java.lang.String, java.lang.Long)
*/
#Override
#ObjectifyTransactional
public void mutateDataWithException(String usernameToMutate, Long userIdToDelete) throws Exception {
//update one
LOGGER.info("Attempting to update UserEntity with username={}", "username_1");
List<UserEntity> mutatedUsersList = new ArrayList<UserEntity>();
List<UserEntity> users = dummyDao.findByUsername(usernameToMutate);
for (UserEntity userEntity : users) {
userEntity.setUsername(userEntity.getUsername() + "_updated");
mutatedUsersList.add(userEntity);
}
dummyDao.update(mutatedUsersList);
//delete another
UserEntity user = dummyDao.findByUserId(userIdToDelete).get(0);
LOGGER.info("Attempting to delete UserEntity with userId={}", user.getUserId());
dummyDao.delete(user.getUserId());
throw new RuntimeException("Dummy Exception");
}
/* (non-Javadoc)
* #see com.bbb.core.objectify.test.services.DummyService#findAllUsers()
*/
#Override
public List<UserEntity> findAllUsers() {
return dummyDao.loadAll();
}
Aspect which wraps the method annoted with ObjectifyTransactional as a transact work.
#Aspect
#Component
public class ObjectifyTransactionAspect {
private static final Logger LOGGER = LoggerFactory.getLogger(ObjectifyTransactionAspect.class);
#Around(value = "execution(* *(..)) && #annotation(objectifyTransactional)")
public Object objectifyTransactAdvise(final ProceedingJoinPoint pjp, ObjectifyTransactional objectifyTransactional) throws Throwable {
try {
Object result = null;
Work<Object> work = new Work<Object>() {
#Override
public Object run() {
try {
return pjp.proceed();
} catch (Throwable throwable) {
throw new ObjectifyTransactionExceptionWrapper(throwable);
}
}
};
switch (objectifyTransactional.propagation()) {
case REQUIRES_NEW:
int limitTries = objectifyTransactional.limitTries();
if(limitTries <= 0) {
Exception illegalStateException = new IllegalStateException("limitTries must be more than 0.");
throw new ObjectifyTransactionExceptionWrapper(illegalStateException);
} else {
if(limitTries == Integer.MAX_VALUE) {
result = ObjectifyService.ofy().transactNew(work);
} else {
result = ObjectifyService.ofy().transactNew(limitTries, work);
}
}
break;
case NOT_SUPPORTED :
case NEVER :
case MANDATORY :
result = ObjectifyService.ofy().execute(objectifyTransactional.propagation(), work);
break;
case REQUIRED :
case SUPPORTS :
ObjectifyService.ofy().transact(work);
break;
default:
break;
}
return result;
} catch (ObjectifyTransactionExceptionWrapper e) {
String packageName = pjp.getSignature().getDeclaringTypeName();
String methodName = pjp.getSignature().getName();
LOGGER.error("An exception occured while executing [{}.{}] in a transactional context."
, packageName, methodName, e);
throw e.getCause();
} catch (Throwable ex) {
String packageName = pjp.getSignature().getDeclaringTypeName();
String methodName = pjp.getSignature().getName();
String fullyQualifiedmethodName = packageName + "." + methodName;
throw new RuntimeException("Unexpected exception while executing ["
+ fullyQualifiedmethodName + "] in a transactional context.", ex);
}
}
}
Now the problem code part that i see is as follows in com.googlecode.objectify.impl.TransactorNo:
#Override
public <R> R transact(ObjectifyImpl<O> parent, Work<R> work) {
return this.transactNew(parent, Integer.MAX_VALUE, work);
}
#Override
public <R> R transactNew(ObjectifyImpl<O> parent, int limitTries, Work<R> work) {
Preconditions.checkArgument(limitTries >= 1);
while (true) {
try {
return transactOnce(parent, work);
} catch (ConcurrentModificationException ex) {
if (--limitTries > 0) {
if (log.isLoggable(Level.WARNING))
log.warning("Optimistic concurrency failure for " + work + " (retrying): " + ex);
if (log.isLoggable(Level.FINEST))
log.log(Level.FINEST, "Details of optimistic concurrency failure", ex);
} else {
throw ex;
}
}
}
}
private <R> R transactOnce(ObjectifyImpl<O> parent, Work<R> work) {
ObjectifyImpl<O> txnOfy = startTransaction(parent);
ObjectifyService.push(txnOfy);
boolean committedSuccessfully = false;
try {
R result = work.run();
txnOfy.flush();
txnOfy.getTransaction().commit();
committedSuccessfully = true;
return result;
}
finally
{
if (txnOfy.getTransaction().isActive()) {
try {
txnOfy.getTransaction().rollback();
} catch (RuntimeException ex) {
log.log(Level.SEVERE, "Rollback failed, suppressing error", ex);
}
}
ObjectifyService.pop();
if (committedSuccessfully) {
txnOfy.getTransaction().runCommitListeners();
}
}
}
transactOnce is by code / design always using a single transaction to do things. It will either commit or rollback the transaction. there is no provision to chain transactions like a normal enterprise app would want.... service -> calls multiple dao methods in a single transaction and commits or rollbacks depending on how things look.
keeping this in mind, i removed all annotations and transact method calls in my dao methods so that they don't start an explicit transaction and the aspect in service wraps the service method in transact and ultimately in transactOnce...so basically the service method is running in a transaction and no new transaction is getting fired again. This is a very basic scenario, in actual production apps services can call other service methods and they might have the annotation on them and we could still end up in a chained transaction..but anyway...that is a different problem to solve....
I know NoSQLs dont support write consistency at table or inter table levels so am I asking too much from google cloud datastore?
I use for debugging reasons the java.lang.reflect.Proxy stuff to have a generic way to implement all possible interfaces... but this seems to be difficult to get it working with proguard. Any suggestions?
THX
-Marco
public class DebugLogListenerFactory {
public static IAirplaneListenerAll createStreamHandle(ICAirplane plane) {
DebugLogListenerHandler handler = new DebugLogListenerHandler(plane);
IAirplaneListenerAll proxy = (IAirplaneListenerAll) Proxy
.newProxyInstance(IAirplaneListenerAll.class.getClassLoader(),
new Class[] { IAirplaneListenerAll.class }, handler);
plane.addListener(proxy);
return proxy;
}
private static class DebugLogListenerHandler implements InvocationHandler {
private final Level levDef = Level.FINE;
public DebugLogListenerHandler() {
}
public Object invoke(Object proxy, Method method, Object[] args)
throws Throwable {
System.out.println("invoked" + method);
String methodName = method.getName();
String msg = methodName + ": ";
if (args != null) {
boolean first = true;
for (Object o : args) {
if (first) {
first = false;
} else {
msg += " ,";
}
msg += o.toString();
}
}
CDebug.getLog().log(levDef, msg);
return null;
}
}
}
The easiest solution is probably to avoid shrinking/optimizing/obfuscating the interface and its methods:
-keep interface some.package.IAirplaneListenerAll {
<methods>;
}
You might allow shrinking:
-keep,allowshrinking interface some.package.IAirplaneListenerAll {
<methods>;
}
If the InvocationHandler can deal with obfuscated method names, you might also allow obfuscation:
-keep,allowshrinking,allowobfuscation interface some.package.IAirplaneListenerAll {
<methods>;
}