What does "RemoteException" mean in general in HDFS? - hadoop

1) May anyone help me about the concept of "Remoteexception"? What does it mean in general?
2) Also what does it mean by unwrapRemoteException? Was not sure it means by "if this remote exception wraps up one of the lookupTypes"
/**
* If this remote exception wraps up one of the lookupTypes
* then return this exception.
* <p>
* Unwraps any IOException.
*
* #param lookupTypes the desired exception class.
* #return IOException, which is either the lookupClass exception or this.
*/
public IOException unwrapRemoteException(Class<?>... lookupTypes) {
if(lookupTypes == null)
return this;
for(Class<?> lookupClass : lookupTypes) {
if(!lookupClass.getName().equals(getClassName()))
continue;
try {
return instantiateException(lookupClass.asSubclass(IOException.class));
} catch(Exception e) {
// cannot instantiate lookupClass, just return this
return this;
}
}
// wrapped up exception is not in lookupTypes, just return this
return this;
}
(Hadoop_HDFS_Open_Source: https://github.com/apache/hadoop)
Thanks in advance! Any help will be much appreciated!

1) May anyone help me about the concept of "Remoteexception"? What does it mean in general?
Remote Exception is thrown(created) in the server side. Server throws such exception because client sent invalid request, or server has internal error or something else. The RPC server in the server side serialized the exception, and sent it to client side. Client side deserializes the exception, and gets the exception.
2) Also what does it mean by unwrapRemoteException?
The question is "why we need to wrap exception". First, RemoteException is a wrapper not the real exception. The real exception thrown by server could be AccessControlException(the user has no previleage), FileNotFoundException(invalid request). We wrap them up inside a RemoteException. But why? Because the code is cleaner and more readable.
Was not sure it means by "if this remote exception wraps up one of the lookupTypes"
For example, in DFSClient.java
public HdfsFileStatus getFileInfo(String src) throws IOException {
checkOpen();
try {
return namenode.getFileInfo(src);
} catch(RemoteException re) {
throw re.unwrapRemoteException(AccessControlException.class,
FileNotFoundException.class,
UnresolvedPathException.class);
}
}
If the re wraps a FileNotFoundException, then the getFileInfo only returns the FileNotFoundException. Then the user can see a shorter & cleaner exception message. The user only need to know the file is not found, don't care if it's remote or not.
But if the re wraps a SafeModeException or some unknown exception. It's probably some server's internal bug or configuration error. We throw the re(RemoteException) exactly. So the user can know the error is from remote(server side), even though the user doesn't know the wrapped SafeModeException(or some unknown exception). The user can ask help from tech support.

Related

Why freemarker.cache.TemplateCache has storeNegativeLookup?

I have template_update_delay=24h for caching the templates for 24 hours. If my URLTemplateLoader gets IOException due to temporary outage(http status 429) then freemarker.cache.TemplateCache will call storeNegativeLookup and cached the exception too.
cachedTemplate.templateOrException = e
// Template source was removed
if (!newLookupResult.isPositive()) {
if(debug) {
LOG.debug(debugName + " no source found.");
}
storeNegativeLookup(tk, cachedTemplate, null);
return null;
}
private void storeNegativeLookup(TemplateKey tk,
CachedTemplate cachedTemplate, Exception e) {
cachedTemplate.templateOrException = e;
cachedTemplate.source = null;
cachedTemplate.lastModified = 0L;
storeCached(tk, cachedTemplate);
}
Later even if the URL endpoint is up and available, freemarker.cache.TemplateCache:getTemplate() will keep picking the cachedTemplate with the IOException and will keep rethrowing the exception till the cache is not expired.
else if(t instanceof IOException) {
rethrown = true;
throwLoadFailedException((IOException)t);
}
This is causing application of fail all the time ((.
How can I force Freemarker to retry fetching the template from the source instead of cache if there was an exception happened last time?
Failed lookups are cached for the same reason successful ones are. A failed lookup can take as much or even more time (if for example a connection timeout is involved) as a successful one, and so can jam the application if it's for a frequently used template (by consuming the whole thread pool for example).
The problem is that there's only a single templateUpdateDelay. But it's possible that you want to use a different one depending what the cause of the error was. You should open a feature request for that on Jira.
What can you do right now though. You can catch the exception thrown by Configuration.getTemplate, and walk the cause trace to find out if the root cause is an exception that you don't want to cache, and then call Configuration.removeTemplateFromCache (but consider if you should only do it once in a certain time interval). I'm not sure if the exception with HTTP 429 can be recognized reliably out-of-the-box, or you have to customize the TemplateLoader so that the thrown exception contains the necessary information.

How to create a custom Sonar rule to check if a method throws a certain exception?

So we want to check a certain method, say findOne() in certain java classes if it throws a specific exception or not. If it doesn't throw the exception, then an issue to be reported at method level.
We could use
public void visitThrowStatement(ThrowStatementTree tree)
but this only gets called when there is a statement that throws the exception, how can we check if it's not thrown?
You need to keep a context in your visitor to know in which method you are currently visiting throw statements.
Basically, if you are within a findOne method, then you will visit the code of the method, if it has a correct throw statement,then don't raise an issue but if it has not then raise an issue.
Something along the lines of (this is pseudo code and should of course be adapted but that will explain the concept):
LinkedList<MethodTree> stack;
int throwCount = 0;
void visitMethod(MethodTree methodTree) {
stack.push(methodTree);
throwCount = 0;
super.visitMethod(methodTree);
if(throwCount == 0) {
//raise Issue
}
}
void visit throwStatement(ThrowStatementTree tree) {
if(isCorrectExceptionThrown(tree)) {
throwCount++;
}
}

Identifying database errors and query errors separate in spring

I am writing a RESTful web service using spring 3 and I noticed that when I implemented my DAO's (I am using spring-jdbc for database access), the exceptions that get thrown are pretty generic, so i am not able to identify if the exception occurred because my database is down or my query failed.
sample code:
try {
Q q = jdbcTemplate.queryForObject(MY_QUERY, new Object[]{id}, new MyMapper());
return q;
} catch (DataAccessException e) {
// What is this exception ? database down ? query failed ?
}
Unless I know what exception is this during runtime, I can't send back reasonable error message to service client.
Any help is appreciated. Thanks
You shouldn't be trying to catch every single possible exception; code so that you don't run into the possibility of multiple user or programmer error type exceptions. Generally there are three type of exceptions that occur in every DAO regardless of what you do. (1) your database is down... imo if this is the issue you've got bigger problems. (2) user authentication error which is easy enough to catch and deal with (however you should probably be handling that situation on your RESTful front-end. (3) improper data. If you have bad data, just send the attempted data back and the reason for the exception.
try {
Q q = jdbcTemplate.queryForObject(MY_QUERY, new Object[]{id}, new MyMapper());
return q;
} catch (DataAccessException e) {
throw new DaoException("Could not retrieve q with ID: " + qID, e);
}
The method queryForObject only throws DataAccessException or IncorrectResultSizeDataAccessException. You shouldn't catch exception due to database connection failure at this level. However, in your DAO class, you normally have code to establish the dataSource which can be injected by Spring's IoC container. That part of code will throw exception if the database connection fails. You should catch DB failure exception there.

How can I tread OpenDolphin client send HttpHostConnectException?

Is there way to handle situation when message is not delivered to server? Dolphin log infors about situation clearly, but I'would like to catch it from code. I was looking for some method like: onError to override like onFinished:
clientDolphin.send(message, new OnFinishedHandlerAdapter() {
#Override
public void onFinished(List<ClientPresentationModel> presentationModels) {
// Do something useful
}
}
});
, but there is nothing like that. Also wrapping send call in try/catch does not work(not suprising since send is not blocking its caller code).
I thing there is definitely some easy way to get informed about undelivered message, but I cant see it.
Thaks, in advace, for answers!
You can assign an onException handler to the ClientConnector - and you are actually supposed to do so. The exception handler will get the exception object passed in that happened in the asynchronous send action.
Below is the default handler that even tells you, what you should do ;-)
Closure onException = { Throwable up ->
def out = new StringWriter()
up.printStackTrace(new PrintWriter(out))
log.severe("onException reached, rethrowing in UI Thread, consider setting ClientConnector.onException\n${out.buffer}")
uiThreadHandler.executeInsideUiThread { throw up } // not sure whether this is a good default
}

About Spring Transaction Manager

Currently i am using spring declarative transaction manager in my application. During DB operations if any constraint violated i want to check the error code against the database. i mean i want to run one select query after the exception happened. So i am catching the DataIntegrityViolationException inside my Catch block and then i am trying to execute one more error code query. But that query is not get executed . I am assuming since i am using the transaction manager if any exception happened the next query is not getting executed. Is that right?. i want to execute that error code query before i am returning the results to the client. Any way to do this?
#Override
#Transactional
public LineOfBusinessResponse create(
CreateLineOfBusiness createLineOfBusiness)
throws GenericUpcException {
logger.info("Start of createLineOfBusinessEntity()");
LineOfBusinessEntity lineOfBusinessEntity =
setLineOfBusinessEntityProperties(createLineOfBusiness);
try {
lineOfBusinessDao.create(lineOfBusinessEntity);
return setUpcLineOfBusinessResponseProperties(lineOfBusinessEntity);
}
// Some db constraints is failed
catch (DataIntegrityViolationException dav) {
String errorMessage =
errorCodesBd.findErrorCodeByErrorMessage(dav.getMessage());
throw new GenericUpcException(errorMessage);
}
// General Exceptions handling
catch (Exception exc) {
logger.debug("<<<<Coming inside General >>>>");
System.out.print("<<<<Coming inside General >>>>");
throw new GenericUpcException(exc.getMessage());
}
}
public String findErrorCodeByErrorMessage(String errorMessage)throws GenericUpcException {
try{
int first=errorMessage.indexOf("[",errorMessage.indexOf("constraint"));
int last=errorMessage.indexOf("]",first);
String errorCode=errorMessage.substring(first+1, last);
//return errorCodesDao.find(errorCode);
return errorCode;
}
catch(Exception e)
{
throw new GenericUpcException(e.getMessage());
}
}
Please help me.
I don't think problem you're describing has anything to do with Transaction management. If DataIntegrityViolationException happens within your try() block you code within catch() should execute. Perhaps exception different from DataIntegrityViolationException happens or your findErrorCodeByErrorMessage() throwing another exception. In general, Transaction logic would be applied only once you return from your method call, until then you could do whatever you like using normal Java language constructs. I suggest you put breakpoint in your error error handler or some debug statements to see what's actually happening.

Resources