I am trying to expose a rest endpoint with camel. It will show a json data which is inside some .json files stored in s3 bucket. Also, it will filter by a date range.
First, I got some s3 objects informations in my Camel routes. (I am using kotlin)
//expose the endpoint
from("jetty:http://0.0.0.0:8080/getObjects")
.routeId("list-objects-on-bucket")
.to("aws-s3://[bucket-name]?amazonS3Client=#s3Client&operation=listObjects")
.process(ListObjects())
.to("direct:filter-list-from-s3")
then, I filter the data. (Till here everything is alright)
from("direct:filter-list-from-s3")
.routeId("filter-list-from-s3")
.process(FilterObjects())
.to("log:info")
But in my FilterObject class I do not know how to download every files that matches (look the if statement) and pass it to the next route that will treat them
class SaoMateusFilterObjects : Processor {
override fun process(exchange: Exchange?) {
val start_date = exchange!!.getIn().getHeader("start_date") as String
val end_date = exchange.getIn().getHeader("end_date") as String
val formatter = DateTimeFormatter.ofPattern("dd-MM-yyyy")
val start = LocalDate.parse(start_date).format(formatter)
val end = LocalDate.parse(end_date).format(formatter)
val objectsNames = exchange!!.getIn().body as LinkedList<String>
for (objectName in objectsNames) {
if(objectName.contains(start) && objectName.contains(end) && objectName.contains(".json")) {
exchange.getIn() to "aws-s3://[bucket-name]?amazonS3Client=#s3Client&operation=getObject&fileName=$objectName"
}
}
}
}
Some problems are:
1 - I want to read. By I think I can't use the from() method. Because it can be use just once. So, the to() method is used to read.
2 - exchange.getIn().to("[s3-uri]") maybe/must be converted in S3Object(). How??
Can Someone help me with this?
Thank you
Instead of .to route, use .bean() and use the s3.getObject method to get the S3Object.
always Prefer using .bean() over .processor().
offical_s3_java_object operation sample.
Related
I have created an ASP.NET Core Web Api backend with an Angular 7 frontend. One of the methods in the Api can return either an object or an array to an Angular service. How do I route to specific observable, based on the data type returned? I am a noob to Angular, so any kind assistance would be appreciated.
Angular service call to Api:
getLinksFromSitus(situs: any) {
this.http.post(this.baseUrl + 'getLinksFromSitus', situs).subscribe(data =>
this.apiData.next(data)
);
}
Portion of Web Api that returns array if more than one APN present:
// if more than one item in list, get status information for each and return list to user to select appropriate apn
if (propApn.Count > 1)
{
return Ok(propApn);
}
Portion of same method to return object if only one value for APN:
var resultsModel = new Results
{
ArcGisLink = arcGisLink,
HistInfoLink = histInfoLink,
PropInfoLink = propInfoLink
};
return Ok(resultsModel);
You can't do this. Typescript can only type things based on static analysis at build time, what your describing would require Typescript to know the result of your API call at build time, which it doesn't do.
The best you can do is indicating that your API call can return both of your them:
public myApiFunc(req: MyRequestModel): Observable<any>
But that will still require you to figure out which type returned at runtime.
I was able to find a solution that worked...
getLinksFromSitus(situs: any) {
this.http.post(this.baseUrl + 'getLinksFromSitus', situs).subscribe(data => {
if (data.hasOwnProperty('arcGisLink')) {
this.apiData.next(data);
} else {
let vals = [];
vals = this.apiPropApn.getValue();
const item = vals.concat(data);
this.apiPropApn.next(item);
}
});
}
So, after subscribing to the HttpResponse, I am able to check if the data in the response contains a known property. If it doesn't contain the known property, then it concatenates the data to a BehaviorSubject array. It works perfectly.
I'm using SpringBoot and Spring REST.
I would like to understand the HTTP PATCH method to update properties of my Model
Is there any good tutorial explaining how to make it works ?
HTTP PATCH method and body to be Send
Controller method and how to manage the update operation
I've noticed that many of the provided answers are all JSON patching or incomplete answers. Below is a full explanation and example of what you need with functioning real world code
First, PATCH is a selective PUT. You use it to update any number of fields for an object or list of objects. In a PUT you typically send the entire object with whatever updates.
PATCH /object/7
{
"objId":7,
"objName": "New name"
}
PUT /object/7
{
"objId":7,
"objName": "New name",
"objectUpdates": true,
"objectStatus": "ongoing",
"scoring": null,
"objectChildren":[
{
"childId": 1
},
............
}
This allows you to update records without huge amounts of endpoints. For example, with above, to update scoring you need object/{id}/scoring, then to update name you need object/{id}/name. Literally one endpoint for every item or you require the front end to post the entire object for every update. If you have a huge object, this can take a lot of network time or mobile data that is unnecessary. The patch lets you have 1 endpoint with the minimal object property sends that a mobile platform should use.
here is an example of a real world use for patch:
#ApiOperation(value = "Patch an existing claim with partial update")
#RequestMapping(value = CLAIMS_V1 + "/{claimId}", method = RequestMethod.PATCH)
ResponseEntity<Claim> patchClaim(#PathVariable Long claimId, #RequestBody Map<String, Object> fields) {
// Sanitize and validate the data
if (claimId <= 0 || fields == null || fields.isEmpty() || !fields.get("claimId").equals(claimId)){
return new ResponseEntity<>(HttpStatus.BAD_REQUEST); // 400 Invalid claim object received or invalid id or id does not match object
}
Claim claim = claimService.get(claimId);
// Does the object exist?
if( claim == null){
return new ResponseEntity<>(HttpStatus.NOT_FOUND); // 404 Claim object does not exist
}
// Remove id from request, we don't ever want to change the id.
// This is not necessary, you can just do it to save time on the reflection
// loop used below since we checked the id above
fields.remove("claimId");
fields.forEach((k, v) -> {
// use reflection to get field k on object and set it to value v
// Change Claim.class to whatver your object is: Object.class
Field field = ReflectionUtils.findField(Claim.class, k); // find field in the object class
field.setAccessible(true);
ReflectionUtils.setField(field, claim, v); // set given field for defined object to value V
});
claimService.saveOrUpdate(claim);
return new ResponseEntity<>(claim, HttpStatus.OK);
}
The above can be confusing for some people as newer devs don't normally deal with reflection like that. Basically, whatever you pass this function in the body, it will find the associated claim using the given ID, then ONLY update the fields you pass in as a key value pair.
Example body:
PATCH /claims/7
{
"claimId":7,
"claimTypeId": 1,
"claimStatus": null
}
The above will update claimTypeId and claimStatus to the given values for claim 7, leaving all other values untouched.
So the return would be something like:
{
"claimId": 7,
"claimSrcAcctId": 12345678,
"claimTypeId": 1,
"claimDescription": "The vehicle is damaged beyond repair",
"claimDateSubmitted": "2019-01-11 17:43:43",
"claimStatus": null,
"claimDateUpdated": "2019-04-09 13:43:07",
"claimAcctAddress": "123 Sesame St, Charlotte, NC 28282",
"claimContactName": "Steve Smith",
"claimContactPhone": "777-555-1111",
"claimContactEmail": "steve.smith#domain.com",
"claimWitness": true,
"claimWitnessFirstName": "Stan",
"claimWitnessLastName": "Smith",
"claimWitnessPhone": "777-777-7777",
"claimDate": "2019-01-11 17:43:43",
"claimDateEnd": "2019-01-11 12:43:43",
"claimInvestigation": null,
"scoring": null
}
As you can see, the full object would come back without changing any data other than what you want to change. I know there is a bit of repetition in the explanation here, I just wanted to outline it clearly.
There is nothing inherently different in PATCH method as far as Spring is concerned from PUT and POST. The challenge is what you pass in your PATCH request and how you map the data in the Controller. If you map to your value bean using #RequestBody, you'll have to figure what is actually set and what null values mean. Others options would be limit PATCH requests to one property and specify it in url or map the values to a Map.
See also Spring MVC PATCH method: partial updates
Create a rest template using -
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
RestTemplate rest = new RestTemplate(new HttpComponentsClientHttpRequestFactory());
now make the PATCH call
ResponseEntity<Map<String, Object>> response = rest.exchange(api, HttpMethod.PATCH, request,
responseType);
I am trying to learn Scala using Spring framework. I have to implement conditional get logic in my code. I understand it could be done using etag or Last-Modified option.
Here is my piece of code:
var lastModifiedTime: Long = _;
#RequestMapping(value= Array("/users/{id}"),method=Array(RequestMethod.GET),headers = Array("Content-Type=application/json"))
#ResponseBody
def getmeth(request: User_details, web: WebRequest): User_details = {
if (web.checkNotModified(lastModifiedTime)) {
return null
} else {
lastModifiedTime = System.currentTimeMillis()
}
Could you please help me to fix this code?
Disclaimer: I don't know Spring Web.
But according to the documentation fist of all you should take action if request is modified so you should remove bang (!) from condition. Also lastModifiedTime should be computed from the outside of the getmeth method.
Notice that unlike in Java if statement is an expression and it returns value so you shouldn't use return statement.
As it was said in comment conditional code can be easy and safely done using Scala's Option. In Scala you should always avoid null, as it is hard to distinguish it from incorrect behavior of your code, and it is very easy to forget or don't know that it is required to write logic dealing with it - you must always read the javadoc (assuming it exists and it is up to date). When you use Option type compiler will force you to deal with "nullability".
def getmeth(request: User_details, web: WebRequest): Option[User_details] =
if (web.checkNotModified(lastModifiedTime)) {
None
} else {
val userDetails = yourLogic()
Some(userDetails)
}
Then you can perform an action when option is a Some instance. To do that you can use map method.
getmeth(req, web) map { userDetails =>
userDetails.getName
}
EDIT: #optimus Now when you gave wider scope I see that your method signature is forced by framework and yon can't wrap your value with Option. I think that your problem may be that you update lastModifiedTime on every request so it seems reasonable to me that checkNotModified is always false. I think that you should use that feature only on requests that not always update checkNotModified to current time. It becomes pointless otherwise.
Update lastModifiedTime once your resource has become outdated.
#ResponseStatus(HttpStatus.OK)
#RequestMapping(value = Array("/users/{user_id}"),method = Array(RequestMethod.GET))
def getUser(#PathVariable("user_id") user_id: String,
#Context req : Request ): Any = {
val u = hm.get(user_id).asInstanceOf[User]
val tag = u.hashCode().asInstanceOf[EntityTag]
if (req.getMethod().equals("GET")) {
val rb : Response.ResponseBuilder = req.evaluatePreconditions(tag);
if (rb != null)
{
rb
}
else
{
// val u = hm.get(user_id).asInstanceOf[User]
u
}
}
How to force URIBuilder.path(...) to encode parameters like "%AD"?
The methods path, replacePath and segment of URIBuilder do not always encode parameters with percentage, correctly.
When a parameter contains the character "%" followed by two characters that together form an URL-encoded character, the "%" is not encoded as "%25".
For example
URI uri = UriBuilder.fromUri("https://dummy.com").queryParam("param", "%AD");
String test = uri.build().toString();
"test" is "https://dummy.com?param=%AD"
But it should be "https://dummy.com?param=%25AD" (with the character "%" encoded as "%25")
The method UriBuilderImpl.queryParam(...) behaves like this when the two characters following the "%" are hexadecimal. I.e, the method "com.sun.jersey.api.uri.UriComponent.isHexCharacter(char)" returns true for the characters following the "%".
I think the behavior of UriBuilderImpl is correct because I guess it tries to not encode parameters that already are encoded. But in my scenario, I will never try to create URLs with parameters that already encoded.
What should I do?
My Web application uses Jersey and in many places I build URIs using the class UriBuilder or invoke the method getBaseUriBuilder of UriInfo objects.
I can replace "%" with "%25", every time I invoke the methods queryParam, replaceQueryParam or segment. But I am looking for a less cumbersome solution.
How can I make Jersey to return my own implementation of UriBuilder?
I thought of creating a class that extends UriBuilderImpl that overrides these methods and that perform this replacing before invoking super.queryParam(...) or whatever.
Is there any way of making Jersey to return my own UriBuilder instead of UriBuilderImpl, when invoking UriBuilder.fromURL(...), UriInfo.getBaseUriBuilder(...), etc?
Looking at the method RuntimeDelegate, I thought of extending RuntimeDelegateImpl. My implementation would override the method createUriBuilder(...), which would return my own UriBuilder, instead of UriBuilderImpl.
Then, I would add the file META-INF/services/javax.ws.rs.ext.RuntimeDelegate and in it, a the full class name of my RuntimeDelegateImpl.
The problem is that the jersey-bundle.jar already contains a META-INF/services/javax.ws.rs.ext.RuntimeDelegate that points to com.sun.jersey.server.impl.provider.RuntimeDelegateImpl, so the container loads that file instead of my javax.ws.rs.ext.RuntimeDelegate. Therefore, it does not load my RuntimeDelegateimplementation.
Is it possible to provide my own implementation of RuntimeDelegate?
Should I take a different approach?
UriBuilder
This is possible with help of UriComponent from Jersey or URLEncoder directly from Java:
UriBuilder.fromUri("https://dummy.com")
.queryParam("param",
UriComponent.encode("%AD",
UriComponent.Type.QUERY_PARAM_SPACE_ENCODED))
.build();
Which result in:
https://dummy.com/?param=%25AD
Or:
UriBuilder.fromUri("https://dummy.com")
.queryParam("param", URLEncoder.encode("%AD", "UTF-8"))
.build()
Will result in:
https://dummy.com/?param=%25AD
For a more complex examples (i.e. encoding JSON in query param) this approach is also possible. Let's assume you have a JSON like {"Entity":{"foo":"foo","bar":"bar"}}. When encoded using UriComponent the result for query param would look like:
https://dummy.com/?param=%7B%22Entity%22:%7B%22foo%22:%22foo%22,%22bar%22:%22bar%22%7D%7D
JSON like this could be even injected via #QueryParam into resource field / method param (see JSON in Query Params or How to Inject Custom Java Types via JAX-RS Parameter Annotations).
Which Jersey version do you use? In the tags you mention Jersey 2 but in the RuntimeDelegate section you're using Jersey 1 stuff.
See if the following examples help. The thread linked below has an extensive discussion on the available functions and their differing outputs.
The following:
UriBuilder.fromUri("http://localhost:8080").queryParam("name", "{value}").build("%20");
UriBuilder.fromUri("http://localhost:8080").queryParam("name", "{value}").buildFromEncoded("%20");
UriBuilder.fromUri("http://localhost:8080").replaceQuery("name={value}).build("%20");
UriBuilder.fromUri("http://localhost:8080").replaceQuery("name={value}).buildFromEncoded("%20");
Will output:
http://localhost:8080?name=%2520
http://localhost:8080?name=%20
http://localhost:8080?name=%2520
http://localhost:8080?name=%20
via http://comments.gmane.org/gmane.comp.java.jsr311.user/71
Also, based on the Class UriBuilder documentation, the following example shows how to obtain what you're after.
URI templates are allowed in most components of a URI but their value
is restricted to a particular component. E.g.
UriBuilder.fromPath("{arg1}").build("foo#bar");
would result in encoding of the '#' such that the resulting URI is
"foo%23bar". To create a URI "foo#bar" use
UriBuilder.fromPath("{arg1}").fragment("{arg2}").build("foo", "bar")
instead. URI template names and delimiters are never encoded but their
values are encoded when a URI is built. Template parameter regular
expressions are ignored when building a URI, i.e. no validation is
performed.
It is possible to overwrite the default behavior in jersey manually at start up e.g. with a static helper that calls RuntimeDelegate.setInstance(yourRuntimeDelegateImpl).
So if you want to have an UriBuilder that encodes percents even if they look like they are part of an already encoded sequence, this would look like:
[...]
import javax.ws.rs.core.UriBuilder;
import javax.ws.rs.ext.RuntimeDelegate;
import com.sun.jersey.api.uri.UriBuilderImpl;
import com.sun.ws.rs.ext.RuntimeDelegateImpl;
// or for jersey2:
// import org.glassfish.jersey.uri.internal.JerseyUriBuilder;
// import org.glassfish.jersey.internal.RuntimeDelegateImpl;
public class SomeBaseClass {
[...]
// this is the lengthier custom implementation of UriBuilder
// replace this with your own according to your needs
public static class AlwaysPercentEncodingUriBuilder extends UriBuilderImpl {
#Override
public UriBuilder queryParam(String name, Object... values) {
Object[] encValues = new Object[values.length];
for (int i=0; i<values.length; i++) {
String value = values[i].toString(); // TODO: better null check here, like in base class
encValues[i] = percentEncode(value);
}
return super.queryParam(name, encValues);
}
private String percentEncode(String value) {
StringBuilder sb = null;
for (int i=0; i < value.length(); i++) {
char c = value.charAt(i);
// if this condition is is true, the base class will not encode the percent
if (c == '%'
&& i + 2 < value.length()
&& isHexCharacter(value.charAt(i + 1))
&& isHexCharacter(value.charAt(i + 2))) {
if (sb == null) {
sb = new StringBuilder(value.substring(0, i));
}
sb.append("%25");
} else {
if (sb != null) sb.append(c);
}
}
return (sb != null) ? sb.toString() : value;
}
// in jersey2 one can call public UriComponent.isHexCharacter
// but in jersey1 we need to provide this on our own
private static boolean isHexCharacter(char c) {
return ('0' <= c && c <= '9')
|| ('A' <=c && c <= 'F')
|| ('a' <=c && c <= 'f');
}
}
// here starts the code to hook up the implementation
public static class AlwaysPercentEncodingRuntimeDelegateImpl extends RuntimeDelegateImpl {
#Override
public UriBuilder createUriBuilder() {
return new AlwaysPercentEncodingUriBuilder();
}
}
static {
RuntimeDelegate myDelegate = new AlwaysPercentEncodingRuntimeDelegateImpl();
RuntimeDelegate.setInstance(myDelegate);
}
}
Caveat: Of course that way it is not very configurable, and if you do that in some library code that might be reused by others, this might cause some irritation.
For example I had the same problem as the OP when writing a rest client in a Confluence plugin, and ended up with the "manual encode every parameter" solution instead, as the plugins are loaded via OSGi and thus are simply not able to touch the RuntimeDelegateImpl (getting java.lang.ClassNotFoundException: com.sun.ws.rs.ext.RuntimeDelegateImpl at runtime instead).
(And just for the record, in jersey2 this looks very similar; especially the code to hook the custom RuntimeDelegateImpl is the same.)
We built a large insurance policy and claim management system using Grails and Groovy. Performance problems are slowing down the site because all 'READS' fetch from the database, which is not necessary since most data is static. We want to introduce a simple key/value cache in the Grails layer, but we don't want to litter the existing code with cache.get() and cache.set() code, we want to use aspects instead.
Here is a sample from our main controller....
InsuranceMainController {
def customer {
//handles all URI mappings for /customer/customerId
}
def policy {
//handles all URI mappings for /policy/policyId,
}
def claim {
//handles all URL mappings for /claim/claimId
}
As far as the cache goes, assume for the moment it's a simple Map named "cache" that's available as a globally-scoped object, and objects in the cache are keyed by request URI...
cache.put("/customer/99876", customerObject)
cache.put("/policy/99-33-ARYT", policyObject)
Going back to the controller, if we just litter the code with cache.get()/set(), which is what we want to avoid using Spring AOP, we'll end up with messy code. We want to achieve the following functionality with apsects, or with just a simpler and cleaner implementation...
InsuranceMainController {
def customer {
Object customer = cache.get(request.getRequestURI())
if ( customer != null)
//render response with customer object
}else
//get the customer from the database, then add to cache
CustomerPersistenceManager customerPM = ...
customer = customerPM.getCustomer(customerId)
cache.put(request.getRequestURI(), customer)
}
}
We need examples that show how we can achieve the above functionality using Spring AOP or something simpler in Grails while avoiding the littering of the code with cache.get()/set(). Suggestions to refactor the existing controller are welcome if it's required to get AOP working properly.
Thanks in advance
Rather than using AOP, you could adapt Mr Paul Woods' controller simplification pattern to move the cache handling out to a single method?
Something like this might work:
class InsuranceMainController {
def customer = {
Object customer = withCachedRef( 'customerId' ) { customerId ->
CustomerPersistenceManager customerPM = ...
customerPM.getCustomer(customerId)
}
}
def policy = {
//handles all URI mappings for /policy/policyId,
Object policy = withCachedRef( 'policyId' ) { policyId ->
PolicyPersistenceManager policyPM = ...
policyPM.getPolicy(policyId)
}
}
// ...
private def withCachedRef( String id, Closure c ) {
Object ret = cache.get( request.requestURI )
if( !ret ) {
ret = c.call( params[ id ] )
cache.put( request.requestURI, ret )
}
ret
}
}
However, I haven't tested it at all :-( Just a suggestion of an alternative to AOP