Android Volley Caching with different POST requests - android-volley

I am using Android Volley to cache requests this works fine when I was using GET but I switched to use POST for some reasons. Now I want to cache the same URL with different POST data.
Request 1 -> URL1, POST Data = "Cat=1"
Request 2 -> URL1, POST Data = "Cat=2"
Request 3 -> URL1, POST Data = "Cat=3"
is this can be done with Android Volley

as the Volley.Request.getCacheKey() returns the URL which in my case is the same; this did not work for me.
Instead I had to override getCacheKey() in my child class to return URL+POST(key=Value)
That way I was able to cache all the POST requests made to the same URL with different POST data.
when you try to retrieve the cached request you need to construct the cache key with the same way.
so here is a snapshot of my code:
public class CustomPostRequest extends Request<String> {
.
.
private Map<String, String> mParams;
.
.
public void SetPostParam(String strParam, String strValue)
{
mParams.put(strParam, strValue);
}
#Override
public Map<String,String> getParams() {
return mParams;
}
#Override
public String getCacheKey() {
String temp = super.getCacheKey();
for (Map.Entry<String, String> entry : mParams.entrySet())
temp += entry.getKey() + "=" + entry.getValue();// not do another request
return temp;
}
}
When ever you construct a new request you can use getCacheKey() to search for the cached request first before putting it in the requests queue.
I hope this helps.

Also if you don't want to use one of the existing Request classes you can follow this code (I'm using JsonArrayRequest here, you can use whatever you want)
Map<String, String> params = yourData;
JsonArrayRequest request = new JsonArrayRequest(Request.Method.POST, url,
new Response.Listener<JSONArray>() {
... Needed codes
},
new Response.ErrorListener() {
...
}
){
#Override
protected Map<String, String> getParams() throws AuthFailureError {
return params;
}
#Override
public String getCacheKey() {
return generateCacheKeyWithParam(super.getCacheKey(), params);
}
};
and based on Mahmoud Fayez's answer, here the generateCacheKeyWithParam() method:
public static String generateCacheKeyWithParam(String url, Map<String, String> params) {
StringBuilder urlBuilder = new StringBuilder(url);
for (Map.Entry<String, String> entry : params.entrySet()) {
urlBuilder.append(entry.getKey()).append("=").append(entry.getValue());
}
url = urlBuilder.toString();
return url;
}

Related

Configuring AWS Signing in Reactive Elasticsearch Configuration

In one of our service I tried to configure AWS signing in Spring data Reactive Elasticsearch configuration.
Spring provides the configuring the webclient through webclientClientConfigurer
ClientConfiguration clientConfiguration = ClientConfiguration.builder()
.connectedTo("localhost:9200")
.usingSsl()
.withWebClientConfigurer(
webClient -> {
return webClient.mutate().filter(new AwsSigningInterceptor()).build();
})
. // ... other options to configure if required
.build();
through which we can configure to sign the requests but however AWS signing it requires url, queryparams, headers and request body(in case of POST,POST) to generate the signed headers.
Using this I created a simple exchange filter function to sign the request but in this function I was not able to access the request body and use it.
Below is the Filter function i was trying to use
#Component
public class AwsSigningInterceptor implements ExchangeFilterFunction
{
private final AwsHeaderSigner awsHeaderSigner;
public AwsSigningInterceptor(AwsHeaderSigner awsHeaderSigner)
{
this.awsHeaderSigner = awsHeaderSigner;
}
#Override
public Mono<ClientResponse> filter(ClientRequest request, ExchangeFunction next)
{
Map<String, List<String>> signingHeaders = awsHeaderSigner.createSigningHeaders(request, new byte[]{}, "es", "us-west-2"); // should pass request body bytes in place of new byte[]{}
ClientRequest.Builder requestBuilder = ClientRequest.from(request);
signingHeaders.forEach((key, value) -> requestBuilder.header(key, value.toArray(new String[0])));
return next.exchange(requestBuilder.build());
}
}
I also tried to access the request body inside ExchangeFilterFunction using below approach but once i get the request body using below approach.
ClientRequest.from(newRequest.build())
.body(
(outputMessage, context) -> {
ClientHttpRequestDecorator loggingOutputMessage =
new ClientHttpRequestDecorator(outputMessage) {
#Override
public Mono<Void> writeWith(Publisher<? extends DataBuffer> body) {
log.info("Inside write with method");
body =
DataBufferUtils.join(body)
.map(
content -> {
// Log request body using
// 'content.toString(StandardCharsets.UTF_8)'
String requestBody =
content.toString(StandardCharsets.UTF_8);
Map<String, Object> signedHeaders =
awsSigner.getSignedHeaders(
request.url().getPath(),
request.method().name(),
multimap,
requestHeadersMap,
Optional.of(
requestBody.getBytes(StandardCharsets.UTF_8)));
log.info("Signed Headers generated:{}", signedHeaders);
signedHeaders.forEach(
(key, value) -> {
newRequest.header(key, value.toString());
});
return content;
});
log.info("Before returning the body");
return super.writeWith(body);
}
#Override
public Mono<Void>
setComplete() { // This is for requests with no body (e.g. GET).
Map<String, Object> signedHeaders =
awsSigner.getSignedHeaders(
request.url().getPath(),
request.method().name(),
multimap,
requestHeadersMap,
Optional.of("".getBytes(StandardCharsets.UTF_8)));
log.info("Signed Headers generated:{}", signedHeaders);
signedHeaders.forEach(
(key, value) -> {
newRequest.header(key, value.toString());
});
return super.setComplete();
}
};
return originalBodyInserter.insert(loggingOutputMessage, context);
})
.build();
But with above approach I was not able to change the request headers as adding headers throws UnsupportedOperationException inside writewith method.
Has anyone used the spring data reactive elastic search and configured to sign with AWS signed headers?
Any help would be highly appreciated.

How to read/modify form data that goes through Spring Cloud Gateway?

I am trying to validate and log form data that goes through Spring Cloud Gateway. I have tried a few methods and encounter a few problems and I could not read it properly. I have tried:
#Component
public class GatewayRequestFilter {
#Bean
public GlobalFilter apply() {
return (exchange, chain) -> {
MediaType contentType = exchange.getRequest().getHeaders().getContentType();
ModifyRequestBodyGatewayFilterFactory.Config modifyRequestConfig = new ModifyRequestBodyGatewayFilterFactory.Config();
/// Method 1
if (contentType.includes(MediaType.MULTIPART_FORM_DATA)) {
modifyRequestConfig.setRewriteFunction(String.class, String.class, (exchange1, originalRequestBody) -> {
validateAndAuditLog(exchange1, originalRequestBody);
return Mono.just(originalRequestBody);
});
}
/// Method 2
if (contentType.includes(MediaType.MULTIPART_FORM_DATA)) {
return exchange.getMultipartData().flatMap(originalRequestBody -> {
validateAndAuditLog(exchange1, originalRequestBody);
return chain.filter(exchange);
});
}
/// Method 3:
/// https://github.com/spring-cloud/spring-cloud-gateway/issues/1307#issuecomment-553910834
return new ModifyRequestBodyGatewayFilterFactory().apply(modifyRequestConfig).filter(exchange, chain);
};
}
}
For the 1st and 3rd method, if I set inClass as String.class then I can see data in some kind of http format. The problem is that I don't know how to parse it into hashMap or LinkedMultiValueMap to access each of value using key. Here is the output I get:
----------------------------162653831591335516327921
Content-Disposition: form-data; name="simple-text"
text
----------------------------162653831591335516327921
Content-Disposition: form-data; name="simple-file"; filename="simple-file"
Content-Type: application/octet-stream
Simple file
----------------------------162653831591335516327921--
If I change inClass as Object.class then there is another error:
{
"timestamp": "2020-04-03T02:37:57.096+0000",
"path": "/tc/test/test",
"status": 500,
"error": "Internal Server Error",
"message": "Content type 'multipart/form-data;boundary=--------------------------537619313111072161580699' not supported for bodyType=java.lang.Object",
"requestId": "0592497a-1"
}
For the 2nd method I can get data in LinkedMultiValueMap which is good because I can read each data using key value and I can also get uploaded files name, but the problem is that, it hang for 10s before pass the request to down stream.
Anyone has any idea what should I do to read or modify form data that goes through Spring Cloud Gateway?
Rewriting the answer with example.
Basic approach is defined here, though it needs lot of refinement to work for multi-part.
https://developpaper.com/question/how-to-modify-the-request-parameters-of-multipart-form-data-format-in-spring-cloud-gateway/
For any approach to work once you read the data, you need to set a modified request object to exchange downstream to be processed again. Setting the new multi-part object downstream is bit tricky because there is not a straightforward way to convert string->multi-part->string.
Here is a sample code based on the approach. Note that this for now works only if multi-part contains form fields and not file type fields, because in later case we are dealing with a stream, which can be embedded anywhere within the entire multi-part request, and it is not possible to modify such request without blocking calls, which the netty does not allow.
private final List<HttpMessageReader<?>> messageReaders = HandlerStrategies.withDefaults().messageReaders();
public GatewayFilter apply(Config config) {
return new OrderedGatewayFilter((exchange, chain) -> {
ServerRequest serverRequest = ServerRequest.create(exchange, messageReaders);
// get modified body from original body o
Mono<MultiValueMap<String, String>> modifiedBody = serverRequest.bodyToMono(String.class).flatMap(o -> {
// create mock request to read body
SynchronossPartHttpMessageReader synchronossReader = new SynchronossPartHttpMessageReader();
MultipartHttpMessageReader reader = new MultipartHttpMessageReader(synchronossReader);
MockServerHttpRequest request = MockServerHttpRequest.post("").contentType(exchange.getRequest().getHeaders().getContentType()).body(o);
Mono<MultiValueMap<String, Part>> monoRequestParts = reader.readMono(MULTIPART_DATA_TYPE, request, Collections.emptyMap());
// modify parts
return monoRequestParts.flatMap(requestParts -> {
Map<String, List<String>> modifedBodyArray = requestParts.entrySet().stream().map(entry -> {
String key = entry.getKey();
LOGGER.info(key);
List<String> entries = entry.getValue().stream().map(part -> {
LOGGER.info("{}", part);
// read the input part
String input = ((FormFieldPart) part).value();
// return the modified input part
return new String(modifyRequest(config, exchange, key, input));
}).collect(Collectors.toList());
return new Map.Entry<String, List<String>>() {
#Override
public String getKey() {
return key;
}
#Override
public List<String> getValue() {
return entries;
}
#Override
public List<String> setValue(List<String> param1) {
return param1;
}
};
}).collect(Collectors.toMap(k -> k.getKey(), k -> k.getValue()));
return Mono.just(new LinkedMultiValueMap<String, String>(modifedBodyArray));
});
});
// insert the new modified body
BodyInserter bodyInserter = BodyInserters.fromPublisher(modifiedBody, new ParameterizedTypeReference<MultiValueMap<String, String>>() {});
HttpHeaders headers = new HttpHeaders();
headers.putAll(exchange.getRequest().getHeaders());
// the new content type will be computed by bodyInserter
// and then set in the request decorator
headers.remove(HttpHeaders.CONTENT_LENGTH);
CachedBodyOutputMessage outputMessage = new CachedBodyOutputMessage(exchange, headers);
return bodyInserter.insert(outputMessage, new BodyInserterContext())
.then(Mono.defer(() -> {
ServerHttpRequest decorator = decorate(exchange, headers, outputMessage);
return chain.filter(exchange.mutate().request(decorator).build());
}));
}, RouteToRequestUrlFilter.ROUTE_TO_URL_FILTER_ORDER + 1);
}
// some of the helper methods
private String modifyRequest(Config config, ServerWebExchange exchange, String key, String input) {
// do your thing in here !!!
return input;
}
private ServerHttpRequestDecorator decorate(ServerWebExchange exchange, HttpHeaders headers, CachedBodyOutputMessage outputMessage) {
return new ServerHttpRequestDecorator(exchange.getRequest()) {
#Override
public HttpHeaders getHeaders() {
long contentLength = headers.getContentLength();
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.putAll(headers);
if (contentLength > 0) {
httpHeaders.setContentLength(contentLength);
} else {
// TODO: this causes a 'HTTP/1.1 411 Length Required' // on httpbin.org
httpHeaders.set(HttpHeaders.TRANSFER_ENCODING, "chunked");
}
return httpHeaders;
}
#Override
public Flux<DataBuffer> getBody() {
return outputMessage.getBody();
}
};
}

problem with volley Post request with header token

i have laravel site and i want create android app for this site
and im beginner in java .
In summary...
for authentication api route i use one middleware .
this middleware have one token .
in the postman program all thing is good...
but when i want send one post request with header token in volley android
my function cant give username or some variable ...
please help me to resolve this problem
this is my function send volly , dont forget all thing is good in postman request
int selectedId = statusregister.getCheckedRadioButtonId();
radioselect=(RadioButton)findViewById(selectedId);
send=0;
String url="my http api site...";
requestQueue= Volley.newRequestQueue(RegisterActivity.this);
StringRequest stringRequest=new StringRequest(Request.Method.POST, url, new Response.Listener<String>() {
#Override
public void onResponse(String response) {
Log.i("response",response);
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
Log.e("HttpClient", "error: " + error.toString());
}
}){
#Override
protected Map<String, String> getParams() {
Map<String, String> params = new HashMap<String, String>(); params.put("username",username.getText().toString().trim());
params.put("email",mailuser.getText().toString().trim());
params.put("password",password.getText().toString().trim());
params.put("status",radioselect.getText().toString().trim());
return params;
}
#Override
public Map<String, String> getHeaders() throws AuthFailureError {
Map<String, String> params2 = new HashMap<String, String>();
params2.put("Content-Type", "application/json; charset=UTF-8");
params2.put("APP_KEY", "7ACFdsd328BEA81sssdfgg556B91");
return params2;
}
};
requestQueue.add(stringRequest);
}
401 erorr ...
when i remove middleware its ok ...
this is screenshot begore set APP_KEY in header
enter image description here
and this screenshot after set APP_KEY in header
enter image description here
401 is an authorization issue
The default API token name in laravel is access_token not APP_KEY
params2.put("access_token", "7ACFdsd328BEA81sssdfgg556B91");
The API_KEY is an environment variable used for encryption in Laravel and you're sending it as a header in a post request

HttpURLConnnection request failures on Android 6.0 (MarshMallow)

I am using Google's Volley library for my application project , that targets minimum api level 14.So, Volley library uses HttpURLConnection as the NetworkClient.
Therefore , there should not be any issue related to Removal of Apache HTTPClient. However, I have done the configuration required for 6.0 Sdk i.e compileSdkVersion 23, build-tool-version 23.0.1 and build:gradle:1.3.1' and even tried adding useLibrary 'org.apache.http.legacy'. Have updated the same for Volley library project in my application.
Recently ,I tried to run my app on Android 6.0 (MarshMallow), my project compiles and runs. But those requests that require authentication headers are failing on MarshMallow with:
BasicNetwork.performRequest: Unexpected response code 401 com.android.volley.AuthFailureError
However the same is running on all Api level below 23.
I have checked the headers many times.Strangely, those requests that do not require authentication are giving response with 200 OK.
Right now I am totally clueless what is breaking the requests, does anybody have any idea what has changed in new Version that HttpURLConnection request fails for only Api level 23? Is anybody else using Volley and facing similar issue?
Here is my CustomRequest Class
public class CustomRequest extends Request<Void> {
int id, cmd;
Map<String, String> params;
BaseModel model;
public CustomRequest(int method, int cmd, String url, Map<String, String> params, int id, BaseModel model) {
super(method, url, null);
this.id = id;
this.cmd = cmd;
this.params = params;
this.model = model;
if (method == Method.GET) {
setUrl(buildUrlForGetRequest(url));
}
Log.v("Volley", "Making request to: " + getUrl());
}
private String buildUrlForGetRequest(String url) {
if (params == null || params.size() == 0) return url;
StringBuilder newUrl = new StringBuilder(url);
Set<Entry<String, String>> paramPairs = params.entrySet();
Iterator<Entry<String, String>> iter = paramPairs.iterator();
newUrl.append("?");
while (iter.hasNext()) {
Entry<String, String> param = iter.next();
newUrl
.append(param.getKey())
.append("=")
.append(param.getValue());
if (iter.hasNext()) newUrl.append("&");
}
return newUrl.toString();
}
#Override
public Map<String, String> getHeaders() throws AuthFailureError {
Map<String, String> headers = new HashMap<String, String>();
headers.put("X-Api-Version", Contract.API_VERSION);
headers.put("X-Client", "android");
String accessToken = APP.getInstance().getToken();
if (!accessToken.equals("")) {
headers.put("Authorization", "Bearer " + accessToken);
}
return headers;
}
#Override
protected Response<Void> parseNetworkResponse(NetworkResponse response) {
Exception ex;
try {
String jsonString = new String(
response.data, HttpHeaderParser.parseCharset(response.headers));
JsonNode json = new ObjectMapper().readTree(jsonString);
if (model != null) model.parse(id, json);
EventBus.getDefault().post(new EventResponse(cmd, true, null));
return Response.success(null, HttpHeaderParser.parseCacheHeaders(response)); //Doesn't return anything. BaseModel.parse() does all the storage work.
} catch (NoMoreDataException e) {
ex = e;
EventBus.getDefault().post(new NoMoreDataModel(cmd, e));
EventBus.getDefault().post(new EventResponse(cmd, false, null));
return Response.success(null, HttpHeaderParser.parseCacheHeaders(response));
} catch (Exception e) {
ex = e;
Log.e("CustomRequest", Log.getStackTraceString(e));
String message = APP.getInstance().getString(R.string.failedRequest);
if (!TextUtils.isEmpty(e.getMessage()))
message = e.getMessage();
EventBus.getDefault().post(new ErrorEventModel(cmd, message, e));
return Response.error(new ParseError(ex));
}
}
#Override
protected Map<String, String> getParams() throws AuthFailureError {
return params;
}
#Override
protected void deliverResponse(Void response) {
Log.v("Volley", "Delivering result: " + getUrl());
}
#Override
public void deliverError(VolleyError error) {
Log.e("CustomRequest", "Delivering error: Request=" + getUrl() + " | Error=" + error.toString());
String message = APP.getInstance().getString(R.string.failedRequest);
EventBus.getDefault().post(new ErrorEventModel(cmd, message, error));
}
}
Only difference I found between Api 23 and others is the HostNameVerifier.
For Api level 23 : com.android.okhttp.internal.tls.OkHostnameVerifier
For Api level <23 : javax.net.ssl.DefaultHostnameVerifier.
After checking the Server side of my application, found the reason behind the issue.
For Android 6.0(MarshMallow) the headers were becoming empty and this was not the case with other versions.
So the fix that worked for me:
Created and Added a new custom header X-Proxy-No-Redirect => 1 and passed along with other headers.
Theory behind it:
There is a API server to which we send request, then the API server redirects the request to corresponding site based on the oAuth token
While redirecting
For the redirection to happen, there are two ways to do that
1 - We just send a response back to the caller stating to redirect to a certain page. Then the caller(Networking library) takes care of redirection
2 - API server will itself makes the redirect request and get the response and then pass to caller
Earlier we were using the Method 1.
On Android 6.0 - The networking lib(Volley) doesn't seem to set all the headers while making the redirection request.
Once this new header is set, Method 2 will come into effective.
P.S This fix was applicable for my application project , maybe not for others.Just providing what was wrong and what helped me.

Volley's NetworkImageView with images on FTP

The scenario is: "Have adapters that work for HTTP, been asked to add FTP support" - sounds reasonable.
Is there a way to do it in an equally reasonable timeframe?
Or a similar mechanism/customized Volley library to swap Volley for?
The FTP is password-protected, if this makes any difference.
Tried so far:
HttpClientStack: Scheme 'ftp' not registered
HurlStack" libcore.net.url.FtpURLConnection cannot be cast to java.net.HttpURLConnection
attempted tunneling through a custom URLonnection (multiple difficulties, like libcore package not being public)
custom ImageCache that takes FTP handling off Volley's hands (what's the point, really?)
Only thing I've come up with thus far: override performRequest to do something completely different for ftp links.
Mind, the code here is bad, it just barely serves my purpose:
public class HurlStackFtp extends HurlStack {
#Override
public HttpResponse performRequest(Request<?> request, Map<String, String> additionalHeaders)
throws IOException, AuthFailureError {
String urlString = request.getUrl();
if (urlString != null && urlString.startsWith("ftp://")) {
return performFTPRequest(request, additionalHeaders);
} else {
return super.performRequest(request, additionalHeaders);
}
}
public HttpResponse performFTPRequest(Request<?> request, Map<String, String> additionalHeaders)
throws IOException, AuthFailureError {
String url = request.getUrl();
HashMap<String, String> map = new HashMap<String, String>();
map.putAll(request.getHeaders());
map.putAll(additionalHeaders);
// UrlRewriter not supported
InputStream input = getStreamFromFTP(url);
StatusLine responseStatus = new BasicStatusLine(new ProtocolVersion("HTTP", 1, 1),
/*connection.getResponseCode()*/200, /*connection.getResponseMessage()*/"OK");
BasicHttpResponse response = new BasicHttpResponse(responseStatus);
response.setEntity(makeEntity(input));
return response;
}
public static InputStream getStreamFromFTP(String url) throws IOException {
URL parsedUrl = new URL(url);
URLConnection cn = parsedUrl.openConnection();
cn.connect();
return cn.getInputStream();
}
private static HttpEntity makeEntity(InputStream inputStream) throws IOException {
BasicHttpEntity entity = new BasicHttpEntity();
entity.setContent(inputStream);
entity.setContentLength(inputStream.available());
entity.setContentType("binary/octet-stream"); // sigh...
return entity;
}
}

Resources