I am running a job which fetches data from an API. Sometimes the request fails with
error making http request: Get "http://XXX/0xFA103c21ea2DF71DFb92B0652F8B1D795e51cdEf": context deadline exceeded
Error Screenshot
Is there a way I could extend the deadline?
I tried adding maxTaskDuration and timeout, that doesn't seem to be working either.
This is the part of my job description fetching requests -
schemaVersion = 1
name = "score multiple"
contractAddress = "0x502B4BadDe549E5338676bF9caF9C415463A4c74"
maxTaskDuration = "5m0s"
observationSource = """
decode_log [type="ethabidecodelog"
abi="OracleRequest(bytes32 indexed specId, address requester, bytes32 requestId, uint256 payment, address callbackAddr, bytes4 callbackFunctionId, uint256 cancelExpiration, uint256 dataVersion, bytes data)"
data="$(jobRun.logData)"
topics="$(jobRun.logTopics)"]
decode_cbor [type="cborparse" data="$(decode_log.data)"]
fetch [type="http" method=GET url="$(decode_cbor.get)" timeout="300s"]
Solved by updating DEFAULT_HTTP_TIMELINE in .env
refrence:
https://docs.chain.link/docs/configuration-variables/#default_http_timeout
Related
After upgrading helidon to 2.5.4 and jersey to 2.35, I am getting the following exception in one of the API. With previous version of helidon(2.0.2) and jersey(2.29.1), it used to work fine. Can you please help to resolve this issue.
{"level":"ERROR","logger":"org.glassfish.jersey.server.ServerRuntime$Responder","thread":"jersey-thread-7","ts":1671639547627,"x-request-id":"","msg":"An I/O error has occurred while writing a response message entity to the container output stream.:\norg.glassfish.jersey.server.internal.process.MappableException: java.io.IOException: Bad news: the stream has been closed\n\tat org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:67)\n\tat org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)\n\tat org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116)\n\tat org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:635)\n\tat org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:373)\n\tat org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:363)\n\tat org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:258)\n\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)\n\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)\n\tat org.glassfish.jersey.internal.Errors.process(Errors.java:292)\n\tat org.glassfish.jersey.internal.Errors.process(Errors.java:274)\n\tat org.glassfish.jersey.internal.Errors.process(Errors.java:244)\n\tat org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)\n\tat org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234)\n\tat org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)\n\tat io.helidon.webserver.jersey.JerseySupport$JerseyHandler.lambda$doAccept$4(JerseySupport.java:335)\n\tat io.helidon.common.context.Contexts.runInContext(Contexts.java:117)\n\tat io.helidon.common.context.ContextAwareExecutorImpl.lambda$wrap$7(ContextAwareExecutorImpl.java:154)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread.java:834)\nCaused by: java.io.IOException: Bad news: the stream has been closed\n\tat io.helidon.webserver.jersey.ResponseWriter$DataChunkOutputStream.awaitRequest(ResponseWriter.java:327)\n\tat io.helidon.webserver.jersey.ResponseWriter$DataChunkOutputStream.write(ResponseWriter.java:186)\n\tat org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200)\n\tat org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276)\n\tat com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2171)\n\tat com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeBytes(UTF8JsonGenerator.java:1260)\n\tat com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeFieldName(UTF8JsonGenerator.java:284)\n\tat com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:726)\n\tat com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)\n\tat com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
I also see the below warning whenever the mappableexception is encountered.
{"level":"WARN","logger":"io.helidon.webserver.BareResponseImpl","thread":"jersey-thread-1","ts":1671689369384,"x-request-id":"","msg":"Entity was requested and not fully consumed before a response is sent. This is not supported. Connection will be closed. Please fix your route for POST /demo/api/v2/test/","podname":"deployment-demo-batch-0"}
Below is the signature of the api which errored,
#post
#path("test")
#produces(MediaType.APPLICATION_JSON + "; charset=UTF-8")
#consumes(MediaType.APPLICATION_JSON + "; charset=UTF-8")
#timed(name="app_kpi",
absolute = true,
reusable = true,
unit=MetricUnits.NANOSECONDS,displayName="test",
tags = {"api=test"})
#tag(name = "Version2")
#APIResponse(responseCode = "200", description = "Success", content = #content(schema = #Schema(type=SchemaType.ARRAY, implementation = OutputRecord.class)))
#APIResponse(responseCode = "403", description = "Forbidden")
#APIResponse(responseCode = "401", description = "Unauthorized")
#APIResponse(responseCode = "500", description = "An unexpected error occurred during the request.")
#authenticated
public Vector<OutputRecords> test(#requestbody(description = "An array of input records to be processed.", required = true,
content = #content(schema = #Schema(type = SchemaType.ARRAY, implementation = InputRecord.class))) Vector <InputRecords> inputRecords)
This has been reported as issue 5775 for Helidon.
gspread.exceptions.APIError: {'code': 429, 'message': "Quota exceeded for quota metric 'Read requests' and limit 'Read requests per minute per user' of service 'sheets.googleapis.com' for consumer 'project_number:81537768804'.", 'status': 'RESOURCE_EXHAUSTED', 'details': [{'#type': 'type.googleapis.com/google.rpc.ErrorInfo', 'reason': 'RATE_LIMIT_EXCEEDED', 'domain': 'googleapis.com', 'metadata': {'consumer': 'projects/81537768804', 'quota_metric': 'sheets.googleapis.com/read_requests', 'quota_limit': 'ReadRequestsPerMinutePerUser', 'service': 'sheets.googleapis.com'}}]}
I'm trying to implement the EPIC FHIR SMART Backend Services (Backend OAuth 2.0)
on go programming language.
I've created my dev account, uploaded the public key there, and selecting the backend system as the application audience.
I'm pretty sure my jwt token is correct. I've inspected it on jwt.io, the signature is correct. However, I always get this error:
{ "error": "invalid_client", "error_description": null }
I've tried other possible solutions as well such as:
ensuring the expiration date within the jet claim is below 5 minutes
placing the payload in the body with the correct content type, which is application/x-www-form-urlencoded
ensuring to use the sandbox client_id
using the correct jwt sign in method (RS384)
What should I do to resolve this issue?
Btw, I also saw several discussions on the google groups saying that it's worth to wait for one or two days after the dev account is created.
Below is my code. Appreciate the help!
var (
oauth2TokenUrl = "https://fhir.epic.com/interconnect-fhir-oauth/oauth2/token"
sandboxClientID = "..."
privateKey = "..."
)
// load private key
signKey, err := jwt.ParseRSAPrivateKeyFromPEM([]byte(privateKey))
So(err, ShouldBeNil)
// construct jwt claims
now := time.Now()
claims := jwt.MapClaims{
"iss": sandboxClientID,
"sub": sandboxClientID,
"aud": oauth2TokenUrl,
"jti": uuid.New().String(), // fill with reference id
"exp": now.Add(1 * time.Minute).Unix(), // cannot be more than 5 minutes!
}
log.Info(" => claims:", utility.ToJsonString(claims))
// generate signed token using private key with RS384 algorithm
alg := jwt.SigningMethodRS384
signedToken, err := jwt.NewWithClaims(alg, claims).SignedString(signKey)
So(err, ShouldBeNil)
log.Info(" => signed token", signedToken)
// prepare api call payload
payload := map[string]string{
"grant_type": "client_credentials",
"client_assertion_type": "urn:ietf:params:oauth:client-assertion-type:jwt-bearer",
"client_assertion": signedToken,
}
// dispatch the api call
req := resty.New().
R().
EnableTrace().
SetFormData(payload)
res, err := req.Post(oauth2TokenUrl)
So(err, ShouldBeNil)
log.Info(" => response status:", res.StatusCode())
log.Info(" => response header:", res.Header())
log.Info(" => response body:", string(res.Body()))
// parse response
resBody := make(map[string]interface{})
err = json.Unmarshal(res.Body(), &resBody)
So(err, ShouldBeNil)
Fantastic, I got it working now.
The solution is simply waiting! it was confusing because I can't find any explanation about this on the doc, and also the error message is not quite friendly.
in summary, after creating dev app and the public key is uploaded there, we have to wait for a few hours/days, and then the credentials will eventually be usable.
The waiting part is applied to both open epic and app orchard dev accounts.
It seems that Epic has some kind of synchronising mechanism which runs once a day. So waiting after account create is the only solution. Please also note that, in app settings after Endpoint URI change you also have to wait some time.
Error { "error": "invalid_client", "error_description": null } also shows up when redirect_uri param is set to something like localhost:3000.
I encountered this problem too. In my case, I was using "Patients" as the "Application Audience" selected for the Epic SMART on FHIR app. I was able to successfully obtain an authorization code on the test server, but when I attempted to exchange it for an access token I received "invalid_client" error message.
The mistake I made is that the redirect_uri in the HTTP POST must be an absolute URL and must match a redirect URI you have specified for your app. If the redirect URI is invalid, the resulting error message will say "invalid client" (which is misleading).
Here is a sample of the Python code I was using...
data = {
'grant_type': 'authorization_code',
'code': request.GET.get('code'),
'redirect_uri': 'http://127.0.0.1:8000/ehr_connection_complete/', # THIS MUST BE AN ABSOLUTE URL
'client_id': '11111111-2222-3333-4444-555555555555',
}
response = post(url, data)
It felt odd to me that an error with the redirect_uri parameter generates an error message about invalid_client, but it's true with Epic's test FHIR server.
I hope this information helps others.
I have a Spring boot app running on Tomcat. I have to resolve each ip to its Geolocation : city , province and Country . However,sometimes I receive ip address as a comma separated String instead of a single ip address. For example , 1.39.27.224, 8.37.225.221 .
The code to extract ip from a Http request that I am using :
public static String getIp(final HttpServletRequest request) {
PreConditions.checkNull(request, "request cannot be null");
String ip = request.getHeader("X-FORWARDED-FOR");
if (!StringUtils.hasText(ip)) {
ip = request.getRemoteAddr();
}
return ip;
}
The X-Forwarded-For can be used to identify the originating IP address of a client connecting to a web server through an HTTP proxy or load balancer.
The general format of this field is
X-Forwarded-For: client, proxy1, proxy2
In above example you can see that the request is passed through proxy1 and proxy2.
In your case you should parse this comma separated string and read the first value which is client's IP address.
Warning - It is easy to forge an X-Forwarded-For field so you might get wrong information.
Please take a look at https://en.wikipedia.org/wiki/X-Forwarded-For to read more about this.
Here is what I use in my servlet (running on Jetty behind HAProxy) -
I just try to get the first IP address in the X-Forwarded-For header:
Pattern FIRST_IP_ADDRESS = Pattern.compile("^(\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})");
public static String parseXff(HttpServletRequest httpReq) {
String xff = httpReq.getHeader("X-Forwarded-For");
if (xff != null) {
Matcher matcher = FIRST_IP_ADDRESS.matcher(xff);
if (matcher.find()) {
return matcher.group(1);
}
}
// return localhost when servlet is accessed directly, without HAProxy
return "127.0.0.1";
}
I eventually want to send a GET request to my Amazon EC2 server, but for now I just need to get the GET request to work in some fashion. To do that, I am trying to send a GET request to google as a test.
I am using a Arduino Uno with an Ethernet shield connected to the internet via DHCP.
My issue is that although client.connect() seems to work, client.available doesn't work either with google or with my EC2 server. Although, if my issue lies elsewhere, please tell me.
I am able to ping google and do a Telnet GET request simulation using:
-telnet www.google.com 80
-GET /search?q=arduino HTTP/1.0
Arduino Code:
#include <SPI.h>
#include <Ethernet.h>
// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network:
byte mac[] = { 0x90, 0xA2, 0xDA, 0x0F, 0xD2, 0xAF };
//byte mac[] = {0xf0, 0x1f, 0xaf, 0x33, 0x62, 0x2f };
IPAddress ip(192,168,1,11);
// initialize the library instance:
EthernetClient client;
//char server[] = "ec2-54-69-168-77.us-west-2.compute.amazonaws.com";
char server[] = "www.google.com";
double dummyValue = 7.5;
void setup()
{
Serial.begin(9600);
// attempt a DHCP connection:
Serial.println("Attempting to get an IP address using DHCP:");
if (!Ethernet.begin(mac)) {
// if DHCP fails, start with a hard-coded address:
Serial.println("failed to get an IP address using DHCP, trying manually");
Ethernet.begin(mac, ip);
}
Serial.print("My address:");
Serial.println(Ethernet.localIP());
}
void loop()
{
// connect to the server
for(int i = 0;i <100 ; i++) {
if (client.connect(server, 80)) {
// print to serial monitor
Serial.println("connected...");
Serial.println("ARDUINO: forming HTTP request message");
// send data the server through GET request
//client.print("GET /~sclaybon3/firstdatatest.php?reading=3 HTTP/1.0\r\n");
client.print("GET /search?q=arduino HTTP/1.0\r\n");
Serial.println("Get request");
//client.print(dummyValue);
//client.print(" HTTP/1.1");
//client.print("Host: ec2-54-69-168-77.us-west-2.compute.amazonaws.com\r\n");
client.print("Host: www.google.com\r\n");
//Serial.println("Host:www.google.com");
//client.print("Connection: close\r\n\r\n");
client.print("\r\n");
Serial.println("ARDUINO: HTTP message sent");
// give server some time to receive and store data
// before asking for response from it
delay(1000);
// get the response from the page and print it to serial port
// to ascertain that data was received properly
if(client.available())
{
Serial.println("ARDUINO: HTTP message received");
Serial.println("ARDUINO: printing received headers and script response...\n");
while(client.available())
{
char c = client.read();
Serial.print(c);
}
}
else
{
Serial.println("ARDUINO: no response received / no response received in time");
}
client.stop();
}
}
// do nothing forever after:
while(true);
}
Arduino output:
Attempting to get an IP address using DHCP:
My address:192.168.1.11
connected...
ARDUINO: forming HTTP request message
Get request
ARDUINO: HTTP message sent
ARDUINO: no response received / no response received in time