Wanted to send a file to an http end point url using Data fusion.
Making this http call as a pipeline alert at the completion of the pipeline.
This is not working. Getting 500 response from API. Can someone help me on how do I send the file?
If this can be achieved with http plugin as sink, please let me know the steps.
I doubt you can send files using HTTP. You can create a request using the HTTP plugin as I answered one of your previous question - Data Fusion - Issue with http post plugin.
In HTTP Callback you can find information:
This action can be used when you want to perform an HTTP request at the end of a pipeline run.
Using this plugin, you can send a request or text, not a file.
If you would like to send a file, you should use File like it's mentioned in this thread: Some questions about google Data fusion
Regarding Error 500
The HyperText Transfer Protocol (HTTP) 500 Internal Server Error server error response code indicates that the server encountered an unexpected condition that prevented it from fulfilling the request.
It means that there is an issue with your endpoint. It might be a firewall or HTTP server issue. Too many possibilities of what could be wrong with current details.
In addition, you can find some additional information in doc Ingest CSV data to BigQuery using Cloud Data Fusion - Batch ingestion
Related
I recorded uploading an attachment (pdf file) using blazemeter plugin, then open the jmx file using Jmeter. I modified uploads tab as in the following image and placed pdf file under bin directly. when i run the script i got 400 bad request , could you please advise what might be the issue?
As per 400 Bad Request status code description:
The HyperText Transfer Protocol (HTTP) 400 Bad Request response status code indicates that the server cannot or will not process the request due to something that is perceived to be a client error (for example, malformed request syntax, invalid request message framing, or deceptive request routing).
So I would recommend using a 3rd-party sniffer tool like Fiddler or Wireshark to capture the requests originating from JMeter and the real browser and compare them.
Requests must be exactly the same (apart from dynamic parameters which need to be correlated), once you amend your JMeter configuration so it will send the same request as the real browser does your file upload will be successful.
We cannot suggest anything meaningful unless we see successful and failing requests dumps fully (URL, headers and body)
I have a NiFi flow which needs to post the data to some other server, before posting it needs to fetch a value from my internal server or Nifi Distributed cache.
And add that Value in to Http header before posting the data, also update when gets particular error code 401.
Below is the flow I am trying, all green processor I could do but stuck at getting the value and adding in invoke http header . What are other process I can make us to achieve this .
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.9.0/org.apache.nifi.processors.standard.InvokeHTTP/
according to documentation on http code 401 InvikeHTTP routes file to No Retry relation.
also flow file should contain some response attributes including invokehttp.status.code
so, you could use RouteOnAttribute processor to redirect flow file to your external token processor only when invokehttp.status.code=401.
I have given two url in csv file and trying to load that in jmeter. I am getting 503 response code.
As per HTTP 503 status code description
The HyperText Transfer Protocol (HTTP) 503 Service Unavailable server error response code indicates that the server is not ready to handle the request.
Common causes are a server that is down for maintenance or that is overloaded. This response should be used for temporary conditions and the Retry-After HTTP header should, if possible, contain the estimated time for the recovery of the service.
Note: together with this response, a user-friendly page explaining the problem should be sent.
Caching-related headers that are sent along with this response should be taken care of, as a 503 status is often a temporary condition and responses shouldn't usually be cached.
So I can think of at least 3 possible issues:
Your server is really overloaded, you can try checking Response Data tab of the View Results Tree listener - if you're lucky - you will the the aforementioned "user-friendly page explaining the problem"
Your request is malformed and server is sending 503 response instead of 4xx which indicates client-side error. Double check what URL(s) you're trying to open by looking into Request tab of the View Results Tree listener and try to open it in the browser. If you're building the URL from parts (i.e. host, port, protocol, query string, etc.) - you might want to check if the values are correctly read from the CSV file using Debug Sampler
I fail to see HTTP Header Manager in your Test Plan, some servers check the client Headers, for example User-Agent or Accept and may report an error if the required header is missing (i.e. indicating that the browser is not supported)
I have read and tried the solutions shared in the previous posts for this error but nothing helped me to fix this. Kindly help.
I am making a HTTPS API call. A very simple call which accepts a 2 KB JSON message via POST method and sends a one word acknowledgement. It works perfectly fine in Postman tool. In JMeter, no. of threads (users) is kept as 25. It works perfectly fine one time with all 25 success response and at times getting few failure response as
Response code: Non HTTP response code: java.net.SocketException
Response message: Non HTTP response message: Unexpected end of file from server
No consistency at all. I have also tried with both enabling and disabling "use keepalive" checkbox. Both giving me all success one time ; few/all failure at another time with the above error. Please help. Thank you.
Below are the JMeter settings:
HTTP Header Manager : Content-Type - application/json
HTTPRequest sampler : protocol - HTTPS
Server Name or IP : project server name
Method : POST
PATH : The required path with https authentication details passed as parameters
IMPLEMENTATION : Set to JAVA (HTTPCLIENT4 was giving me “443 failed to respond” error)
Add header
Connection Keep-Alive
Both errors (Java implementation and Apache HTTPClient4 implementation), are essentially saying the same thing: The server closed the connection, without providing any response. I think there could be the following reasons:
Authentication problem. If server side checks authentication before processing a request (e.g. using Spring), it may be rejecting your request, it may be not bothering with any response if authentication is not considered correct.
Request Issue. Some less noticeable properties of the request you send via Jmeter are different from what you send in Postman. It could be some minor thing with formatting, or some headers server expects. Some of such inconsistencies can also cause Load Balancer (if you use one) to reject request before it's delivered to the server.
Certificate issues. Since you are using HTTPS, you need to make sure your certificate is setup correctly on JMeter side.
So I suggest:
Review server logs and see if your request makes it to the server. If yes, you might be able to see how it was rejected. If not, you need to trace back and see who rejected it (LB, authentication, etc)
Compare headers and body sent by JMeter vs Postman line to line (use TCPDump for example to obtain it).
I am using yahoo YUI Ajax call to post request.
Page is developed in JSP and server is Tomcat 6 and using Struts 2.x.
While i am sending small amount of data using Ajax call post request it is working fine.But when i am sending huge amount of data i am getting 400 bad request error.
The 400 error comes when The Web server thinks that the data stream sent by the client (e.g. your Web browser ) was 'malformed' i.e. did not respect the HTTP protocol completely. So the Web server was unable to understand the request and process it.
There could be possibilities that data is too large, so you should better encode the data using java script inbuilt function for example escape().
Please check maxPostSize attribute of Connector in conf/server.xml.
See the following doc for the details
In Tomcat, when the post data exceeds the maximum specified in maxPostSize (server.xml) it returns a 400 as error code