I'm trying to save cookies in a post request. Here is my code :
CookieContainer myCookieContainer = new CookieContainer();
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(url);
myHttpWebRequest.ContentType = "application/x-www-form-urlencoded";
myHttpWebRequest.UserAgent = userAgent;
myHttpWebRequest.CookieContainer = myCookieContainer;
myHttpWebRequest.Method = "POST";
byte[] postdata = encoding.GetBytes(submitString);
myHttpWebRequest.BeginGetRequestStream(async1 =>
{
using (Stream stream = myHttpWebRequest.EndGetRequestStream(async1))
stream.Write(postdata, 0, postdata.Length);
myHttpWebRequest.BeginGetResponse(async2 =>
{
HttpWebResponse rep = (HttpWebResponse)myHttpWebRequest.EndGetResponse(async2);
CookieCollection cookies = rep.Cookies;
using (Stream stream = rep.GetResponseStream())
using (StreamReader sr = new StreamReader(stream))
{
String content = sr.ReadToEnd();
if (pageDownloadedEventHandler != null)
pageDownloadedEventHandler(content);
}
}, null);
}, null);
Alaways the CookieContainer is empty.
How to get the cookies?
Your code seems to be perfect, if the server sends you back any cookies you should see them in rep.Cookies, as well as in myCookieContainer.
If you want to be sure use Fiddler or Wireshark to analyze the HTTP network traffic and look for the cookies, but if I'm right you won't find them. In this case my idea is to analyze the network traffic doing the same request with your browser, maybe the php/asp.net/other app decided not to set cookies due to some missing request headers.
Related
I have an API end-point that I need to post to.
Using Postman I do the following:
Set the method to be POST
Add the URL
In the Headers I set: Content-Type to be application/json
In the Body I add my json string
I hit [Send] and get a 200 response with the expected response.
However, in C# .Net Framework 4.8 (LinqPad 5) I do
var c = new HttpClient(); // Disposed of later
c.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
string data = #"{ ""a"": ""b"", ""c"": ""d"" }"; // copied from Postman.
HttpContent payload = new StringContent(data, Encoding.UTF8, "application/json");
var msg = new HttpRequestMessage(HttpMethod.Post, new Uri("https://the.url"), UriKind.Absolute)
{
Content = payload,
};
var response = c.SendAsync(msg).GetAwaiter().GetResult(); // It's a synchronous flow
And this responds with a 502 Bad Gateway.
What am I missing...?
I should point out that I need to use the HttpClient and not RestSharp.
I am trying to download attachments from Jira. I use /rest/api/2/attachment/{id} to get json response of the attachment. It has field "content" that is the attachment url. I use this url and construct HttpGet and execute to get response that always gives me html content asking for login. I am sending Basic Authorization in the httpGet header. Apparently this works for downloading .png files but not any other file types. I am using java spring rest to connect to Jira horizon.
Closeable httpclient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet(jira attachmenturl);
httpGet.setHeader("Authorization", "Basic <base64Credentials>);
CloseableHttpResponse response = httpclient.execute(httpGet)
The response for .txt, .jpeg, Microsoft documents is different from response that I get for .png files.
This works:
#GetMapping(value = "/getAttachment")
public String getAttachment(#RequestParam("id") String attachmentID) throws Exception {
Client client = Client.create();
WebResource webResource = client.resource(jiraBaseURLRest + "attachment/" +
attachmentID);
ClientResponse response = webResource.header("Authorization", "Basic " +
base64Creds).type("application/json")
.accept("application/json").get(ClientResponse.class);
String result = response.getEntity(String.class);
JSONObject jsonObj = new JSONObject(result);
System.out.println("JSON Object = "+jsonObj);
URL url = new URL(jsonObj.getString("content"));
Closeable httpclient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet(url.toURI());
httpGet.setHeader("Authorization", "Basic " + base64Creds);
CloseableHttpResponse response1 = ((CloseableHttpClient) httpclient).execute(httpGet);
if(response1.getStatusLine().getStatusCode() == 200)
{
HttpEntity entity = response1.getEntity();
if (entity.isStreaming())
{
System.out.println("Streaming...");
byte data[] = EntityUtils.toByteArray(entity);
FileOutputStream fout = new FileOutputStream(new File("D://pdf1.pdf"));
fout.write(data);
fout.close();
System.out.println("Done!!");
}
}
return "Success";
}
Javascript gives error: "DOMException: Failed to read the 'cookie' property from 'Document': Access is denied for this document."
Iv'e set all cookies to use use HttpOnly = false but it do not seem to help.
The page works perfectly if I call the page without puppeteer.
I currently copy all session variables from the request I am initializing puppeteer with:
var cookies = new List<CookieParam>();
foreach (var key in request.Cookies.Keys)
{
var cookie = request.Cookies.Get(key.ToString());
cookies.Add(new CookieParam {
Name = cookie.Name,
Value = cookie.Value,
HttpOnly = false,
Domain = cookie.Domain,
Url = baseUrl,
Path = cookie.Path,
Secure = cookie.Secure
});
}
await page.SetCookieAsync(cookies.ToArray());
I were using page.SetContent(...) to create the page content. This do not seem to allow usage of cookies client side.
I switched to use page.GoToAsync(...) which avoided the problem.
I am getting this error only on my local workstation and prod server.
In Dev and Cert it is working fine.
local workstation - 20 GB memory, Win 7 64 bit, IIS Express, VS
2013 dev, cert & prod - 8 GB memory , 2008 R2 64 Bit, IIS 7.5
I have a web api (.net 4.0) which takes the incoming request body and uploads it to a storage server. configured web api as per this website.
I have these in my web.config
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="2147483648" />
</requestFiltering>
</security>
</system.webServer>
<system.web>
<httpRuntime maxRequestLength="2097152" />
</system.web>
I also have an implementation of IHostBufferPolicySelector which returns false for PUT & POST requests. so the request to this web api for PUt & POST are not buffered.
for any files < ~350 MB it is working fine. but web api is throwing out of memory exceptions when file size >= ~ 400 MB and this is happening only on Local workstation and Prod server.
Web Api controller calls below code to stream the request to the destination server
public async Task<HttpResponseMessage> StoreObjectAsync(Uri namespaceUrl, string userName, string password, string objectName, Stream objectContent, string contentType = "application/octet-stream", IDictionary<string, string> systemMetadata = null)
{
Uri namespaceRootUrl = Utilities.GetNamespaceRootUrl(namespaceUrl);
using (var request = new HttpRequestMessage() { Method = HttpMethod.Put })
{
request.RequestUri = Utilities.CreateRequestUri(namespaceRootUrl, objectName);
request.Content = new StreamContent(objectContent);
request.Content.Headers.ContentType = new MediaTypeHeaderValue(contentType);
HttpResponseMessage response;
response = await this.httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
return response;
}
}
After doing some research online, i understand from this link & this link that HttpClient on .Net 4.0 buffers the request body and because of that behavior it seemed to me that it is throwing outofmemory exception
so I changed my code to below this time using HttpWebRequest using which I have the control to specify that request should be streamed but not buffered.
public async Task<HttpResponseMessage> StoreObjectAsync(Uri namespaceUrl, string userName, string password, string objectName, Stream content, long contentLength, string contentType = "application/octet-stream", IDictionary<string, string> systemMetadata = null)
{
Uri namespaceRootUrl = Utilities.GetHCPNamespaceRootUrl(namespaceUrl);
HttpWebRequest httpWebRequest = ((HttpWebRequest)WebRequest.Create(requestUri));
httpWebRequest.Method = "PUT";
httpWebRequest.KeepAlive = true;
httpWebRequest.AllowWriteStreamBuffering = false;
httpWebRequest.ContentType = contentType;
httpWebRequest.ContentLength = contentLength;
using (Stream requestStream = await httpWebRequest.GetRequestStreamAsync())
{
await content.CopyToAsync(requestStream);
}
var webResponse = await httpWebRequest.GetResponseAsync();
HttpWebResponse httpWebResponse = (HttpWebResponse)webResponse;
Stream httpWebResponseContent = httpWebResponse.GetResponseStream();
HttpResponseMessage response = new HttpResponseMessage()
{
StatusCode = httpWebResponse.StatusCode,
ReasonPhrase = httpWebResponse.StatusDescription,
Content = new StreamContent(httpWebResponseContent)
};
return response;
}
Now it is working fine on my local machine. I am able to upload files around 1GB without any errors or memory exceptions. Havent pushed this to Prod yet.
But I still dont understand why the same code using HttpClient on .net 4.0 worked on Dev and Cert servers but not on Prod and my local.
please help me in understanding
How to find out why it worked on Dev and Cert? What system/server
configurations will affect the memory allocations to this api?
I wrote the following code in Xamarin to connect the Web Server:
var request = WebRequest.Create( "http://srv21.n-software.de/authentication.json") as HttpWebRequest;
// request.Method = "GET";
request.Method = "POST";
request.Headers.Add("name", "demo");
request.Headers.Add("password", "demo");
request.ContentType = "application/x-www-form-urlencoded";
HttpWebResponse Httpresponse = (HttpWebResponse)request.GetResponse();
It connects to the web server and the web server gets the request for "authentication.json", but doesn't get the parameters of the header ("name" and "password").
What is wrong with my code?
Most likely your parameters need to be in the body of the POST request instead of in the headers. Alternatively you might try to use a GET request instead and provide the parameters through the URL, if your server supports it (i.e. http://srv21.n-software.de/authentication.json?name=demo&password=demo).
This worked for me
using System.Net.Http;
string URL = "http://www.here.com/api/postForm.php";
string DIRECT_POST_CONTENT_TYPE = "application/x-www-form-urlencoded";
HttpClient client = new HttpClient();
string postData = "username=usernameValueHere&password=passwordValueHere");
StringContent content = new StringContent(postData, Encoding.UTF8, DIRECT_POST_CONTENT_TYPE);
HttpResponseMessage response = await client.PostAsync(DIRECT_GATEWAY_URL, content);
string result = await response.Content.ReadAsStringAsync();