My flutter app connects to a socket via https and I am using aqueduct to get secure data. the socket data is full length string such as;
2_7#a_b_c_d_e_f_g#h_i_j_k_l_m_n#
I convert the data to json and my json data looks like:
"{data:2_7#a_b_c_d_e_f_g#h_i_j_k_l_m_n#}"
and sent to flutter app. the 2_7# means I have 2 rows and 7 column.
The original server socket data 152_7# which means I have 152 rows with 7 column.
when I try to get this data (152_7#) using socket in aqueduct I get only 12 rows or sometimes 25 rows.
If the server data short I get all of them, but can get big string data.
my question is how to get full data using aqueduct and socket?
import 'package:aqueduct/aqueduct.dart';
import 'package:ntmsapi/ntmsapi.dart';
Socket socket;
String _reply;
var _secureResponse;
var _errorData;
class NtmsApiController extends Controller {
#override
Future<RequestOrResponse> handle(Request request) async {
try {
String _xRequestValue = "";
_reply = "";
_errorData = "Server_Error";
if (request.path.remainingPath != null) {
_xRequestValue = request.path.remainingPath;
// returns a string like 152 row with 11 column
socket = await Socket.connect('192.168.xxx.xxx’, 8080);
socket.write('$_xRequestValue\r\n');
socket.handleError((data) {
_secureResponse = "$_errorData";
});
await for (var data in socket) {
// _cant get all the data from reply, reply shows about 25 row data
_reply = new String.fromCharCodes(data).trim();
_secureResponse = "$_reply";
socket.close();
socket.destroy();
}
} else {
_secureResponse = "$_errorData";
}
} catch (e) {
_secureResponse = "$_errorData";
}
return new Response.ok("$_secureResponse")
..contentType = ContentType.json;
}
}
Related
I am hoping there is someone here who is familiar with a Real Estate data standard known as RETS. The National Association of Realtors provides a dll for interfacing with their services called libRETS, but it is not being supported like it once was and recent events have prompted us to create our own as a replacement. For logistics reasons, we can't do this in Core and are using the current C#.Net 4.7.2.
There are 2 or 3 different "security levels" for connecting to a RETS Server, with the method being a per case basis from one MLS to the next. We can successfully connect to those who only require a login and password, but are hitting a wall on those who also require what is called a UserAgent and UserAgentPassword, which must passed somehow using Md5 encryption. The server is returning:
The remote server returned an error: (401) Unauthorized.
private WebResponse GetLoginBasicResponse()//*** THIS ONE WORKS ***
{
try
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
var request = (HttpWebRequest)WebRequest.Create(new Uri(_cred.loginUri));
request.Method = "GET";
request.Headers.Add("RETS-Version", _retsVersion);
request.Credentials = new NetworkCredential(_login, _password);
return request.GetResponse();
}
catch (Exception ex)
{
string ignore = ex.Message;
return null;
}
}
private WebResponse GetLoginWithUserAgentResponse()//*** THIS ONE DOES NOT WORK ***
{
try
{
// ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
var request = (HttpWebRequest)WebRequest.Create(new Uri(_cred.loginUri));
request.Method = "GET";
request.Headers.Add("RETS-Version", _retsVersion);
if (!string.IsNullOrEmpty(_cred.userAgent))
{
request.UserAgent = Md5(_cred.userAgent + ":" + _cred.userAgentPassword);
//request.Headers.Add("RETS-UA-Authorization", "Digest " + Md5(_cred.userAgent + ":" + _cred.userAgentPassword));
}
request.Credentials = new NetworkCredential(_login, _password);
return request.GetResponse();
}
catch (Exception ex)
{
string ignore = ex.Message;
return null;
}
}
public string Md5(string input) //*** Borrowed this from from .NET Core Project and presume it works
{
// Use input string to calculate MD5 hash
using (System.Security.Cryptography.MD5 md5 = System.Security.Cryptography.MD5.Create())
{
byte[] inputBytes = Encoding.ASCII.GetBytes(input);
byte[] hashBytes = md5.ComputeHash(inputBytes);
// Convert the byte array to hexadecimal string
StringBuilder sb = new StringBuilder();
for (int i = 0; i < hashBytes.Length; i++)
{
sb.Append(hashBytes[i].ToString("X2"));
}
return sb.ToString();
}
}
Page 20 of this document describes how to build the UA header: https://www.ranww.org/documents/resources/rets_1_8.pdf
There’s a few other fields you need to include.
We were not able to solve the issue in .NET but found a .NET Core project in GitHub that we are using instead. https://github.com/CrestApps/RetsConnector
This case can be closed
Not seeing an option to "Mark as Answer". Have tried both MS Edge and Google Chrome
I've created a method to add members in a Batch Request to a google group using .NET core and google's .NET client library. The code looks like this:
private void InitializeGSuiteDirectoryService()
{
_directoryServiceCredential = GoogleCredential
.FromJson(GlobalSettings.Instance.GSuiteSettings.Credentials)
.CreateScoped(_scopes)
.CreateWithUser(GlobalSettings.Instance.GSuiteSettings.User);
_directoryService = new DirectoryService(new BaseClientService.Initializer()
{
HttpClientInitializer = _directoryServiceCredential,
ApplicationName = _applicationName
});
}
public OperationResult<int> AddGroupMembers(Group group, IEnumerable<Member> members)
{
var result = new OperationResult<int>();
var memberList = members.ToList();
var batchRequestCount = 0;
if (memberList.Any())
{
var request = new BatchRequest(_directoryService);
foreach (var member in memberList)
{
batchRequestCount++;
request.Queue<Members>(_directoryService.Members.Insert(member, group.Id), (content, error, i, message) =>
{
if (message.IsSuccessStatusCode)
{
//log OK
}
else
{
// Implement Exponential backoff only on the request that failed.
}
});
if (batchRequestCount == 30|| member.Equals(memberList.Last()))
{
request.ExecuteAsync().Wait();
request = new BatchRequest(_directoryService); //Clear queue
}
}
}
return result;
}
The logic works fine if the amount of members is small; however, when the members count is let's say 100( this is the max amount of users in my google's test instance), I get an Error from Google that reads: "quotaExceeded". According to Google's documentation, the limit for a batch request on their Admin SDK is 1000 and I've set my logic to Execute when we reach a limit of 30.
The QUESTION is: How do I implement error handling to retry whenever I get this error? Their documentation suggests implementing 'Exponential Backoff' with a response that contains a 'retry-able error'(I don't see this when I inspect my response).
So here's what I ended up doing to implement Exponential Backoff on my call to add members to a Gsuite group. Since I'm using dotnet core, I was able to use 'Polly', which is a resilience and transient-fault-handling library that offers this functionality out of the box. There may be some need for refactoring, but here's what the code looks like for now:
public OperationResult<int> AddGroupMembers(Group group, IEnumerable<Member> members)
{
var result = new OperationResult<int>();
var memberList = members.ToList();
var batchRequestCount = 0;
if (memberList.Any())
{
var request = new BatchRequest(_directoryService);
foreach (var member in memberList)
{
retryRequest = false; // This variable needs to be declared at the class level to guarantee the value is available to the original thread running the process.
batchRequestCount++;
request.Queue<Members>(_directoryService.Members.Insert(member, group.Id), (content, error, i, message) =>
{
// If error code is 'quotaExceeded' retry the request ( You can add as many error codes as you'd like to retry here)
if (error.Code == 403)
{
retryRequest = true;
}
});
// Execute batch request to add members in batches of 30 member max
if (batchRequestCount == 30|| member.Equals(memberList.Last()))
{
// Below is what the code to retry using polly looks like
var response = Policy
.HandleResult<HttpResponseMessage>(message => message.StatusCode == HttpStatusCode.Conflict)
.WaitAndRetry(new[]
{
TimeSpan.FromSeconds(1),
TimeSpan.FromSeconds(2),
TimeSpan.FromSeconds(4)
}, (results, timeSpan, retryCount, context) =>
{
// Log Warn saying a retry was required.
})
.Execute(() =>
{
var httpResponseMsg = new HttpResponseMessage();
// Execute batch request Synchronously
request.ExecuteAsync().Wait();
if (retryRequest)
{
httpResponseMsg.StatusCode = HttpStatusCode.Conflict;
retryRequest = false;
}
else
{
httpResponseMsg.StatusCode = HttpStatusCode.OK;
}
return httpResponseMsg;
});
if (response.IsSuccessStatusCode)
{
// Log info
}
else
{
// Log warn
}
requestCount = 0;
request = new BatchRequest(_directoryService);
batchCompletedCount++;
}
}
}
return result;
}
I am inserting 3000 plus data from my server to my SQLite Database. The problem is the inserting process is very slow. Is there a better way to insert the data efficiently and effectively? What I am doing is I converted the data I got from my server to JSON Object and insert it one-by-one. I know what I am doing is inefficient. How can I fix this?
public class AndroidSQLiteDb : ISQLiteDB
{
public SQLiteAsyncConnection GetConnection()
{
var dbFileName = "backend.db3";
var documentsPath = System.Environment.GetFolderPath(System.Environment.SpecialFolder.Personal);
var path = Path.Combine(documentsPath, dbFileName);
return new SQLiteAsyncConnection(path);
}
}
public async void FirstSyncContacts(string host, string database, string contact)
{
try
{
var db = DependencyService.Get<ISQLiteDB>();
var conn = db.GetConnection();
var sql = "SELECT * FROM tblContacts WHERE Coordinator = '" + contact + "'";
var getContacts = conn.QueryAsync<ContactsTable>(sql);
var resultCount = getContacts.Result.Count;
var current_datetime = DateTime.Now.ToString("yyyy-MM-dd hh:mm:00");
//Check if the retailer has been sync
if (resultCount < 1)
{
try
{
syncStatus.Text = "Syncing Retailer";
var link = Constants.requestUrl + "Host=" + host + "&Database=" + database + "&Contact=" + contact + "&Request=9DpndD";
string contentType = "application/json";
JObject json = new JObject
{
{ "ContactID", contact }
};
HttpClient client = new HttpClient();
var response = await client.PostAsync(link, new StringContent(json.ToString(), Encoding.UTF8, contentType));
if (response.IsSuccessStatusCode)
{
var content = await response.Content.ReadAsStringAsync();
if (content != "")
{
var contactsresult = JsonConvert.DeserializeObject<List<ContactsData>>(content);
foreach (var item in contactsresult)
{
// update only the properties that you have to...
item.LastSync = Convert.ToDateTime(current_datetime);
item.ServerUpdate = Convert.ToDateTime(item.ServerUpdate);
item.MobileUpdate = Convert.ToDateTime(item.MobileUpdate);
}
await conn.InsertAsync(contactsresult);
}
}
//Proceed to next function
FirstSyncRetailerGroup(host, database, contact);
}
catch (Exception ex)
{
Console.Write("Syncing Retailer Error " + ex.Message);
}
}
//If not get the retailer
else
{
SyncContacts(host, database, contact);
}
}
catch (Exception ex)
{
Console.Write("Syncing Retailer Error " + ex.Message);
}
}
Use the non-async Insert in one background thread, instead of 3000 separate async calls...
Re-use the List from your DeserializeObject step instead of creating new local objects that will just be thrown away on each loop iteration.
No need to assign all those json properties (item.XXX) to another local variable, just update the properties of each existing ContactsData as needed before inserting it into the DB.
Example using SQLiteConnection:
// Use the non-async version of SQLiteConnection
var conn = new SQLiteConnection(dbPath, true, null);
// code removed for example...
await System.Threading.Tasks.Task.Run(() =>
{
var contactsresult = JsonConvert.DeserializeObject<List<ContactsData>>(content);
// start a transaction block so all 3000 records are committed at once.
conn.BeginTransaction();
// Use `foreach` in order shortcut the need to retrieve the object from the list via its index
foreach (var item in contactsresult)
{
// update only the properties that you have to...
item.LastSync = Convert.ToDateTime(current_datetime);
item.ServerUpdate = Convert.ToDateTime(item.ServerUpdate);
item.MobileUpdate = Convert.ToDateTime(item.MobileUpdate);
conn.Insert(item);
}
conn.Commit();
});
Example using SQLiteAsyncConnection:
var db = DependencyService.Get<ISQLiteDB>();
var conn = db.GetConnection();
~~~
var contactsresult = JsonConvert.DeserializeObject<List<ContactsData>>(content);
foreach (var item in contactsresult)
{
// update only the properties that you have to...
item.LastSync = Convert.ToDateTime(current_datetime);
item.ServerUpdate = Convert.ToDateTime(item.ServerUpdate);
item.MobileUpdate = Convert.ToDateTime(item.MobileUpdate);
}
conn.InsertAsync(contactsresult); // Insert the entire list at once...
I had the same problem so even the answer is some years late, maybe can be usefull for somebody.
This is how I did.
First: I get the data all from server as json
var response = await client.GetAsync("your_server_url");
var content = await response.Content.ReadAsStringAsync();
ResponseData = JsonConvert.DeserializeObject<DataModel>(content);
Second: Save data to database
await conn.InsertAllAsync(ResponseData)
But in my case, because our app works offline data, I first insert all data in a temp table, then I get all new records comparing main table with temp table.
NewDataFromTemp = await conn.QueryAsync<DataModel>("SELECT * FROM [TableTemp] t WHERE t.[TABLE_ID] NOT IN (SELECT g.[TABLE_ID] FROM [MainTable] g)");
And insert new records in Main table
await conn.InsertAllAsync(NewDataFromTemp)
Then I check for updated records
UpdatedDataFromTemp = await conn.QueryAsync<DataModel>("SELECT t.* FROM [TableTemp] t, [MainTable] o WHERE t.[TABLE_ID]=o.[TABLE_ID] AND t.[TABLE_UPDATED]>o.[TABLE_UPDATED]");
And update all record in main table
await conn.UpdateAllAsync(UpdatedDataFromTemp);
I use logical delete so when updating the logical delete will be updated too.
I am using dart powerSNMP for .Net.
I am trying to query a table using GetTable(), it does not work for me.
Below C# code does not return any row,
const string address = "xxx.xxx.xx.x";
using (var mgr = new Manager())
{
var slave = new ManagerSlave(mgr);
slave.Socket.ReceiveTimeout = 13000;
try
{
//Retrieve table using GetNext requests
Variable[,] table = slave.GetTable("1.3.6.1.4.1.14823.2.2.1.1.1.9",
SnmpVersion.Three,
null,
new Security()
{
AuthenticationPassword = "mypassword1",
AuthenticationProtocol = AuthenticationProtocol.Md5,
PrivacyPassword = "mypassword2",
PrivacyProtocol = PrivacyProtocol.Des
},
new IPEndPoint(IPAddress.Parse(address), 161),
0);
}catch(Exception ez)
{
}
}
This is supposed to return a set of records from given OID. but It does not return me anything. When I use MIB Browser, I see GetBulk operation fetches all the records for me.
But what is wrong with GetTable() here?
When I use the the ObjectContent object to create the HttpContent to send a request via HttpClient to a Web API service I am getting the following error:
Cannot write more bytes to the buffer than the configured maximum buffer size: 65536
The following code is being used to send the request. The Card object has about 15 properties.
var client = new HttpClient();
var content = new ObjectContent<IEnumerable<Card>>(cards, "application/xml");
MessageBox.Show(content.ReadAsString()); //This line gives me the same error.
var response = client.Post("http://localhost:9767/api/cards", content);
How do I change the configured size to something greater than 65,536?
Since the problem resides in the ReadAsString extension method I would suggest that you create your own extension method to solve the maximum buffer size issue.
Here’s an example of a ReadAsLargeString extension method that maybe solves the problem.
public static string ReadAsLargeString(this HttpContent content)
{
var bufferedContent = new MemoryStream();
content.CopyTo(bufferedContent);
if (bufferedContent.Length == 0)
{
return string.Empty;
}
Encoding encoding = DefaultStringEncoding;
if ((content.Headers.ContentType != null) && (content.Headers.ContentType.CharSet != null))
{
encoding = Encoding.GetEncoding(content.Headers.ContentType.CharSet);
}
return encoding.GetString(bufferedContent.GetBuffer(), 0, (int)bufferedContent.Length);
}
There is a thread about this. Try using HttpCompletionOption.ResponseContentRead:
var message = new HttpRequestMessage(HttpMethod.Post, "http://localhost:9767/api/cards");
message.Content = content;
var client = new HttpClient();
client.Send(message, HttpCompletionOption.ResponseContentRead);
Try this for the client:
HttpClient client = new HttpClient("http://localhost:52046/");
// enable support for content up to 10 MB size
HttpClientChannel channel = new HttpClientChannel() {
MaxRequestContentBufferSize = 1024 * 1024 * 10
};
client.Channel = channel;
On the Server (snippet is based preview 4 but you should get the clue):
public class CustomServiceHostFactory : HttpConfigurableServiceHostFactory {
public override ServiceHostBase CreateServiceHost(string constructorString, Uri[] baseAddresses) {
var host = base.CreateServiceHost(constructorString, baseAddresses);
foreach (HttpEndpoint endpoint in host.Description.Endpoints) {
endpoint.TransferMode = TransferMode.Streamed;
endpoint.MaxReceivedMessageSize = 1024 * 1024 * 10;
}
return host;
}
}