I am working to establish a CI pipeline for SSIS, with the first step automating the creation of ISPAC deployment packages. I am using the code from: https://github.com/rtumaykin/ssis-build which creates ssisbuild.exe. The executable reads the XML from dtproj, unencrypting secured information using CrptoAPIs for TripleDES as documented here
These two lines of code
var passwordDeriveBytes = new PasswordDeriveBytes(password, rgbSalt);
cryptoServiceProvider.Key = passwordDeriveBytes.CryptDeriveKey("TripleDES", "SHA1", 192, cryptoServiceProvider.IV);
Supply the password and salt from the dtproj file (converted to byte array with Convert.FromBase64String(saltXMLAttributeNode.Value)
The TripleDES conversion generates the exceptions:
"Specified initialization {IV} does not match the block size for this algorithm."
Looking into the VS2017 15.5 dtproj XML I find this section:
SSIS:Algorithm="http://www.w3.org/2001/04/xmlenc#aes256-cbc">
Indicating the DTPROJ Encryption Algorithm is now aes256-cbc.
And in fact, changing the code to use:
cryptoServiceProvider.Key = passwordDeriveBytes.CryptDeriveKey("AES", "SHA1", 256, cryptoServiceProvider.IV);
Removes the error.
But the next lines:
var passwordDeriveBytes = new PasswordDeriveBytes(password, rgbSalt);
cryptoServiceProvider.Key = passwordDeriveBytes.CryptDeriveKey("AES", "SHA1", 256, cryptoServiceProvider.IV)
Generates the new non-descript error:
"Object identifier (OID) is unknown"
Rewriting the decryption process using the examples in the AESCrptoServiceProvder Class from System.Security.Cryptography.Aes
using:
static string DecryptStringFromBytesAes(byte[] ciphertext, byte[] key, byte[] aesiv)
{
if (ciphertext == null || ciphertext.Length <= 0)
throw new ArgumentNullException("cypherText");
if (key == null || key.Length <=0)
throw new ArgumentNullException("Key");
if (aesiv == null || aesiv.Length <=0)
throw new ArgumentNullException("AESIV");
string plaintext = null;
using (AesCryptoServiceProvider aesAlg = new AesCryptoServiceProvider())
{
aesAlg.Key = key;
aesAlg.IV = aesiv;
ICryptoTransform decryptor = aesAlg.CreateDecryptor(aesAlg.Key, aesAlg.IV);
using (MemoryStream msDecrypt = new MemoryStream(ciphertext))
{
using (CryptoStream csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read))
{
using (StreamReader srDecrtypt = new StreamReader(csDecrypt))
{
plaintext = srDecrtypt.ReadToEnd();
}
}
}
}
return plaintext;
}
where Key = the Salt value from dtproj generates an error
"The specified key is not a valid size for this algorithm."
when loading the key at line:
using (AesCryptoServiceProvider aesAlg = new AesCryptoServiceProvider())
{
aesAlg.Key = key;
I am either supplying the incorrect value for the key or using the incorrect CryptoServiceProvider.
Does anyone know how encryption has been changed in VS2017 SSDT 15.5.1?
Related
All,
I am using a Spring boot application to store data in DB. I am getting this data from IBM MQ through Kafka topic.
I am getting messages in EBCDIC format, so used cobol copybook, JRecord, cb2xml jars to convert to readable format and store in DB.
Now i am getting another file also in the same manner, but after conversion the data looks like this:
10020REFUNDONE
10021REFUNDTWO ·" ÷/
10022REFUNDTHREE oú^ "
10023REFUNDFOUR ¨jÄ ò≈
Here is how i am converting to readable format from ebcdic:
AbstractLineReader reader = null;
StringBuffer finalBuffer = new StringBuffer();
try {
String copybook = "/ds_header.cbl";
reader = CustomCobolProvider.getInstance().getLineReader(copybook, Convert.FMT_MAINFRAME, new BufferedInputStream(new ByteArrayInputStream(salesData)));
AbstractLine line;
while ((line = reader.read()) != null) {
if (null != line.getFieldValue(REC_TYPE)){
finalBuffer.append(line.getFullLine());
}
}
}
and this is my getLineReader method:
public AbstractLineReader getLineReader(String copybook, int numericType, InputStream fileStream) throws Exception {
String font = "";
if (numericType == 1) {
font = "cp037";
}
InputStream stream = CustomCobolProvider.class.getResourceAsStream(copybook);
if(stream == null ) throw new RuntimeException("Can't Load the Copybook Metadata file from Resource....");
LayoutDetail copyBook = ((ExternalRecord)this.copybookInt.loadCopyBook(stream, copybook, CopybookLoader.SPLIT_REDEFINE, 0, font, CommonBits.getDefaultCobolTextFormat(), Convert.FMT_MAINFRAME, 0, (AbsSSLogger)null).setFileStructure(Constants.IO_FIXED_LENGTH)).asLayoutDetail();
AbstractLineReader ret = LineIOProvider.getInstance().getLineReader(copyBook, (LineProvider)null);
ret.open(fileStream, copyBook);
return ret;
}
I am stuck here with the numeric conversion, i got to know it is coming in packed decimal.
I have nil knowledge on cobol and mainframe, referred few sites and got to know how to convert from ebcdic to readable format. Please help!
The problem is getFullLine() method does not do any field translation; you need to access individual fields. You can use the line.getFieldIterator(0) to get a field iterator for the line.
Also unless you are using an ancient version of JRecord, you are better off using the JRecordInterface1 class.
Some thing like the following should work:
StringBuffer finalBuffer = new StringBuffer();
try {
ICobolIOBuilder iob = JRecordInterface1.COBOL .newIOBuilder(copybookName)
.setFont("cp037")
.setFileOrganization(Constants.IO_FIXED_LENGTH)
;
AbstractLineReader reader = iob.newReader(dataFile);
while ((line = reader.read()) != null) {
String sep = "";
for (AbstractFieldValue fv : line.getFieldIterator(0)) {
finalBuffer.append(sep).append(fv);
sep = "\t";
}
finalBuffer.append("\n");
}
reader.close();
} catch (Exception e) {
// what ever ....
}
Other points
With MQ data source you do not need to create line-readers. You can create lines directly from a byte array:
ICobolIOBuilder iob = JRecordInterface1.COBOL .newIOBuilder(copybookName)
.setFont("cp037")
.setFileOrganization(Constants.IO_FIXED_LENGTH)
;
AbstractLine line = iob.newLine(byteArrayFromMq);
for (AbstractFieldValue fv : line.getFieldIterator(0)) {
// what ever
}
I am hoping there is someone here who is familiar with a Real Estate data standard known as RETS. The National Association of Realtors provides a dll for interfacing with their services called libRETS, but it is not being supported like it once was and recent events have prompted us to create our own as a replacement. For logistics reasons, we can't do this in Core and are using the current C#.Net 4.7.2.
There are 2 or 3 different "security levels" for connecting to a RETS Server, with the method being a per case basis from one MLS to the next. We can successfully connect to those who only require a login and password, but are hitting a wall on those who also require what is called a UserAgent and UserAgentPassword, which must passed somehow using Md5 encryption. The server is returning:
The remote server returned an error: (401) Unauthorized.
private WebResponse GetLoginBasicResponse()//*** THIS ONE WORKS ***
{
try
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
var request = (HttpWebRequest)WebRequest.Create(new Uri(_cred.loginUri));
request.Method = "GET";
request.Headers.Add("RETS-Version", _retsVersion);
request.Credentials = new NetworkCredential(_login, _password);
return request.GetResponse();
}
catch (Exception ex)
{
string ignore = ex.Message;
return null;
}
}
private WebResponse GetLoginWithUserAgentResponse()//*** THIS ONE DOES NOT WORK ***
{
try
{
// ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
var request = (HttpWebRequest)WebRequest.Create(new Uri(_cred.loginUri));
request.Method = "GET";
request.Headers.Add("RETS-Version", _retsVersion);
if (!string.IsNullOrEmpty(_cred.userAgent))
{
request.UserAgent = Md5(_cred.userAgent + ":" + _cred.userAgentPassword);
//request.Headers.Add("RETS-UA-Authorization", "Digest " + Md5(_cred.userAgent + ":" + _cred.userAgentPassword));
}
request.Credentials = new NetworkCredential(_login, _password);
return request.GetResponse();
}
catch (Exception ex)
{
string ignore = ex.Message;
return null;
}
}
public string Md5(string input) //*** Borrowed this from from .NET Core Project and presume it works
{
// Use input string to calculate MD5 hash
using (System.Security.Cryptography.MD5 md5 = System.Security.Cryptography.MD5.Create())
{
byte[] inputBytes = Encoding.ASCII.GetBytes(input);
byte[] hashBytes = md5.ComputeHash(inputBytes);
// Convert the byte array to hexadecimal string
StringBuilder sb = new StringBuilder();
for (int i = 0; i < hashBytes.Length; i++)
{
sb.Append(hashBytes[i].ToString("X2"));
}
return sb.ToString();
}
}
Page 20 of this document describes how to build the UA header: https://www.ranww.org/documents/resources/rets_1_8.pdf
There’s a few other fields you need to include.
We were not able to solve the issue in .NET but found a .NET Core project in GitHub that we are using instead. https://github.com/CrestApps/RetsConnector
This case can be closed
Not seeing an option to "Mark as Answer". Have tried both MS Edge and Google Chrome
I'm wondering whether anybody can run the code below and see why firefox cannot talk to the owin SSL server correctly? It worked well in Chrome & IE.
I've tried to manually import the pfx into firefox but it still cannot talk to the owin server correctly.
Refer to the code below, I created an owin http server with SSL support and it can generate & install the certificate & bind the certificate to port automatically.
However, I noticed an issue when I test in different browsers. The firefox browser always display an error message indicating that:
The connection to the server was reset while the page was loading.
The page you are trying to view cannot be shown because the
authenticity of the received data could not be verified. Please
contact the website owners to inform them of this problem.
Chrome & IE can display the page correctly.
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Security;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Threading.Tasks;
using CERTENROLLLib;
using Microsoft.Owin.Hosting;
using AppFunc = System.Func<System.Collections.Generic.IDictionary<string, object>, System.Threading.Tasks.Task>;
using System.Security.Permissions;
using System.Security.Policy;
namespace Owin.Startup
{
class Program
{
static void Main(string[] args)
{
int port = 7878;
var cert = GetCert("localhost", TimeSpan.FromDays(3650), "devpwd", AppDomain.CurrentDomain.BaseDirectory + "cert.pfx");
ActivateCert((X509Certificate2)cert, port, GetAppId());
StartOptions so = new StartOptions();
so.Urls.Add($"https://+:{port}/");
using (WebApp.Start<Startup>(so))
{
Console.WriteLine($"Hosted on port: {port}");
Console.ReadLine();
}
}
static private string GetAppId()
{
Assembly assembly = Assembly.GetExecutingAssembly();
//The following line (part of the original answer) is misleading.
//**Do not** use it unless you want to return the System.Reflection.Assembly type's GUID.
//Console.WriteLine(assembly.GetType().GUID.ToString());
// The following is the correct code.
var attribute = (GuidAttribute)assembly.GetCustomAttributes(typeof(GuidAttribute), true)[0];
var id = attribute.Value;
return id;
}
static public X509Certificate GetCert(string cn, TimeSpan expirationLength, string pwd = "", string filename = null)
{
// http://stackoverflow.com/questions/18339706/how-to-create-self-signed-certificate-programmatically-for-wcf-service
// http://stackoverflow.com/questions/21629395/http-listener-with-https-support-coded-in-c-sharp
// https://msdn.microsoft.com/en-us/library/system.security.cryptography.x509certificates.storename(v=vs.110).aspx
// create DN for subject and issuer
System.Security.Cryptography.X509Certificates.X509Certificate cert = null;
if (filename != null && File.Exists(filename))
{
cert = new X509Certificate2(filename, pwd, X509KeyStorageFlags.Exportable | X509KeyStorageFlags.MachineKeySet | X509KeyStorageFlags.PersistKeySet);
}
else
{
var base64encoded = string.Empty;
base64encoded = CreateCertContent(cn, expirationLength, pwd);
cert = new System.Security.Cryptography.X509Certificates.X509Certificate2(
System.Convert.FromBase64String(base64encoded), pwd,
// mark the private key as exportable (this is usually what you want to do)
// mark private key to go into the Machine store instead of the current users store
X509KeyStorageFlags.Exportable | X509KeyStorageFlags.MachineKeySet | X509KeyStorageFlags.PersistKeySet
);
File.WriteAllBytes(filename, cert.Export(X509ContentType.Pfx, pwd));
}
// instantiate the target class with the PKCS#12 data (and the empty password)
return cert;
}
private static string CreateCertContent(string cn, TimeSpan expirationLength, string pwd)
{
string base64encoded = string.Empty;
var dn = new CX500DistinguishedName();
dn.Encode("CN=" + cn, X500NameFlags.XCN_CERT_NAME_STR_NONE);
CX509PrivateKey privateKey = new CX509PrivateKey();
privateKey.ProviderName = "Microsoft Strong Cryptographic Provider";
privateKey.Length = 2048;
privateKey.KeySpec = X509KeySpec.XCN_AT_KEYEXCHANGE;
privateKey.KeyUsage = X509PrivateKeyUsageFlags.XCN_NCRYPT_ALLOW_DECRYPT_FLAG |
X509PrivateKeyUsageFlags.XCN_NCRYPT_ALLOW_KEY_AGREEMENT_FLAG;
privateKey.MachineContext = true;
privateKey.ExportPolicy = X509PrivateKeyExportFlags.XCN_NCRYPT_ALLOW_PLAINTEXT_EXPORT_FLAG;
privateKey.Create();
// Use the stronger SHA512 hashing algorithm
var hashobj = new CObjectId();
hashobj.InitializeFromAlgorithmName(ObjectIdGroupId.XCN_CRYPT_HASH_ALG_OID_GROUP_ID,
ObjectIdPublicKeyFlags.XCN_CRYPT_OID_INFO_PUBKEY_ANY,
AlgorithmFlags.AlgorithmFlagsNone, "SHA512");
// Create the self signing request
var cert = new CX509CertificateRequestCertificate();
cert.InitializeFromPrivateKey(X509CertificateEnrollmentContext.ContextMachine, privateKey, "");
cert.Subject = dn;
cert.Issuer = dn; // the issuer and the subject are the same
cert.NotBefore = DateTime.Now.Date;
// this cert expires immediately. Change to whatever makes sense for you
cert.NotAfter = cert.NotBefore + expirationLength;
cert.HashAlgorithm = hashobj; // Specify the hashing algorithm
cert.Encode(); // encode the certificate
// Do the final enrollment process
var enroll = new CX509Enrollment();
enroll.InitializeFromRequest(cert); // load the certificate
enroll.CertificateFriendlyName = cn; // Optional: add a friendly name
string csr = enroll.CreateRequest(); // Output the request in base64
// and install it back as the response
enroll.InstallResponse(InstallResponseRestrictionFlags.AllowUntrustedCertificate,
csr, EncodingType.XCN_CRYPT_STRING_BASE64, pwd); // no password
// output a base64 encoded PKCS#12 so we can import it back to the .Net security classes
base64encoded = enroll.CreatePFX(pwd, // no password, this is for internal consumption
PFXExportOptions.PFXExportChainWithRoot);
return base64encoded;
}
private static void ActivateCert(X509Certificate2 rlt, int port, string appId)
{
X509Store store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
store.Open(OpenFlags.ReadWrite);
if (!store.Certificates.Contains(rlt))
{
store.Add(rlt);
ProcessStartInfo psi = new ProcessStartInfo();
psi.FileName = "netsh";
psi.Arguments = $"http delete sslcert ipport=0.0.0.0:{port}";
Process procDel = Process.Start(psi);
procDel.WaitForExit();
psi.Arguments = $"http add sslcert ipport=0.0.0.0:{port} certhash={rlt.Thumbprint} appid={{{appId}}}";
Process proc = Process.Start(psi);
proc.WaitForExit();
psi.Arguments = $"http delete sslcert ipport=[::]:{port}";
Process procDelV6 = Process.Start(psi);
procDelV6.WaitForExit();
psi.Arguments = $"http add sslcert ipport=[::]:{port} certhash={rlt.Thumbprint} appid={{{appId}}}";
Process procV6 = Process.Start(psi);
procV6.WaitForExit();
psi.Arguments = $"http add urlacl url=https://+:{port}/ user={Environment.UserDomainName}\\{Environment.UserName}";
Process procAcl = Process.Start(psi);
procAcl.WaitForExit();
}
store.Close();
}
}
public class Startup
{
private IAppBuilder app;
public void Configuration(IAppBuilder app)
{
#if DEBUG
app.UseErrorPage();
#endif
app.Use(new Func<AppFunc, AppFunc>(next => (async env =>
{
Console.WriteLine("Begin Request");
foreach (var i in env.Keys)
{
Console.WriteLine($"{i}\t={(env[i] == null ? "null" : env[i].ToString())}\t#\t{(env[i] == null ? "null" : env[i].GetType().FullName)}");
}
if (next != null)
{
await next.Invoke(env);
}
else
{
Console.WriteLine("Process Complete");
}
Console.WriteLine("End Request");
})));
app.UseWelcomePage("/");
this.app = app;
}
}
}
In firefox, type about:config and search for dhe.
The list shown all the cipher algorithms that firefox supported.
However, I noticed you used SHA512 which happened to be not supported by the latest firefox.
Please modify the algorithm to sha256 and it should work in firefox :)
My XML file:
<?xml version="1.0" encoding="utf-8"?>
<dskh>
<khachhang maso="kh01">
<ten_kh>thehung</ten_kh>
<tuoi_kh>15</tuoi_kh>
<dchi_kh>hochiminh</dchi_kh>
</khachhang>
<khachhang maso="kh02">
<ten_kh>hung</ten_kh>
<tuoi_kh>15</tuoi_kh>
<dchi_kh>hcm</dchi_kh>
</khachhang>
</dskh>
My Encrypt and Decrypt code:
class Program
{
// Call this function to remove the key from memory after use for security.
[System.Runtime.InteropServices.DllImport("KERNEL32.DLL", EntryPoint = "RtlZeroMemory")]
public static extern bool ZeroMemory(ref string Destination, int Length);
// Function to Generate a 64 bits Key.
static string GenerateKey()
{
// Create an instance of Symetric Algorithm. Key and IV is generated automatically.
DESCryptoServiceProvider desCrypto = (DESCryptoServiceProvider)DESCryptoServiceProvider.Create();
// Use the Automatically generated key for Encryption.
return ASCIIEncoding.ASCII.GetString(desCrypto.Key);
}
static void EncryptFile(string sInputFilename,
string sOutputFilename,
string sKey) {
FileStream fsInput = new FileStream(sInputFilename,FileMode.Open,FileAccess.Read);
FileStream fsEncrypted = new FileStream(sOutputFilename,FileMode.Create,FileAccess.Write);
DESCryptoServiceProvider DES = new DESCryptoServiceProvider();
DES.Key = Encoding.UTF8.GetBytes(sKey);
DES.IV = Encoding.UTF8.GetBytes(sKey);
ICryptoTransform desencrypt = DES.CreateEncryptor();
CryptoStream cryptostream = new CryptoStream(fsEncrypted,desencrypt,CryptoStreamMode.Write);
byte[] bytearrayinput = new byte[fsInput.Length - 1];
fsInput.Read(bytearrayinput, 0, bytearrayinput.Length);
cryptostream.Write(bytearrayinput, 0, bytearrayinput.Length);
}
static void DecryptFile(string sInputFilename,
string sOutputFilename,
string sKey)
{
DESCryptoServiceProvider DES = new DESCryptoServiceProvider();
//A 64 bit key and IV is required for this provider.
//Set secret key For DES algorithm.
DES.Key = Encoding.UTF8.GetBytes(sKey);
//Set initialization vector.
DES.IV = Encoding.UTF8.GetBytes(sKey);
//Create a file stream to read the encrypted file back.
FileStream fsread = new FileStream(sInputFilename,
FileMode.Open,
FileAccess.Read);
//Create a DES decryptor from the DES instance.
ICryptoTransform desdecrypt = DES.CreateDecryptor();
//Create crypto stream set to read and do a
//DES decryption transform on incoming bytes.
CryptoStream cryptostreamDecr = new CryptoStream(fsread,
desdecrypt,
CryptoStreamMode.Read);
//Print the contents of the decrypted file.
StreamWriter fsDecrypted = new StreamWriter(sOutputFilename);
fsDecrypted.Write(new StreamReader(cryptostreamDecr).ReadToEnd());
fsDecrypted.Flush();
fsDecrypted.Close();
}
static void Main(string[] args)
{
// Must be 64 bits, 8 bytes.
// Distribute this key to the user who will decrypt this file.
string sSecretKey;
// Get the key for the file to encrypt.
sSecretKey = GenerateKey();
// For additional security pin the key.
GCHandle gch = GCHandle.Alloc(sSecretKey, GCHandleType.Pinned);
// Encrypt the file.
EncryptFile(#"C:\xml_kh.xml",
#"C:\xml_encry.xml",
sSecretKey);
//Decrypt the file.
DecryptFile(#"C:\xml_encry.xml",
#"C:\xml_decry.xml",
sSecretKey);
}
}
When i excute Decrypt code, i get Cryptographicexception: Bad Data here:
//Print the contents of the decrypted file.
StreamWriter fsDecrypted = new StreamWriter(sOutputFilename);
fsDecrypted.Write(new StreamReader(cryptostreamDecr).ReadToEnd());
fsDecrypted.Flush();
fsDecrypted.Close();
Please help me!!!
Add two lines at the end of EncryptFile function:
cryptostream.Flush();
cryptostream.Close();
EDIT
I missed there is one more error:
byte[] bytearrayinput = new byte[fsInput.Length - 1];
s/b
byte[] bytearrayinput = new byte[fsInput.Length];
The code working fine when run out side the CRM2011 application. When I embedded my Code Inside the CRM it will return blank pdf file with no data.
following is my code
public static Byte[] RenderReport(string ServiceURL, string ReportName, ParameterValue[] parameters)
{
byte[] result;
string encoding;
string mimeType;
Warning[] warnings = null;
string[] streamids;
string extension;
BasicHttpBinding binding = new BasicHttpBinding(BasicHttpSecurityMode.TransportCredentialOnly);
binding.Security.Message.ClientCredentialType = BasicHttpMessageCredentialType.UserName;
binding.Security.Mode = BasicHttpSecurityMode.TransportCredentialOnly;
binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.Ntlm;
binding.Security.Transport.ProxyCredentialType = HttpProxyCredentialType.None;
binding.Security.Transport.Realm = string.Empty;
binding.MaxReceivedMessageSize = 2147483647;
string deviceInfo = #"<DeviceInfo><Toolbar>False</Toolbar></DeviceInfo>";
EndpointAddress endpoint = new EndpointAddress(ServiceURL);
ReportExecutionServiceSoapClient client = new ReportExecutionServiceSoapClient(binding, endpoint);
client.ClientCredentials.Windows.AllowedImpersonationLevel = TokenImpersonationLevel.Impersonation;
ExecutionInfo execInfo = new ExecutionInfo();
ExecutionHeader execHeader = new ExecutionHeader();
ServerInfoHeader serverInfoHeader;
ExecutionInfo executionInfo;
ExecutionHeader executionHeader = client.LoadReport(null, ReportName, null, out serverInfoHeader, out executionInfo);
// Attach Report Parameters
//executionHeader.ExecutionID = executionInfo.ExecutionID;
if (parameters != null)
client.SetExecutionParameters(executionHeader, null, parameters, null, out executionInfo);
ServerInfoHeader Info= client.Render(executionHeader, null, "PDF", deviceInfo, out result, out extension, out mimeType, out encoding, out warnings, out streamids);
return result;
}
Thanks
I changed the Async processor to run under a user that has access to the Report Server database and it worked instantly.it was set to use network service so I changed it to Administrator in services.