we have a security requirement to use private link when connecting to Elastic cluster in elastic cloud.
However when using private link our old solution using Cloud Id connection won't work anymore. We tested it and it is confirmed in this discussion - link
We could switch to using Uri connection however I noticed this sentence in the official docs - link
We recommend using a Cloud ID whenever possible because your client
will be automatically configured for optimal use with Elastic Cloud,
including HTTPS and HTTP compression.
This would be the connection setting we would be using
ConnectionSettings settings =
new ConnectionSettings(new Uri("https://someUri.privatelink.westeurope.azure.elastic-cloud.com:443"))
.DefaultIndex(DefaultIndex)
.EnableDebugMode()
.MaximumRetries(MaxRetries)
.MaxRetryTimeout(MaxRetryTimeout)
.ApiKeyAuthentication(new ApiKeyAuthenticationCredentials("someApiKey"))
.EnableApiVersioningHeader();
Now I'm not sure what the authors in the official docs meant by optimal use with Elastic cloud.
Are there any settings you can think of we should add here for production ready scenarios ? Are there some things that might not work if we go with out the use of Cloud Id ?
Related
I can not connect to Google Services from client application if it is trying to communicate with oauth2.googleapis.com (which is probably blocked in my corporate network - I dont know how to test it for sure).
I tried BigQuery with JDBC driver in Dbeaver. With basic settings.
User-based login does this:
It generates link for OAUTH. I open the browser and login with the right google account. Then I insert generated code into the Dbeaver and I recieve that AUTH has failed.
Service-based login does this:
It does not want me to visit any webpage. It just tells me:
[Simba][BigQueryJDBCDriver](100004) HttpTransport IO error : oauth2.googleapis.com.
I also tried to use ODBC, where PROXY can be filled in. But no luck.
When I take a look into 'Proxy Options' the proxy port is always rewritten by proxy host. Weird.
This is what happens when i click on 'catalog' or 'dataset' drop-down field. I cant do any further steps.
BUT!
When I set my HTTP PROXY in GCLOUD CLI APP then communication works. And I can call BQ from it.
Does it mean that GCLOUD communicates through HTTP Proxy and DBeaver or ODBC does not? Or does it mean that GCLOUD does not need oauth2.googleapis.com but ODBC and JDBC do and it is blacklisted? I am confused.
We need to migrate from our internal environment to GCP. We would love to use various applications. I would ask for whitelisting oauth2.googleapis.com but i am not sure this is the only problem as GCLOUD app works without any flaws.
I am not-experienced with networking so i am more than happy to update / correct this question or add any info (if you need) to help me understand this issue. Thank you
According to your description, your corporate network is using a Proxy to reach out Internet, this is the reason why gcloud is capable to reach out BigQuery service when Proxy settings are configured in your system; through Cloud SDK Proxy settings or HTTP PROXY environment variable.
You require to setup the proxy settings within the JDBC connection string as described in Simba JDBC driver documentation, e.g.:
jdbc:bigquery:DataSetId=MyDataSetId;ProjectId=MyProjectId;OAuthType=1;ProxyHost=MyProxyHost;ProxyPort=MyProxyPort;ProxyUID=MyProxyUsername;ProxyPWD=MyProxyPassword
This connection string will indicate the Proxy settings to Simba JDBC driver.
I have an existing Datastax Cassandra setup that is working. We just added authentication to the system and now we can log in with our AD accounts. This is very nice and certainly works. However applications need to use a hard-coded username/password in order to connect.
In SQL Server we were able to setup a user to run the service as and then it would connect and work through AD. However in Cassandra it is not the same.
If I don't want to include usernames and especially passwords in my app.config files what are my options?
You can use authentication via LDAP with DSE (Datastax Enterprise), so the authentication stage is done with LDAP instead of the internal authentication in DSE which you're using at the moment. Note that my comments here apply to DSE5.0 onwards but you can use LDAP auth with earlier versions of DSE from 4.6 onwards.
The documentation (link below) covers this. The basic steps are as follows:
Configure your authenticator in the cassandra.yaml to use the DSE authenticator
authenticator: com.datastax.bdp.cassandra.auth.DseAuthenticator
Create an internal role in cassandra to map to the LDAP group(s) in your LDAP server using the CREATE ROLE command
Ensure all the users you need to use map to the relevant LDAP group (part of your LDAP config)
Configure your dse.yaml to have the correct settings for your LDAP server
Restart the DSE process for the settings to take effect
The following documentation gives some good examples and background information:
https://docs.datastax.com/en/latest-dse/datastax_enterprise/unifiedAuth/unifiedAuthConfig.html
https://docs.datastax.com/en/latest-dse/datastax_enterprise/sec/authLdapConfig.html
Note: when configuring the dse.yaml note the comment in the docs regarding user_search_filter:
When using Active Directory set the filter to (sAMAccountName={0})
My application will capture private customer data, requiring secure access for my clients.
Is it possible to use HTTPS/SSL in Wakanda Cloud?
HTTPS/SSL is not a current feature of Wakanda Cloud. However, HTTPS/SSL is a feature of Wakanda Server when you host it yourself.
The Wakanda Cloud Roadmap is on Trello, which lists SSL/HTTPS Support as a proposed feature.
You can vote on proposed features at Trello.
You can also submit additional feature requests at Trello
HTTPS is available in the Wakanda Cloud. I have a medium instance using HTTPS right now.
Need to know if implementation of Jest client available for Https URL (not http- since it is available), or accessing Jest client only based on cluster name or based on DNS name.
suggest me if i am not aware of these implementations.
Issue opened on Git hub: https://github.com/searchbox-io/Jest/issues/270
Edited:
Based on the below reference article , https://github.com/searchbox-io/Jest/blob/master/jest/README.md
it is clear Jest supports Https/SSL connections.
Just curious to know if it can support cluster name based configuration i.e without providing host or port details?
I am using Jest client version 0.1.4. Proxy and SSL configurations were not introduced in this version.
But with this following code, able to successfully access cluster URL without providing host and port details.
List<String> connectionUri = new ArrayList<String>();
connectionUri.add("https://search-testing.es.samplews.com");
jestFactory.setHttpClientConfig(new HttpClientConfig
.Builder(connectionUri)
.maxTotalConnection(maxTotalConnection)
.readTimeout(readTimeout)
.connTimeout(connTimeout)
.multiThreaded(true)
.build());
values:
maxTotalConnection = 50, readTimeout=60000, connTimeout=60000
Is there any cloud computing service like e.g. Google's AppEngine or Amazon's S3 out there, that allows me to use JDBC without using some sort of proxy? I do have a MySQL server, that I can't move into the cloud, so I want to access this server directly via a JDBC connection.
If you can't migrate your database, don't move to the cloud. It would be a performance and security nightmare to have your app communicate with the database over the internet.
To answer your question more specifically:
S3 is a storage platform, so it is irrelevant to your question. Google's App Engine is only designed to interface with it's own storage system.
If you really want to do this you could use a full fledged virtual instance such as Amazon EC2, or Rackspace Cloud.
EDIT
I forgot to mention, Google now offers a MySQL service. I would assume that Java apps use jdbc to connect to it.