I'm installing ELK stack for my company, my cousin uses it for his company too, he's a programmer so I asked him if he bought the Xpack, he says no since the mysql logs he processes aren't of value. I know that I can buy XPack or use Nginx to add authentication, but let's assume that I won't do any authentication, like many elastic users, I have a couple of questions about that scenario.
So I have filebeat that ships MySQL logs to logstash, which feeds them to Elastic search and analytics is done in Kibana.
How to make sure that no information of value end up in the logs while still having meaningful analytics. My company develops an ERP, and has many companies as customers, so at the very least, you'll have the company ID and the user ID in the logs in order to have any meaningful data, isn't this considered sensitive data?
How to make sure that no unauthorized user send a Post request to elastic search or access Kibana? Do you run them locally, not on the internet?
Do you filter the logs of any sensitive before sending it to Filebeat?
I'm just trying to understand how many users manage to run ELK without authentication, while still being able to get meaningful data.
.
How to make sure that no information of value end up in the logs while still having meaningful analytics. My company develops an ERP, and has many companies as customers, so at the very least, you'll have the company ID and the user ID in the logs in order to have any meaningful data, isn't this considered sensitive data?
If you don't want sensitive data stored in your elasticsearch you need to filter it out or anonymize it, for example, you can use a logstash filter to create a fingerprint combining the company id and user id fields, or you can remove any field with sensitive data from your message.
How to make sure that no unauthorized user send a Post request to elastic search or access Kibana? Do you run them locally, not on the internet?
Without authentication this is almost impossible, you will need full control of who knows about your elasticsearch instance and who can access it, if someone besides you has access, they can send requests to your instance, to avoid that you can use a firewall on your servers and only allow access to specific IPs.
Even if you take some precautions, running a Elasticsearch instance in production without any kind of access control is not recommended and is very risky.
You should use an access control method, it could be X-Pack, NGINX or a plugin like Search Guard.
Please check out Search Guard (https://search-guard.com/). The basic version (which is sufficient for most use cases and definitely better than nothing) is free and open source (Apache 2 License).
Disclaimer: I work for Search Guard/floragunn GmbH
If you need to grant some access / privilege, you can use grafana instead of kibana for free.
For ES access, this is like any DB security. Configure your server to allow only some IPs on 9200 and 9300.
You can also look at: https://github.com/sscarduzio/elasticsearch-readonlyrest-plugin to secure delete query (still free).
Related
I had a few questions in regards to elasticsearch architecture and associated services and/or products that is not clear to me.
The idea is to setup an elasticsearch instance for searching through file shares, Exchange mailboxes, Sharepoint sites and even Teams conversations if possible.
How would I setup the elasticsearch instance to support the following requirements:
Security filtering results from these sources for users
Develop on a simple and clean web search page like SearchUI from Elastic themselves.
Active Directory or ADFS authentication
Use nodejs on a separate server to proxy to elastic, as elastic user management means that users get access to all search results
I can find tutorials and blogs on some of these items, but no comprehensive description of how the architecture would actually work specifically with the SearchUI and proxying of data to ES.
Please have a look at this new product released by Elastic guys using same elastic search framework
https://www.elastic.co/workplace-search
it closley matches your requirement.
I am exploring the idea that capturing certain telescope entries (e.g. type = request) and save to 3rd party service such as Elastic Search.
Why? The hosting I am using does not support straight Logstash (or equivalent integration) and I need a way to access to request history. I can do this with MySQL but other tools such as Elastic Search provides more insights.
Any ideas where do I get started?
I am building a service for checking for phishing or malware URLs for one of my applications. This service will be running on google app engine. Now, I want to use google safe browsing's Update API (v4) to have local database of URL hashes. But I am having hard time to understand the setup process of the local database they have mentioned.
https://developers.google.com/safe-browsing/v4/local-databases
They do provide a Go source code to do something of this sort but its not descriptive enough to have my own implementation.
I want to setup the db on google cloud itself. Can anyone point me to any good documentation or some ways to do the same if you have tried this before.
you can use:
"https://safebrowsing.googleapis.com/v4/threatListUpdates:fetch?key=" + apiKey;
to download hash version database.
you may need send a post request to get the database according to https://developers.google.com/safe-browsing/v4/reference/rest/v4/threatListUpdates/fetch
We have a Java Spring application with lot's of contacts inside a database. Now we'd like to provide these contacts via CardDAV in order to access them via external devices.
As far as I understood CardDAV, it uses the 'well-known' protocol. Which means, it'll look up http://mydomain.com/.well-known/carddav
This might be a problem, because we have a Tomcat Server running, and multiple applications running on it and each of them should provide a CardDAV server. This means, our URLs look like:
http://mydomain.com/appOne/
http://mydomain.com/appTwo/
http://mydomain.com/appThree/
Each of those applications has a completely different set of users and data. Though each of those CardDAV repositories has to lookup its own data source and has to use its own authentication mechanism.
The question is of course: How can I get multiple different CardDAV servers with a single domain?
Btw: Is there any REAL information about CardDAV (not just WebDAV or is it all the same?!)?
For example I couldn't find anything about multiple repositories / access right restrictions. Maybe I want to have a single CardDAV server with multiple different Users, where each user has an own address book and there are some common address books.
The well-known url is used for clients to automatically discover the root of the carddav server, when a user just types in a domainname. You can only redirect to 1 server per domain, but you could setup multiple domains to redirect to multiple carddav servers.
If you can't use multiple sub-domains, you simply cannot use well-known. Instead, you will have to ask users to fill in a full url to their principal to setup their acccounts.
As to your question if there's 'real' information. rfc6352 is the official documentation. It's definitely a lot more than just WebDAV.
Effectively, iOS only supports well-known. If an iOS device cant connect via well-known it will allow the user to enter a complete principal address, BUT thats only AFTER displaying an error message to the user, at which point most users will give up.
However, the redirect occurs after authentication, so as long as you're able to authenticate at the root (eg with a username scheme that incorporates the sub-site, like 'appOne:brad') then you should be able to do it. Alternatively, as mentioned above, just use subdomains.
We have a custom Active Directory integrated web app that helps users perform some self-service on their accounts (e.g. update photo, change phone number, reset password etc.) Our app runs on domain-joined servers, as Local System, and is thus able to authenticate to the AD using the server account(s).
We use a service connection point, that the app's clients use to locate an instance of our app. (Our app clients are hard-coded to look for certain keywords which are published on the servie connection point's keywords attribute.)
We recently had a situation wherein someone (we believe accidentally) changed the keywords on one of the service connection points resulting in an outage, since the clients could no longer find our SCP when querying the AD for our keyword(s).
The customer is a bit upset about this and wishes for us to provide them the ability to determine who can change the keywords on our SCPs. This feedback was passed on from our sales guys to us, and now we need to provide some way of helping them figure out who can change the keywords on our SCPs.
So, we're looking for an API to help us to determine Effective Permissions on our Active Directory service connection point objects, so we can alleviate this situation for the customer. We couldn't quite find an Effective Permissions / Access API that could help us list all the users who have effective write access to the keyword and other attributes on our SCPs.
Is there an API/other way that one can use to determine Effective Permissions on an Active Directory object?
It needs to be able to list all the users who have a specified access on a specified set of attributes of an Active Directory object.
This stack overflow post may be able to help you. LINQ to LDAP should also allow you to access the information pretty easily as well.