Is there a way to authorize destinations with Apache Apollo MQ?
What I would like is to make it so that
1) users may write only to a shared topic but restrict read to an server/admin. This topic is to send messages to a server.
2) Users may read from their own private topic but no one but the server/admin may write to it.
For Example:
Topic User rights Server/Admin rights
/public Write only Read only
/user/foo ONLY the user foo may read Write only
/user/bar ONLY the user bar may read Write only
/user/<username> ONLY the <username> may read Write only
Now for the interesting part. This must work with dynamic topics. The user's name is NOT known ahead of time.
I had this working with Apache ActiveMQ using a custom BrokerFilter but am not sure how to do with with Apollo.
Thanks for any help.
After a lot of head scratching I figured it out.
In apollo.xml:
<broker xmlns="http://activemq.apache.org/schema/activemq/apollo" security_factory="com.me.MyAuthorizationPlugin">
In com.me.MyAuthorizationPlugin:
package com.me
import org.fusesource.hawtdispatch.DispatchQueue.QueueType
import org.apache.activemq.apollo.broker.security._
import org.apache.activemq.apollo.broker.{ Queue, Broker, VirtualHost }
import java.lang.Boolean
class MyAuthorizationPlugin extends SecurityFactory {
def install(broker: Broker) {
DefaultSecurityFactory.install(broker)
}
def install(virtual_host: VirtualHost) {
DefaultSecurityFactory.install(virtual_host)
val default_authorizer = virtual_host.authorizer
virtual_host.authorizer = new Authorizer() {
def can(ctx: SecurityContext, action: String, resource: SecuredResource): Boolean = {
println("Resource: " + resource.id + " User: " + ctx.user)
resource.resource_kind match {
case SecuredResource.TopicKind =>
val id = resource.id
println("Topic Resource: " + id + " User: " + ctx.user)
var result : Boolean = id.startsWith("user." + ctx.user) || id.startsWith("MDN." + ctx.user + ".")
println("Result: " + result)
return result
case _ =>
return default_authorizer.can(ctx, action, resource)
}
}
}
}
}
The following URLs seemed VERY useful and indeed nearly a perfect match:
https://github.com/apache/activemq-apollo/blob/trunk/apollo-stomp/src/test/resources/apollo-stomp-custom-security.xml#L18
https://github.com/apache/activemq-apollo/blob/trunk/apollo-stomp/src/test/scala/org/apache/activemq/apollo/stomp/test/UserOwnershipSecurityFactory.scala#L29
Now I only need to clean up my nasty scala and put it in Git.
I am thinking of doing two tests:
Speed of EXACTLY what I need
A Regex pattern matcher with username / clientID replacements and +/*/?/etc This pattern will be pulled from the config file.
If they are nearly identical I may see about adding it to Apollo by contacting commiters.
Related
Our production instance of NiFi is version 1.8.0. We have a custom processor that continually looks at it's downstream connections in order to route flow files based on the connection's queue size.
Here is the salient snippet of how we do this . . .
String processorId = this.getIdentifier();
ProcessorGroupStatus processGroupStatus = ((EventAccess) getControllerService()).getContollerStatus();
Collection<ConnectionStatus> groupConnections = processGroupStatus.getConnectionStatus();
ArrayList connections = new ArrayList<>(groupConnections);
for (Object processorConnection : connections) {
ConnectionStatus connection = (ConnectionStatus) processorConnection;
if(connection.getSourceId().equals(processorId){
//do stuff with connection.getQueuedCount() & connection.getQueuedBytes()
break;
}
}
Everything has been working as expected for the last couple of years. However, upgrading our NiFi instance to version 1.11.4 has broken this approach. The exception thrown is:
class org.apache.nifi.contoller.serviceStandardContollerServiceProvider cannot be cast to class org.apache.nifi.reporting.EventAccess
Is there another way to retrieve connections from processContext?
One approach that may be more upwardly compatible (and easier to maintain) than a custom Java processor would be to use the ExecuteGroovyScript processor.
The Groovy script in this case would look something like:
ff = session.get()
if (ff) {
me = context.procNode
processorId = me.identifier
connections = me.processGroup.connections
connections.each { connection ->
if(connection.source.identifier.equals(processorId)) {
ff[connection.identifier] = "I am the source " +
"[" + connection.flowFileQueue.size().objectCount + "]" +
"[" + connection.flowFileQueue.size().byteCount + "]"
}
else {
ff[connection.identifier] = "I am NOT the source; my name is [" + connection.name + "]"
}
}
REL_SUCCESS << ff
}
To find out what is available to the Groovy script, I use a combination of the NiFi JavaDocs (https://javadoc.io/static/org.apache.nifi/nifi-api/1.12.0/index.html) and the Github code for NiFi (https://github.com/apache/nifi/tree/c396927299586b896df4ebc745793b4c451f3898/nifi-api/src/main/java/org/apache/nifi).
As a side note, we converted our custom Java processors to Groovy script, because of an upgrade incompatibility when going to (ironically) 1.8.0. We have not had an issue with NiFi upgrades since then, and are currently running v 1.11.4.
I tried creating a method in Postman and got really close but am having issues with the signature. We are trying to query the IP ranges for VPCs to add to a WAF rule, in order to allow traffic to a secure application.
Postman allows a pre-request script, in Javascript, and supports a handful of included JS libraries, including CryptoJS.
The code here creates exactly the request that Ali Cloud says needs to be signed. It signs with HMAC-SHA1 from CryptoJS and encodes to base 64.
All of the variables are included in the request parameters. I'm not sure what else it could be complaining about.
var dateIso = new Date().toISOString();
var randomString = function(length) {
var text = "";
var possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
for(var i = 0; i < length; i++) {
text += possible.charAt(Math.floor(Math.random() * possible.length));
}
return text;
}
var accesskeyid = "LTAI4GC7VEijsm5bV3zwcZxZ"
var action = "DescribePublicIPAddress"
var format = "XML"
var regionid = "cn-shanghai-eu13-a01"
var signaturemethod = "HMAC-SHA1"
var signaturenonce = randomString(16)
var signatureversion = "1.0"
var timestamp = dateIso.replace(/:/gi, "%253A")
var version = "2016-04-28"
pm.environment.set("AccessKeyId", accesskeyid)
pm.environment.set("Action", action)
pm.environment.set("Format", format)
pm.environment.set("RegionID", regionid)
pm.environment.set("SignatureMethod", signaturemethod)
pm.environment.set("SignatureNonce", signaturenonce)
pm.environment.set("SignatureVersion", signatureversion)
pm.environment.set("Timestamp", dateIso)
pm.environment.set("Version", version)
var request = "GET&%2F&" + "AccessKeyID%3D" + accesskeyid + "%26Action%3D" + action + "%26Format%3D" + format + "%26RegionID%3D" + regionid + "%26SignatureMethod%3D" + signaturemethod + "%26SignatureNonce%3D" + signaturenonce + "%26SignatureVersion%3D" + signatureversion + "%26Timestamp%3D" + timestamp + "%26Version%3D" + version
pm.environment.set("Request", request)
var hash = CryptoJS.HmacSHA1(request, "spwH5dNeEm4t4dlpqvYWVGgf7aEAxB&")
var base64 = CryptoJS.enc.Base64.stringify(hash)
var encodesig = encodeURIComponent(base64)
pm.environment.set("Signature", encodesig);
console.log(base64)
console.log(request)
The console output shows:
Signature: XbVi12iApzZ0rRgJLBv0ytJJ0LY=
Parameter string to be signed:
GET&%2F&AccessKeyID%3DLTAI4GC7VEijsm5bV3zwcZvC%26Action%3DDescribePublicIPAddress%26Format%3DXML%26RegionID%3Dcn-shanghai-eu13-a01%26SignatureMethod%3DHMAC-SHA1%26SignatureNonce%3DiP1QJtbasQNSOxVY%26SignatureVersion%3D1.0%26Timestamp%3D2020-06-01T15%253A38%253A12.266Z%26Version%3D2016-04-28
Request sent:
GET https://vpc.aliyuncs.com/?AccessKeyID=LTAI4GC7VEijsm5bV3zwcZvC&Action=DescribePublicIPAddress&Format=XML&RegionID=cn-shanghai-eu13-a01&SignatureMethod=HMAC-SHA1&SignatureNonce=iP1QJtbasQNSOxVY&SignatureVersion=1.0&Timestamp=2020-06-01T15:38:12.266Z&Version=2016-04-28&Signature=XbVi12iApzZ0rRgJLBv0ytJJ0LY%3D
Response Received:
<?xml version='1.0' encoding='UTF-8'?><Error><RequestId>B16D216F-56ED-4D16-9CEC-633C303F2B61</RequestId><HostId>vpc.aliyuncs.com</HostId><Code>IncompleteSignature</Code><Message>The request signature does not conform to Aliyun standards. server string to sign is:GET&%2F&AccessKeyID%3DLTAI4GC7VEijsm5bV3zwcZvC%26Action%3DDescribePublicIPAddress%26Format%3DXML%26RegionID%3Dcn-shanghai-eu13-a01%26SignatureMethod%3DHMAC-SHA1%26SignatureNonce%3DiP1QJtbasQNSOxVY%26SignatureVersion%3D1.0%26Timestamp%3D2020-06-01T15%253A38%253A12.266Z%26Version%3D2016-04-28</Message><Recommend><![CDATA[https://error-center.aliyun.com/status/search?Keyword=IncompleteSignature&source=PopGw]]></Recommend></Error>
When I check the "server string to sign" from the response and the parameter string that was signed in a compare, they are identical.
It looks like everything is built as needed but the signature is still barking. Guessing I missed something simple but haven't found it yet.
Note: The accesskeyID and key posted are for example purposes and not a real account so this code will not copy and paste to execute in Postman.
PS - I learned quite a bit from the other few threads on this topic, which is how I got to this point. akostadinov was super helpful on another thread.
I believe you have double encoded &. I have implemented other Alibaba Cloud REST APIs successfully. Could you please check this.
Following is the expected string to sign format:
GET&%2F&AccessKeyId%3Dtestid&Action%3DDescribeVpcs&Format%3DXML&
SignatureMethod%3DHMAC-SHA1&SignatureNonce%3D3ee8c1b8-83d3-44af-a94f-4e0ad82fd6cf&SignatureVersion%3D1.0&TimeStamp%3D2016-02-23T12%253A46%
253A24Z&Version%3D2014-05-15
A bit late to the party, but as this is the first result when googling for the IncompleteSignature error, I thought I might comment and hopefully save someone else the grief I have been through.
For me, the subtle detail that I missed in the official documentation here is that the key used for the signature requires an ampersand & to be added to the end, before being used.
As soon as I caught that, everything else worked perfectly.
I am trying to integrate QnAmaker knowledge base with Azure Bot Service.
I am unable to find knowledge base id on QnAMaker portal.
How to find the kbid in QnAPortal?
The Knowledge Base Id can be located in Settings under “Deployment details” in your knowledge base. It is the guid that is nestled between “knowledgebases” and “generateAnswer” in the POST (see image below).
Hope of help!
Hey you can also use python to get this by take a look at the following code.
That is if you wanted to write a program to dynamically get the kb ids.
import http.client, os, urllib.parse, json, time, sys
# Represents the various elements used to create HTTP request path for QnA Maker
operations.
# Replace this with a valid subscription key.
# User host = '<your-resource-name>.cognitiveservices.azure.com'
host = '<your-resource-name>.cognitiveservices.azure.com'
subscription_key = '<QnA-Key>'
get_kb_method = '/qnamaker/v4.0/knowledgebases/'
try:
headers = {
'Ocp-Apim-Subscription-Key': subscription_key,
'Content-Type': 'application/json'
}
conn = http.client.HTTPSConnection(host)
conn.request ("GET", get_kb_method, None, headers)
response = conn.getresponse()
data = response.read().decode("UTF-8")
result = None
if len(data) > 0:
result = json.loads(data)
print
#print(json.dumps(result, sort_keys=True, indent=2))
# Note status code 204 means success.
KB_id = result["knowledgebases"][0]["id"]
print(response.status)
print(KB_id)
except :
print ("Unexpected error:", sys.exc_info()[0])
print ("Unexpected error:", sys.exc_info()[1])
TL;DR: What Database.FileFormat constant should I use for an MS Access 2000-2003 database, when creating the Database object?
I have built a SAMBA test application using jCIFS. It allows me to create/overwrite files if given the correct authentication credentials, regardless of on which PC in the domain I use it.
I also have an application that uses uCanAccess/jackcess to connect to an MDB on a network share. However (from what I understand), it uses the credentials of the logged-in user, a number of whom have read-only access. Only system/network administrators have write permission.
The database in question is not password-protected. (I don't need to enter a password when opening it.)
My intention is to have the app ask for the administrator's Samba credentials before it writes to the DB, using those in the uCanAccess connection, so that it doesn't throw a java.nio.channels.NonWritableChannelException, as per the below stack trace:
java.nio.channels.NonWritableChannelException
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:747)
at com.healthmarketscience.jackcess.impl.PageChannel.writePage(PageChannel.java:310)
at com.healthmarketscience.jackcess.impl.PageChannel.writePage(PageChannel.java:247)
at com.healthmarketscience.jackcess.impl.TableImpl.writeDataPage(TableImpl.java:1980)
at com.healthmarketscience.jackcess.impl.TableImpl.addRows(TableImpl.java:2229)
at com.healthmarketscience.jackcess.impl.TableImpl.addRow(TableImpl.java:2067)
at net.ucanaccess.converters.UcanaccessTable.addRow(UcanaccessTable.java:44)
at net.ucanaccess.commands.InsertCommand.insertRow(InsertCommand.java:101)
at net.ucanaccess.commands.InsertCommand.persist(InsertCommand.java:148)
at net.ucanaccess.jdbc.UcanaccessConnection.flushIO(UcanaccessConnection.java:315)
at net.ucanaccess.jdbc.UcanaccessConnection.commit(UcanaccessConnection.java:205)
at net.ucanaccess.jdbc.AbstractExecute.executeBase(AbstractExecute.java:217)
at net.ucanaccess.jdbc.Execute.execute(Execute.java:46)
at net.ucanaccess.jdbc.UcanaccessPreparedStatement.execute(UcanaccessPreparedStatement.java:228)
at myapp.db.Digger.addTransaction(Digger.java:993)
at myapp.tasks.TransactionRunnable.run(TransactionRunnable.java:42)
at java.lang.Thread.run(Thread.java:745)
Update: I have tried using the smbFileChannel class by Gord Thompson and J. T. Alhborn, shown here. My code, based off the main class shown in that answer, looks like this:
// Ask the user for login credentials and the path to the database
String smbURL = (chosenDir.endsWith("/") ? chosenDir : chosenDir + '/')
+ dbName;
System.out.println("DB Path to use for URL: " + smbURL);
URL u = new URL(smbURL);
try (
// construct the SMB DB URL
SmbFileChannel sfc = new SmbFileChannel(smbURL);
Database db = new DatabaseBuilder().setChannel(sfc)
.setFileFormat(Database.FileFormat.GENERIC_JET4).create();
) {
// Model the table
Table tbl = new TableBuilder("Transactions")
.addColumn(new ColumnBuilder("TransactionID", DataType.LONG).setAutoNumber(true))
.addColumn(new ColumnBuilder("ControllerID", DataType.LONG).setAutoNumber(false))
.addColumn(new ColumnBuilder("ReaderID", DataType.LONG).setAutoNumber(false))
.addColumn(new ColumnBuilder("Event", DataType.LONG).setAutoNumber(false))
.addColumn(new ColumnBuilder("Timestamp", DataType.SHORT_DATE_TIME).setAutoNumber(false))
.addColumn(new ColumnBuilder("Number", DataType.LONG).setAutoNumber(false))
.addIndex(new IndexBuilder(IndexBuilder.PRIMARY_KEY_NAME).addColumns("TransactionID").setPrimaryKey())
.toTable(db);
// Add the row
Map<String, Object> values = new HashMap<>();
values.put("ControllerID", cid);
values.put("ReaderID", rid);
values.put("Event", evtNum);
values.put("Timestamp", ts); // Long; must be converted to DataType.SHORT_DATE_TIME
values.put("Number", accNum);
tbl.addRowFromMap(values);
} catch (IOException IOEx) {
System.err.println(
"Failed to write record to Transactions table in database: "
+ IOEx.getMessage()
);
IOEx.printStackTrace(System.err);
} catch (Exception ex) {
System.err.println(
'[' + ex.getClass().getSimpleName() + "]: Failed to write record to "
+ "Transactions table in database: " + ex.getMessage()
);
ex.printStackTrace(System.err);
}
Executing the above code results in the following output:
DB Path to use for URL: smb://machine.vpnName/Storage/me/dbs/DBName.mdb
Failed to write record to Transactions table in database: Logon failure: account currently disabled.
jcifs.smb.SmbAuthException: Logon failure: account currently disabled.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:546)
at jcifs.smb.SmbTransport.send(SmbTransport.java:663)
at jcifs.smb.SmbSession.sessionSetup(SmbSession.java:390)
at jcifs.smb.SmbSession.send(SmbSession.java:218)
at jcifs.smb.SmbTree.treeConnect(SmbTree.java:176)
at jcifs.smb.SmbFile.doConnect(SmbFile.java:911)
at jcifs.smb.SmbFile.connect(SmbFile.java:957)
at jcifs.smb.SmbFile.connect0(SmbFile.java:880)
at jcifs.smb.SmbFile.open0(SmbFile.java:975)
at jcifs.smb.SmbFile.open(SmbFile.java:1009)
at jcifs.smb.SmbRandomAccessFile.<init>(SmbRandomAccessFile.java:57)
at jcifs.smb.SmbRandomAccessFile.<init>(SmbRandomAccessFile.java:42)
at samba.SmbFileChannel.<init>(SmbFileChannel.java:30)
at samba.SambaLanWriteTest.writeTest(SambaLanWriteTest.java:130)
at samba.SambaLanWriteTest.main(SambaLanWriteTest.java:181)
I have write access to a test copy of the database file in question when using Windows File Explorer. I am choosing that one when prompted.
Update 2: I realised that I neglected to add my username and password to the smb:// URL, as Thompson's example shows. I changed to code to this:
String smbCred = "smb://" + auth.getUsername() + ":" + auth.getPassword() + "#",
fixer = chosenDir.replace("\\", "/").replace("smb://", smbCred),
smbURL = fixer + dbName;
System.out.println("DB Path to use for URL: " + smbURL);
// URL u = new URL(smbURL);
The next problem I had was that my password contains special illegal characters (such as '#', ':', ';', '=' and '?'). I escaped these by using java.net.URLEncoder.encode() on auth.getUsername() and auth.getPassword() so the code doesn't throw a MalformedURLException when creating the SmbChannel. However, the next exception I encountered is as follows:
Failed to write record to Transactions table in database: File format GENERIC_JET4 [VERSION_4] does not support file creation for null
java.io.IOException: File format GENERIC_JET4 [VERSION_4] does not support file creation for null
at com.healthmarketscience.jackcess.impl.DatabaseImpl.create(DatabaseImpl.java:444)
What Database.FileFormat constant should I use for an MS Access 2000-2003 database, when creating the Database object?
It turns out that I needed to use Database.FileFormat.V2000.
After that, it was all plain sailing (although I still need to work out how to get the Long timestamp to convert correctly).
here is a part of my perl cgi script (which is working..):
use Net::LDAP;
use Net::LDAP::Entry;
...
$edn = "DC=xyz,DC=com";
$quser ="(&(objectClass=user)(cn=$username))";
$ad = Net::LDAP->new("ip_address...");
$ldap_msg=$ad->bind("$username\#xyz.com", password=>$password);
my $result = $ad->search( base=>$edn,
scope=>"sub",
filter=>$quser);
my $entry;
my $myname;
my $emailad;
my #entries = $result->entries;
foreach $entry (#entries) {
$myname = $entry->get_value("givenName");
$emailad = $entry->get_value("mail");
}
So basically, there is no admin/manager account for AD, users credentials are used for binding. I need to implement the same thing in grails..
+Is there a way to configure the plugin to search several ADs, I know I can add more ldap IPs in context.server but for each server I need a different search base...
++ I dont wanna use my DB, just AD. User logins through ldap > I get his email, and use the email for another ldap query but that will probably be another topic :)
Anyway the code so far is:
grails.plugin.springsecurity.ldap.context.managerDn = ''
grails.plugin.springsecurity.ldap.context.managerPassword = ''
grails.plugin.springsecurity.ldap.context.server = 'ldap://address:389'
grails.plugin.springsecurity.ldap.authorities.ignorePartialResultException = true
grails.plugin.springsecurity.ldap.search.base = 'DC=xyz,DC=com'
grails.plugin.springsecurity.ldap.authenticator.useBind=true
grails.plugin.springsecurity.ldap.authorities.retrieveDatabaseRoles = false
grails.plugin.springsecurity.ldap.search.filter="sAMAccountName={0}"
grails.plugin.springsecurity.ldap.search.searchSubtree = true
grails.plugin.springsecurity.ldap.auth.hideUserNotFoundExceptions = false
grails.plugin.springsecurity.ldap.search.attributesToReturn =
['mail', 'givenName']
grails.plugin.springsecurity.providerNames=
['ldapAuthProvider',anonymousAuthenticationProvider']
grails.plugin.springsecurity.ldap.useRememberMe = false
grails.plugin.springsecurity.ldap.authorities.retrieveGroupRoles = false
grails.plugin.springsecurity.ldap.authorities.groupSearchBase ='DC=xyz,DC=com'
grails.plugin.springsecurity.ldap.authorities.groupSearchFilter = 'member={0}'
And the error code is: [LDAP: error code 1 - 000004DC: LdapErr: DSID-0C0906E8, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, v1db1
And it's the same code for any user/pass I try :/
Heeeeelp! :)
The most important thing with grails and AD is to use ActiveDirectoryLdapAuthenticationProvider rather than LdapAuthenticationProvider as it will save a world of pain. To do this, just make the following changes:
In resources.groovy:
// Domain 1
ldapAuthProvider1(ActiveDirectoryLdapAuthenticationProvider,
"mydomain.com",
"ldap://mydomain.com/"
)
// Domain 2
ldapAuthProvider2(ActiveDirectoryLdapAuthenticationProvider,
"mydomain2.com",
"ldap://mydomain2.com/"
)
In Config.groovy:
grails.plugin.springsecurity.providerNames = ['ldapAuthProvider1', 'ldapAuthProvider2']
This is all the code you need. You can pretty much remove all other grails.plugin.springsecurity.ldap.* settings in Config.groovy as they don't apply to this AD setup.
Documentation:
http://docs.spring.io/spring-security/site/docs/3.1.x/reference/springsecurity-single.html#ldap-active-directory