I am using Airflow for a few weeks now and I use a lot xcoms to send informations from one task to another.
I just figured out that these xcoms ( as the return values of the Python Operators tasks) are accessible directly from the web UI.
However, I send data like credentials, etc that I don't want to be displayed in the web UI of Airflow.
I have tried to look into the airflow config file but the only option that can be changed is the possibility to pickle xcoms.
Does anyone have a solution to my problem ?
Thank you
for data credentails, you can do it in different mode based on the security strategy.
hide the xcom menu and pages, you can change the app.py in www folder. just comments the xcom related parts. it should be 1-3 line of codes.
encrypt the xcom raw message, change you code when push or pull messages.
delete the historical xcom messages as above metioned.
If the credentials are static, then add it in variables and call them via Variables by importing from airflow.models import Variable. The other option that comes to my mind is adding a PostgresOperator to delete the XCOM once the dag is completed, something like below,
delete_xcom = PostgresOperator(
task_id='task_id',
postgres_conn_id='airflow_db',
sql="delete from xcom where dag_id=dag.dag_id and
task_id='your_task_id' and execution_date={{ ds }}",
dag=dag)
Related
I have an ORACLE 12c form which displays a list of files stored in the database. The form allows a user to import and export these files using webutils. It all works fine, but if I change the form to automatically export a file on startup (using exactly the same code as a user would use if they pressed the Export button, but without having the user to press anything), the form complains that it can't find webutils. What is causing this behaviour?
Since I posted this question, I ran across two sites which said that you can't call webutils via WNFI unless you use a timer. I am yet to try this at work, but it certainly sounds like the way to go.
I have been crawling through various forums and blogs for an AuthorizationPlugin example or understanding which can show me how to create a mac authorization plugin that do not affect any UI components. I want to use it for a remote access kind of solution. I have been able to get NameAndPasswordPlugin example work. But I am not able to achieve below requirements:
Do not change the default UI. i.e not have any custom UI components
Ability to read and write into default UI fields, especially username (if any) and password
Work on need basis. i.e. I need the mechanism to pass through when remote access session is not ON. In that case I want it to fall back to loginwindow:login mechanism
Also how would it communicate with outside world ? I was not able to read or write into files from plugin. I saw an example where some pipes where used. not sure what the recommended method
You don't need a SFAuthorizationPluginView, you just need an authorization plugin. You insert your plugin into the list of plugins and it can read from contexts set by previous plugins and write to or create contexts for later plugins.
For example, if you are working with console login this bash command shows you what mechanisms are configured (mechanisms are instances of a plugin)
security authorizationdb read system.login.console
If you add your plugin after builtin:authenticate,privileged then you can use this code in your mechanismInvoke function to read the values.
err = mechanism->fPlugin->fCallbacks->GetHintValue(mechanism->fEngine, "username", &value);
if (err == noErr) {
//Log the event
os_log(OS_LOG_DEBUG, "Login for user '%{public}s'.",(const char *)value->data);
}
where mechanism->fPlugin->fCallbacks->GetHintValue and mechanism->fEngine are the callback and engineref you setup as part of your plugin. There is also a "SetContextValue" function for writing the username or password.
You will need to write an authorization plugin which will set the context values "username" (kAuthorizationEnvironmentUsername) and "password" (kAuthorizationEnvironmentPassword). Then set result as kAuthorizationResultAllow. You would also need to place your plugin just before loginwindow:login.
I currently build web app which is using external MongoDb via Mongolabs.
The api is based on personal key using in urls. As docs says e.g.:
Here’s an example of a complete Resource URL:
https://api.mongolab.com/api/1/databases?apiKey=**2E81PUmPFI84t7UIc_5YdldAp1ruUPKye**
So the question is how to securely store such api key 2E81PUmPFI84t7UIc_5YdldAp1ruUPKye
Reading Django docs about Cross Site Request Forgery but stil do not understand where the key is recorded.
There are two ways to do this.
One way is to have a local_settings.py file that's imported in the main settings.py file and put into .gitignore so it's not in git. Some people however think this isn't good practice, because it might tempt to put complex things in there that aren't in VCS, so people effectively have different environments. I however am fine with it.
try:
from local_settings import *
except ImportError:
pass # No local_settings file
The other way (recommended by dislikers of the first way) is by setting it via environment variables, and reading these in settings.py.
MONGO_API_KEY = os.environ['MONGO_API_KEY']
You'd then have to pass the environment variable somehow though. E.g. via uwsgi's environ setting, or by setting it in your bash with export, or via another way.
I would load it in the settings file from an environment variable. Have a look at the Django Settings
One alternative is to use the library django-fernet-fields that uses the library cryptography.
The usage is very simple. In your model you need to add a new field:
from django.db import models
from fernet_fields import EncryptedTextField
class MyModel(models.Model):
apikey = EncryptedTextField()
By default, the field is going to be encrypted using the SECRET_KEY from your settings. So if you change it or lose it, you will not be able to access your data.
For better security, you can save your SECRET_KEY as an environment variable, and then pass it to the settings file.
import os
SECRET_KEY = os.environ.get('APP_SECRET_KEY', 'unsafe-secret-key')
django-fernet-fields
Quick answer:
Store in .env
Read in settings.py
After a research it seems that Meteor Sessions are reset after refreshing page or opening the website in new tab, i.e. they are not usual server-side sessions but something like global javascript variables on client-side. Some people advice to use AmplifyJS, but I'm not sure that it will work like usual session in other frameworks/languages and also it is a third party library, so is there any normal way to use sessions in Meteor, i.e. keep user-specific data on server?
At this moment I'm handling that by using custom Collections, but it is not an ideal way of doing that because it is needed to remove expired values from Collection manually, which makes additional troubles.
Yes this is correct. Despite the name Session is nothing like a cookie, but just a reactive form of a variable stored in a hashmap
To keep data persistent across tabs you need to use a Collections (as this is the only way to reactively share data across tabs) - Cookies can't work because they can't be made reactive as data needs to be sent to the server to notify the client when there is a change. There really wouldn't be another way at the moment as the publish/subscribe methods can only send down data from collections at the moment.
You can use your setup you have now with your custom collection. You can use a server side cron job to remove expired data (either with Meteor.setInterval or Tom Coleman's cron.
There is a package developed just for that: https://atmospherejs.com/u2622/persistent-session
After installation you can use the following functions to set sessions which are persistent:
//store a persistent session variable which is stored across templates
Session.setPersistent(key, value);
//same as above, but automatically deletes session data when user logs out
Session.setAuth(key, value);
I've tried the package and it works like charm.
I would like to have a workflow create a task, then email the assigned user that they have a new task and include a link to the newly created task in the body of the email. I have client side code that will correctly create the edit URL, using the entities GUID and stores it in a custom attribute. However, when the task is created from within a workflow, the client script isn't run.
So, I think a plug-in should work, but I can't figure out how to determine the URL of the CRM installation. I'm authoring this in a test environment and definitely don't want to have to change things when I move to production. I'm sure I could use a config file, but seems like the plug-in should be able to figure this out at runtime.
Anyone have any ideas how to access the URL of the crm service from within a plug-in? Any other ideas?
There is no simple way to do this. However, there is one.
The MSCRM_Config is the deployment database that handle physical deployment properties, like the URL from which users are accessing the CRM deployment. The url that you might want is the one stored in "ADWebApplicationRootDomain", in the MSCRM_CONFIG.dbo.DeploymentProperties table. You may need some permission to access this database.
Note that this doesn't work in a deployment that is an Internet Facing Deployment.
Another way could be to query the discovery service to retrieve the same information (in the case that you are on the Online edition of MSCRM4).
What do you mean by "change things"?
If you create a custom workflow assembly, you can give it a server url input. Once you register it with CRM, you can simply type in the server url when you configure the workflow. You'll have to update the url for any workflows that use the custom workflow assembly once you move to production, but you'll only have to do that once.
My apologies if this is what you meant you wanted to avoid.
Edit: Sounds like you may be able to use the CustomConfiguration attribute when you register the plugin. Here's some more info.
http://blogs.msdn.com/crm/archive/2008/10/24/storing-configuration-data-for-microsoft-dynamics-crm-plug-ins.aspx
String Url = ((string)(Registry.LocalMachine.OpenSubKey(
"Software\\Microsoft\\MSCRM").GetValue("ServerUrl"))
).Replace("MSCRMServices", "");