I am getting an error on writing a script in an attempt to get data into influxDB. The following creates executeScript syntax error which seems to work fine sending the line through the API.
test_measurement,inputType=json
aID="102185",cType="text",pub="2018-07-24
20:17:13Z",fetch="2018-07-24 16:17:23-04:00" 1532541609
Inserted the following script in nifi ExecuteScript but it keeps throwing the following error:
Failed to process session due to javax.script.ScriptException: <eval> Missing space after numeric literal line = line + inputType="json" + " " + ",aID="2312321" + ",cType="text" + ",pub="2018-07-24 13:09:12" + ",fetch="2018-07-24 13:20:02" + " " + Date.now() * 1000000;
Any help or thoughts please.
Related
I'm quite stumped here. I'm using pandas version 1.3.5 and am trying to replace an excel tab with new data from a dataframe. Seems extremely simple using the below:
Workbook = load_workbook(file_path + '\\' + file_nm + '.xlsx')
writer = pd.ExcelWriter(file_path + '\\' + file_nm + '.xlsx', mode="a", engine='openpyxl',if_sheet_exists='replace')
writer.book = Workbook
df.to_excel(writer,sheet_name='Positions_auto', index=False)*
However, when trying to replace I'm getting the below error:
ValueError: <Worksheet "Positions_auto"> is not in list
I'm googled far and wide and haven't seen similar issues with to_excel.
Any thoughts? I have no clue why it's not working as the documents say.
I feel like i've tried everything
Im using SlackAPIPostOperator in Airflow to send Slack messages on task failures.
I wondered if there's a smart way to add a link to the airflow UI logs page of the failed task to the slack message.
The following is an example I want to achieve:
http://myserver-uw1.myaws.com:8080/admin/airflow/graph?execution_date=...&arrange=LR&root=&dag_id=MyDAG&_csrf_token=mytoken
The current message is:
def slack_failed_task(context):
failed_alert = SlackAPIPostOperator(
task_id='slack_failed',
channel="#mychannel",
token="...",
text=':red_circle: Failure on: ' +
str(context['dag']) +
'\nRun ID: ' + str(context['run_id']) +
'\nTask: ' + str(context['task_instance']))
return failed_alert.execute(context=context)
You can build the url to the UI with the config value base_url under the [webserver] section and then use Slack's message format <http://example.com|stuff> for links.
from airflow import configuration
def slack_failed_task(context):
link = '<{base_url}/admin/airflow/log?dag_id={dag_id}&task_id={task_id}&execution_date={execution_date}|logs>'.format(
base_url=configuration.get('webserver', 'BASE_URL'),
dag_id=context['dag'].dag_id,
task_id=context['task_instance'].task_id,
execution_date=context['ts'])) # equal to context['execution_date'].isoformat())
failed_alert = SlackAPIPostOperator(
task_id='slack_failed',
channel="#mychannel",
token="...",
text=':red_circle: Failure on: ' +
str(context['dag']) +
'\nRun ID: ' + str(context['run_id']) +
'\nTask: ' + str(context['task_instance']) +
'\nSee ' + link + ' to debug')
return failed_alert.execute(context=context)
We can also do this using the log_url attribute in the Task Instance
def slack_failed_task(context):
failed_alert = SlackAPIPostOperator(
task_id='slack_failed',
channel="#mychannel",
token="...",
text=':red_circle: Failure on: ' +
str(context['dag']) +
'\nRun ID: ' + str(context['run_id']) +
'\nTask: ' + str(context['task_instance']) +
'\nLogs: <{url}|to Airflow UI>'.format(url=context['task_instance'].log_url)
)
return failed_alert.execute(context=context)
I know this is available since version 1.10.4 at the very least.
I have a Jython script calling wsadmin libraries to configure a WAS server.
I have these functions:
def createWasObject(was_object_type, was_path, object_params):
if isinstance(was_path, basestring):
was_path = AdminConfig.getid(was_path)
str_params = '['
for k,v in object_params.items():
str_params = str_params + '[' + k + ' "' + v + '"] '
str_params = str_params + ']'
return AdminConfig.create(was_object_type, was_path, str_params)
def createJdbcProviders(was_path, jdbc_providers):
was_object_type = 'JDBCProvider'
for jdbc_provider in jdbc_providers:
jdbc = createWasObject(was_object_type, was_path, jdbc_provider['params'])
print jdbc
for datasource in jdbc_provider['datasources']:
ds = createWasObject('Datasource', jdbc, datasource['params'])
print
The "print jdbc" prints:
Teradata JDBC Provider(cells/jsr-websphere-1Cell01/nodes/jsr-websphere-1Node01/servers/jsr-business|resources.xml#JDBCProvider_1444648929602)"
Which looks like a correct object ID
However, when using it to create a datasource, I get the following error:
WASX7017E: Exception reçue lors de l'exécution du fichier "/root/jsr_auto_deployment/jsr.py" ; informations sur l'exception : com.ibm.ws.scripting.ScriptingException: Invalid object name: "Teradata JDBC Provider(cells/jsr-websphere-1Cell01/nodes/jsr-websphere-1Node01/servers/jsr-business|resources.xml#JDBCProvider_1444648929602)"
I am using Jython 2.7 through a Thin client. Reusing an object returned by AdminConfig.create() was working well with a Jython script ran through wsadmin.sh
My problem was on that line:
was_path = AdminConfig.getid(was_path)
Most of the time I was passing a string but this time I was already using an ID. So I removed the AdminConfig.getid in the function and added it in the calls when necessary only.
I have a log file which generates following data:
2015-07-06 11:07:24 +0522 [ERROR]
2015-07-06 11:07:29 +0522 [ERROR] index=healthe-int-legacy host=kdatamap.abc.com com.rp.keplar.collector.CollectorException: Could not process additional data, connection lost to data collector service
I want to store data in different section like date, time, index value and error related information like 'Could not process additional data, connection lost to data collector service' into database. How to parse so that I can easily store in DB? Please guide me.
You want to read up the manual on the really powerful File and String classes.
Consider this rather quick hack:
aFile = File.new("/your/file.dat")
aFile.each_line { |line|
arr = line.split
print "date = " + arr[0] + "\n"
print "time = " + arr[1] + "\n"
print "index = " + arr[4].split('=')[1]
}
It does not take into accout that the file might not exist or that the lines might be aligned differently. Have a look at '''regular''' expressions for implementation of a more robust (but unfortunately more difficult to read) matching algorighm.
Basic I/O is described at http://ruby-doc.com/docs/ProgrammingRuby/html/tut_io.html.
I have a jython script to create a server, deploy and application and then start the server.
But I am geeting the following exception while running it.
WASX7017E: Exception received while running file "myfile.py"; exception information: javax.management.MBeanException[[ com.ibm.websphere.management.exception.AdminException: Server, SERVERNAME, not found.
Here is the entire code... http://snipt.org/BMaf4
Update: He is the entire log http://snipt.org/BNZ1
Can't figure out where I am going wrong.
But When I am issuing a start all servers in wsadminlib.. the server gets started
The problem might be in the way how you deploying and/or configuring your application.
# Your script sample
# Deploy the WAR
APP_PATH= APP_HOME + '/MYAPP/CycleWAR/war/MYAPPCycle.war'
ARGS_LIST='.....'
AdminApp.install(APP_PATH, ARGS_LIST)
You need to add another parameter into your arguments list ARGS_LIST : args = "[-server " + SERVER_NAME + "]"
# Since 6.0.X version
# Server Deployment
args = "[-server " + serverName + "]"
# Cluster Deployment
# args = "[-cluster " + clusterName + "]"
AdminApp.install(applicationFilePath, args)
After the server configurations...
#Sync the Nodes
Sync1 = AdminControl.completeObjectName( "type=NodeSync,node="+ NODE_NAME +",*")
print "Getting Sync Info.. " + "\n" + Sync1
AdminControl.invoke(Sync1, 'sync')
print NODE_NAME + " Sync Completed.. "
We have to sync the nodes first before trying to start the server.