I have an OpenDaylight controller set up as the mechanism driver in an OpenStack cloud. The installation was done with DevStack in an all-in-one virtual machine.
Everything is working fine. I have two VM in my OpenStack cloud and they can ping each other. Then I want to add a new flow to my Open vSwitch, I have the following error:
curl -u admin:admin -H 'Content-Type: application/yang.data+xml' -X PUT -d #flow_data.xml http://192.168.100.100:8181/restconf/config/opendaylight-inventory:nodes/node/openflow:1/table/234/flow/100770 | python -m json.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1347 0 638 100 709 42078 46761 --:--:-- --:--:-- --:--:-- 50642
{
"errors": {
"error": [
{
"error-info": "Data from case (urn:opendaylight:flow:inventory?revision=2013-08-19)output-action-case are specified but other data from case (urn:opendaylight:flow:inventory?revision=2013-08-19)drop-action-case were specified earlier. Data aren't from the same case.",
"error-message": "Error parsing input: Data from case (urn:opendaylight:flow:inventory?revision=2013-08-19)output-action-case are specified but other data from case (urn:opendaylight:flow:inventory?revision=2013-08-19)drop-action-case were specified earlier. Data aren't from the same case.",
"error-tag": "malformed-message",
"error-type": "protocol"
}
]
}
}
flow_data.xml :
<?xml version="1.0"?>
<flow xmlns="urn:opendaylight:flow:inventory">
<priority>14865</priority>
<flow-name>jpsampleFlow</flow-name>
<idle-timeout>12000</idle-timeout>
<match>
<ethernet-match>
<ethernet-type>
<type>2048</type>
</ethernet-type>
</ethernet-match>
<ipv4-source>10.0.0.1/32</ipv4-source>
<ipv4-destination>10.0.0.2/32</ipv4-destination>
<ip-match>
<ip-dscp>28</ip-dscp>
</ip-match>
</match>
<id>9</id>
<table_id>0</table_id>
<instructions>
<instruction>
<order>6555</order>
</instruction>
<instruction>
<order>0</order>
<apply-actions>
<action>
<order>0</order>
<drop-action/>
<output-action>
<output-node-connector>1</output-node-connector>
</output-action>
</action>
</apply-actions>
</instruction>
</instructions>
</flow>
Any idea what am I doing wrong ? Thank you.
you have one action to drop and another action to forward. aren't
those mutually exclusive? try without one or the other.
Related
I tried adding service principal to azure databricks workspace using cloud shell but getting error. I am able to look at all the clusters in the work space and I was the one who created that workspace. Do I need to be in admin group if I want to add Service Principal to workspace?
curl --netrc -X POST \ https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.net/api/2.0/preview/scim/v2/ServicePrincipals \ --header 'Content-type: application/scim+json' \ --data #create-service-principal.json \ | jq .
file has following info:
{ "displayName": "sp-name", "applicationId": "a9217fxxxxcd-9ab8-dxxxxxxxxxxxxx", "entitlements": [ { "value": "allow-cluster-create" } ], "schemas": [ "urn:ietf:params:scim:schemas:core:2.0:ServicePrincipal" ], "active": true }
Here is the error I am getting: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 279 100 279 0 0 5166 0 --:--:-- --:--:-- --:--:-- 5264 parse error: Invalid numeric literal at line 2, column 0
Do I need to be in admin group if I want to add Service Principal to workspace?
Issue is with JSON file not with access to admin group.
You need to check double quotes in line number 2 of your JSON file.
You can refer this github link
Try this code in Python that you can run in a Databricks notebook:
import pandas
import json
import requests
# COMMAND ----------
# MAGIC %md ### define variables
# COMMAND ----------
pat = 'EnterPATHere' # paste PAT. Get it from settings > user settings
workspaceURL = 'EnterWorkspaceURLHere' # paste the workspace url in the format of 'https://adb-1234567.89.azuredatabricks.net'
applicationID = 'EnterApplicationIDHere' # paste ApplicationID / ClientID of Service Principal. Get it from Azure Portal
friendlyName = 'AddFriendlyNameHere' # paste FriendlyName of ServicePrincipal. Get it from Azure Portal
# COMMAND ----------
# MAGIC %md ### add service principal
# COMMAND ----------
payload_raw = {
'schemas':
['urn:ietf:params:scim:schemas:core:2.0:ServicePrincipal'],
'applicationId': applicationID,
'displayName': friendlyName,
'groups':[],
'entitlements':[]
}
payload = json.loads(json.dumps(payload_raw))
response = requests.post(workspaceURL + '/api/2.0/preview/scim/v2/ServicePrincipals',\
headers = {'Authorization' : 'Bearer '+ pat,\
'Content-Type': 'application/scim+json'},\
data=json.dumps(payload))
response.content
I have actually published a blog post where a Python script is provided to fully manage service principals and access control in Databricks workspaces.
I am using requests library to connect to elasticsearch for fetching data. I have
26 indices,
spread across 2 nodes,
with 1st node having 16GB RAM / 8 vCPU and the
2nd 8GB RAM / 4 vCPU.
All my nodes are in AWS EC2.
In all I have around 200 GB of data. I am primarily using the database for aggregation exercises.
A typical data record would look like this
SITE_ID DATE HOUR MAID
123 2021-05-05 16 m434
I am using the following python3 definition to send the request and get the data.
def elasticsearch_curl(uri, json_body='',verb='get'):
headers={'Content-Type': 'application/json',}
try:
resp = requests.get(uri, headers=headers, data=json_body)
try:
resp_text = json.loads(resp.text)
except:
print("Error")
except Exception as error:
print('\nelasticsearch_curl() error:', error)
return resp_text
##Variables
tabsite : ['r123','r234'] ##names of indices
siteid : [123,124,125] ##name of sites
I am using the following code to get the data:
for key,value in tabsite.items():
k=key.replace('_','')
if es.indices.exists(index=k):
url="http://localhost:9200/"+str(k)+"/_search"
jb1='{"size":0,"query": {"bool" : {"filter" : [{"terms" : {"site_id": ' + str(siteid) + '}},{"range" : {"date" : \
{"gte": "'+str(st)+'","lte": "'+str(ed)+'"}}}]}}, "aggs" : {"group_by" : {"terms": {"field": "site_id","size":100},"aggs" : {"bydate" : {"terms" : \
{"field":"date","size": 10000},"aggs" : {"uv" : {"cardinality": {"field": "maid"}}}}}}}}'
try:
r2=elasticsearch_curl(url, json_body=jb1)
k1=r2.get('aggregations',{}).get('group_by',{}).get('buckets')
print(k1)
except:
pass
The above code returns the data from r123 which has 18GB of data while it fails to get it from r234 which has 55 GB of data.
I am getting the following error:
elasticsearch_curl() error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
I have tried the following:
Try running the above code in a machine which has only r234 index with around 45GB of data. It worked.
I tried increasing the RAM size of the 2nd machine in production from 8GB to 16GB - it failed.
When I searched for options here, I understood I need to close the headers. I am not sure how.
I have the following questions:
How do I keep my elasticsearch nodes stable without getting them shutdown automatically?
How do I get rid of the above error which shuts down one of the nodes or both.
Is there any optimimal configuration setting ratio for volume of data : number of nodes : amount of RAM / vCPUs.
I'm new to Elastic Search and I'm having an index in red-state due to can't assign shard 0 error.
I found a way to get the explanation but I'm still lost on understanding and fixing it. The server's version is 7.5.2.
curl -XGET 'http://localhost:9200/_cluster/allocation/explain' returns
{
"index":"event_tracking",
"shard":0,
"primary":false,
"current_state":"unassigned",
"unassigned_info":{
"reason":"CLUSTER_RECOVERED",
"at":"2020-12-22T14:51:08.943Z",
"last_allocation_status":"no_attempt"
},
"can_allocate":"no",
"allocate_explanation":"cannot allocate because allocation is not permitted to any of the nodes",
"node_allocation_decisions":[
{
"node_id":"cfsLU-nnRTGQG1loc4hdVA",
"node_name":"xxx-clustername",
"transport_address":"127.0.0.1:9300",
"node_attributes":{
"ml.machine_memory":"7992242176",
"xpack.installed":"true",
"ml.max_open_jobs":"20"
},
"node_decision":"no",
"deciders":[
{
"decider":"replica_after_primary_active",
"decision":"NO",
"explanation":"primary shard for this replica is not yet active"
},
{
"decider":"same_shard",
"decision":"NO",
"explanation":"the shard cannot be allocated to the same node on which a copy of the shard already exists [[event_tracking][0], node[cfsLU-nnRTGQG1loc4hdVA], [P], recovery_source[existing store recovery; bootstrap_history_uuid=false], s[INITIALIZING], a[id=TObxz0EFQbylZsyTiIH7SA], unassigned_info[[reason=CLUSTER_RECOVERED], at[2020-12-22T14:51:08.943Z], delayed=false, allocation_status[fetching_shard_data]]]"
},
{
"decider":"throttling",
"decision":"NO",
"explanation":"primary shard for this replica is not yet active"
}
]
}
]
}
I, more or less, understand the error message but I can't find the proper way to fix it. This server is not running on Docker, it's directly installed in the Linux machine.
curl -XGET 'http://localhost:9200/_cat/recovery/event_tracking?v' result
index shard time type stage source_host source_node target_host target_node repository snapshot files files_recovered files_percent files_total bytes bytes_recovered bytes_percent bytes_total translog_ops translog_ops_recovered translog_ops_percent
event_tracking 0 54.5m existing_store translog n/a n/a 127.0.0.1 xxx-cluster n/a n/a 0 0 100.0% 106 0 0 100.0% 2857898852 7061000 6489585 91.9%
What can I try to resolve this?
I have 2 data nodes and 3 master nodes in an ES cluster. I was doing a rolling upgrade as ES suggested moving from 5.6.10 to 6.8.10.
As there should be zero downtime, I was testing that and getting one error.
I have upgraded the 1 data node and do basic search testing. It is working fine. When I have upgraded 2nd node search is breaking with the below Error.
java.lang.IllegalArgumentException: Top hits result window is too large, the top hits aggregator [top]'s from + size must be less than or equal to: [100] but was [999]. This limit can be set by changing the [index.max_inner_result_window] index level setting.
index.max_inner_result_window -- This property was introduced in the 6.X version, and the master node is still on 5.6.10. So what will be the solution with 0 downtimes?
Note: My indexing is stopped completely. My 2 data nodes are now on 6.8.10 and master nodes are on 5.6.
Thanks
1 - Change the parameter on current indexes:
curl -X PUT "http://localhost:9200/_all/_settings?pretty" -H 'Content-Type: application/json' -d'
{
"index.max_inner_result_window": "2147483647"
}
'
2 - Create a template to further indexes:
curl -X PUT "http://localhost:9200/_index_template/template_max_inner_result?pretty" -H 'Content-Type: application/json' -d'
{
"index_patterns": ["*"],
"template": {
"settings": {
"index":{
"max_inner_result_window": 2147483647
}
}
}
}
'
I am trying to use a Raspberry Pi to poll the interface MIB (IF:MIB) of a TP-LINK router and then send the metrics to Librato.
Setting up collectd and integrating it with Librato is no problem at all - I am successfully tracking other metrics (cpu, memory, etc.). The challenge I have is with the collectd-snmp plugin configuration.
I installed net-snmp and can "see" the router:
pi#raspberrypi ~ $ snmpwalk -v 1 -c public 192.168.0.1 IF-MIB::ifInOctets
IF-MIB::ifInOctets.2 = Counter32: 1206812646
IF-MIB::ifInOctets.3 = Counter32: 1548296842
IF-MIB::ifInOctets.5 = Counter32: 19701783
IF-MIB::ifInOctets.10 = Counter32: 0
IF-MIB::ifInOctets.11 = Counter32: 0
IF-MIB::ifInOctets.15 = Counter32: 0
IF-MIB::ifInOctets.16 = Counter32: 0
IF-MIB::ifInOctets.22 = Counter32: 0
IF-MIB::ifInOctets.23 = Counter32: 0
The Pi is on 192.168.0.20, the router on 192.168.0.1.
My collectd.conf is as follows:
<Plugin snmp>
<Data "ifmib_if_octets32">
Type "if_octets"
Table true
Instance "IF-MIB::ifDescr"
Values "IF-MIB::ifInOctets" "IF-MIB::ifOutOctets"
</Data>
<Host "localhost">
Address "192.168.0.1"
Version 1
Community "public"
Collect "ifmib_if_octets32"
Interval 60
</Host>
</Plugin>
When I restart collectd I get the following error:
pi#raspberrypi ~ $ sudo service collectd restart
[....] Restarting statistics collection and monitoring daemon: collectdNo log handling enabled - turning on stderr logging
MIB search path: $HOME/.snmp/mibs:/usr/share/mibs/site:/usr/share/snmp/mibs:/usr/share/mibs/iana:/usr/share/mibs/ietf:/usr/share/mibs/netsnmp
Cannot find module (IF-MIB): At line 0 in (none)
[2015-01-24 23:01:31] snmp plugin: read_objid (IF-MIB::ifDescr) failed.
[2015-01-24 23:01:31] snmp plugin: No such data configured: `ifmib_if_octets32'
No log handling enabled - turning on stderr logging
MIB search path: $HOME/.snmp/mibs:/usr/share/mibs/site:/usr/share/snmp/mibs:/usr/share/mibs/iana:/usr/share/mibs/ietf:/usr/share/mibs/netsnmp
Cannot find module (IF-MIB): At line 0 in (none)
[2015-01-24 23:01:33] snmp plugin: read_objid (IF-MIB::ifDescr) failed.
[2015-01-24 23:01:33] snmp plugin: No such data configured: `ifmib_if_octets32'
. ok
It obviously can't find the MIB, it doesn't even seem to be looking at the router's IP. Any suggestions on how to configure this correctly?
I figured it out:
<Plugin snmp>
<Data "if_Octets">
Type "if_octets"
Table true
Values "IF-MIB::ifInOctets" "IF-MIB::ifOutOctets"
</Data>
<Host "tp-link">
Address "192.168.0.1"
Version 1
Community "public"
Collect "if_Octets"
Interval 60
</Host>
</Plugin>