SNMP - Decode Hex String Value - snmp

This is my first question here, so hope it's correctly done.
Im trying to get some information from a ZTE C300 OLT.
The thing is when i try to get the SN of one of the ONTS I get the response in HEX-String
snmpwalk -cpublic -v2c [OLTIP] 1.3.6.1.4.1.3902.1082.500.10.2.2.5.1.2
And this is the response that I get
SNMPv2-SMI::enterprises.3902.1082.500.10.2.2.5.1.2.285278736.1 = Hex-STRING: 5A 54 45 47 C8 79 9B 27
This is the SN that i have on the OLT ZTEGC8799B27, but im trying to convert the HEX-STRING into text and i don't get that SN text.
Indeed i have a python script for SNMP and the response that i get for that OID is
{'1.3.6.1.4.1.3902.1082.500.10.2.2.5.1.2.285278736.1': "ZTEGÈy\x9b'"}
Can someone give me a hand on this?. I'm new on SNMP and this is giving me some headache.
Thanks in advace!

This is a 8 octet hex string, the first 4 octets are ASCII.
Just convert hex 2 ascii.

Indeed it was easier. The firts 4 bytes were encoded, and the other 4 is the actual serial number splitted every 2 digits. So i only need to decode the first part and concatenate the rest.

Works with OLT ZTE C320
def hex_str(str):
str = str.strip()
str = str.split(' ')
vendor_id = ''
serial = str[4:]
serial = "".join(serial)
for hex_byte in str[:4]:
vendor_id += chr(int(hex_byte, 16))
normalized_serial = vendor_id + serial
return normalized_serial
def ascii_to_hex(str):
arr = []
hex_byte = ''
for i in range(len(str)):
hex_byte += hex(ord(str[i]))
hex_byte = hex_byte.replace('0x', ' ')
hex_byte = hex_str(hex_byte)
return hex_byte
# value = f"5A 54 45 47 C8 79 9B 27 "
# value = f"49 54 42 53 8B 69 A2 45 "
# value = f"ZTEGÈy\x9b'"
value = f"ITBS2Lz/"
# value = f"ITBS2HP#"
if(len(value) == 24):
print(hex_str(value))
else:
print(ascii_to_hex(value))

Related

removing bad data from a data file using pig

I have a data file like this
1943 49 1
1975 91 L
1903 56 3
1909 52 3
1953 96 3
1912 82
1976 66 3
1913 35
1990 45 1
1927 92 A
1912 2
1924 22
1971 2
1959 94 E
now using pig script I want to remove the bad data like removing those rows which have characters and empty fields
I tried this way
records = load '/user/a106524609/test.txt' using PigStorage(' ') as
(year:chararray, temperature:int, quality:int);
rec1 = filter records by temperature != 'null' and (quality != 'null ')
Load it as lines
A = load 'data.txt' using PigStorage('\n') as (line:chararray);
Split on all whitespaces
B = FOREACH A GENERATE FLATTEN(STRSPLIT(line, '\\s+')) as (year:int,temp:int,quality:chararray);
Filter by valid strings
C = FILTER B BY quality IN ('0','1','2','3','4','5','6','7','8','9');
(Optionally) Cast to an int
D = FOREACH C GENERATE year,temp,(int)quality;
In Spark, I would start with a regex match of the expected format.
val cleanRows = sc.textFile("data.txt")
.filter(line => line.matches("(?:\\d+\\s+){2}\\d+"))

How to use StudentT distribution in pymc3?

I'm not sure whether this counts as a question or a bug report. I posted a GitHub gist here: https://gist.github.com/jbwhit/a9012e04b0f48e582c22
I found this question (pymc3: hierarchical model with multiple obsesrved variables) to be an excellent starting point for my own hierarchical model, but ran into difficulties as soon as I tried to modify it in any substantial way.
First, the model and setup that works:
import numpy as np
import pymc3 as pm
n_individuals = 200
points_per_individual = 10
means = np.random.normal(30, 12, n_individuals)
observed = np.random.normal(means, 1, (points_per_individual, n_individuals))
model = pm.Model()
with model:
hyper_mean = pm.Normal('hyper_mean', mu=0, sd=100)
hyper_sigma = pm.HalfNormal('hyper_sigma', sd=3)
means = pm.Normal('means', mu=hyper_mean, sd=hyper_sigma, shape=n_individuals)
sigmas = pm.HalfNormal('sigmas', sd=100)
ye = pm.Normal('ye', mu=means, sd=sigmas, observed=observed)
trace = pm.sample(10000)
All of the above works as expected (and the traces look nice). The next piece of code makes one change (swapping a T distribution for the Normal):
model = pm.Model()
with model:
hyper_mean = pm.Normal('hyper_mean', mu=0, sd=100)
hyper_sigma = pm.HalfNormal('hyper_sigma', sd=3)
### Changed to a T distribution ###
means = pm.StudentT('means', nu=hyper_mean, sd=hyper_sigma, shape=n_individuals)
sigmas = pm.HalfNormal('sigmas', sd=100)
ye = pm.Normal('ye', mu=means, sd=sigmas, observed=observed)
trace = pm.sample(10000)
The following is the output:
Assigned NUTS to hyper_mean
Assigned NUTS to hyper_sigma_log
Assigned NUTS to means
Assigned NUTS to sigmas_log
---------------------------------------------------------------------------
PositiveDefiniteError Traceback (most recent call last)
<ipython-input-12-69f59e2f3d47> in <module>()
18 ye = pm.Normal('ye', mu=means, sd=sigmas, observed=observed)
19
---> 20 trace = pm.sample(10000)
/Users/jonathan/miniconda2/envs/pymc3/lib/python3.5/site-packages/pymc3/sampling.py in sample(draws, step, start, trace, chain, njobs, tune, progressbar, model, random_seed)
121 """
122 model = modelcontext(model)
--> 123
124 step = assign_step_methods(model, step)
125
/Users/jonathan/miniconda2/envs/pymc3/lib/python3.5/site-packages/pymc3/sampling.py in assign_step_methods(model, step, methods)
66 selected_steps[selected].append(var)
67
---> 68 # Instantiate all selected step methods
69 steps += [s(vars=selected_steps[s]) for s in selected_steps if selected_steps[s]]
70
/Users/jonathan/miniconda2/envs/pymc3/lib/python3.5/site-packages/pymc3/sampling.py in <listcomp>(.0)
66 selected_steps[selected].append(var)
67
---> 68 # Instantiate all selected step methods
69 steps += [s(vars=selected_steps[s]) for s in selected_steps if selected_steps[s]]
70
/Users/jonathan/miniconda2/envs/pymc3/lib/python3.5/site-packages/pymc3/step_methods/nuts.py in __init__(self, vars, scaling, step_scale, is_cov, state, Emax, target_accept, gamma, k, t0, model, profile, **kwargs)
76
77
---> 78 self.potential = quad_potential(scaling, is_cov, as_cov=False)
79
80 if state is None:
/Users/jonathan/miniconda2/envs/pymc3/lib/python3.5/site-packages/pymc3/step_methods/quadpotential.py in quad_potential(C, is_cov, as_cov)
33 return QuadPotential_SparseInv(C)
34
---> 35 partial_check_positive_definite(C)
36 if C.ndim == 1:
37 if is_cov != as_cov:
/Users/jonathan/miniconda2/envs/pymc3/lib/python3.5/site-packages/pymc3/step_methods/quadpotential.py in partial_check_positive_definite(C)
56 if len(i):
57 raise PositiveDefiniteError(
---> 58 "Simple check failed. Diagonal contains negatives", i)
59
60
PositiveDefiniteError: Scaling is not positive definite. Simple check failed. Diagonal contains negatives. Check indexes [202]
Any suggestion on how to get this to work?
As I mentioned in the comment, try running:
model = pm.Model()
with model:
hyper_mean = pm.Normal('hyper_mean', mu = 0, sd = 100)
hyper_sigma = pm.HalfNormal('hyper_sigma', sd = 3)
nu = pm.Exponential('nu', 1./10, testval = 5.)
### Changed to a T distribution ###
means = pm.StudentT('means', nu = nu, mu = hyper_mean, sd = hyper_sigma, shape = n_individuals)
sigmas = pm.HalfNormal('sigmas', sd = 100)
ye = pm.Normal('ye', mu = means, sd = sigmas, observed = observed)
trace = pm.sample(10000)
In other words: use the mu argument of the pm.StudentT method for hyper_mean and nu for the degrees of freedom.
Once it starts working, you might also try to add the pm.find_MAP method (as suggested by #Chris Fonnesbeck).
Try finding the MAP estimate and use that as the starting point for the MCMC run:
start = pm.find_MAP()
trace = pm.sample(10000, start=start)

Pig Scripting - Cast STRING to INT

Beginner in Pig, Need help
For all NON - AlphaNumeric, Cast the STRING TO INT
- To be handled without passing each field name separately.
Sample data -
00013425731998101620140402300032736901 00000000AAA001200X111685V00000000
00283335542006120920131010300030003105 00000000AAA001200X117407 00000000
00000000331998101620140402300033128107 00000000AAA001200X111685 00000000
00003902331999090620140402300032545208 00000000AAA001200X111685 00000000
Its a fixedwidth file, mapping details as follow -
orderNumber 1 9
origin 10 10
Startdate 11 18
ModDate 19 26
Identifier 27 36
Code 37 38
CodeType 39 40
Number 41 48
Num 49 114
Either use substr to extract the parts and then cast them or use a regexp. For example for the first two fields:
input = load ... as (line:chararray);
a = foreach input generate SUBSTRING(line, 0, 9) as orderNumber:long, SUBSTRING(line, 9, 10) as origin:chararray;
This way you should be able to convert each part of the input line into the desired components.
Alternatively you could write a UDF that takes a string as input and does the splitting and returns a bag or tuple.

ArgumentError: In `load': marshal data too short

I want to realize multiple processes. I have to send the data which bubble-sorted in different child processes back to parent process then merge data. This is part of my code:
rd1,wt1 = IO.pipe # reader & writer
pid1 = fork {
rd1.close
numbers = Marshal.load(Marshal.dump(copylist[0,p]))
bubble_sort(numbers)
sList[0] = numbers.clone
wt1.write Marshal.dump(sList[0])
Process.exit!(true)
}
Process.waitpid(pid1)
Process.waitpid(pid2)
wt1.close
wt2.close
pid5 = fork {
rd5.close
a = Marshal.load(rd1.gets)
b = Marshal.load(rd2.gets)
mList[0] = merge( a,b).clone
wt5.write Marshal.dump(mList[0])
Process.exit!(true)
}
There are pid1...pid7, rd1...rd7, wt1...wt7. pid1...pid4 are bubble-sort 4 part of data. pid5 and 6 merge data from pid1, 2 and pid 3, 4. Finally, pid7 merges the data from pid5 and 6.
When data size is small, it succeeds, but when I input larger data (10000):
Data example : 121 45 73 89 11 452 515 32 1 99 4 88 41 53 159 482 2013 2 ...
then, errors occur: :in 'load': marshal data too short (ArgumentError) and another kind error: in 'load': instance of IO needed (TypeError). The first error line is in pid5: a = ... and pid6: b = .... The other kind of error line is in pid7: b = .... Are my data too big for this method?
Marshal.load and Marshal.dump work with binary data. The problem with the short reads is here:
a = Marshal.load(rd1.gets)
b = Marshal.load(rd2.gets)
#gets reads up to a new-line (or end of file) and then stops. The trouble is that new-line may be present in the binary data created by Marshal.dump.
Change gets to read in both lines.

VB6 RS232 commands not working

I have the following code:
MSCommProj.CommPort = 6
MSCommProj.RThreshold = 1
MSCommProj.Settings = "19200,N,8,1"
MSCommProj.InputLen = 0
MSCommProj.PortOpen = True
And it opens just fine and connects but when i try sending the command:
MSCommProj.Output = "21 8901 5057 31 0A" & Chr$(13)
and
MSCommProj.Output = "21 89 01 50 57 31 0A" & Chr$(13)
and
MSCommProj.Output = "3F 89 01 50 57 0A" & Chr$(13)
as instructed by the user manual, it does not come on.
Here is the pages in the manual that shows this. Maybe i am just doing it wrong?:
Are you sure that you're meant to be sending character data to the RS232 interface for that? Those look like binary sequences to me.
Rather than:
MSCommProj.Output = "3F 89 01 50 57 0A" & Chr$(13)
I'd be looking at transmitting the binary data thus:
MSCommProj.Output = chr$(63) & chr$(137) & chr$(1) & chr$(80) & chr$(87) & chr(10)
You'll note that there's no chr$(13) at the end, the spec doesn't call for that.
If you want to know what the conversions are for those hex values, start up the Windows calculator, change the view to scientific, switch to hex mode, enter the value, the switch to decimal mode.
Or you can download an ASCII table for this purpose. Or view one of my voluminous essays on the subject here.
You are required to send bytes given.
You instead send string representation of those.
Send actual bytes.
chr$(&h21) & chr$(&h89) & chr$(&h01) & chr$(&h50) etc.
It was because i did not use a cross-over cable... All the rs232 codes were correct. Blah.

Resources