Myhdl: assigning a bitslice to a signed variable fails with negative values - slice

The code added fails with a ValueError and i have no idea whats going wrong. Here is what i want to do:
In my fpga i receive data via spi.
Data is a bipolar signal (in the sense of a measurement signal) arriving in a 16 bit register rxdata.
the task is to interpret this signal as being signed and only the upper 12 bits (including sign) are needed. The variable bipolar is therefore 12bit wide and signed i.e. declared like in the code. Then i assign:
bipolar.next=rxdata[:4].signed()
my problem is, that the assignment of the 12bit slice fails as soon as data becomes negative (i.e most significant bit becomes 1).
With data 0x8fff for instance at runtime i get:
'ValueError: intbv value 2303 >= maximum 2048'
I don't expect this, as both sides are declared signed and data fits into the variable bipolar.
Is there another way to do this?
(by the way: bipolar[:].next=rxdata[:4].signed() i get 0 as a result which i would not expect either)
#testcase.py sk 09.12.2020
#assign a slice to a signed signal (bipolar) fails at runtime with negative numbers
from myhdl import *
nspi=16
n=12
tend=1e-6
#block
def testcase():
CLK = Signal(bool(0))
RESET = ResetSignal(1,active = 0, isasync=True)
bipolar=Signal(intbv(0,min=-2**(n-1),max=2**(n-1)))
rxdata = Signal(intbv(0)[nspi:0]) #received data is bipolar, transferred via spi into rxdata
''' Clock driver 16MHz'''
#always(delay(31))
def driver():
CLK.next = not CLK
#instance
def init():
rxdata.next=0x8fff #0x7fff i.e. positive passes, 0x8fff i.e negative fails runtime check
yield delay(100)
#always_seq(CLK.negedge,reset=RESET)
def assign():
#bipolar[:].next=rxdata[:(nspi-n)].signed() #this passes - but result is 0! (unexpected for me)
bipolar.next=rxdata[:(nspi-n)].signed() #this fails with negative numbers (unexpected for me)
print(bipolar, 'bipolar=', int(str(bipolar),16))
return instances()
tc = testcase()
tc.run_sim(tend*1e9)
print('Simulated to tend='+str(tend))

Just found the way to do it: use shadow signals! i.e use round brackets () instead of square ..[]
#testcase.py sk 12.12.2020
#assign a slice to a signed signal (bipolar) fails at runtime with negative numbers
#-> use shadow signals instead! #bipolar[:].next=rxdata[:(nspi-n)].signed() #this passes - but result is 0! (unexpected for me)
from myhdl import *
nspi=16
n=12
tend=1e-6
#block
def testcase():
CLK = Signal(bool(0))
RESET = ResetSignal(1,active = 0, isasync=True)
bipolar=Signal(intbv(0,min=-2**(n-1),max=2**(n-1)))
#rxdata = Signal(intbv(0)[nspi:0]) #received data is bipolar, transferred via spi into rxdata
rxdata = Signal(intbv(0,min=0,max=2**nspi))
''' Clock driver 16MHz'''
#always(delay(31))
def driver():
CLK.next = not CLK
#instance
def init():
rxdata.next=0xffff #0x7fff i.e. positive passes, 0x8fff i.e negative fails runtime check when not using shadow
yield delay(100)
#always_seq(CLK.negedge,reset=RESET)
def assign():
#bipolar.next=rxdata[:(nspi-n)].signed() #this fails in runtime check -> number too big
#bipolar[:].next=rxdata[:(nspi-n)].signed() #this passes - but result is 0! (unexpected for me)
bipolar.next=rxdata(nspi,(nspi-n)).signed() #this is the way: use shadow signal !
print(bipolar, 'bipolar=', int(str(bipolar),16))
return instances()
tc = testcase()
tc.run_sim(tend*1e9)
print('Simulated to tend='+str(tend))

Related

With ruamel.yaml how can I conditionally convert flow maps to block maps based on line length?

I'm working on a ruamel.yaml (v0.17.4) based YAML reformatter (using the RoundTrip variant to preserve comments).
I want to allow a mix of block- and flow-style maps, but in some cases, I want to convert a flow-style map to use block-style.
In particular, if the flow-style map would be longer than the max line length^, I want to convert that to a block-style map instead of wrapping the line somewhere in the middle of the flow-style map.
^ By "max line length" I mean the best_width that I configure by setting something like yaml.width = 120 where yaml is a ruamel.yaml.YAML instance.
What should I extend to achieve this? The emitter is where the line-length gets calculated so wrapping can occur, but I suspect that is too late to convert between block- and flow-style. I'm also concerned about losing comments when I switch the styles. Here are some possible extension points, can you give me a pointer on where I'm most likely to have success with this?
Emitter.expect_flow_mapping() probably too late for converting flow->block
Serializer.serialize_node() probably too late as it consults node.flow_style
RoundTripRepresenter.represent_mapping() maybe? but this has no idea about line length
I could also walk the data before calling yaml.dump(), but this has no idea about line length.
So, where should I and where can I adjust the flow_style whether a flow-style map would trigger line wrapping?
What I think the most accurate approach is when you encounter a flow-style mapping in the dumping process is to first try to emit it to a buffer and then get the length of the buffer and if that combined with the column that you are in, actually emit block-style.
Any attempt to guesstimate the length of the output without actually trying to write that part of a tree is going to be hard, if not impossible to do without doing the actual emit. Among other things the dumping process actually dumps scalars and reads them back to make sure no quoting needs to be forced (e.g. when you dump a string that reads back like a date). It also handles single key-value pairs in a list in a special way ( [1, a: 42, 3] instead of the more verbose [1, {a: 42}, 3]. So a simple calculation of the length of the scalars that are the keys and values and separating comma, colon and spaces is not going to be precise.
A different approach is to dump your data with a large line width and parse the output and make a set of line numbers for which the line is too long according to the width that you actually want to use. After loading that output back you can walk over the data structure recursively, inspect the .lc attribute to determine the line number on which a flow style mapping (or sequence) started and if that line number is in the set you built beforehand change the mapping to block style. If you have nested flow-style collections, you might have to repeat this process.
If you run the following, the initial dumped value for quote will be on one line.
The change_to_block method as presented changes all mappings/sequences that are too long
that are on one line.
import sys
import ruamel.yaml
yaml_str = """\
movie: bladerunner
quote: {[Batty, Roy]: [
I have seen things you people wouldn't believe.,
Attack ships on fire off the shoulder of Orion.,
I watched C-beams glitter in the dark near the Tannhäuser Gate.,
]}
"""
class Blockify:
def __init__(self, width, only_first=False, verbose=0):
self._width = width
self._yaml = None
self._only_first = only_first
self._verbose = verbose
#property
def yaml(self):
if self._yaml is None:
self._yaml = y = ruamel.yaml.YAML(typ=['rt', 'string'])
y.preserve_quotes = True
y.width = 2**16
return self._yaml
def __call__(self, d):
pass_nr = 0
changed = [True]
while changed[0]:
changed[0] = False
try:
s = self.yaml.dumps(d)
except AttributeError:
print("use 'pip install ruamel.yaml.string' to install plugin that gives 'dumps' to string")
sys.exit(1)
if self._verbose > 1:
print(s)
too_long = set()
max_ll = -1
for line_nr, line in enumerate(s.splitlines()):
if len(line) > self._width:
too_long.add(line_nr)
if len(line) > max_ll:
max_ll = len(line)
if self._verbose > 0:
print(f'pass: {pass_nr}, lines: {sorted(too_long)}, longest: {max_ll}')
sys.stdout.flush()
new_d = self.yaml.load(s)
self.change_to_block(new_d, too_long, changed, only_first=self._only_first)
d = new_d
pass_nr += 1
return d, s
#staticmethod
def change_to_block(d, too_long, changed, only_first):
if isinstance(d, dict):
if d.fa.flow_style() and d.lc.line in too_long:
d.fa.set_block_style()
changed[0] = True
return # don't convert nested flow styles, might not be necessary
# don't change keys if any value is changed
for v in d.values():
Blockify.change_to_block(v, too_long, changed, only_first)
if only_first and changed[0]:
return
if changed[0]: # don't change keys if value has changed
return
for k in d:
Blockify.change_to_block(k, too_long, changed, only_first)
if only_first and changed[0]:
return
if isinstance(d, (list, tuple)):
if d.fa.flow_style() and d.lc.line in too_long:
d.fa.set_block_style()
changed[0] = True
return # don't convert nested flow styles, might not be necessary
for elem in d:
Blockify.change_to_block(elem, too_long, changed, only_first)
if only_first and changed[0]:
return
blockify = Blockify(96, verbose=2) # set verbose to 0, to suppress progress output
yaml = ruamel.yaml.YAML(typ=['rt', 'string'])
data = yaml.load(yaml_str)
blockified_data, string_output = blockify(data)
print('-'*32, 'result:', '-'*32)
print(string_output) # string_output has no final newline
which gives:
movie: bladerunner
quote: {[Batty, Roy]: [I have seen things you people wouldn't believe., Attack ships on fire off the shoulder of Orion., I watched C-beams glitter in the dark near the Tannhäuser Gate.]}
pass: 0, lines: [1], longest: 186
movie: bladerunner
quote:
[Batty, Roy]: [I have seen things you people wouldn't believe., Attack ships on fire off the shoulder of Orion., I watched C-beams glitter in the dark near the Tannhäuser Gate.]
pass: 1, lines: [2], longest: 179
movie: bladerunner
quote:
[Batty, Roy]:
- I have seen things you people wouldn't believe.
- Attack ships on fire off the shoulder of Orion.
- I watched C-beams glitter in the dark near the Tannhäuser Gate.
pass: 2, lines: [], longest: 67
-------------------------------- result: --------------------------------
movie: bladerunner
quote:
[Batty, Roy]:
- I have seen things you people wouldn't believe.
- Attack ships on fire off the shoulder of Orion.
- I watched C-beams glitter in the dark near the Tannhäuser Gate.
Please note that when using ruamel.yaml<0.18 the sequence [Batty, Roy] never will be in block style
because the tuple subclass CommentedKeySeq does never get a line number attached.

Why are my byte arrays not different even though print() says they are?

I am new to python so please forgive me if I'm asking a dumb question. In my function I generate a random byte array for a given number of bytes called "input_data", then I add bytewise some bit errors and store the result in another byte array called "output_data". The print function shows that it works exactly as expected, there are different bytes. But if I compare the byte arrays afterwards they seem to be identical!
def simulate_ber(packet_length, ber, verbose=False):
# generate input data
input_data = bytearray(random.getrandbits(8) for _ in xrange(packet_length))
if(verbose):
print(binascii.hexlify(input_data)+" <-- simulated input vector")
output_data = input_data
#add bit errors
num_errors = 0
for byte in range(len(input_data)):
error_mask = 0
for bit in range(0,7,1):
if(random.uniform(0, 1)*100 < ber):
error_mask |= 1 << bit
num_errors += 1
output_data[byte] = input_data[byte] ^ error_mask
if(verbose):
print(binascii.hexlify(output_data)+" <-- output vector")
print("number of simulated bit errors: " + str(num_errors))
if(input_data == output_data):
print ("data identical")
number of packets: 1
bytes per packet: 16
simulated bit error rate: 5
start simulation...
0d3e896d61d50645e4e3fa648346091a <-- simulated input vector
0d3e896f61d51647e4e3fe648346001a <-- output vector
number of simulated bit errors: 6
data identical
Where is the bug? I am sure the problem is somewhere between my ears...
Thank you in advance for your help!
output_data = input_data
Python is a referential language. When you do the above, both variables now refer to the same object in memory. e.g:
>>> y=['Hello']
>>> x=y
>>> x.append('World!')
>>> x
['Hello', 'World!']
>>> y
['Hello', 'World!']
Cast output_data as a new bytearray and you should be good:
output_data = bytearray(input_data)

Runtime error claiming a negative or zero argument to the logarithm function in Box-Mueller algorithm

The following code is a part of a Fortran 90 program that I wrote in Plato IDE:
It is just the Box-Mueller algorithm to generate Gaussian random numbers.
Program brownstep2_single_stage
Integer:: i,j,m,n,countsucc!,a
Real:: dt,D,epsa,r1,r2,w,fptsum,fptdef1,fptdef2
Real,Dimension(0:100002) :: fx !gt
!T=1000.0 and n*dt=T
dt=0.001
m=100000
n=100000
D=1.0
!a=7
w=2
epsa=0.00001
fx(0)=6.0
!gt(0)=0
fptsum=0
countsucc=0
Call random_seed()
Do i=0,m
!Call random_seed(a)
Do j=0,n
Do while (w>=1.0.and.w<0.0)
Call random_number(r1)
Call random_number(r2)
!r=rand()
r1=2.0*r1-1
r2=2.0*r2-1
w=r1*r1+r2*r2
End do
w=sqrt((-2.0*log(w))/w)
r1=r1*w
r2=r2*w
If(mod(j,2)==0) then
w=r1
Else if(mod(j,2)==1) then
w=r2
End if
fx(j+1)=fx(j)+w*sqrt(2.0*D*dt)
If(fx(j+1)<epsa) then
fptsum=fptsum+(j+1)*dt
countsucc=countsucc+1
exit
End if
print *,i,j
End do
End do
fptdef1=fptsum/m
fptdef2=fptsum/countsucc
print *,'The value of fpt by 1st definition is:',fptdef1
print *,'The value of fpt by 2nd definition is:',fptdef2
print *,'The number of successful events is:',countsucc
print *,'The total number of events is:',m
End program brownstep2_single_stage
During compilation, it shows no error, but when run, it shows the following runtime error, claiming a negative or zero argument to the logarithm function.
Runtime error from program:e:\my files\sample2brownstep_gauss.exe
Run-time Error
Error: Negative or zero argument to logarithm routine
BROWNSTEP2_SINGLE_STAGE - in file sample2brownstep_gauss.f90 at line 31 [+02cc]
What should I do to avoid this?
The changed code above still has problems. w is still not set before the do while loop is reached for the first time and w is used in the condition. Use an 'infinite' do loop with an exit statement. This ensures that one attempt at w is always attempted. This would be better:
do
Call random_number(r1)
Call random_number(r2)
r1=2.0*r1-1
r2=2.0*r2-1
w=r1*r1+r2*r2
if (w .lt. 1.0) exit
End do
w=sqrt((-2.0*log(w))/w)
r1=r1*w
r2=r2*w

Why do I need to run this function twice to get the expected output?

In system-verilog i have a random number generator like this:
task set_rand_value(output bit [7:0] rand_frms, output bit [13:0] rand_bcnt);
begin
bit [7:0] rand_num;
// Set the random value between 2 and 254
rand_num=$urandom_range(254,2);
rand_bcnt=$urandom_range(2047,1);
// Check to ensure number is even
if ((rand_num/2)*2 != rand_num) begin
rand_num=rand_num+1;
$display("Generated %d frames. 8'h%h", rand_num, rand_num);
$display("Packets generated with 0x%4h bytes.", rand_bcnt);
end
// Set the random frame number
else begin
rand_frms = rand_num;
$display("Generated %d frames. 8'h%h", rand_num, rand_num);
$display("Packets generated with 0x%4h bytes.", rand_bcnt);
end
end
endtask // set_rand_value
For some reason the first time I run this (and the third and fifth... etc) it returns a value of zero even though a number is properly generated.
I am calling it like this:
bit [ 7:0] num_frms;
bit [13:0] size_val;
// Generate random values
set_rand_value(num_frms, size_val); // Run twice to correct initial write error
$display("Error correction for packets sent: 0x%h", num_frms);
set_rand_value(num_frms, size_val);
$display("The number of packets being sent: 0x%h", num_frms);
Which gives me this output:
Generated 214 frames. 8'hd6
Packets generated with 0x05b2 bytes.
Error correction for packets sent: 0x00
Generated 252 frames. 8'hfc
Packets generated with 0x011f bytes.
The number of packets being sent: 0xfc
0x80fc011f // Expected number
I have tried quite a few things to correct it but for some reason the only way i can get it to behave the way i expect it to is to just run it twice every time i need to use it.
You only set num_frms if the random number is even. Why not just do
function void set_rand_value(output bit [7:0] rand_frms, output bit [13:0] rand_bcnt);
begin
// Set the random value between 2 and 254
rand_frms=$urandom*2;
rand_bcnt=$urandom_range(2047,1);
endfunction
P.S Only use tasks for routines that may block.
The reason why the code above did not work was you were not assigning rand_frms = rand_num; in the first half of the if condition . So only for 50% of the times read_frm would have a value in it. [ 50 % just been approximate ] .

mpi4py: Internal Error: invalid error code 409e0e (Ring ids do not match)

I am coding in python and using mpi4py to do some optimization in parallel. I am using Ordinary Least Squares, and my data is too large to fit on one processor, so I have a master process that then spawns other processes. These child processes each import a section of the data that they respectively work with throughout the optimization process.
I am using scipy.optimize.minimize for the optimization, so the child processes receive a coefficient guess from the parent process, and then report the sum of squared error (SSE) to the parent process, and then scipy.optimize.minimize goes through iterations, trying to find a minimum for the SSE. After each iteration of the minimize function, the parent broadcasts the new coefficient guesses to the child processes, who then calculate the SSE again. In the child processes, this algorithm is set up in a while loop. In the parent process, I simply call scipy.optimize.minimize.
On the part that is giving me a problem, I am doing a nested optimization, or an optimization within an optimization. The inner optimization is an OLS regression as described above, and then the outer optimization is minimizing another function that uses the coefficient of the inner optimization (the OLS regression).
So in my parent process, I have two functions that I minimize, and the second function calls on the first and does a new optimization for every iteration of the second function's optimization. The child processes have a nested while loop for those two optimizations.
Hopefully that all makes sense. If more information is needed, please let me know.
Here is the relevant code for the parent process:
comm = MPI.COMM_SELF.Spawn(sys.executable,args = ['IVQTparallelSlave_cdf.py'],maxprocs=processes)
# First stage: reg D on Z, X
def OLS(betaguess):
comm.Bcast([betaguess,MPI.DOUBLE], root=MPI.ROOT)
SSE = np.array([0],dtype='d')
comm.Reduce(None,[SSE,MPI.DOUBLE], op=MPI.SUM, root = MPI.ROOT)
comm.Bcast([np.array([1],'i'),MPI.INT], root=MPI.ROOT)
return SSE
# Here is the CDF function.
def CDF(yguess, delta_FS, tau):
# Calculate W(y) in the slave process
# Solving the Reduced form after every iteration: reg W(y) on Z, X
comm.Bcast([yguess,MPI.DOUBLE], root=MPI.ROOT)
betaguess = np.zeros(94).astype('d')
###########
# This calculates the reduced form coefficient
coeffs_RF = scipy.minimize(OLS,betaguess,method='Powell')
# This little block is to get the slave processes to stop
comm.Bcast([betaguess,MPI.DOUBLE], root=MPI.ROOT)
SSE = np.array([0],dtype='d')
comm.Reduce(None,[SSE,MPI.DOUBLE], op=MPI.SUM, root = MPI.ROOT)
cont = np.array([0],'i')
comm.Bcast([cont,MPI.INT], root=MPI.ROOT)
###########
contCDF = np.array([1],'i')
comm.Bcast([contCDF,MPI.INT], root=MPI.ROOT) # This is to keep the outer while loop going
delta_RF = coeffs_RF.x[1]
return abs(delta_RF/delta_FS - tau)
########### This one finds Y(1) ##############
betaguess = np.zeros(94).astype('d')
######### First Stage: reg D on Z, X #########
coeffs_FS = scipy.minimize(OLS,betaguess,method='Powell')
print coeffs_FS
# This little block is to get the slave processes' while loops to stop
comm.Bcast([betaguess,MPI.DOUBLE], root=MPI.ROOT)
SSE = np.array([0],dtype='d')
comm.Reduce(None,[SSE,MPI.DOUBLE], op=MPI.SUM, root = MPI.ROOT)
cont = np.array([0],'i')
comm.Bcast([cont,MPI.INT], root=MPI.ROOT)
delta_FS = coeffs_FS.x[1]
######### CDF Function #########
yguess = np.array([3340],'d')
CDF1 = lambda yguess: CDF(yguess, delta_FS, tau)
y_minned_1 = scipy.minimize(CDF1,yguess,method='Powell')
Here is the relevant code for the child processes:
#IVQTparallelSlave_cdf.py
comm = MPI.Comm.Get_parent()
.
.
.
# Importing data. The data is the matrices D, and ZX
.
.
.
########### This one finds Y(1) ##############
######### First Stage: reg D on Z, X #########
cont = np.array([1],'i')
betaguess = np.zeros(94).astype('d')
# This corresponds to 'coeffs_FS = scipy.minimize(OLS,betaguess,method='Powell')' of the parent process
while cont[0]:
comm.Bcast([betaguess,MPI.DOUBLE], root=0)
SSE = np.array(((D - np.dot(ZX,betaguess).reshape(local_n,1))**2).sum(),'d')
comm.Reduce([SSE,MPI.DOUBLE],None, op=MPI.SUM, root = 0)
comm.Bcast([cont,MPI.INT], root=0)
if rank==0: print '1st Stage OLS regression done'
######### CDF Function #########
cont = np.array([1],'i')
betaguess = np.zeros(94).astype('d')
contCDF = np.array([1],'i')
yguess = np.array([0],'d')
# This corresponds to 'y_minned_1 = spicy.minimize(CDF1,yguess,method='Powell')'
while contCDF[0]:
comm.Bcast([yguess,MPI.DOUBLE], root=0)
# This calculates the reduced form coefficient
while cont[0]:
comm.Bcast([betaguess,MPI.DOUBLE], root=0)
W = 1*(Y<=yguess)*D
SSE = np.array(((W - np.dot(ZX,betaguess).reshape(local_n,1))**2).sum(),'d')
comm.Reduce([SSE,MPI.DOUBLE],None, op=MPI.SUM, root = 0)
comm.Bcast([cont,MPI.INT], root=0)
#if rank==0: print cont
comm.Bcast([contCDF,MPI.INT], root=0)
My problem is that after one iteration through the outer minimization, it spits out the following error:
Internal Error: invalid error code 409e0e (Ring ids do not match) in MPIR_Bcast_impl:1328
Traceback (most recent call last):
File "IVQTparallelSlave_cdf.py", line 100, in <module>
if rank==0: print 'CDF iteration'
File "Comm.pyx", line 406, in mpi4py.MPI.Comm.Bcast (src/mpi4py.MPI.c:62117)
mpi4py.MPI.Exception: Other MPI error, error stack:
PMPI_Bcast(1478).....: MPI_Bcast(buf=0x2409f50, count=1, MPI_INT, root=0, comm=0x84000005) failed
MPIR_Bcast_impl(1328):
I haven't been able to find any information about this "ring id" error or how to fix it. Help would be much appreciated. Thanks!

Resources