I'm using the Spartan 3E Starter Kit and I'm trying to receive Ethernet frames on it via a 100 MBit link.
For those who don't know, the board features a PHY chip, exposing the receiving clock with 25 MHz. I have (pretty much) verified that receiving works fine by buffering the received frames and resending them via a serial link.
Furthermore, I'm using a CRC32 generator from outputlogic.com. I aggregate the received nybbles to bytes and forward them to the CRC. At the end of the frame, I latch the generated CRC and display it on the LCD, together with the CRC I found in the ethernet frame.
However, (as you might have guessed) the two numbers do not match.
527edb0d -- FCS extracted from the frame
43a4d833 -- calculated using the CRC32 generator
The first one can also be verified by running the package through pythons crc32 function, both with the frame captured by wireshark and the frame captured and retrieved via serial port from the FPGA.
I guess it must be something more or less trivial. I pasted the receiving process over here. I stripped off everything which was not neccessary. When capturing the output via serial, I added a fifo (readily made unit from Xilinx) which latched at the same time as the CRC generator to get exactly the same bytes.
Does anyone have an idea what's wrong with that?
I started working on an ethernet MAC a while back, and although I never got round to finishing it I do have a working CRC generator that you can use here:
CRC.vhd
Its based on a Xilinx App note on the IEEE 802.3 CRC, which you can find here.
The CRC is instantiated in the ethernet receieve component, if you look at the ETH_RECEIVE_SM process you can see how the FCS is loaded into the checker.
Hopefully you can spot your mistake by comparing with my code.
Edit:
I took the sample ethernet frame from fpga4fun and passed it through the CRC checker, see the simulation screenshot below (right click, copy URL and view in a new browser tab for full resolution):
You can see the residual C704DD7B at the end there, try doing the same with your own CRC checker and see what you get.
The generator you used may not be pre-processing and post-processing the data. If that generator takes a string of zeros and produces a zero crc, then that's the problem. A string of zeros should not produce zero. (What it produces depends on the number of zeros.)
The processing for the Ethernet crc is to invert the crc, then apply the crc algorithm, then invert the crc again.
Related
This is an architectural question regarding gnuradio(-companion) and since I am not sure how to tackle this problem in the first place I first describe what I want to achieve and then how I think I would to it.
Problem
I implement a special form of an RFID reader with an Ettus X310 SDR: The transmitter sends an OOK/AM modulated (PIE encoded) request, followed by a pure Sine wave. The RFID tag backscatters its response onto this sine wave using OOK/AM modulation in FM0 or "Miller subcarrier" coding (a form of a differential Manchester coding). I want to receive its response, translate it into bits (and form a PDU), buffer different responses in a FIFO and send them for further processing. The properties of the tag response are:
It is asynchronuous. I do not know when the response is coming and if it does, when the proper sampling times are: I cannot simply filter, sample, decimate the signal and use a simple slicer because I do not know what the sample points are.
The response comes into very small "bursts" (say, 100 bits). Hence I cannot afford performing timing recovery on bits and waste them (except I buffer the entire signal somehow which I do not think is the way to do it).
The signal starts with a small preamble (UHF RFID Gen2 preamble) which is 6 bits (~8 bit transitions). This may not be enough for for time recovery but can be used to identify the start of a response somehow.
It uses mentioned FM0 encoding, so I have a guaranteed transition every bit. For that reason, I do not have to sample them but could detect the transitions and convert them into bits. I would not need conventional clock recovery (e.g. M&M) either.
My Thoughts
"Ordinary" gnuradio preprocessing brings me to the received oversampled bits: Downconversion, filtering; possibly a slicer which uses a lowpass filter to subtract the mean value and a comparator (note that even this may be challenging because the lowpass filter may have a large settling time of few bits until it obtains the right mean value).
In order to detect the actual transmission, I do not think I have much choice other than a simple squelch that detects a higher signal level than the noise floor (is this true or is there a way to detect the transmission using the preamble only?)
Once the squelch block detects a transmission, I could use a differentiator (or similar) to get the edges. But my understanding of the transition between this "baseband land" and "bits/PDUs" ends: I would need a block that triggers asynchronously (rather than samples at fixed intervals). In an actual system, the edges from the described detector could act as clock input of a flip flop. However, I do not see which standard gnuradio block would allow me to do this.
Once in "bits land", the bits (or PDUs) would be processed at a much lower rate. However, two clock domains are crossed: the normal baseband sampling rate, an irregular rate by which the transitions are detected and the rate at which the bits are read. For that reason, I would be looking for a FIFO or shift register, in which the detected bits are shifted in at whichever edge transition rate they come in and read out at the regular bit rate on the other side.
Question
What is the correct architecture/approach to implement this in gnuradio?
I could imagine to implement this with my own blocks. But as much as possible I would like to use standard block, gnuradio-companion. I would like to resort to own blocks (in particular C++) only as last resort if either not possible otherwise or if it would really not be the right way to so it otherwise.
I need to use keyboard as input for musical notes, and digilent speaker as output.
I plan to use only one octave.
My most intriguing questions are:
How do I represent the musical notes in VHDL code.
How do I (or do I need to) implement a DAC module that uses Spartan 3E Starter's built-in DAC? I have read on other forums that it can't be implemented. I need to use it in order to transmit the note to the speaker. The teacher who supervises my and my colleagues' projects suggested me to look into PWM for that(but all I've found is explained in electronic manner, no accompanying code, or explanation on implementation).
Besides keyboard controller, a processing module(for returning the note corresponding to the pressed key from the notes vector) and DAC, that I have figured out so far that I need, what else do I need.
There is a DAC (see comments)
There is no DAC on the Spartan-3E Starter Kit. Using a low-pass PWM signal is a common way to generate analog signal level from digital output.
You need to define a precision for your PWM, let's say 8 bits or 256 levels. For each audio sample you want to output, you need to count from 0 to 255. When the counter is less than the desired sample level, output 1, otherwise output 0. When the counter reach 255, reset it and go to the next sample.
Thus, if you want 8 bits precision (256 levels) and 8KHz signal, the counter will have to run at 256*8000 = 2.048MHz.
For your other questions, there is no easy answer. It's your job, as designer, to figure that out.
I want to implement a Morse code to ASCII converter in vhdl and then to transmit that ASCII character to PC terminal through UART. I have completed the UART part.
But i don't know how to implement the converter part. The problem is with varying symbol rate of input Morse code. I want to detect dot, dash, character space and word space.
Please help me for the following implementation:
detection of Morse code symbols with variable symbol rate
binary search tree to traverse the Morse code tree to find the corresponding ASCII character.
It seems as if you're planning to do the converter in two stages, which sounds like a decent idea - first stage for decoding dots, dashes and pauses, second stage for assembling these into characters.
For the first stage, you'll need to define some minimum and maximum timings for your input - that is, what is the shortest and longest a dot / dash / etc can last (I'd take it there are some standards for this?).
With these timings it should be possible to use a counter and an edge detector to decode your input morse signal.
For the second stage, you can create a finite state machine or maybe even just a look-up-table to decode the incoming dots and dashes of a single character, and send this along to your UART.
You need to create a module which sample your input and from the pattern you record, use some fixed pattern to out the corresponding letter. You need to study the width of your received signal to estimate how much your sample rate should be. You can try to read and understand the following shared code on github. https://github.com/altugkarakurt/Morse-Code-Converter which will do the reverse process of generating morse code. This can be used as a test generator for your Morse Reader. The code ASCII extract from Morse detector can be feed into a FIFO, which is a standard circuit and connected to your UART. The Morse detector write the ASCII into the FIFO if the FIFO is not full, while the UART read the ASCII from the FIFO and transmit it to the UART. The FIFO acts as a buffer. If you need some standard UART code look at the Xilinx website which has one UART part of the Microblaze design, it is very simple to interface it.
I was trying to understand how the CRC-32 for ethernet works and am having some issues. I looked at an ARP request that had the hex values:
00000024e8cc96beffffffffffff080600010800060400010024e8cc96be0a3307fa0000000000000a3307fb0000000000000000000000000000000000000ff0fdca
and threw that into one of the many online generators and could not get the CRC to match up. It looks to me like the first two bytes (0x00 0x00) are the beginning of the frame, and that the last four bytes (0x0F 0xF0 0xFD 0xCA) are the CRC, but I cannot calculate that when I put the middle bytes into the online generators.
Any idea what I am assuming incorrectly?
I had a mistake in my original post. I was looking at the FPGA's loop-backed response, the real ARP looked like:
0000ffffffffffff0024e8cc96be080600010800060400010024e8cc96be0a3307fa0000000000000a3307fb0000000000000000000000000000000000000ff0fdca
(the src and dst mac are swapped). If I put everything after 0000 and before ff0fdca into the second link I had listed in the question, I get 0xCAFDF00F as a CRC, which is the same as originally sent out in the ARP request (just in reverse byte order). So it makes sense if you use the right data from the get go....
TASK:
I'm building a set of x86 assembly reverse engineering challenges, of which I have twenty or so already completed. They're just for fun / education.
The current challenge is one of the more advanced ones, and involves some trickery that makes it look like the EP is actually in the normal program, but it's actually packed away in another PE section.
Heres' the basic flow:
Starts out as if it were a normal MSVC++ application.
Injected a sneaky call away to a bunch of anti-debugger tricks.
If they pass, a DWORD in memory is set to 1.
Later in the program flow, it checks for that value being 1, and if it works it decrypts a small call table. If it fails, it sends them off on a wild goose chase of fake anti-debug tricks and eventually just crashes.
The call table points to the real decryption routines that decrypt the actual program code section.
The decryption routines are called, and they decrypt using a basic looped xor (C^k^n where C is ciphertext, k is a 32-bit key and n is the current data offset)
VirtualProtect is used to switch the section's protection flags from RW to RX.
Control flow is redirected to OEP, program runs.
The idea is that since they think they're in normal program flow, it makes them miss the anti-debug call and later checks. Anyway, that all works fine.
PROBLEM:
The current problem is that OllyDbg and a few other tools look at the packed section and see that it has high entropy, and throw up a warning that it's packed. The code section pointer in the PE header is correctly set, so it doesn't get this from having EP outside code - it's purely an entropy analysis thing.
QUESTION:
Is there an encryption method I can use that preserves low entropy, but is still easy to implement in x86 asm? I don't want to use a plain xor, since it's too easy, but I also don't want it to catch it as packed and give the game away.
I thought of something like a shuffler (somehow produce a keystream and use it to swap 4-byte blocks of code around), but I'm not sure that this is going to work, or even be simple.
Anyone got any ideas?
Actually, OllyDbg works like this pseudocode:
useful_bytes = number_of_bytes_in_section - count_bytes_with_values(0x00, 0x90, 0xCC)
warn about compression if useful_bytes > 0x2000 and count_bytes_with_values(0xFF, 0xE8, 0x8B, 0x89, 0x83) / useful_bytes < 0.075
So, the way to avoid that warning is to use enough bytes with the values 0xFF 0xE8 0x8B 0x89 0x83 in the compressed section.
Don't pack/encrypt your entire program code. Just encrypt a small percentage of bytes, randomly selected from your program code. If they're not decrypted, the program will soon crash if it tries to run the code anyway - and because the majority of the program is unchanged, entropy-based checks won't be set off.
What about simply reversing the bytes (from last to first)? Intel assembler instructions aren't fixed length, so this would shuffle them a little. Or you could simply rotate each byte by a fixed amount...
EDIT: Wrong guess, this is not how Olly works. See my other answer. This still applies to tools other than OllyDbg that calculates entropy.
Expanding on ninjaljs comment:
While I haven't checked, the entropy value OllyDbg calculates is likely bytewise, without context. See How to calculate the entropy of a file? for a common algorithm for doing this.
This algorithm gives that the sequence 0 1 2 ... 254 255 have the maximum entropy possible, despite being completely predictable. A sequence of random bytes between 0 and 255 would get slightly lower entropy, since it won't have exactly the same number of each possible value.
Some quick checks on uncompressed executables with pefile tells me that uncompressed x86 code has entropy of about 6.3 to 6.6. Compressed code with entropy 8.0, encoded with base64, has entropy 6.0. Thus, base64 is easily enough to stop this algorithm from finding compressed code.