I'm trying to implement a method , calculating the CRC and adding it to a input- bitstring . For this I have found the CRC32 class in Java. Since I'm not familiar with this subject, I'm not sure how i can add the CRC.
The getValue() method of CRC32 would return a long type, but adding a long to a bitstring makes not much of sense.
public byte[] getData() {
byte[] bytes = bits.toByteArray();
return bytes ;
}
public void addCRC() {
crc = new CRC32();
crc.update(getData(), 0, getBitArrayLength());
long crcLong = crc.getValue();
}
Related
i first pushed my struct of clothes through my function addclothes in an dynamic array and it worked easily , but dont know how to do it with static arrays can someone please help someone. i beleive it is easy to figure it out
struct clothes {
string name;
string color;
}
clothes[5] public Clothes;
function addCloth(string calldata _name, string calldata _color) public {
Clothes.push(clothes(_name,_color));
}
i did it pretty easily before it the code is here but with dynamic array
'struct clothes {
string name;
string color;
}
clothes[] public Clothes;
function addCloth(string calldata _name, string calldata _color) public {
Clothes.push(clothes(_name,_color));
}'
You can push to dynamic arrays because they are dynamic. Pushing allocates memory for a new slot on the blockchain for the array slot, and appends a new element to that array.
Static arrays don't support pushing because when you initialize them you have to define their size, which means that you'll allocate the memory for those slots when you deploy the contract, and it doesn't allow you to modify its size nor append new elements.
So, instead of pushing, you'll have to iterate and set those slots.
In my example I use a counter to iterate over the array and give them a value.
contract Cloth {
struct Clothes {
string name;
string color;
}
Clothes[5] public clothes;
uint counter = 0;
function addCloth(string calldata _name, string calldata _color) public {
require(counter < 5, "Can't add more clothes. Limit of the array reached.");
clothes[counter] = Clothes(_name,_color);
counter++;
}
}
We're trying to encode several input strings with the same length (32) applying Blowfish + Hex encoding.
The problem is that not always the final coded strings have the same length as we expect (32 length strings).
Below you find the code used. Please, can you help to suggest what's wrong?
public static String encrypt(String clear, String key)
{
try
{
Security.setProperty("crypto.policy", "unlimited");
byte [] keyBytes = key.getBytes("ASCII");//toByteArray(key);
filelogger.info("Key coded in bytes "+keyBytes);
SecretKeySpec skey = new SecretKeySpec(keyBytes, "Blowfish");
byte [] clearBytes = clear.getBytes();//toByteArray(clear);
filelogger.info("Input string coded in bytes "+clearBytes);
Cipher ci = Cipher.getInstance("Blowfish");
ci.init(Cipher.ENCRYPT_MODE, skey);
// encrypt the clear bytes value
byte[] encoded = ci.doFinal(clearBytes);
filelogger.info("Blowfish output "+encoded);
return Base64.getEncoder().encodeToString(encoded);
}
catch (Exception e)
{
filelogger.error("Error while encrypting: " + e.toString());
logger.error("Error while encrypting: " + e.toString());
return Base64.getEncoder().encodeToString(new byte[0]);
}
}
Best regards
We have solved using the following solution:
Passing the option "Blowfish/ECB/NoPadding" to the getInstance function.
public byte[] encryptBlowfishECBNopadding(byte[] key, byte[] dati) {
byte[] output = null;
try {
SecretKeySpec KS = new SecretKeySpec(key, "Blowfish");
Cipher cipher = Cipher.getInstance("Blowfish/ECB/NoPadding");
cipher.init(Cipher.ENCRYPT_MODE, KS);
output = cipher.doFinal(dati);
return output;
} catch (Exception ee) {
logger.error(ee.getMessage());
filelogger.error(ee.toString());
return new byte[0];
}
}
Encoding the result of the method as below:
byte[] encryptresult=encryptBlowfishECBNopadding(toByteArray(decriptedki),toByteArray(criptokeyhlr.getKeydata()));
String stringencriptedki=Hex.encodeHexString(encryptresult).toUpperCase();
In this manner every output string has the same length.
Thanks to all for the support!
I'm using a custom AVIOContext to bridge FFMpeg with java IO. The function avformat_open_input() never returns. I have searched the web for similar problems, all of which were caused by faulty network or wrong server configurations. However, I'm not using network at all, as you can see in the following little program:
package com.example;
import org.bytedeco.javacpp.*;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import static org.bytedeco.javacpp.avcodec.*;
import static org.bytedeco.javacpp.avformat.*;
import static org.bytedeco.javacpp.avutil.*;
import static org.bytedeco.javacpp.avdevice.*;
import static org.bytedeco.javacpp.avformat.AVFormatContext.*;
public class Test {
public static void main(String[] args) throws Exception {
File dir = new File(System.getProperty("user.home"), "Desktop");
File file = new File(dir, "sample.3gp");
final RandomAccessFile raf = new RandomAccessFile(file, "r");
Loader.load(avcodec.class);
Loader.load(avformat.class);
Loader.load(avutil.class);
Loader.load(avdevice.class);
Loader.load(swscale.class);
Loader.load(swresample.class);
avcodec_register_all();
av_register_all();
avformat_network_init();
avdevice_register_all();
Read_packet_Pointer_BytePointer_int reader = new Read_packet_Pointer_BytePointer_int() {
#Override
public int call(Pointer pointer, BytePointer buf, int bufSize) {
try {
byte[] data = new byte[bufSize]; // this is inefficient, just use as a quick example
int read = raf.read(data);
if (read <= 0) {
System.out.println("EOF found.");
return AVERROR_EOF;
}
System.out.println("Successfully read " + read + " bytes of data.");
buf.position(0);
buf.put(data, 0, read);
return read;
} catch (Exception ex) {
ex.printStackTrace();
return -1;
}
}
};
Seek_Pointer_long_int seeker = new Seek_Pointer_long_int() {
#Override
public long call(Pointer pointer, long offset, int whence) {
try {
raf.seek(offset);
System.out.println("Successfully seeked to position " + offset + ".");
return offset;
} catch (IOException ex) {
return -1;
}
}
};
int inputBufferSize = 32768;
BytePointer inputBuffer = new BytePointer(av_malloc(inputBufferSize));
AVIOContext ioContext = avio_alloc_context(inputBuffer, inputBufferSize, 1, null, reader, null, seeker);
AVInputFormat format = av_find_input_format("3gp");
AVFormatContext formatContext = avformat_alloc_context();
formatContext.iformat(format);
formatContext.flags(formatContext.flags() | AVFMT_FLAG_CUSTOM_IO);
formatContext.pb(ioContext);
// This never returns. And I can never get result.
int result = avformat_open_input(formatContext, "", format, null);
// all clean-up code omitted for simplicity
}
}
And below is my sample console output:
Successfully read 32768 bytes of data.
Successfully read 32768 bytes of data.
Successfully read 32768 bytes of data.
Successfully read 32768 bytes of data.
Successfully read 32768 bytes of data.
Successfully read 7240 bytes of data.
EOF found.
I've checked the sum of bytes, which corresponds to the file size; EOF is also hit, meaning the file is completely read. Actually I am a bit skeptical as why avformat_open_input() would even read the entire file and still without returning? There must be something wrong with what I am doing. Can any expert shed some lights or point me to the right direction? I'm new to javacv and ffmpeg and especially to programming with Buffers and stuff. Any help, suggestion or criticism is welcome. Thanks in advance.
Ok, now I have found the problem. I have misinterpreted the docs and overlooked most of the examples I found. My bad.
According to the documentation on ffmpeg:
AVIOContext* avio_alloc_context (unsigned char* buffer,
int buffer_size,
int write_flag,
void* opaque,
int(*)(void *opaque, uint8_t *buf, int buf_size) read_packet,
int(*)(void *opaque, uint8_t *buf, int buf_size) write_packet,
int64_t(*)(void *opaque, int64_t offset, int whence) seek
)
The third parameter, write_flag is used in the following fashion:
write_flag - Set to 1 if the buffer should be writable, 0 otherwise.
Actually, it means if the AVIOContext is for data output (i.e. writing), write_flag should be set to 1. Otherwise, if the context is for data input (i.e. reading), it should be set to 0.
In the question I posted, I passed 1 as the write_flag and it is causing the problem when reading. Passing 0 instead solves the problem.
Later I re-read all the examples I found, all the avio_alloc_context() calls uses 0, not 1 when reading. So that further indicates why I'm having the problem.
To conclude, I will post the revised code with the problems corrected as a future reference.
package com.example;
import org.bytedeco.javacpp.*;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import static org.bytedeco.javacpp.avformat.*;
import static org.bytedeco.javacpp.avutil.*;
import static org.bytedeco.javacpp.avformat.AVFormatContext.*;
public class Test {
public static void main(String[] args) throws Exception {
File dir = new File(System.getProperty("user.home"), "Desktop");
File file = new File(dir, "sample.3gp");
final RandomAccessFile raf = new RandomAccessFile(file, "r");
Loader.load(avformat.class);
Loader.load(avutil.class);
av_register_all();
avformat_network_init();
Read_packet_Pointer_BytePointer_int reader = new Read_packet_Pointer_BytePointer_int() {
#Override
public int call(Pointer pointer, BytePointer buf, int bufSize) {
try {
byte[] data = new byte[bufSize]; // this is inefficient, just use as a quick example
int read = raf.read(data);
if (read <= 0) {
// I am still unsure as to return '0', '-1' or 'AVERROR_EOF'.
// But according to the following link, it should return 'AVERROR_EOF',
// http://www.codeproject.com/Tips/489450/Creating-Custom-FFmpeg-IO-Context
// btw 'AVERROR_EOF' is a nasty negative number, '-541478725'.
return AVERROR_EOF;
}
buf.position(0);
buf.put(data, 0, read);
return read;
} catch (Exception ex) {
ex.printStackTrace();
return -1;
}
}
};
Seek_Pointer_long_int seeker = new Seek_Pointer_long_int() {
#Override
public long call(Pointer pointer, long offset, int whence) {
try {
if (whence == AVSEEK_SIZE) {
// Returns the entire file length. If not supported, simply returns a negative number.
// https://www.ffmpeg.org/doxygen/trunk/avio_8h.html#a427ff2a881637b47ee7d7f9e368be63f
return raf.length();
}
raf.seek(offset);
return offset;
} catch (IOException ex) {
ex.printStackTrace();
return -1;
}
}
};
int inputBufferSize = 32768;
BytePointer inputBuffer = new BytePointer(av_malloc(inputBufferSize));
AVIOContext ioContext = avio_alloc_context(inputBuffer,
inputBufferSize,
0, // CRITICAL, if the context is for reading, it should be ZERO
// if the context is for writing, then it is ONE
null,
reader,
null,
seeker);
AVInputFormat format = av_find_input_format("3gp");
AVFormatContext formatContext = avformat_alloc_context();
formatContext.iformat(format);
formatContext.flags(formatContext.flags() | AVFMT_FLAG_CUSTOM_IO);
formatContext.pb(ioContext);
// Now this is working properly.
int result = avformat_open_input(formatContext, "", format, null);
System.out.println("result == " + result);
// all clean-up code omitted for simplicity
}
}
References:
AVSEEK_SIZE documentation
avio_alloc_context() documentation
Additional References: (I do not have enough reputation points for more links but I found these examples critical in helping me so I pasted them in plain text anyway)
Creating Custom FFmpeg IO-Context (CodeProject Example) at:
http://www.codeproject.com/Tips/489450/Creating-Custom-FFmpeg-IO-Context
Another example showing the use of write_flag in avio_alloc_context() at:
https://www.ffmpeg.org/doxygen/2.5/avio_reading_8c-example.html#a20
Your seek code needs to handle AVSEEK_SIZE as whence, and your read should return 0 on EOF ("Upon reading end-of-file, zero is returned." - literal quote from man 2 read), not AVERROR_EOF.
I'm quite new to programming and I'd like help/guidance on this concept please - "separating the user interface from logic". I'm assigned a simple task of designing a currency converter (it's pretty much a very basic code as it asks for the conversion rate - but that's the given spec) and below is the code I've used:
public class CurrencyConverter
{
public static void main(String[] args)
{
System.out.println("Welcome to Currency converter!" + "\n");
System.out.println("Please select an option below: ");
System.out.println("1 >> Pound to Euro " +
"\n2 >> Euro to Pound");
Scanner s = new Scanner(System.in);
String selection = s.next();
switch (selection)
{
case "1":
System.out.println("Enter the conversion rate from Pound to Euro");
break;
case "2":
System.out.println("Enter the conversion rate from Euro to Pound");
break;
}
Scanner rate = new Scanner(System.in);
double conversionRate = rate.nextDouble();
System.out.println("Now enter the amount to be converted: ");
Scanner input = new Scanner(System.in);
double amount = input.nextDouble();
double totalValue = conversionRate * amount;
System.out.println(totalValue);
}
There's a specific instruction to not do it this way (i.e. just one class) and also to not use GUI. Two classes are to be used, one for the user interface and the other for the conversion (logic). I've tried putting the code below in a second class "Conversion" but it doesn't work.
double totalValue = conversionRate * amount;
Any tip/help would be much appreciated! By the way, if you know how to do it using GUI, that'd be helpful too. But of course my main problem is the one above.
Thanks.
Separating ui from logic allows you to increase scalability and readability of your code. In your case one class should provide methods to support interaction with user in order to provide necessary data and other that will handle convertion process. Both classes should not be aware of each other.
For example ui can be implemented as follows:
public class ConverterUI {
public Double askForConversionRate() {
System.out.println("Welcome to Currency converter!" + "\n");
System.out.println("Please select an option below: ");
System.out.println("1 >> Pound to Euro "
+ "\n2 >> Euro to Pound");
Scanner s = new Scanner(System.in);
String selection = s.next();
switch (selection) {
case "1":
System.out.println("Enter the conversion rate from Pound to Euro");
break;
case "2":
System.out.println("Enter the conversion rate from Euro to Pound");
break;
}
Scanner rate = new Scanner(System.in);
return rate.nextDouble();
}
public Double askForAmountToConvert() {
System.out.println("Now enter the amount to be converted: ");
Scanner input = new Scanner(System.in);
return input.nextDouble();
}
public void showResult(Double result) {
System.out.println("Convertion result is: " + result);
}
}
It has three simple methods that you can use to get data from user but there is no information about what to do with it. Converter class is responsible for this part:
public class Converter {
private Double conversionRate;
private Double amount;
public Double convert() {
return conversionRate * amount;
}
public void setConversionRate(Double conversionRate) {
this.conversionRate = conversionRate;
}
public void setAmount(Double amount) {
this.amount = amount;
}
}
It allows you to set convertion rate and amount values and make some calculations with convert() method. It doesn't need to know what is the source of data - you should support valid arguments and it will provide response.
In this situation main class can look like this:
public class Main {
public static void main(String[] args) {
ConverterUI ui = new ConverterUI();
Converter converter = new Converter();
Double convertionRate = ui.askForConversionRate();
converter.setConversionRate(convertionRate);
Double amount = ui.askForAmountToConvert();
converter.setAmount(amount);
Double result = converter.convert();
ui.showResult(result);
}
}
We create two objects - ui and converter. Calling ui methods provides data which are set to converter, and result of convertion is passed back to ui to inform user.
This approach allows you to add new UI (defining an interfaces would be a nice idea) and converter implementations without need of editing existing ones.
If you make another class called Conversion, it will not have access to the conversionRate and amount variables, since those variables only exist within the main method of the CurrencyConverter object.
You would have to create an object of the Conversion class and pass conversionRate and amount as method parameters. You might think about passing conversionRate to the constructor so that you could use the same rate to convert many different amounts, only passing the amount each time:
// Construct an object of the Conversion class with
// conversionRate as a parameter.
Conversion c = new Conversion(conversionRate);
// Call the convert method of the Conversion object using amount as a param.
System.out.println("Value of " + amountOne +
" Pounds in Euros: " + c.convert(amountOne));
System.out.println("Value of " + amountTwo +
" Pounds in Euros: " + c.convert(amountTwo));
You could later just assign c to a new Conversion object if you wanted to change the rate:
c = new Conversion(aDifferentRate);
The same goes for your Scanner objects. You can reuse the same variable over and over again. In your example, I don't think you even need to assign it to a new object, but you could:
Scanner in = new Scanner(System.in);
String str = in.next();
in = new Scanner(3.14159);
double d = in.nextDouble();
I wanted to know where I can find an implementation of the Bloom filter, with some explanation about the choice of the hash functions.
Additionally I have the following questions:
1) the Bloom filter is known to have false positives. Is it possible to reduce them by using two filters, one for used elements, and one for non-used elements (assuming the set is finite and known a-priori) and compare the two?
2) are there other similar algorithms in the CS literature?
My intuition is that you'll get a better reduction in false positives by using the additional space that the anti-filter would have occupied to just expand the positive filter.
As for resources, the papers referenced for March 8 from my course syllabus would be useful.
An Java implementation of the Bloom filter can be found from here. In case you cannot view it, I will paste the code in the following (with comments in Chinese).
import java.util.BitSet;
publicclass BloomFilter
{
/* BitSet初始分配2^24个bit */
privatestaticfinalint DEFAULT_SIZE =1<<25;
/* 不同哈希函数的种子，一般应取质数 */
privatestaticfinalint[] seeds =newint[] { 5, 7, 11, 13, 31, 37, 61 };
private BitSet bits =new BitSet(DEFAULT_SIZE);
/* 哈希函数对象 */
private SimpleHash[] func =new SimpleHash[seeds.length];
public BloomFilter()
{
for (int i =0; i < seeds.length; i++)
{
func[i] =new SimpleHash(DEFAULT_SIZE, seeds[i]);
}
}
// 将字符串标记到bits中
publicvoid add(String value)
{
for (SimpleHash f : func)
{
bits.set(f.hash(value), true);
}
}
//判断字符串是否已经被bits标记
publicboolean contains(String value)
{
if (value ==null)
{
returnfalse;
}
boolean ret =true;
for (SimpleHash f : func)
{
ret = ret && bits.get(f.hash(value));
}
return ret;
}
/* 哈希函数类 */
publicstaticclass SimpleHash
{
privateint cap;
privateint seed;
public SimpleHash(int cap, int seed)
{
this.cap = cap;
this.seed = seed;
}
//hash函数，采用简单的加权和hash
publicint hash(String value)
{
int result =0;
int len = value.length();
for (int i =0; i < len; i++)
{
result = seed * result + value.charAt(i);
}
return (cap -1) & result;
}
}
}
In terms of designing Bloom filter, the number of hash functions your bloom filter need can be determined as in here also refering the Wikipedia article about Bloom filters, then you find a section Probability of false positives. This section explains how the number of hash functions influences the probabilities of false positives and gives you the formula to determine k from the desired expected prob. of false positives.
Quote from the Wikipedia article:
Obviously, the probability of false
positives decreases as m (the number
of bits in the array) increases, and
increases as n (the number of inserted
elements) increases. For a given m and
n, the value of k (the number of hash
functions) that minimizes the
probability is
It's very easy to implement a Bloom filter using Java 8 features. You just need a long[] to store the bits, and a few hash functions, which you can represent with ToIntFunction<T>. I made a brief write up on doing this from scratch.
The part to be careful about is selecting the right bit from the array.
public class BloomFilter<T> {
private final long[] array;
private final int size;
private final List<ToIntFunction<T>> hashFunctions;
public BloomFilter(long[] array, int logicalSize, List<ToIntFunction<T>> hashFunctions) {
this.array = array;
this.size = logicalSize;
this.hashFunctions = hashFunctions;
}
public void add(T value) {
for (ToIntFunction<T> function : hashFunctions) {
int hash = mapHash(function.applyAsInt(value));
array[hash >>> 6] |= 1L << hash;
}
}
public boolean mightContain(T value) {
for (ToIntFunction<T> function : hashFunctions) {
int hash = mapHash(function.applyAsInt(value));
if ((array[hash >>> 6] & (1L << hash)) == 0) {
return false;
}
}
return true;
}
private int mapHash(int hash) {
return hash & (size - 1);
}
public static <T> Builder<T> builder() {
return new Builder<>();
}
public static class Builder<T> {
private int size;
private List<ToIntFunction<T>> hashFunctions;
public Builder<T> withSize(int size) {
this.size = size;
return this;
}
public Builder<T> withHashFunctions(List<ToIntFunction<T>> hashFunctions) {
this.hashFunctions = hashFunctions;
return this;
}
public BloomFilter<T> build() {
return new BloomFilter<>(new long[size >>> 6], size, hashFunctions);
}
}
}
I think we should look at the application of Bloom Filters, and the secret is in the name, it is a Filter and not a data-structure. It is mostly used for saving resources by checking whether items are not part of a set. If you want to minimize false positives to 0, you will have to insert all the items that aren't apart of the set, since there are no false negatives for a well designed Bloom Filter, except that bloom filter would be gigantic and unpractical, might as well just store the items in a skip-list :) I wrote a simple tutorial on Bloom Filters if anyone is interested.
http://techeffigy.wordpress.com/2014/06/05/bloom-filter-tutorial/