I want to classify sentences as positive, negative or neutral. I have got the code written in java but i am not understanding how to use the code for my file.In the above code where i should use my file and the results for each sentence should come as positive, negative and neutral. Please guide me for this.
public class SWN3 {
private String pathToSWN = "data"+File.separator+"SentiWordNet_3.0.0.txt";
private HashMap<String, String> _dict;
public SWN3(){
_dict = new HashMap<String, String>();
HashMap<String, Vector<Double>> _temp = new HashMap<String, Vector<Double>>();
try{
BufferedReader csv = new BufferedReader(new FileReader(pathToSWN));
String line = "";
while((line = csv.readLine()) != null)
{
String[] data = line.split("\t");
Double score = Double.parseDouble(data[2])- Double.parseDouble(data[3]);
String[] words = data[4].split(" ");
for(String w:words)
{
String[] w_n = w.split("#");
w_n[0] += "#"+data[0];
int index = Integer.parseInt(w_n[1])-1;
if(_temp.containsKey(w_n[0]))
{
Vector<Double> v = _temp.get(w_n[0]);
if(index>v.size())
for(int i = v.size();i<index; i++)
v.add(0.0);
v.add(index, score);
_temp.put(w_n[0], v);
}
else
{
Vector<Double> v = new Vector<Double>();
for(int i = 0;i<index; i++)
v.add(0.0);
v.add(index, score);
_temp.put(w_n[0], v);
}
}
}
Set<String> temp = _temp.keySet();
for (Iterator<String> iterator = temp.iterator(); iterator.hasNext();) {
String word = (String) iterator.next();
Vector<Double> v = _temp.get(word);
double score = 0.0;
double sum = 0.0;
for(int i = 0; i < v.size(); i++)
score += ((double)1/(double)(i+1))*v.get(i);
for(int i = 1; i<=v.size(); i++)
sum += (double)1/(double)i;
score /= sum;
String sent = "";
if(score>=0.75)
sent = "strong_positive";
else
if(score > 0.25 && score<=0.5)
sent = "positive";
else
if(score > 0 && score>=0.25)
sent = "weak_positive";
else
if(score < 0 && score>=-0.25)
sent = "weak_negative";
else
if(score < -0.25 && score>=-0.5)
sent = "negative";
else
if(score<=-0.75)
sent = "strong_negative";
_dict.put(word, sent);
}
}
catch(Exception e){e.printStackTrace();}
}
public String extract(String word, String pos)
{
return _dict.get(word+"#"+pos);
}
}
Related
in this methode there is only one instruction that not working:
private static byte[] encodePassword(String password,String salt) throws UnsupportedEncodingException
{
String mergedPasswordAndSalt =mergePasswordAndSalt(password, salt);
SHA512Digest digester =new SHA512Digest();
byte[] hash = new byte[digester.getDigestSize()];
digester.update(hash, 0, mergedPasswordAndSalt.length());
digester.doFinal(hash, 0);
System.out.println("init hash= "+Base64.encode(hash));
for (int i = 1; i < ITERATIONS; ++i) {
digester.update(hash, 0, mergedPasswordAndSalt.length());
digester.doFinal(Bytes.concat(hash, mergedPasswordAndSalt.getBytes("UTF-8")),0);
}
System.out.println("FINAL hash= "+Base64.encode(hash));
return hash;
}
that instructions: is the equivalent of this one in the java api that is way:
for (int i = 1; i < ITERATIONS; ++i) {
hash = digester.digest(Bytes.concat(hash, mergedPasswordAndSalt.getBytes("UTF-8")));
}
I have the solution:
To concat bytes i used this method:
public static byte[] concat(byte[]... arrays) {
int length = 0;
byte[][] arr$ = arrays;
int pos = arrays.length;
for(int i$ = 0; i$ < pos; ++i$) {
byte[] array = arr$[i$];
length += array.length;
}
byte[] result = new byte[length];
pos = 0;
byte[][] arr$$=arrays;
arr$=arr$$;
int len$ = arrays.length;
for(int i$ = 0; i$ < len$; ++i$) {
byte[] array = arr$[i$];
System.arraycopy(array, 0, result, pos, array.length);
pos += array.length;
}
return result;
}
To do 4999 iteration on the digest we need a method that takes the hash after every iteration and works in the current digest concatenated with the first diegest(generate out of loop):
private static byte[] encodePassword(String password,String salt) throws UnsupportedEncodingException
{
String mergedPasswordAndSalt =mergePasswordAndSalt(password, salt);
byte[] hash = new byte[88];
hash=digestt(mergedPasswordAndSalt.getBytes("UTF-8"));
for (int i = 1; i < ITERATIONS; ++i) {
hash=digestt(concat(hash,mergedPasswordAndSalt.getBytes("UTF-8")));
}
return hash;
}
public static byte[] digestt(byte[] bytes) {
Digest digest = new SHA512Digest();
byte[] resBuf = new byte[digest.getDigestSize()];
digest.update(bytes, 0, bytes.length);
digest.doFinal(resBuf, 0);
return resBuf;
}
thank you very much i asked many questions and you are always there for help.
I'm trying to make a decoder for images in processing, but it returns þ no matter what I put in. Help would be much appreciated.
void setup() {
//size(255, 255);
colorMode(HSB);
String result = decode();
println(result);
saveStrings("output.txt", new String[]{result});
}
String decode() {
PImage img = loadImage("code.png");
//image(img, 0, 0);
img.loadPixels();
String decoded = "";
int[] chars = new int[255 * 255];
for (int i = 0; i < 255; i++) {
for (int j = 0; j < 255; j++) {
color col = img.pixels[i * 255 + j];//get(i, j);
if (hue(col) == 0 && saturation(col) == 0) {
chars[floor(hue(col) * 255 + saturation(col))] = j;
println((char) j);
}
}
}
decoded = new String(chars, 0, chars.length);
String[] list = decoded.split("\u0000");
StringBuilder builder = new StringBuilder();
for (String string : list) {
builder.append(string);
}
decoded = builder.toString();
return decoded;
}
The main problem is likely at if (hue(col) == 0 && saturation(col) == 0) {
The next line will always have 0 for hue(col) and saturation(col). This means you are always executing chars[0] = j;, and your final result has only one character (the þ you described).
I am writing code for apriori algorithm in data mining my code takes as long as 60 seconds for a pretty small dataset which is solved by other code i got from internet in just 2 seconds but i am not getting where am i doing wrong, can someone tell me why the other code is fast over mine.
My code:
import java.util.*;
import java.io.*;
public class Apriori_p {
double support;
ArrayList<String> trans;
Map<String, Integer> map;
long start;
void print(ArrayList<String> temp) {
for (int i = 0; i < temp.size(); i++) {
System.out.println(temp.get(i));
}
System.out.println("Count :" + temp.size());
}
void run() throws FileNotFoundException {
start = System.currentTimeMillis();
trans = new ArrayList<>();
ArrayList<String> temp = new ArrayList<>();
map = new HashMap<>();
Scanner sc = new Scanner(System.in);
System.out.println("Enter support %");
support = sc.nextDouble();
System.out.println("Enter file name");
String file = sc.next();
sc = new Scanner(new File(file));
int lines = 0;
while (sc.hasNextLine()) {
String s = sc.nextLine();
if (s.matches("\\s*")) {
continue;
}
lines++;
String[] spl = s.split("\\s+");
ArrayList<Integer> elem = new ArrayList<>();
for (int i = 0; i < spl.length; i++) {
String cand;
int n = Integer.parseInt(spl[i]);
cand = spl[i].trim();
if (!elem.contains(n)) {
elem.add(n);
}
if (map.containsKey(cand)) {
int count = map.get(cand);
map.put(cand, count + 1);
} else {
map.put(cand, 1);
}
}
Collections.sort(elem);
String con = " ";
for (int i = 0; i < elem.size(); i++) {
con = con + elem.get(i) + " ";
String s1 = String.valueOf(elem.get(i)).trim();
if(!temp.contains(s1))
temp.add(s1);
}
trans.add(con);
}
support = (support * lines) / 100;
System.out.println(System.currentTimeMillis() - start);
apriori(temp, 1);
}
public static void main(String[] args) throws FileNotFoundException {
new Apriori_p().run();
}
public void apriori(ArrayList<String> temp, int m) {
Set<String> diff = null;
if (m == 1) {
diff = new HashSet<>();
}
for (int i = 0; i < temp.size(); i++) {
if (map.get(temp.get(i)) < support) {
if (m == 1) {
diff.add(temp.get(i));
}
temp.remove(i);
i--;
}
}
for (int i = 0; i < trans.size() && m == 1; i++) {
for (String j : diff) {
String rep = " " + j + " ";
trans.get(i).replace(rep, " ");
}
}
if (temp.size() == 0) {
return;
}
System.out.println("Size " + m + " :");
print(temp);
ArrayList<String> ntemp = new ArrayList<>();
int n = temp.size();
for (int i = 0; i < n; i++) {
for (int j = i + 1; j < n; j++) {
StringTokenizer st1 = new StringTokenizer(temp.get(i), " ");
StringTokenizer st2 = new StringTokenizer(temp.get(j), " ");
String str1 = "", str2 = "";
for (int s = 0; s < m - 2; s++) {
str1 = str1 + " " + st1.nextToken();
str2 = str2 + " " + st2.nextToken();
}
if (str2.compareToIgnoreCase(str1) == 0) {
int s1 = Integer.parseInt(st1.nextToken()), s2 = Integer.parseInt(st2.nextToken());
String s3;
if (s1 <= s2) {
s3 = (str1 + " " + s1 + " " + s2).trim();
} else {
s3 = (str1 + " " + s2 + " " + s1).trim();
}
if(!ntemp.contains(s3)){
ntemp.add(s3);
}
}
}
}
temp.clear();
for (int j = 0; j < ntemp.size(); j++) {
int c = 0;
for (int i = 0; i < trans.size(); i++) {
int check = 0;
String tr = trans.get(i);
StringTokenizer st1 = new StringTokenizer(ntemp.get(j)," ");
while(st1.hasMoreElements()){
String str = st1.nextToken();
if(!tr.contains(" " + str + " ")){
check = 1;
break;
}
}
if(check == 0){
c= 1;
if (map.containsKey(ntemp.get(j))) {
int count = map.get(ntemp.get(j));
map.put(ntemp.get(j), count + 1);
} else {
map.put(ntemp.get(j), 1);
}
}
}
if (c == 0) {
ntemp.remove(j);
j--;
}
}
apriori(ntemp, m + 1);
}
}
Fast code:
import java.io.*;
import java.util.*;
public class Apriori3{
public static void main(String[] args) throws Exception {
Apriori3 ap = new Apriori3(args);
}
private List<int[]> itemsets;
private String transaFile;
private int numItems;
private int numTransactions;
private double minSup;
private boolean usedAsLibrary = false;
public Apriori3(String[] args) throws Exception {
configure(args);
go();
}
private void go() throws Exception {
long start = System.currentTimeMillis();
createItemsetsOfSize1();
int itemsetNumber = 1;
int nbFrequentSets = 0;
while (itemsets.size() > 0) {
calculateFrequentItemsets();
if (itemsets.size() != 0) {
nbFrequentSets += itemsets.size();
log("Found " + itemsets.size() + " frequent itemsets of size " + itemsetNumber + " (with support " + (minSup * 100) + "%)");;
createNewItemsetsFromPreviousOnes();
}
itemsetNumber++;
}
long end = System.currentTimeMillis();
log("Execution time is: " + ((double) (end - start) / 1000) + " seconds.");
log("Found " + nbFrequentSets + " frequents sets for support " + (minSup * 100) + "% (absolute " + Math.round(numTransactions * minSup) + ")");
log("Done");
}
private void foundFrequentItemSet(int[] itemset, int support) {
if (usedAsLibrary) {
} else {
System.out.println(Arrays.toString(itemset) + " (" + ((support / (double) numTransactions)) + " " + support + ")");
}
}
private void log(String message) {
if (!usedAsLibrary) {
System.err.println(message);
}
}
private void configure(String[] args) throws Exception {
if (args.length != 0) {
transaFile = args[0];
} else {
transaFile = "chess.dat"; // default
}
if (args.length >= 2) {
minSup = (Double.valueOf(args[1]).doubleValue());
} else {
minSup = .8;
}
if (minSup > 1 || minSup < 0) {
throw new Exception("minSup: bad value");
}
numItems = 0;
numTransactions = 0;
BufferedReader data_in = new BufferedReader(new FileReader(transaFile));
while (data_in.ready()) {
String line = data_in.readLine();
if (line.matches("\\s*")) {
continue;
}
numTransactions++;
StringTokenizer t = new StringTokenizer(line, " ");
while (t.hasMoreTokens()) {
int x = Integer.parseInt(t.nextToken());
if (x + 1 > numItems) {
numItems = x + 1;
}
}
}
outputConfig();
}
private void outputConfig() {
log("Input configuration: " + numItems + " items, " + numTransactions + " transactions, ");
log("minsup = " + minSup + "%");
}
private void createItemsetsOfSize1() {
itemsets = new ArrayList<int[]>();
for (int i = 0; i < numItems; i++) {
int[] cand = {i};
itemsets.add(cand);
}
}
private void createNewItemsetsFromPreviousOnes() {
int currentSizeOfItemsets = itemsets.get(0).length;
log("Creating itemsets of size " + (currentSizeOfItemsets + 1) + " based on " + itemsets.size() + " itemsets of size " + currentSizeOfItemsets);
HashMap<String, int[]> tempCandidates = new HashMap<String, int[]>(); //temporary candidates
for (int i = 0; i < itemsets.size(); i++) {
for (int j = i + 1; j < itemsets.size(); j++) {
int[] X = itemsets.get(i);
int[] Y = itemsets.get(j);
assert (X.length == Y.length);
int[] newCand = new int[currentSizeOfItemsets + 1];
for (int s = 0; s < newCand.length - 1; s++) {
newCand[s] = X[s];
}
int ndifferent = 0;
for (int s1 = 0; s1 < Y.length; s1++) {
boolean found = false;
for (int s2 = 0; s2 < X.length; s2++) {
if (X[s2] == Y[s1]) {
found = true;
break;
}
}
if (!found) {
ndifferent++;
newCand[newCand.length - 1] = Y[s1];
}
}
assert (ndifferent > 0);
if (ndifferent == 1) {
Arrays.sort(newCand);
tempCandidates.put(Arrays.toString(newCand), newCand);
}
}
}
itemsets = new ArrayList<int[]>(tempCandidates.values());
log("Created " + itemsets.size() + " unique itemsets of size " + (currentSizeOfItemsets + 1));
}
private void line2booleanArray(String line, boolean[] trans) {
Arrays.fill(trans, false);
StringTokenizer stFile = new StringTokenizer(line, " ");
while (stFile.hasMoreTokens()) {
int parsedVal = Integer.parseInt(stFile.nextToken());
trans[parsedVal] = true;
}
}
private void calculateFrequentItemsets() throws Exception {
log("Passing through the data to compute the frequency of " + itemsets.size() + " itemsets of size " + itemsets.get(0).length);
List<int[]> frequentCandidates = new ArrayList<int[]>();
boolean match;
int count[] = new int[itemsets.size()];
BufferedReader data_in = new BufferedReader(new InputStreamReader(new FileInputStream(transaFile)));
boolean[] trans = new boolean[numItems];
for (int i = 0; i < numTransactions; i++) {
String line = data_in.readLine();
line2booleanArray(line, trans);
for (int c = 0; c < itemsets.size(); c++) {
match = true;
int[] cand = itemsets.get(c);
for (int xx : cand) {
if (trans[xx] == false) {
match = false;
break;
}
}
if (match) {
count[c]++;
}
}
}
data_in.close();
for (int i = 0; i < itemsets.size(); i++) {
if ((count[i] / (double) (numTransactions)) >= minSup) {
foundFrequentItemSet(itemsets.get(i), count[i]);
frequentCandidates.add(itemsets.get(i));
}
}
itemsets = frequentCandidates;
}
}
I have to set up a queue class that implements from a deque class. I need to use this to set up two deck cards with a random order. I have the code below, it works when the first deck is created but for some reason it does not work with the second deck, its the same code that im reusing.
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 5
at prog.pkg4.Deque.insertOnBack(Prog4.java:93)
at prog.pkg4.Queue.insert(Prog4.java:153)
at prog.pkg4.Prog4.createDeck(Prog4.java:465)
at prog.pkg4.Prog4.topTrump(Prog4.java:444)
at prog.pkg4.Prog4.main(Prog4.java:287)
initiates the two decks
Queue player = new Queue();
Queue computer = new Queue();
player = createDeck(player, cards);
computer = createDeck(computer, cards);
code to create random deck
public static Queue createDeck(Queue queue, GreekHero[] cards){
Random rand = new Random();
int temp = 0;
int r;
for(int i = 0; i < 30; i++){
r = rand.nextInt(30);
cards[temp] = cards[i];
cards[i] = cards[r];
cards[r] = cards[temp];
}
for(int i = 0; i < 29; i++){
queue.insert(cards[i]);
System.out.println(queue.insertions());
System.out.println(queue);
}
return queue;
}
class Queue{
private Deque queue;
public Queue(){
queue = new Deque();
}
public void insert(Object o){
queue.insertOnBack(o);
}
public Object delete(){
return queue.deleteFromFront();
}
public boolean isEmpty(){
return queue.isEmpty();
}
public String toString(){
return queue.toString();
}
public int insertions(){
return queue.getInsertions();
}
}
i've tested out the deque code several times i know it works, as demonstrated by the first deck that is created, im just not sure what could be causing the problem for the second deck.
EDIT: I've added the Deque class code below, the way i have this set up is that if the number of insertions equals the size of the array, it should double in size, as mentioned before it works with the first deque but on the second deque it stops at size of array - 1. I've increased the size to test out and I could make it bigger to satisfy this project but I need to create a deque with an increasing array.
class Deque{
private Object[] arrayObject;
private int beggining; //tracks first element in array
private int insertions; //counts the items in the array
private static int SIZE = 30; //size of array
public Deque(){
arrayObject = new Object[SIZE];
beggining = 0;
insertions = 0;
}
// displays position of first element in circular array
public Object getBeggining(){
int temp = beggining + 1;
if(temp == SIZE)
temp = 0;
return temp;
}
public int getInsertions(){
return insertions;
}
public Object indexOne(){
int temp = beggining + 1;
if(temp == SIZE)
temp = 0;
return arrayObject[temp];
}
public String toString(){
if(isEmpty())
return "Empty";
int temp = beggining + 1;
if( temp >= SIZE)
temp = 0;
String s = "Current Index:\n[("+arrayObject[temp]+")";
int loops = 0;
for(int i = temp + 1; loops < insertions - 1; i++){
if(i >= SIZE)
i = 0;
s += ", ("+arrayObject[i]+")";
loops++;
}
s += "]";
return s;
}
public String toStore(){
String s = "Store Index:\n[(1: "+arrayObject[1]+")";
for(int i = 1; i <= SIZE - 1; i++)
s += ", ("+(i+1)+": "+arrayObject[i]+")";
s += "]";
return s;
}
public void insertOnFront(Object o){
if(insertions == SIZE)
arrayObject = increaseArray();
arrayObject[beggining] = o;
beggining--;
if(beggining < 0)
beggining = SIZE - 1;
insertions++;
}
public Object deleteFromFront(){
if(isEmpty())
return null;
int count = beggining + 1;
if(count >= SIZE)
count = 0;
Object temp = arrayObject[count];
beggining += 1;
insertions--;
if(insertions > 0)
insertions = 0;
return temp;
}
public void insertOnBack(Object o){
int temp = beggining + insertions + 1;
if(insertions == SIZE - 1)
arrayObject = increaseArray();
if(temp >= SIZE)
temp = 0 + (temp - SIZE);
arrayObject[temp] = o;
insertions++;
}
public Object deleteFromBack(){
if(isEmpty())
return null;
int count = beggining + insertions;
Object temp = arrayObject[count];
insertions--;
if(insertions >= 0)
insertions = 0;
return temp;
}
public boolean isEmpty(){
if(insertions > 0)
return false;
else
return true;
}
public Object[] increaseArray(){
SIZE *= 2;
int loops = 0;
int j = beggining;
Object[] newArray = new Object[SIZE];
for(int i = j; loops <= SIZE/2; i++){
if(j >= SIZE/2)
j = 0;
newArray[i] = arrayObject[j];
loops++;
j++;
}
return newArray;
}
}
I solved the issue by moving the SIZE variable as an instance variable of the class and removed static from it. I don't know why the issue popped in on the second iteration rather than on the first try, ill look it up later, if anyone knows please post it here.
I have a custom class MW. MW gets 2 matrices-(ke matrix and val matrix).
And I am trying to sumup all the matrices coming into reducer.
So I need to first parse my string and I stored them into 2 double array.
I am geting all the ke matrix and val matrix in reducer.
But I am not able to sumup.
Any suggestion.
Inorder to get the sum outside the forloop,i declared them as static.
public class Reducer extends
Reducer<IntWritable, MW, Text, Text> {
static double[][] key;
static double[][] value;
public void reduce(IntWritable keys,
Iterable<MW> values, Context context)
throws IOException, InterruptedException {
for (MW c : values)
{
String data = c.toString();
data = data.trim();
String[] parts = data.split("#");
String part1 = parts[0];
String part2 = parts[1];
/*
* Parse key
*/
String[] keyrows = part1.split(",");
String[][] keymatrix = new String[keyrows.length][];
int keyr = 0;
for (String keyrow : keyrows) {
keymatrix[keyr++] = keyrow.split("\\|");
}
double[][] ke = new double[keymatrix.length][keymatrix[0].length];
for (int i = 0; i<keymatrix.length; i++) {
for (int j = 0; j<keymatrix[0].length; j++) {
ke[i][j] = Double.valueOf(keymatrix[i][j]);
}
}
key = new double[ke.length][ke[0].length];
for(int sumi = 0;sumi<ke.length;sumi++){
for(int sumj=0;sumj<ke[0].length;sumj++){
key[sumi][sumj] += ke[sumi][sumj];
}
}
/*Parsing value
*/
String[] valuerows = part2.split(",");
String[][] valuematrix = new String[valuerows.length][];
int valr = 0;
for (String valuerow : valuerows) {
valuematrix[valr++] = valuerow.split("\\|");
}
double[][] val = new double[valuematrix.length][valuematrix[0].length];
for (int i = 0; i<valuematrix.length; i++) {
for (int j = 0; j<valuematrix[0].length; j++) {
val[i][j] = Double.valueOf(valuematrix[i][j]);
}
}
//calculating sum for value
value = new double[val.length][val[0].length];
for(int sumi = 0;sumi<val.length;sumi++){
for(int sumj=0;sumj<val[0].length;sumj++){
value[sumi][sumj] += val[sumi][sumj];
}
}
}
System.out.println("sum 1");
for(int diai=0;diai<key.length;diai++){
for(int diaj=0;diaj<key[0].length;diaj++){
System.out.print(key[diai][diaj]+"\t");
}
System.out.println("");
}
System.out.println("sum 2");
for(int diai=0;diai<value.length;diai++){
for(int diaj=0;diaj<value[0].length;diaj++){
System.out.print(value[diai][diaj]+"\t");
}
System.out.println("");
}
UPDATE
I think the problem is with in line
key = new double[ke.length][ke[0].length];
and
value = new double[val.length][val[0].length];
before summing I am rebuilding the matrix key and value inside the loop.
It should build it once before the loop and then add to it.
But to do that I should do
double[][] key = new double[ke.length][ke[0].length];
double[][] value = new double[val.length][val[0].length];
before
for (MW c : values)
{
but
How will I get the dimensions outside the for loop?
yes i solved the problem .
i emitted the dimensions as key to reducer. It worked.