I'm playing around with CompletableFuture and streams in Java 8 and I get different printouts each time I run this. Just curious, why?
public class DoIt {
public static class My {
private long cur = 0;
public long next() {
return cur++;
}
}
public static long add() {
long sum = 0;
for (long i=0; i<=100;i++) {
sum += i;
}
return sum;
}
public static long getResult(CompletableFuture<Long> f) {
long l = 0;
try {
f.complete(42l);
l = f.get();
System.out.println(l);
} catch (Exception e) {
//...
}
return l;
}
public static void main(String[] args){
ExecutorService exec = Executors.newFixedThreadPool(2);
My my = new My();
long sum = Stream.generate(my::next).limit(20000).
map(x -> CompletableFuture.supplyAsync(()-> add(), exec)).
mapToLong(f->getResult(f)).sum();
System.out.println(sum);
exec.shutdown();
}
}
If I skip the f.complete(42l) call I always get the same result.
http://download.java.net/jdk8/docs/api/java/util/concurrent/CompletableFuture.html#complete-T-
by the time the complete(42l) call happens, some add()'s may have already completed.
Related
public class Grade {
private int [] array = {2,3,1,4,5,7,1};
public int findSum() {
int sum;
sum = 0;
for(int i =0; i <array.length; i++)
{
sum = sum +array[i];
}
return sum;
}
public double findAverage() {
double average;
average = findSum()/array.length;
return average;
}
}
class ExamClient {
public static void main(String[] args) {
double answer;
answer = findAverage();
System.out.println("Average of all elements in the array is" + answer);
}
}
In the main you have to create a instance of the class
Create instance
public static void main(String[] args)
{
double answer;
Grade g= new Grade();
answer = g.findAverage();
System.out.println("Average of all elements in the array is" + answer);
}
Also you can make the method static
I need a specialized queue-like data structure. It can be used by multiple consumers, but each item in queue must be removed from queue after k consumers read it.
Is there any production ready implementation? Or Should I implement a queue with read-counter in each item, and handle item removal myself?
Thanks in advance.
I think this is what you are looking for. Derived from the source code for BlockingQueue. Caveat emptor, not tested.
I tried to find a way to wrap Queue, but Queue doesn't expose its concurrency members, so you can't get the right semantics.
public class CountingQueue<E> {
private class Entry {
Entry(int count, E element) {
this.count = count;
this.element = element;
}
int count;
E element;
}
public CountingQueue(int capacity) {
if (capacity <= 0) {
throw new IllegalArgumentException();
}
this.items = new Object[capacity];
this.lock = new ReentrantLock(false);
this.condition = this.lock.newCondition();
}
private final ReentrantLock lock;
private final Condition condition;
private final Object[] items;
private int takeIndex;
private int putIndex;
private int count;
final int inc(int i) {
return (++i == items.length) ? 0 : i;
}
final int dec(int i) {
return ((i == 0) ? items.length : i) - 1;
}
private static void checkNotNull(Object v) {
if (v == null)
throw new NullPointerException();
}
/**
* Inserts element at current put position, advances, and signals.
* Call only when holding lock.
*/
private void insert(int count, E x) {
items[putIndex] = new Entry(count, x);
putIndex = inc(putIndex);
if (count++ == 0) {
// empty to non-empty
condition.signal();
}
}
private E extract() {
Entry entry = (Entry)items[takeIndex];
if (--entry.count <= 0) {
items[takeIndex] = null;
takeIndex = inc(takeIndex);
if (count-- == items.length) {
// full to not-full
condition.signal();
}
}
return entry.element;
}
private boolean waitNotEmpty(long timeout, TimeUnit unit) throws InterruptedException {
long nanos = unit.toNanos(timeout);
while (count == 0) {
if (nanos <= 0) {
return false;
}
nanos = this.condition.awaitNanos(nanos);
}
return true;
}
private boolean waitNotFull(long timeout, TimeUnit unit) throws InterruptedException {
long nanos = unit.toNanos(timeout);
while (count == items.length) {
if (nanos <= 0)
return false;
nanos = condition.awaitNanos(nanos);
}
return true;
}
public boolean put(int count, E e) {
checkNotNull(e);
final ReentrantLock localLock = this.lock;
localLock.lock();
try {
if (count == items.length)
return false;
else {
insert(count, e);
return true;
}
} finally {
localLock.unlock();
}
}
public boolean put(int count, E e, long timeout, TimeUnit unit)
throws InterruptedException {
checkNotNull(e);
final ReentrantLock localLock = this.lock;
localLock.lockInterruptibly();
try {
if (!waitNotFull(timeout, unit)) {
return false;
}
insert(count, e);
return true;
} finally {
localLock.unlock();
}
}
public E get() {
final ReentrantLock localLock = this.lock;
localLock.lock();
try {
return (count == 0) ? null : extract();
} finally {
localLock.unlock();
}
}
public E get(long timeout, TimeUnit unit) throws InterruptedException {
final ReentrantLock localLock = this.lock;
localLock.lockInterruptibly();
try {
if (waitNotEmpty(timeout, unit)) {
return extract();
} else {
return null;
}
} finally {
localLock.unlock();
}
}
public int size() {
final ReentrantLock localLock = this.lock;
localLock.lock();
try {
return count;
} finally {
localLock.unlock();
}
}
public boolean isEmpty() {
final ReentrantLock localLock = this.lock;
localLock.lock();
try {
return count == 0;
} finally {
localLock.unlock();
}
}
public int remainingCapacity() {
final ReentrantLock lock= this.lock;
lock.lock();
try {
return items.length - count;
} finally {
lock.unlock();
}
}
public boolean isFull() {
final ReentrantLock localLock = this.lock;
localLock.lock();
try {
return items.length - count == 0;
} finally {
localLock.unlock();
}
}
public void clear() {
final ReentrantLock localLock = this.lock;
localLock.lock();
try {
for (int i = takeIndex, k = count; k > 0; i = inc(i), k--)
items[i] = null;
count = 0;
putIndex = 0;
takeIndex = 0;
condition.signalAll();
} finally {
localLock.unlock();
}
}
}
A memory efficient way that retains the info you need:
Each queue entry becomes a
Set<ConsumerID>
so that you ensure the k times are for k distinct consumers: your app logic checks if the
set.size()==k
and removes it from queue in that case.
In terms of storage: you will have tradeoffs of which Set implementation based on
size and type of the ConsumerID
speed of retrieval requirement
E.g if k is very small and your queue retrieval logic has access to a
Map<ID,ConsumerId>
then you could have simply an Int or even a Short or Byte depending on # distinct ConsumerID's and possibly store in an Array . This is slower than accessing a set since it would be traversed linearly - but for small K that may be reasonable.
No matter how simple I make the compareTo of my complex key, I don't get expected results. With the exception of if I use one key that is the same for every record, it will appropriately reduce to one record. I've also witnessed that this happens only when I process the full load, if I break off a few of the records that didn't reduce and run it on a much smaller scale those records get combined.
The sum of the output records is correct, but there is duplication at the record level of items I would have expected to group together. So where I would expect say 500 records summing up to 5,000, I end up with 1232 records summing up to 5,000 with obvious records that should have been reduced into one.
I've read about the problems with object references and complex keys and values, but I don't see anywhere that I have potential for that left. To that end you will find places that I'm creating new objects that I probably don't need to, but I'm trying everything at this point and will dial it back once it is working.
I'm out of ideas on what to try or where and how to poke to figure this out. Please help!
public static class Map extends
Mapper<LongWritable, Text, IMSTranOut, IMSTranSums> {
//private SimpleDateFormat dtFormat = new SimpleDateFormat("yyyyddd");
#Override
public void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException {
String line = value.toString();
SimpleDateFormat dtFormat = new SimpleDateFormat("yyyyddd");
IMSTranOut dbKey = new IMSTranOut();
IMSTranSums sumVals = new IMSTranSums();
String[] tokens = line.split(",", -1);
dbKey.setLoadKey(-99);
dbKey.setTranClassKey(-99);
dbKey.setTransactionCode(tokens[0]);
dbKey.setTransactionType(tokens[1]);
dbKey.setNpaNxx(getNPA(dbKey.getTransactionCode()));
try {
dbKey.setTranDate(new Date(dtFormat.parse(tokens[2]).getTime()));
} catch (ParseException e) {
}// 2
dbKey.setTranHour(getTranHour(tokens[3]));
try {
dbKey.setStartDate(new Date(dtFormat.parse(tokens[4]).getTime()));
} catch (ParseException e) {
}// 4
dbKey.setStartHour(getTranHour(tokens[5]));
try {
dbKey.setStopDate(new Date(dtFormat.parse(tokens[6]).getTime()));
} catch (ParseException e) {
}// 6
dbKey.setStopHour(getTranHour(tokens[7]));
sumVals.setTranCount(1);
sumVals.setInputQTime(Double.parseDouble(tokens[8]));
sumVals.setElapsedTime(Double.parseDouble(tokens[9]));
sumVals.setCpuTime(Double.parseDouble(tokens[10]));
context.write(dbKey, sumVals);
}
}
public static class Reduce extends
Reducer<IMSTranOut, IMSTranSums, IMSTranOut, IMSTranSums> {
#Override
public void reduce(IMSTranOut key, Iterable<IMSTranSums> values,
Context context) throws IOException, InterruptedException {
int tranCount = 0;
double inputQ = 0;
double elapsed = 0;
double cpu = 0;
for (IMSTranSums val : values) {
tranCount += val.getTranCount();
inputQ += val.getInputQTime();
elapsed += val.getElapsedTime();
cpu += val.getCpuTime();
}
IMSTranSums sumVals=new IMSTranSums();
IMSTranOut dbKey=new IMSTranOut();
sumVals.setCpuTime(inputQ);
sumVals.setElapsedTime(elapsed);
sumVals.setInputQTime(cpu);
sumVals.setTranCount(tranCount);
dbKey.setLoadKey(key.getLoadKey());
dbKey.setTranClassKey(key.getTranClassKey());
dbKey.setNpaNxx(key.getNpaNxx());
dbKey.setTransactionCode(key.getTransactionCode());
dbKey.setTransactionType(key.getTransactionType());
dbKey.setTranDate(key.getTranDate());
dbKey.setTranHour(key.getTranHour());
dbKey.setStartDate(key.getStartDate());
dbKey.setStartHour(key.getStartHour());
dbKey.setStopDate(key.getStopDate());
dbKey.setStopHour(key.getStopHour());
dbKey.setInputQTime(inputQ);
dbKey.setElapsedTime(elapsed);
dbKey.setCpuTime(cpu);
dbKey.setTranCount(tranCount);
context.write(dbKey, sumVals);
}
}
Here is the implementation of the DBWritable class:
public class IMSTranOut implements DBWritable,
WritableComparable<IMSTranOut> {
private int loadKey;
private int tranClassKey;
private String npaNxx;
private String transactionCode;
private String transactionType;
private Date tranDate;
private double tranHour;
private Date startDate;
private double startHour;
private Date stopDate;
private double stopHour;
private double inputQTime;
private double elapsedTime;
private double cpuTime;
private int tranCount;
public void readFields(ResultSet rs) throws SQLException {
setLoadKey(rs.getInt("LOAD_KEY"));
setTranClassKey(rs.getInt("TRAN_CLASS_KEY"));
setNpaNxx(rs.getString("NPA_NXX"));
setTransactionCode(rs.getString("TRANSACTION_CODE"));
setTransactionType(rs.getString("TRANSACTION_TYPE"));
setTranDate(rs.getDate("TRAN_DATE"));
setTranHour(rs.getInt("TRAN_HOUR"));
setStartDate(rs.getDate("START_DATE"));
setStartHour(rs.getInt("START_HOUR"));
setStopDate(rs.getDate("STOP_DATE"));
setStopHour(rs.getInt("STOP_HOUR"));
setInputQTime(rs.getInt("INPUT_Q_TIME"));
setElapsedTime(rs.getInt("ELAPSED_TIME"));
setCpuTime(rs.getInt("CPU_TIME"));
setTranCount(rs.getInt("TRAN_COUNT"));
}
public void write(PreparedStatement ps) throws SQLException {
ps.setInt(1, loadKey);
ps.setInt(2, tranClassKey);
ps.setString(3, npaNxx);
ps.setString(4, transactionCode);
ps.setString(5, transactionType);
ps.setDate(6, tranDate);
ps.setDouble(7, tranHour);
ps.setDate(8, startDate);
ps.setDouble(9, startHour);
ps.setDate(10, stopDate);
ps.setDouble(11, stopHour);
ps.setDouble(12, inputQTime);
ps.setDouble(13, elapsedTime);
ps.setDouble(14, cpuTime);
ps.setInt(15, tranCount);
}
public int getLoadKey() {
return loadKey;
}
public void setLoadKey(int loadKey) {
this.loadKey = loadKey;
}
public int getTranClassKey() {
return tranClassKey;
}
public void setTranClassKey(int tranClassKey) {
this.tranClassKey = tranClassKey;
}
public String getNpaNxx() {
return npaNxx;
}
public void setNpaNxx(String npaNxx) {
this.npaNxx = new String(npaNxx);
}
public String getTransactionCode() {
return transactionCode;
}
public void setTransactionCode(String transactionCode) {
this.transactionCode = new String(transactionCode);
}
public String getTransactionType() {
return transactionType;
}
public void setTransactionType(String transactionType) {
this.transactionType = new String(transactionType);
}
public Date getTranDate() {
return tranDate;
}
public void setTranDate(Date tranDate) {
this.tranDate = new Date(tranDate.getTime());
}
public double getTranHour() {
return tranHour;
}
public void setTranHour(double tranHour) {
this.tranHour = tranHour;
}
public Date getStartDate() {
return startDate;
}
public void setStartDate(Date startDate) {
this.startDate = new Date(startDate.getTime());
}
public double getStartHour() {
return startHour;
}
public void setStartHour(double startHour) {
this.startHour = startHour;
}
public Date getStopDate() {
return stopDate;
}
public void setStopDate(Date stopDate) {
this.stopDate = new Date(stopDate.getTime());
}
public double getStopHour() {
return stopHour;
}
public void setStopHour(double stopHour) {
this.stopHour = stopHour;
}
public double getInputQTime() {
return inputQTime;
}
public void setInputQTime(double inputQTime) {
this.inputQTime = inputQTime;
}
public double getElapsedTime() {
return elapsedTime;
}
public void setElapsedTime(double elapsedTime) {
this.elapsedTime = elapsedTime;
}
public double getCpuTime() {
return cpuTime;
}
public void setCpuTime(double cpuTime) {
this.cpuTime = cpuTime;
}
public int getTranCount() {
return tranCount;
}
public void setTranCount(int tranCount) {
this.tranCount = tranCount;
}
public void readFields(DataInput input) throws IOException {
setNpaNxx(input.readUTF());
setTransactionCode(input.readUTF());
setTransactionType(input.readUTF());
setTranDate(new Date(input.readLong()));
setStartDate(new Date(input.readLong()));
setStopDate(new Date(input.readLong()));
setLoadKey(input.readInt());
setTranClassKey(input.readInt());
setTranHour(input.readDouble());
setStartHour(input.readDouble());
setStopHour(input.readDouble());
setInputQTime(input.readDouble());
setElapsedTime(input.readDouble());
setCpuTime(input.readDouble());
setTranCount(input.readInt());
}
public void write(DataOutput output) throws IOException {
output.writeUTF(npaNxx);
output.writeUTF(transactionCode);
output.writeUTF(transactionType);
output.writeLong(tranDate.getTime());
output.writeLong(startDate.getTime());
output.writeLong(stopDate.getTime());
output.writeInt(loadKey);
output.writeInt(tranClassKey);
output.writeDouble(tranHour);
output.writeDouble(startHour);
output.writeDouble(stopHour);
output.writeDouble(inputQTime);
output.writeDouble(elapsedTime);
output.writeDouble(cpuTime);
output.writeInt(tranCount);
}
public int compareTo(IMSTranOut o) {
return (Integer.compare(loadKey, o.getLoadKey()) == 0
&& Integer.compare(tranClassKey, o.getTranClassKey()) == 0
&& npaNxx.compareTo(o.getNpaNxx()) == 0
&& transactionCode.compareTo(o.getTransactionCode()) == 0
&& (transactionType.compareTo(o.getTransactionType()) == 0)
&& tranDate.compareTo(o.getTranDate()) == 0
&& Double.compare(tranHour, o.getTranHour()) == 0
&& startDate.compareTo(o.getStartDate()) == 0
&& Double.compare(startHour, o.getStartHour()) == 0
&& stopDate.compareTo(o.getStopDate()) == 0
&& Double.compare(stopHour, o.getStopHour()) == 0) ? 0 : 1;
}
}
Implementation of the Writable class for the complex values:
public class IMSTranSums
implements Writable{
private double inputQTime;
private double elapsedTime;
private double cpuTime;
private int tranCount;
public double getInputQTime() {
return inputQTime;
}
public void setInputQTime(double inputQTime) {
this.inputQTime = inputQTime;
}
public double getElapsedTime() {
return elapsedTime;
}
public void setElapsedTime(double elapsedTime) {
this.elapsedTime = elapsedTime;
}
public double getCpuTime() {
return cpuTime;
}
public void setCpuTime(double cpuTime) {
this.cpuTime = cpuTime;
}
public int getTranCount() {
return tranCount;
}
public void setTranCount(int tranCount) {
this.tranCount = tranCount;
}
public void write(DataOutput output) throws IOException {
output.writeDouble(inputQTime);
output.writeDouble(elapsedTime);
output.writeDouble(cpuTime);
output.writeInt(tranCount);
}
public void readFields(DataInput input) throws IOException {
inputQTime=input.readDouble();
elapsedTime=input.readDouble();
cpuTime=input.readDouble();
tranCount=input.readInt();
}
}
Your compareTo is flawed, it will totally fail the sort algorithm, because you seem to break transivity in your ordering.
I would recommend you to use a CompareToBuilder from Apache Commons or a ComparisonChain from Guava to make your comparisons much more readable (and correct!).
I'm trying to make a small square move across the top of the panel. I'm not worried about the seamlessness of the animation or flicker or anything like that. It appears that in the while-loop, repaint() isn't repeatedly calling the paintComponent. Thoughts?
public class NodeMove extends JFrame {
boolean running = true;
public NodeMove() {
widgetNode panel = new widgetNode();
add(panel);
setLocationRelativeTo(null);
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
setSize(400, 400);
setVisible(true);
Runnable node = new widgetNode();
Thread thread1 = new Thread(node);
thread1.start();
}
class widgetNode extends JPanel implements Runnable {
private static final long serialVersionUID = 1L;
private int x = 30;
private int y = 30;
public widgetNode() {
}
public void run(){
while(running){
nodeUpdate();
repaint();
try {
Thread.sleep(500);
} catch (InterruptedException e) {}
}
}
public void nodeUpdate(){
x += 4;
}
protected void paintComponent(Graphics g) {
super.paintComponents(g);
g.drawRect(x, y, 30, 30);
}
}
public static void main(String[] args) {
NodeMove frame = new NodeMove();
for(int i = 0; i < 50; i++){
frame.repaint();
}
}
}
I am a newbie to Trident and I'm looking to create an 'Average' aggregator similar to 'Sum(), but for 'Average'.The following does not work:
public class Average implements CombinerAggregator<Long>.......{
public Long init(TridentTuple tuple)
{
(Long)tuple.getValue(0);
}
public Long Combine(long val1,long val2){
return val1+val2/2;
}
public Long zero(){
return 0L;
}
}
It may not be exactly syntactically correct, but that's the idea. Please help if you can. Given 2 tuples with values [2,4,1] and [2,2,5] and fields 'a','b' and 'c' and doing an average on field 'b' should return '3'. I'm not entirely sure how init() and zero() work.
Thank you so much for your help in advance.
Eli
public class Average implements CombinerAggregator<Number> {
int count = 0;
double sum = 0;
#Override
public Double init(final TridentTuple tuple) {
this.count++;
if (!(tuple.getValue(0) instanceof Double)) {
double d = ((Number) tuple.getValue(0)).doubleValue();
this.sum += d;
return d;
}
this.sum += (Double) tuple.getValue(0);
return (Double) tuple.getValue(0);
}
#Override
public Double combine(final Number val1, final Number val2) {
return this.sum / this.count;
}
#Override
public Double zero() {
this.sum = 0;
this.count = 0;
return 0D;
}
}
I am a complete newbie when it comes to Trident as well, and so I'm not entirely if the following will work. But it might:
public class AvgAgg extends BaseAggregator<AvgState> {
static class AvgState {
long count = 0;
long total = 0;
double getAverage() {
return total/count;
}
}
public AvgState init(Object batchId, TridentCollector collector) {
return new AvgState();
}
public void aggregate(AvgState state, TridentTuple tuple, TridentCollector collector) {
state.count++;
state.total++;
}
public void complete(AvgState state, TridentCollector collector) {
collector.emit(new Values(state.getAverage()));
}
}