create an application that prompts the user for a set of five integer numbers using data structure - data-structures

*Using the ArrayBoundedStack class,create an application named EditNumbers that prompts the user for a set of five integer numbers, push its content into a stack, and then repeatedly prompts the user for changes to numbers, until the user enters an X, indicating the end of changes. Legal change operations are: M, A, R, and C.
• Option M return the maximum value in the set
• Option A v1 means add v1 to each number in the set
• Option R means reverse the numbers in the set
• Option C v1 v2 means change all occurrences of v1 to v2
*
(arrayboundedstack) using this codes bellow
**1)** package stack;
public class ArrayBoundedStack <T> implements StackInterface <T> {
private final int DEFSIZE=100;
private int index=-1;
private T[] arr;
public ArrayBoundedStack()
{
arr=(T[])new Object[DEFSIZE];
}
public ArrayBoundedStack(int size)
{
arr=(T[])new Object[size];
}
public boolean isFull()
{
return index == (arr.length-1);
}
public boolean isEmpty()
{
return index == -1;
}
public void push(T element)
{
if(!isFull())
{
index++;
arr[index]=element;
}
else{
throw new OverflowStackException("The stack is fill , you cannot push");
}
}
public void pop()
{
if(!isEmpty())
{
arr[index]=null;
index--;
}
else{
throw new UnderflowStackException("The stack is empty , you cannot pop");
}
}
public T top()
{
T temp=null;
if (!isEmpty())
temp=arr[index];
else{
throw new UnderflowStackException("The stack is empty , there is no top");
}
return temp;
}
}
**2) ** package stack;
public class MyApp1 {
public static void printStack(ArrayBoundedStack<Integer> st)
{
ArrayBoundedStack<Integer> temp=new ArrayBoundedStack<>(10);
System.out.println("the stack contains:");
while(!st.isEmpty())
{
System.out.println(st.top());
temp.push(st.top());
st.pop();
}
while(!temp.isEmpty())
{
st.push(temp.top());
temp.pop();
}
}
public static void nonNegativeStack(ArrayBoundedStack<Integer> st)
{
ArrayBoundedStack<Integer> temp=new ArrayBoundedStack<>();
while(!st.isEmpty())
{
if(st.top()>=0)
temp.push(st.top());
st.pop();
}
while(!temp.isEmpty())
{
st.push(temp.top());
temp.pop();
}
}
public static void main(String[] args)
{
ArrayBoundedStack<Integer> st=new ArrayBoundedStack<>(10);
st.push(10);
st.push(15);
st.push(30);
printStack(st);
System.out.println("the top of stack is " + st.top());
st.pop();
st.pop();
st.pop();
System.out.println("the top of stack is " + st.top());
}
}
**3) ** package stack;
public class MyApp2 {
public static void main(String[] args)
{
System.out.println("This is a test for exception");
throw new OverflowStackException("the stack is full");
}
}
**4)** package stack;
public class OverflowStackException extends RuntimeException{
public OverflowStackException()
{
super();
}
public OverflowStackException(String msg)
{
super(msg);
}
}
**5) ** package stack;
public interface StackInterface <T> {
public void push(T element) throws OverflowStackException;
public void pop() throws UnderflowStackException;
public T top()throws UnderflowStackException;
public boolean isFull();
public boolean isEmpty();
}
**6)**package stack;
public class UnderflowStackException extends RuntimeException{
public UnderflowStackException()
{
super();
}
public UnderflowStackException(String msg)
{
super(msg);
}
}
i made all the class up and i didnt know how to make a main class

Related

trino udf how to create a aggregate function for the window function

I tried to write a udf function to calculate my data. In the trino's docs, I knew I should to write a function plugin and I succeed to execute my udf aggregate function sql.
But when I write sql with aggregate function and window function, the sql executed failed.
The error log is com.google.common.util.concurrent.ExecutionError: java.lang.NoClassDefFoundError: com/example/ListState.
I think I may implement the interface about the window function.
The ListState.java file code
#AccumulatorStateMetadata(stateSerializerClass = ListStateSerializer.class, stateFactoryClass = ListStateFactory.class)
public interface ListState extends AccumulatorState {
List<String> getList();
void setList(List<String> value);
}
The ListStateSerializer file code
public class ListStateSerializer implements AccumulatorStateSerializer<ListState>
{
#Override
public Type getSerializedType() {
return VARCHAR;
}
#Override
public void serialize(ListState state, BlockBuilder out) {
if (state.getList() == null) {
out.appendNull();
return;
}
String value = String.join(",", state.getList());
VARCHAR.writeSlice(out, Slices.utf8Slice(value));
}
#Override
public void deserialize(Block block, int index, ListState state) {
String value = VARCHAR.getSlice(block, index).toStringUtf8();
List<String> list = Arrays.asList(value.split(","));
state.setList(list);
}
}
The ListStateFactory file code
public class ListStateFactory implements AccumulatorStateFactory<ListState> {
public static final class SingleListState implements ListState {
private List<String> list = new ArrayList<>();
#Override
public List<String> getList() {
return list;
}
#Override
public void setList(List<String> value) {
list = value;
}
#Override
public long getEstimatedSize() {
if (list == null) {
return 0;
}
return list.size();
}
}
public static class GroupedListState implements GroupedAccumulatorState, ListState {
private final ObjectBigArray<List<String>> container = new ObjectBigArray<>();
private long groupId;
#Override
public List<String> getList() {
return container.get(groupId);
}
#Override
public void setList(List<String> value) {
container.set(groupId, value);
}
#Override
public void setGroupId(long groupId) {
this.groupId = groupId;
if (this.getList() == null) {
this.setList(new ArrayList<String>());
}
}
#Override
public void ensureCapacity(long size) {
container.ensureCapacity(size);
}
#Override
public long getEstimatedSize() {
return container.sizeOf();
}
}
#Override
public ListState createSingleState() {
return new SingleListState();
}
#Override
public ListState createGroupedState() {
return new GroupedListState();
}
}
Thanks for help!!!!
And I found the WindowAccumulator class in the trino source code. But I don't know how to use it.
How to create a aggregate function for window function?

Recycler View with Header and Edit Text

I have a recyclerview with a header achieved by using two different element types. In my header there is an edit text which I want to use for filtering the nonheader elements of the list. Below is my current implementation, I have one concern and one problem with it.
My concern is that what I am doing in publishResults with the notifyItemRangeRemoved and notifyItemInserted is the wrong way to update the recycler view. I originally was doing notifyDatasetChanged but his would cause the header row to be refreshed too and the edit text to lose focus. What I really want is a way to refresh only the item rows and leave the header row untouched.
My current problem is that with the existing code if I scroll down too much the edit text looses focus. I want the edit text to keep focus even if I scroll to the bottom of the list.
The code used to use a ListView with setHeaderView and that worked somehow so there must be someway of achieving the goal just not sure what the trick with a recycler view is. Any help is much appreciated.
public class SideListAdapter extends RecyclerView.Adapter<RecyclerView.ViewHolder> implements Filterable {
private static final int TYPE_HEADER = 0;
private static final int TYPE_ITEM = 1;
private final List<String> data;
public List<String> filteredData;
private HeaderActionListener headerActionListener;
public SideListAdapter(Context context, ArrayList<String> data, HeaderActionListener headerActionListener) {
this.data = data;
filteredData = new ArrayList<>(data);
this.context = context;
this.headerActionListener = headerActionListener;
}
#Override
public Filter getFilter() {
return new TestFilter();
}
static class SideListItem extends RecyclerView.ViewHolder {
LinearLayout baseLayout;
public SideListItem(View itemView) {
super(itemView);
baseLayout = (LinearLayout) itemView.findViewById(R.id.settings_defaultcolor);
}
}
class SideListHeader extends SideListHeader {
EditText sort;
public SideListHeaderLoggedIn(View itemView) {
super(itemView);
sort = (EditText) itemView.findViewById(R.id.sort);
}
}
#Override
public RecyclerView.ViewHolder onCreateViewHolder(ViewGroup parent, int viewType) {
if (viewType == TYPE_ITEM) {
View v = LayoutInflater.from(parent.getContext()).inflate(R.layout.list_item, parent, false);
return new SideListItem(v);
} else if (viewType == SideListHeader) {
View v = LayoutInflater.from(parent.getContext()).inflate(R.layout.header, parent, false);
return new SideListHeader(v);
}
throw new RuntimeException("there is no type that matches the type " + viewType + " + make sure your using types correctly");
}
public interface HeaderActionListener {
boolean onSortEditorAction(TextView arg0, int arg1, KeyEvent arg2);
}
#Override
public void onBindViewHolder(RecyclerView.ViewHolder holder, final int position) {
if (holder instanceof SideListHeader) {
final SideListHeader sideListHeader = (SideListHeader) holder;
sideListHeader.sort.setOnEditorActionListener(new TextView.OnEditorActionListener() {
#Override
public boolean onEditorAction(TextView v, int actionId, KeyEvent event) {
}
});
sideListHeader.sort.addTextChangedListener(new TextWatcher() {
#Override
public void beforeTextChanged(CharSequence charSequence, int i, int i2, int i3) {
}
#Override
public void onTextChanged(CharSequence charSequence, int i, int i2, int i3) {
}
#Override
public void afterTextChanged(Editable editable) {
String result = sideListHeader.sort.getText().toString().replaceAll(" ", "");
getFilter().filter(result);
}
});
}
if (holder instanceof SideListItem) {
// Inflate normal item //
}
}
// need to override this method
#Override
public int getItemViewType(int position) {
if (isPositionHeader(position)) {
return TYPE_HEADER;
}
return TYPE_ITEM;
}
private boolean isPositionHeader(int position) {
return position == 0;
}
//increasing getItemcount to 1. This will be the row of header.
#Override
public int getItemCount() {
return filteredData.size() + 1;
}
private class TestFilter extends Filter {
#Override
protected FilterResults performFiltering(CharSequence constraint) {
FilterResults results = new FilterResults();
String prefix = constraint.toString().toLowerCase();
if (prefix.isEmpty()) {
ArrayList<String> list = new ArrayList<>(data);
results.values = list;
results.count = list.size();
} else {
final ArrayList<String> list = new ArrayList<>(data);
final ArrayList<String> nlist = new ArrayList<>();
for (int i = 0 ; i < list.size(); i++) {
String item = list.get(i);
if (item.contains(prefix)) {
nlist.add(item);
}
}
results.values = nlist;
results.count = nlist.size();
}
return results;
}
#SuppressWarnings("unchecked")
#Override
protected void publishResults(CharSequence constraint, FilterResults results) {
notifyItemRangeRemoved(1, getItemCount()-1);
filteredData.clear();
filteredData.addAll((List<String>)results.values);
for(int i = 1; i < getItemCount() - 1; i++){
notifyItemInserted(i);
}
}
}
}
I'm not sure how correct this way is, but in my code I implemented it like that
private var headerList: List<HeaderItem> = listOf(HeaderItem("Title"))
private fun searchItem(items: List<Items>, query: String) {
items.filterIsInstance<MainItem>().filter { filteredItems ->
filteredItems.header.lowercase().contains(query.lowercase())
}.let { searchedItems ->
rvAdapter.submitList(headerList + searchedItems)
}
}
This way I was able to preserve header element when I did my search

TableColumn, How to connect it with property but not fill the cells

I have a TableView and Data class with integer properties for columns. However I would like columns at first show empty cells so user can put value he wants.
Right now its impossible because when creating Data object, integer values has to be created with initial value, so table shows up filled already with numbers.
private ObservableList<MyData> dataList = FXCollections.observableArrayList();
.....edited....
private void buttAddColumnAction(ActionEvent event){
int i = numberOfColumns;// thats the key for lambda expression. Unicate number for column to access its variable;
if(dataList.size() > 0)//resizing each data object with new variable
for(MyData x: dataList)
x.addNew();
TableColumn<MyData, Integer> newColumn = new TableColumn<>("#" + String.valueOf(++numberOfColumns));
newColumn.setCellValueFactory(cellData -> cellData.getValue().getCellValue(i));
// newColumn.setCellFactory(TextFieldTableCell.<MyData, Integer>forTableColumn(new IntegerStringConverter()));
Callback<TableColumn<MyData, Integer>, TableCell<MyData, Integer>> cellFactoryInt = (TableColumn<MyData, Integer> p) -> new EditingCellNumbers(tableView);
newColumn.setCellFactory(cellFactoryInt);
tableView.getColumns().add(newColumn);
}
public class MyData{ //dont forget about public because you wont get acces to properties
private ObservableList<ObjectProperty<Integer>> cellValue = FXCollections.observableArrayList();
public MyData(int howManyColumns) {
for(int i=0; i<howManyColumns; ++i)
this.cellValue.add(new SimpleObjectProperty<Integer>(null));
}
public ObjectProperty<Integer> getCellValue(int whichOne) {
return cellValue.get(whichOne);
}
public void setCellValue(int cellValue, int whichOne) {
this.cellValue.set(whichOne, new SimpleObjectProperty<Integer>(cellValue));
}
public void addNew(){ //ads another variable for another column
cellValue.add(new SimpleObjectProperty<Integer>(null));
}
public void deleteLast(){ //deletes last variable when column is deleted
cellValue.remove(cellValue.size()-1);
}
}
CellFactory
//Klasa ta pozwala na definiowania zachowania komórek, które edytuje użytkownik
public class EditingCellNumbers extends TableCell<MyData, Integer>{
private TextField textField;
private TableView<MyData> parentTableView;
public static int numberOfColumns;
public EditingCellNumbers(TableView<MyData> parent) {
this.parentTableView = parent;
numberOfColumns = parent.getColumns().size();
}
#Override
public void startEdit(){
if (!isEmpty()) {
super.startEdit();
createTextField();
setText(null);
setGraphic(textField);
textField.selectAll();
textField.requestFocus();
}
}
#Override
public void cancelEdit() {
super.cancelEdit();
setText(String.valueOf(getItem()));
setGraphic(null);
}
#Override
public void updateItem(Integer item, boolean empty) {
super.updateItem(item, empty);
if (empty) {
setText(null);
setGraphic(null);
} else {
if (isEditing()) {
if (textField != null) {
textField.setText(getString());
}
setText(null);
setGraphic(textField);
} else {
setText(getString());
setGraphic(null);
}
}
}
private void createTextField() {
textField = new TextField(getString());
textField.setMinWidth(this.getWidth() - this.getGraphicTextGap()* 2);
textField.focusedProperty().addListener(
(ObservableValue<? extends Boolean> arg0,
Boolean arg1, Boolean arg2) -> {
if (!arg2) {
XXX commitEdit(Integer.valueOf(textField.getText()));
}
});
textField.setOnKeyReleased(new EventHandler<Event>() {
#Override
public void handle(Event event) {
try{
int i = Integer.valueOf(textField.getText());
//digit given...
if( (i>=0) && (i<10) ){//making sure cell is filled with just one digit
commitEdit(Integer.valueOf(textField.getText()));
int selectedColumn = parentTableView.getSelectionModel().getSelectedCells().get(0).getColumn(); // gets the number of selected column
int selectedRow = parentTableView.getSelectionModel().getSelectedCells().get(0).getRow();
if(selectedColumn < numberOfColumns-1){
parentTableView.getSelectionModel().selectNext();
parentTableView.edit(selectedRow, parentTableView.getColumns().get(selectedColumn+1));
}else{
parentTableView.getSelectionModel().select(selectedRow+1, parentTableView.getColumns().get(0));
parentTableView.edit(selectedRow+1, parentTableView.getColumns().get(0));
}
}else
textField.clear();
}catch(NumberFormatException e){
textField.clear();
}
}
});
}
private String getString() {
return getItem() == null ? "" : getItem().toString();
}
}
Allow null values in your column by using an ObjectProperty<Integer> instead of an IntegerProperty. This gives a more natural way to define "not initialized" than representing it with 0 (or some other proxy value).
Then you can use the TextFieldTableCell, but just supply a custom StringConverter<Integer>:
public class MyData{ //dont forget about public because you wont get acces to properties
private ObservableList<ObjectProperty<Integer>> cellValue = FXCollections.observableArrayList();
public MyData(int howManyColumns) {
for(int i=0; i<howManyColumns; ++i)
this.cellValue.add(new SimpleObjectProperty<>(new Random().nextInt(10)));
}
// ...
}
and
newColumn.setCellValueFactory(cellData -> cellData.getValue().getCellValue(i));
newColumn.setCellFactory(TextFieldTableCell.<MyData, Integer>forTableColumn(new StringConverter<Integer>() {
#Override
public String toString(Integer i) {
if (i == null) {
return "" ;
} else {
return i.toString();
}
}
#Override
public Integer fromString(String string) {
if (string.trim().length() == 0) {
return null ;
} else {
try {
return Integer.valueOf(string);
} catch (NumberFormatException nfe) {
return null ;
}
}
}
}));
Complete example:
import java.util.Random;
import java.util.function.Function;
import javafx.application.Application;
import javafx.beans.property.ObjectProperty;
import javafx.beans.property.SimpleObjectProperty;
import javafx.beans.property.SimpleStringProperty;
import javafx.beans.property.StringProperty;
import javafx.beans.value.ObservableValue;
import javafx.scene.Scene;
import javafx.scene.control.TableColumn;
import javafx.scene.control.TableView;
import javafx.scene.control.cell.TextFieldTableCell;
import javafx.scene.layout.BorderPane;
import javafx.stage.Stage;
import javafx.util.StringConverter;
public class TableViewWithEmptyIntegerColumn extends Application {
#Override
public void start(Stage primaryStage) {
TableView<Item> table = new TableView<>();
table.setEditable(true);
TableColumn<Item, String> nameCol = createCol("Name", Item::nameProperty);
TableColumn<Item, Integer> valueCol = createCol("Value", Item::valueProperty);
valueCol.setEditable(true);
valueCol.setCellFactory(TextFieldTableCell.forTableColumn(new StringConverter<Integer>() {
#Override
public String toString(Integer i) {
if (i == null) {
return "" ;
} else {
return i.toString() ;
}
}
#Override
public Integer fromString(String string) {
if (string.trim().length() == 0) {
return null ;
} else {
// better to check for a valid int format instead of using try-catch...
try {
return Integer.valueOf(string);
} catch (NumberFormatException nfe) {
return null ;
}
}
}
}));
Random rng = new Random();
for (int i=1; i<=20; i++) {
if (rng.nextDouble() < 0.5) {
table.getItems().add(new Item("Item "+i));
} else {
table.getItems().add(new Item("Item "+i, rng.nextInt(10)+1));
}
}
table.getColumns().addAll(nameCol, valueCol);
primaryStage.setScene(new Scene(new BorderPane(table), 400, 600));
primaryStage.show();
}
private <S,T> TableColumn<S,T> createCol(String title, Function<S, ObservableValue<T>> property) {
TableColumn<S,T> col = new TableColumn<>(title);
col.setCellValueFactory(cellData -> property.apply(cellData.getValue()));
return col ;
}
public static class Item {
private final StringProperty name = new SimpleStringProperty();
private final ObjectProperty<Integer> value = new SimpleObjectProperty<>();
public Item(String name, Integer value) {
setName(name);
setValue(value);
}
public Item(String name) {
this(name, null);
}
public final StringProperty nameProperty() {
return this.name;
}
public final String getName() {
return this.nameProperty().get();
}
public final void setName(final String name) {
this.nameProperty().set(name);
}
public final ObjectProperty<Integer> valueProperty() {
return this.value;
}
public final Integer getValue() {
return this.valueProperty().get();
}
public final void setValue(final Integer value) {
this.valueProperty().set(value);
}
}
public static void main(String[] args) {
launch(args);
}
}

Map-Reduce not reducing as much as expected with complex keys and values

No matter how simple I make the compareTo of my complex key, I don't get expected results. With the exception of if I use one key that is the same for every record, it will appropriately reduce to one record. I've also witnessed that this happens only when I process the full load, if I break off a few of the records that didn't reduce and run it on a much smaller scale those records get combined.
The sum of the output records is correct, but there is duplication at the record level of items I would have expected to group together. So where I would expect say 500 records summing up to 5,000, I end up with 1232 records summing up to 5,000 with obvious records that should have been reduced into one.
I've read about the problems with object references and complex keys and values, but I don't see anywhere that I have potential for that left. To that end you will find places that I'm creating new objects that I probably don't need to, but I'm trying everything at this point and will dial it back once it is working.
I'm out of ideas on what to try or where and how to poke to figure this out. Please help!
public static class Map extends
Mapper<LongWritable, Text, IMSTranOut, IMSTranSums> {
//private SimpleDateFormat dtFormat = new SimpleDateFormat("yyyyddd");
#Override
public void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException {
String line = value.toString();
SimpleDateFormat dtFormat = new SimpleDateFormat("yyyyddd");
IMSTranOut dbKey = new IMSTranOut();
IMSTranSums sumVals = new IMSTranSums();
String[] tokens = line.split(",", -1);
dbKey.setLoadKey(-99);
dbKey.setTranClassKey(-99);
dbKey.setTransactionCode(tokens[0]);
dbKey.setTransactionType(tokens[1]);
dbKey.setNpaNxx(getNPA(dbKey.getTransactionCode()));
try {
dbKey.setTranDate(new Date(dtFormat.parse(tokens[2]).getTime()));
} catch (ParseException e) {
}// 2
dbKey.setTranHour(getTranHour(tokens[3]));
try {
dbKey.setStartDate(new Date(dtFormat.parse(tokens[4]).getTime()));
} catch (ParseException e) {
}// 4
dbKey.setStartHour(getTranHour(tokens[5]));
try {
dbKey.setStopDate(new Date(dtFormat.parse(tokens[6]).getTime()));
} catch (ParseException e) {
}// 6
dbKey.setStopHour(getTranHour(tokens[7]));
sumVals.setTranCount(1);
sumVals.setInputQTime(Double.parseDouble(tokens[8]));
sumVals.setElapsedTime(Double.parseDouble(tokens[9]));
sumVals.setCpuTime(Double.parseDouble(tokens[10]));
context.write(dbKey, sumVals);
}
}
public static class Reduce extends
Reducer<IMSTranOut, IMSTranSums, IMSTranOut, IMSTranSums> {
#Override
public void reduce(IMSTranOut key, Iterable<IMSTranSums> values,
Context context) throws IOException, InterruptedException {
int tranCount = 0;
double inputQ = 0;
double elapsed = 0;
double cpu = 0;
for (IMSTranSums val : values) {
tranCount += val.getTranCount();
inputQ += val.getInputQTime();
elapsed += val.getElapsedTime();
cpu += val.getCpuTime();
}
IMSTranSums sumVals=new IMSTranSums();
IMSTranOut dbKey=new IMSTranOut();
sumVals.setCpuTime(inputQ);
sumVals.setElapsedTime(elapsed);
sumVals.setInputQTime(cpu);
sumVals.setTranCount(tranCount);
dbKey.setLoadKey(key.getLoadKey());
dbKey.setTranClassKey(key.getTranClassKey());
dbKey.setNpaNxx(key.getNpaNxx());
dbKey.setTransactionCode(key.getTransactionCode());
dbKey.setTransactionType(key.getTransactionType());
dbKey.setTranDate(key.getTranDate());
dbKey.setTranHour(key.getTranHour());
dbKey.setStartDate(key.getStartDate());
dbKey.setStartHour(key.getStartHour());
dbKey.setStopDate(key.getStopDate());
dbKey.setStopHour(key.getStopHour());
dbKey.setInputQTime(inputQ);
dbKey.setElapsedTime(elapsed);
dbKey.setCpuTime(cpu);
dbKey.setTranCount(tranCount);
context.write(dbKey, sumVals);
}
}
Here is the implementation of the DBWritable class:
public class IMSTranOut implements DBWritable,
WritableComparable<IMSTranOut> {
private int loadKey;
private int tranClassKey;
private String npaNxx;
private String transactionCode;
private String transactionType;
private Date tranDate;
private double tranHour;
private Date startDate;
private double startHour;
private Date stopDate;
private double stopHour;
private double inputQTime;
private double elapsedTime;
private double cpuTime;
private int tranCount;
public void readFields(ResultSet rs) throws SQLException {
setLoadKey(rs.getInt("LOAD_KEY"));
setTranClassKey(rs.getInt("TRAN_CLASS_KEY"));
setNpaNxx(rs.getString("NPA_NXX"));
setTransactionCode(rs.getString("TRANSACTION_CODE"));
setTransactionType(rs.getString("TRANSACTION_TYPE"));
setTranDate(rs.getDate("TRAN_DATE"));
setTranHour(rs.getInt("TRAN_HOUR"));
setStartDate(rs.getDate("START_DATE"));
setStartHour(rs.getInt("START_HOUR"));
setStopDate(rs.getDate("STOP_DATE"));
setStopHour(rs.getInt("STOP_HOUR"));
setInputQTime(rs.getInt("INPUT_Q_TIME"));
setElapsedTime(rs.getInt("ELAPSED_TIME"));
setCpuTime(rs.getInt("CPU_TIME"));
setTranCount(rs.getInt("TRAN_COUNT"));
}
public void write(PreparedStatement ps) throws SQLException {
ps.setInt(1, loadKey);
ps.setInt(2, tranClassKey);
ps.setString(3, npaNxx);
ps.setString(4, transactionCode);
ps.setString(5, transactionType);
ps.setDate(6, tranDate);
ps.setDouble(7, tranHour);
ps.setDate(8, startDate);
ps.setDouble(9, startHour);
ps.setDate(10, stopDate);
ps.setDouble(11, stopHour);
ps.setDouble(12, inputQTime);
ps.setDouble(13, elapsedTime);
ps.setDouble(14, cpuTime);
ps.setInt(15, tranCount);
}
public int getLoadKey() {
return loadKey;
}
public void setLoadKey(int loadKey) {
this.loadKey = loadKey;
}
public int getTranClassKey() {
return tranClassKey;
}
public void setTranClassKey(int tranClassKey) {
this.tranClassKey = tranClassKey;
}
public String getNpaNxx() {
return npaNxx;
}
public void setNpaNxx(String npaNxx) {
this.npaNxx = new String(npaNxx);
}
public String getTransactionCode() {
return transactionCode;
}
public void setTransactionCode(String transactionCode) {
this.transactionCode = new String(transactionCode);
}
public String getTransactionType() {
return transactionType;
}
public void setTransactionType(String transactionType) {
this.transactionType = new String(transactionType);
}
public Date getTranDate() {
return tranDate;
}
public void setTranDate(Date tranDate) {
this.tranDate = new Date(tranDate.getTime());
}
public double getTranHour() {
return tranHour;
}
public void setTranHour(double tranHour) {
this.tranHour = tranHour;
}
public Date getStartDate() {
return startDate;
}
public void setStartDate(Date startDate) {
this.startDate = new Date(startDate.getTime());
}
public double getStartHour() {
return startHour;
}
public void setStartHour(double startHour) {
this.startHour = startHour;
}
public Date getStopDate() {
return stopDate;
}
public void setStopDate(Date stopDate) {
this.stopDate = new Date(stopDate.getTime());
}
public double getStopHour() {
return stopHour;
}
public void setStopHour(double stopHour) {
this.stopHour = stopHour;
}
public double getInputQTime() {
return inputQTime;
}
public void setInputQTime(double inputQTime) {
this.inputQTime = inputQTime;
}
public double getElapsedTime() {
return elapsedTime;
}
public void setElapsedTime(double elapsedTime) {
this.elapsedTime = elapsedTime;
}
public double getCpuTime() {
return cpuTime;
}
public void setCpuTime(double cpuTime) {
this.cpuTime = cpuTime;
}
public int getTranCount() {
return tranCount;
}
public void setTranCount(int tranCount) {
this.tranCount = tranCount;
}
public void readFields(DataInput input) throws IOException {
setNpaNxx(input.readUTF());
setTransactionCode(input.readUTF());
setTransactionType(input.readUTF());
setTranDate(new Date(input.readLong()));
setStartDate(new Date(input.readLong()));
setStopDate(new Date(input.readLong()));
setLoadKey(input.readInt());
setTranClassKey(input.readInt());
setTranHour(input.readDouble());
setStartHour(input.readDouble());
setStopHour(input.readDouble());
setInputQTime(input.readDouble());
setElapsedTime(input.readDouble());
setCpuTime(input.readDouble());
setTranCount(input.readInt());
}
public void write(DataOutput output) throws IOException {
output.writeUTF(npaNxx);
output.writeUTF(transactionCode);
output.writeUTF(transactionType);
output.writeLong(tranDate.getTime());
output.writeLong(startDate.getTime());
output.writeLong(stopDate.getTime());
output.writeInt(loadKey);
output.writeInt(tranClassKey);
output.writeDouble(tranHour);
output.writeDouble(startHour);
output.writeDouble(stopHour);
output.writeDouble(inputQTime);
output.writeDouble(elapsedTime);
output.writeDouble(cpuTime);
output.writeInt(tranCount);
}
public int compareTo(IMSTranOut o) {
return (Integer.compare(loadKey, o.getLoadKey()) == 0
&& Integer.compare(tranClassKey, o.getTranClassKey()) == 0
&& npaNxx.compareTo(o.getNpaNxx()) == 0
&& transactionCode.compareTo(o.getTransactionCode()) == 0
&& (transactionType.compareTo(o.getTransactionType()) == 0)
&& tranDate.compareTo(o.getTranDate()) == 0
&& Double.compare(tranHour, o.getTranHour()) == 0
&& startDate.compareTo(o.getStartDate()) == 0
&& Double.compare(startHour, o.getStartHour()) == 0
&& stopDate.compareTo(o.getStopDate()) == 0
&& Double.compare(stopHour, o.getStopHour()) == 0) ? 0 : 1;
}
}
Implementation of the Writable class for the complex values:
public class IMSTranSums
implements Writable{
private double inputQTime;
private double elapsedTime;
private double cpuTime;
private int tranCount;
public double getInputQTime() {
return inputQTime;
}
public void setInputQTime(double inputQTime) {
this.inputQTime = inputQTime;
}
public double getElapsedTime() {
return elapsedTime;
}
public void setElapsedTime(double elapsedTime) {
this.elapsedTime = elapsedTime;
}
public double getCpuTime() {
return cpuTime;
}
public void setCpuTime(double cpuTime) {
this.cpuTime = cpuTime;
}
public int getTranCount() {
return tranCount;
}
public void setTranCount(int tranCount) {
this.tranCount = tranCount;
}
public void write(DataOutput output) throws IOException {
output.writeDouble(inputQTime);
output.writeDouble(elapsedTime);
output.writeDouble(cpuTime);
output.writeInt(tranCount);
}
public void readFields(DataInput input) throws IOException {
inputQTime=input.readDouble();
elapsedTime=input.readDouble();
cpuTime=input.readDouble();
tranCount=input.readInt();
}
}
Your compareTo is flawed, it will totally fail the sort algorithm, because you seem to break transivity in your ordering.
I would recommend you to use a CompareToBuilder from Apache Commons or a ComparisonChain from Guava to make your comparisons much more readable (and correct!).

Using a custom Object as key emitted by mapper

I have a situation in which mapper emits as key an object of custom type.
It has two fields an intWritable ID, and a data array IntArrayWritable.
The implementation is as follows.
`
import java.io.*;
import org.apache.hadoop.io.*;
public class PairDocIdPerm implements WritableComparable<PairDocIdPerm> {
public PairDocIdPerm(){
this.permId = new IntWritable(-1);
this.SignaturePerm = new IntArrayWritable();
}
public IntWritable getPermId() {
return permId;
}
public void setPermId(IntWritable permId) {
this.permId = permId;
}
public IntArrayWritable getSignaturePerm() {
return SignaturePerm;
}
public void setSignaturePerm(IntArrayWritable signaturePerm) {
SignaturePerm = signaturePerm;
}
private IntWritable permId;
private IntArrayWritable SignaturePerm;
public PairDocIdPerm(IntWritable permId,IntArrayWritable SignaturePerm) {
this.permId = permId;
this.SignaturePerm = SignaturePerm;
}
#Override
public void write(DataOutput out) throws IOException {
permId.write(out);
SignaturePerm.write(out);
}
#Override
public void readFields(DataInput in) throws IOException {
permId.readFields(in);
SignaturePerm.readFields(in);
}
#Override
public int hashCode() { // same permId must go to same reducer. there fore just permId
return permId.get();//.hashCode();
}
#Override
public boolean equals(Object o) {
if (o instanceof PairDocIdPerm) {
PairDocIdPerm tp = (PairDocIdPerm) o;
return permId.equals(tp.permId) && SignaturePerm.equals(tp.SignaturePerm);
}
return false;
}
#Override
public String toString() {
return permId + "\t" +SignaturePerm.toString();
}
#Override
public int compareTo(PairDocIdPerm tp) {
int cmp = permId.compareTo(tp.permId);
Writable[] ar, other;
ar = this.SignaturePerm.get();
other = tp.SignaturePerm.get();
if (cmp == 0) {
for(int i=0;i<ar.length;i++){
if(((IntWritable)ar[i]).get() == ((IntWritable)other[i]).get()){cmp= 0;continue;}
else if(((IntWritable)ar[i]).get() < ((IntWritable)other[i]).get()){ return -1;}
else if(((IntWritable)ar[i]).get() > ((IntWritable)other[i]).get()){return 1;}
}
}
return cmp;
//return 1;
}
}`
I require the keys with same Id to go to the same reducer with their sort order as coded in the compareTo method.
However when i use this, my job execution status is always map100% reduce 0%.
The reduce never runs to completion. Is there any thing wrong in this implementation?
In general what is the likely problem if reducer status is always 0%.
I think this might be a possible null pointer exception in the read method:
#Override
public void readFields(DataInput in) throws IOException {
permId.readFields(in);
SignaturePerm.readFields(in);
}
permId is null in this case.
So what you have to do is this:
IntWritable permId = new IntWritable();
Either in the field initializer or before the read.
However, your code is horrible to read.

Resources