How to Load Coprocessor Step by Step - hadoop

Question

can anyone shall explain how to load regionCoprocessor trough shell.i can not getting proper information about loading and deploying Coprocessor.Thanks in Advance

Answer 1

Please follow the steps below:
Step 1: Create an interface and extend org.apache.hadoop.hbase.ipc.CoprocessorProtocol
Step 2: Define the method in the interface you want to execute once the co-processor call is made
Step 3: Create an instance of HTable
Step 4: Call the HTable.coprocessorExec() method with all required parameters
Please find the example below:
In the example, we are trying to get list of students whose registration number falls within some range which we are interested in.
Creating Interface Protocol:
public interface CoprocessorTestProtocol extends org.apache.hadoop.hbase.ipc.CoprocessorProtocol{
List<Student> getStudentList(byte[] startRegistrationNumber, byte[] endRegistrationNumber) throws IOException;
}
Sample Student Class:
public class Student implements Serializable{
byte[] registrationNumber;
String name;
public void setRegistrationNumber(byte[] registrationNumber){
this.registrationNumber = registrationNumber;
}
public byte[] getRegistrationNumber(){
return this.registrationNumber;
}
public void setName(String name){
this.name = name;
}
public int getName(){
return this.name;
}
public String toString(){
return "Student[ registration number = " + Bytes.toInt(this.getRegistrationNumber()) + " name = " + this.getName() + " ]"
}
}
Model Class: [Where the business logic to get data from HBase is written]
public class MyModel extends org.apache.hadoop.hbase.coprocessor.BaseEndpointCoprocessor implements CoprocessorTestProtocol{
#Override
List<Student> getStudentList(byte[] startRegistrationNumber, byte[] endRegistrationNumber){
Scan scan = new Scan();
scan.setStartRow(startRegistrationNumber);
scan.setStopRow(endRegistrationNumber);
InternalScanner scanner = ((RegionCoprocessorEnvironment) getEnvironment()).getRegion().getScanner(scan);
List<KeyValue> currentTempObj = new ArrayList<KeyValue>();
List<Student> studentList = new ArrayList<Student>();
try{
Boolean hasNext = false;
Student student;
do{
currentTempObj.clear();
hasNext = scanner.next(currentTempObj);
if(!currentTempObj.isEmpty()){
student = new Student();
for(KeyValue keyValue: currentTempObj){
bytes[] qualifier = keyValue.getQualifier();
if(Arrays.equals(qualifier, Bytes.toBytes("registrationNumber")))
student.setRegistrationNumber(keyValue.getValue());
else if(Arrays.equals(qualifier, Bytes.toBytes("name")))
student.setName(Bytes.toString(keyValue.getValue()));
}
StudentList.add(student);
}
}while(hasNext);
}catch (Exception e){
// catch the exception the way you want
}
finally{
scanner.close();
}
}
}
Client class: [where the call to co-processor is made]
public class MyClient{
if (args.length < 2) {
System.out.println("Usage : startRegistrationNumber endRegistrationNumber");
return;
}
public List<Student> displayStudentInfo(int startRegistrationNumber, int endRegistrationNumber){
final byte[] startKey=Bytes.toBytes(startRegistrationNumber);
final byte[] endKey=Bytes.toBytes(endRegistrationNumber);
String zkPeers = SystemInfo.getHBaseZkConnectString();
Configuration configuration=HBaseConfiguration.create();
configuration.set(HConstants.ZOOKEEPER_QUORUM, zkPeers);
HTableInterface table = new HTable(configuration, TABLE_NAME);
Map<byte[],List<Student>> allRegionOutput;
allRegionOutput = table.coprocessorExec(CoprocessorTestProtocol.class, startKey,endKey,
new Batch.Call<CoprocessorTestProtocol, List<Student>>() {
public List<Student> call(CoprocessorTestProtocol instance)throws IOException{
return instance.getStudentList(startKey, endKey);
}
});
table.close();
List<Student> anotherList = new ArrayList<Student>();
for (List<Student> studentData: allRegionOutput.values()){
anotherList.addAll(studentData);
}
return anotherList;
}
public static void main(String args){
if (args.length < 2) {
System.out.println("Usage : startRegistrationNumber endRegistrationNumber");
return;
}
int startRegistrationNumber = args[0];
int endRegistrationNumber = args[1];
for (Student student : displayStudentInfo(startRegistrationNumber, endRegistrationNumber)){
System.out.println(student);
}
}
}
Please Note: Please have a special look on Scanner.next(Object) method in the example. This returns boolean and stores the current object in Object argument