I've cloned a table and I'm trying to create a Phoenix view for it according to https://phoenix.apache.org/faq.html#How_I_map_Phoenix_table_to_an_existing_HBase_table.
Suppose I have two HBase tables below.
hbase(main):008:0> describe 'USERINFO'
Table USERINFO is ENABLED
USERINFO, {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|', coprocessor$2 => '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|', coprocessor$3 => '
|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|805306366|', coprocessor$4 => '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|805306366|', coprocessor$5 => '|org.apache.phoenix.hbase.index.Indexer|80530
6366|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec', coprocessor$6 => '|org.apache.hadoop.hbase.regionserver.LocalIndexSplitter|80
5306366|'}
COLUMN FAMILIES DESCRIPTION
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'SNAPPY', MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '6553
6', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
hbase(main):006:0> describe 'USERPREFERENCE'
Table USERPREFERENCE is ENABLED
USERPREFERENCE, {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|', coprocessor$2 => '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|', coprocessor$
3 => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|805306366|', coprocessor$4 => '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|805306366|', coprocessor$5 => '|org.apache.phoenix.hbase.index.Indexer
|805306366|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec', coprocessor$6 => '|org.apache.hadoop.hbase.regionserver.LocalIndexSplit
ter|805306366|'}
COLUMN FAMILIES DESCRIPTION
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'SNAPPY', VERSIONS => '1', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '6553
6', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
I would run the follow command to create a table view in Phoenix?
CREATE VIEW USERINFO ( pk VARCHAR PRIMARY KEY, "0".team VARCHAR, "0".firstname, "0".lastname )
CREATE VIEW USERPREFERENCE ( pk VARCHAR PRIMARY KEY, "0".firstname VARCHAR, "0".lastname )
This seems incorrect. How do I create a table view according to this situation?
So I believe what you are missing is that you need to specify what kind of a variable each column would be. So, technically you need to mention the type of column your variable firstname will be and so on.
I tried and tested this code. First I fired up my hbase-shell(Don't worry about the DCDR, its just my namespace, our DBA's have this pretty much locked down):
create 'DCDR:USERINFOV2', {NAME => '0', VERSIONS => 5}
create 'DCDR:USERPREFERENCEV2', {NAME => '0', VERSIONS => 5}
Then I fire up my phoenix shell:
CREATE VIEW "DCDR:USERINFOV2" ( pk VARCHAR PRIMARY KEY, "0".team VARCHAR, "0".firstname VARCHAR, "0".lastname VARCHAR);
CREATE VIEW "DCDR:USERPREFERENCEV2" ( pk VARCHAR PRIMARY KEY, "0".firstname VARCHAR, "0".lastname VARCHAR);
This gives me the views. Hopefully this works for you. Let me know if not.
Related
I'm trying to load Hfiles to an HBase docker container after an upgrade to Hbase 2.2.3.
My load Hfiles function looks as follows:
public void loadHfiles(String hfilesPath) throws IOException{
Path hfiles = new Path(hfilesPath);
Configuration conf = DataContext.getConfig();
BulkLoadHFiles loader = BulkLoadHFiles.create(conf);
loader.bulkLoad(HbaseDataIndex.dataContext.getTableName(), hfiles);
}
The Hfiles load fails with the following error:
[org.apache.hadoop.hbase.tool.LoadIncrementalHFiles] [WARN ] [main]
Skipping non-directory
file:/hfiles/14-02-2023-19-01-43/rHfiles/R/f0994365fa064243b81db927c18600ac
[org.apache.hadoop.hbase.tool.LoadIncrementalHFiles] [WARN ] [main]
Bulk load operation did not find any files to load in directory
/hfiles/14-02-2023-19-01-43/rHfiles/R. Does it contain files in
subdirectories that correspond to column family names?
My HBase table contains the following schema:
COLUMN FAMILIES DESCRIPTION
{NAME => 'BL', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',
NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
CACHE_DATA_ON_WRITE => 'false' , DATA_BLOCK_ENCODING => 'NONE', TTL =>
'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER
=> 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMO RY => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false',
COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}
{NAME => 'R', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',
NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER
=> 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMOR Y => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false',
COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}
I checked and the Hfiles exist on the Hbase container under /hfiles/14-02-2023-19-01-43/rHfiles/R/f0994365fa064243b81db927c18600ac with all the permission r/w/x.
In my previous HBase version 2.1.0 my load Hfiles function looked as follows:
Connection conn = ConnectionFactory.createConnection(DataContext.getConfig());
Admin admin = conn.getAdmin();
Table table = conn.getTable(DataContextataContext.getTableName());
RegionLocator regionLocator = conn.getRegionLocator(HDataContextataContext.getTableName());
LoadIncrementalHFiles load = new LoadIncrementalHFiles(HbaseDataIndex.dataContext.getConfig());
load.doBulkLoad(new Path(hfilesPath), admin, table, regionLocator);
It worked fine. In the new version, I saw that LoadIncrementalHFiles is deprecated, therefore I used the implementation above with BulkLoadHFiles.
What am I doing wrong here?
Must Hfiles be on HDFS/S3 in the new BulkLoadHFiles implementation?
I'm working with magento and I'm trying to add a new category attribute with type varchar(50). I've added the new attribute with:
$installer->addAttribute('catalog_category', 'shortdesc', array(
'type' => 'varchar',
'backend' => '',
'frontend' => '',
'label' => 'Descripción Corta',
'input' => 'textarea',
'class' => '',
'source' => '',
'global' => 1,
'visible' => 1,
'required' => 0,
'user_defined' => 0,
'default' => '',
'searchable' => 0,
'filterable' => 0,
'comparable' => 0,
'visible_on_front' => 0,
'unique' => 0,
'position' => 1,
));
But the max lenth is 255. How can I change attribute length to 50?
Indeed, Magento hardcodes the 255 value (as of Community Edition 1.8.1.0) and doesn't accept parameters to replace it.
This should not be a problem for you, as VARCHAR(255) is the max number of characters that can be stored, but it wont take more space than it needs if you store any less than that. If you really need a hard limit, you can always add code to observe the before save event for categories and strip your string there.
Or, in the extreme case you really want this hard limit on the database, you could alter the table and modify the column.
Iam Using Cakephp 2.4.6 version. Currently I am facing a problem with some fields in the forms. I have a group of horizontal controls in which the first control is a checkbox which hold the id value.if id is selected then only i want to get all other controls in that row.
is there any way other than getting all values into ajax and send thorough ajax.
TFmPlanProgram' => array(
(int) 0 => array(
't_fm_program_id' => '42',
'number_of_time' => '10'
),
(int) 1 => array(
't_fm_program_id' => '43',
'number_of_time' => '10'
),
(int) 2 => array(
't_fm_program_id' => '44',
'number_of_time' => '15'
),
(int) 3 => array(
't_fm_program_id' => '0',
'number_of_time' => ''
),
(int) 4 => array(
't_fm_program_id' => '0',
'number_of_time' => ''
),
This is my array. in that you can see that 3 and 4 having no id value. but it is passed to the server. i want to pass only the selected ID's.
You can try this before calling $this->TFmPlanProgram->save();
foreach($this->request->data['TFmPlanProgram'] as $key => $value){
if(empty($value['number_of_time'])){
unset($this->request->data['TFmPlanProgram'][$key]);
}
}
I just use database session with kohana3.2,and set the config file:
'database' => array(
'name' => 'session_name',
'encrypted' => TRUE,
'lifetime' => 24 * 3600,
'group' => 'write',
'table' => 'sessions',
'columns' => array(
'session_id' => 'session_id',
'last_active' => 'last_active',
'contents' => 'contents'
),
'gc' => 500,
),
But I got error:
Session_Exception [ 1 ]: Error reading session data.SYSPATH\classes\kohana\session.php [ 326 ]
I searched about this,but failed to find out a solution.Has anyone tried database session?
Thanks!
update:
All application config need to be placed in application/config/session.php,so I am wrong,sorry.Both system and modules config shouldnt be modified.
Did you resolve this? I had a similar issue and it was because I needed to set-up the sessions schema:
http://kohanaframework.org/3.2/guide/api/Session_Database
CREATE TABLE `sessions` (
`session_id` VARCHAR( 24 ) NOT NULL,
`last_active` INT UNSIGNED NOT NULL,
`contents` TEXT NOT NULL,
PRIMARY KEY ( `session_id` ),
INDEX ( `last_active` )
) ENGINE = MYISAM ;
Doctrine always includes an ID column in a query, for example:
$new_fees = Doctrine::getTable('Stats')->createQuery('s')
->select('s.sid')->where('s.uid = ?', $this->uid)
->andWhere('s.operation = ?', PPOperationType::FEE_PAID_BY_REFERRED_OWNER)
->andWhere('s.created_at > ?', $lastwd)
->groupBy('s.sid')->execute();
won't work, because s.id is included (which I didn't ask doctrine for). How do I get rid of that id column? Having to use a raw SQL here kills the usefulness of doctrine.
You have to set some column of that table to be the primary in the setTableDefinition, so that doctrine doesn't use default primary as id.
Let's say you sid is you actual primary key.. then...
public function setTableDefinition(){
....
$this->hasColumn('sid', 'decimal', 2, array(
'type' => 'decimal',
'length' => 2,
'unsigned' => 0,
'primary' => true,
'default' => '0',
'notnull' => true,
'autoincrement' => false,
));
}
Notice the 'primary' => true, this prevents doctrine to use id as the default primary key (even when it's not even defined in the table definition file.
This isn't the prettiest solution, but you can call isSubquery(true) on the Doctrine_Query to remove the primary key, in your case s.id.
http://www.doctrine-project.org/api/orm/1.2/doctrine/doctrine_query.html#isSubquery()