Different getIndicesToRecalculate() codes between 21.3 and 21.6 - clickhouse

recently I'm learning about Clickhouse. While excuting alter table test_minmax materialize index idx, I found that the code goes to MergeTreeDataMergerMutator::getIndicesToRecalculate() func.
Here's some code from branch 21.3
std::set<MergeTreeIndexPtr> MergeTreeDataMergerMutator::getIndicesToRecalculate(
BlockInputStreamPtr & input_stream,
const NamesAndTypesList & updated_columns,
const StorageMetadataPtr & metadata_snapshot,
const Context & context)
{
/// Checks if columns used in skipping indexes modified.
const auto & index_factory = MergeTreeIndexFactory::instance();
std::set<MergeTreeIndexPtr> indices_to_recalc;
ASTPtr indices_recalc_expr_list = std::make_shared<ASTExpressionList>();
for (const auto & col : updated_columns.getNames())
{
const auto & indices = metadata_snapshot->getSecondaryIndices();
for (size_t i = 0; i < indices.size(); ++i)
{
// other code
}
// other code
}
If my skipping index has 35 columns, the loop for (const auto & col : updated_columns.getNames()) will push 35 MergeTreeIndexPtrs to indices_to_recalc even if there's only one skipping index on my table. I wonder if it's a bug, then I found the code remains until 21.6.
Here's some code from 21.6, commit message doesn't tell me the reason.
std::set<MergeTreeIndexPtr> MergeTreeDataMergerMutator::getIndicesToRecalculate(
BlockInputStreamPtr & input_stream,
const NameSet & updated_columns,
const StorageMetadataPtr & metadata_snapshot,
ContextPtr context,
const NameSet & materialized_indices,
const MergeTreeData::DataPartPtr & source_part)
{
/// Checks if columns used in skipping indexes modified.
const auto & index_factory = MergeTreeIndexFactory::instance();
std::set<MergeTreeIndexPtr> indices_to_recalc;
ASTPtr indices_recalc_expr_list = std::make_shared<ASTExpressionList>();
const auto & indices = metadata_snapshot->getSecondaryIndices();
for (size_t i = 0; i < indices.size(); ++i)
{
// other code
}
// other code
}
With code in 21.6, indices_to_recalc has only one element in it as expected.
Why here the loop has been modified? Is code from 21.3 a bug? If not, why there are 35 MergeTreeIndexPtrs of same type in the result set?

Related

How to get gtest results?

How to get the result of EXPECT_EXIT() in gtest?
I want to do something according to the results of EXPECT_EXIT. For example:
for(int i = 0; i < 1000; i++) {
for(int j = 0; j < 10; j++) {
auto &result = EXPECT_EXIT(Function(i, inputs[j]));
if (result) {
do something;
break; //jump out of the loop
}
}
}
From your comment:
But once it failed, I need record something and skip rest of the input
check for this function
Short answer:
To stop execution at first mismatch you need to use ASSERT version instead of EXPECT, in your case ASSERT_EXIT.
To output some extra data for failed check you can just use stream operator of check:
ASSERT_EXIT(Function(i, j), testing::ExitedWithCode(0), "") << "Some custom data: " << i << "; " << j;
If you want to do something more on fail - you can use ::testing::Test::HasFailure() function.
Long suggestion:
One of the properties of good unittest is simplicity. That implies absence of any loops or branches. Setup logic (if any) should be moved to fixtures. Also unittests must be executed in isolation.
By switching your test to parametric one you would cover that. Check this:
void Function(int i, int j)
{
if (j != 3)
exit(0);
}
class FunTest : public testing::TestWithParam<std::tuple<int, int>> { };
TEST_P(FunTest, Should_Exit)
{
const auto i = std::get<0>(GetParam());
const auto j = std::get<1>(GetParam());
EXPECT_EXIT(Function(i, j), testing::ExitedWithCode(0), "");
}
INSTANTIATE_TEST_SUITE_P(,
FunTest,
testing::Combine(
testing::Range(1, 20),
testing::Range(1, 10)),
[](const testing::TestParamInfo<std::tuple<int, int>>& info) -> std::string {
const auto i = std::get<0>(info.param);
const auto j = std::get<1>(info.param);
return std::to_string(i) + '_' + std::to_string(j);
});
This approach would help with your problem: you can stop execution by using --gtest_break_on_failure and you can control naming of test cases by implementing custom printer for specific TestParamInfo (lambda in example).
It would also guarantee isolation and provide certain level of simplicity (all complex stuff provided by framework, imho test code is minimalistic and simple).
Also this would allow execution of specific test using gtest_filter
(e.g. --gtest_filter=FunTest.Should_Exit/4_3).

SAFEARRAY data to unsigned char*

I am trying to convert a SAFEARRAY data pointer to unsinged char*. However I am not getting the expected data. Here is a snippet.
SafeArrayLock(psaFrameData);
psaFrameData->rgsabound->cElements;
int nCount = psaFrameData->rgsabound->cElements - psaFrameData->rgsabound->lLbound + 1;
frameData = new unsigned char[nCount];
memset(frameData, 0, nCount);
for (int i = 0; i < nCount; ++i)
{
frameData[i] = ((unsigned char*)(psaFrameData)->pvData)[i];
}
SafeArrayUnlock(psaFrameData);
Do not manually lock the array and then access its pvData (or any of its other data members) directly. Use the various accessors functions instead, such as SafeArrayAccessData():
Increments the lock count of an array, and retrieves a pointer to the array data.
Try something more like this:
// safety check: make sure the array has only 1 dimension...
if (SafeArrayGetDim(psaFrameData) != 1)
{
// handle the error ...
}
else
{
// safety check: make sure the array contains byte elements...
VARTYPE vt = 0;
SafeArrayGetVartype(psaFrameData, &vt);
if (vt != VT_UI1)
{
// handle the error ...
}
else
{
// get a pointer to the array's byte data...
unsigned char *data;
if (FAILED(SafeArrayAccessData(psaFrameData, (void**)&data)))
{
// handle the error ...
}
else
{
// calculate the number of bytes in the array...
LONG lBound, uBound;
SafeArrayGetLBound(psaFrameData, 1, &lBound);
SafeArrayGetUBound(psaFrameData, 1, &uBound);
long nCount = uBound - lBound + 1;
// copy the bytes...
frameData = new unsigned char[nCount];
memcpy(frameData, data, nCount);
// release the pointer to the array's byte data...
SafeArrayUnaccessData(psaFrameData);
}
}
}

CAN communication not working between different PIC

I am working on project, and we need to establish a CAN communication between 4 nodes, 2 using a PIC 18F4580 and 2 using 18F25K80. In all those circuits, I'm using a Crystal oscillator 20MHz. The issue is when I test the communication between same PICs, it's working, but when I try with two different PICs it's not working.
The codes I used to test:
For the emitting PIC 18F4580 : Emitting a CAN message every 1 second :
int i;
unsigned char Can_Init_Flags, Can_Send_Flags, Can_Rcv_Flags; // can flags
unsigned char Rx_Data_Len; // received data length in bytes
char RxTx_Data[8]; // can rx/tx data buffer
char Msg_Rcvd; // reception flag
const long ID_cmd = 3, ID_led1 = 2; // node IDs
long Rx_ID;
void main() {
ADCON1=0xF;
TRISA=0xFF;
TRISD=0;
PORTD=0;
for(i=0;i<10;i++) {
PORTD=0xFF ^ PORTD; //Blinking Leds
Delay_ms(100);
}
Can_Init_Flags = 0; //
Can_Send_Flags = 0; // clear flags
Can_Rcv_Flags = 0; //
Can_Send_Flags = _CAN_TX_PRIORITY_0 & // form value to be used
_CAN_TX_XTD_FRAME & // with CANWrite
_CAN_TX_NO_RTR_FRAME;
Can_Init_Flags = _CAN_CONFIG_SAMPLE_THRICE & // form value to be used
_CAN_CONFIG_PHSEG2_PRG_ON & // with CANInit
_CAN_CONFIG_XTD_MSG &
_CAN_CONFIG_DBL_BUFFER_ON &
_CAN_CONFIG_VALID_XTD_MSG;
CANInitialize(1,3,3,3,1,Can_Init_Flags); // Initialize CAN module
CANSetOperationMode(_CAN_MODE_NORMAL,0xFF); // set NORMAL mode
for(i=0;i<10;i++) {
PORTD=0xFF ^ PORTD; //Blinking Leds
Delay_ms(100);
}
while(1){
PORTD.F7=PORTA.F0;
PORTD.F6=PORTA.F1;
PORTD.F5=PORTA.F2;
PORTD.F4=PORTA.F3; //LEDS := SWITCHS
CANWrite(ID_cmd, RxTx_Data, 1, Can_Send_Flags); // send incremented data back
Delay_ms(1000);
}
}
For the receiving Node PIC 18F25K80 : Blink after receiving any CAN message (Should blink every 1 second) :
unsigned char Can_Init_Flags, Can_Send_Flags, Can_Rcv_Flags; // can flags
unsigned char Rx_Data_Len; // received data length in bytes
char RxTx_Data[8]; // can rx/tx data buffer
char Msg_Rcvd; // reception flag
const long ID_led1 = 2, ID_cmd = 3; // node IDs
long Rx_ID;
void main() {
//OSCCON |= 0b01110010;
TRISC = 0;
Can_Init_Flags = 0; //
Can_Send_Flags = 0; // clear flags
Can_Rcv_Flags = 0; //
Can_Send_Flags = _CAN_TX_PRIORITY_0 & // form value to be used
_CAN_TX_XTD_FRAME & // with CANWrite
_CAN_TX_NO_RTR_FRAME;
Can_Init_Flags = _CAN_CONFIG_SAMPLE_THRICE & // form value to be used
_CAN_CONFIG_PHSEG2_PRG_ON & // with CANInit
_CAN_CONFIG_XTD_MSG &
_CAN_CONFIG_DBL_BUFFER_ON &
_CAN_CONFIG_VALID_XTD_MSG;
CANInitialize(1,3,3,3,1,Can_Init_Flags); // Initialize CAN module
CANSetOperationMode(_CAN_MODE_CONFIG,0xFF); // set CONFIGURATION mode
CANSetMask(_CAN_MASK_B1,-1,_CAN_CONFIG_XTD_MSG); // set all mask1 bits to ones
CANSetMask(_CAN_MASK_B2,-1,_CAN_CONFIG_XTD_MSG); // set all mask2 bits to ones
CANSetFilter(_CAN_FILTER_B2_F4,ID_cmd,_CAN_CONFIG_XTD_MSG);// set id of filter B2_F4 to 2nd node ID
CANSetOperationMode(_CAN_MODE_NORMAL,0xFF); // set NORMAL mode
while(1) { // endless loop
Msg_Rcvd = CANRead(&Rx_ID , RxTx_Data , &Rx_Data_Len, &Can_Rcv_Flags); // receive message
if ((Rx_ID == ID_cmd) && Msg_Rcvd) { // if message received check id
PORTC.F3=!PORTC.F3;
}
}
}
Any help would be greatly appreciated, thanks.
It's me again, it worked, the nodes must have the same oscillator value (in my case : 20MHz Crystal).

How to get a complete row data from VTS?

I am using VTS tables for passing data from 1 script to other. Now, I want to get data from all the column for particular row and print that.
I tried couple of VTC commands but unfortunately that did not work.
Command I tried:-
rc = lrvtc_query_row(vuser);
lr_output_message("Col1:- %s", lr_eval_string("{Col1}"));
can you please suggest where I got wrong or provide me a ready code to print he Row from the VTS table.
First you have to enable API access by click the "Enable" button from VTS Web Admin page, then here's the sample:
Action()
{
char* vts_ip = "127.0.0.1";
int vts_port = 8888;
char **colNames = NULL;
char **rowData = NULL;
int row_index = 1;
int i;
PVCI2 pvci = vtc_connect(vts_ip, vts_port, VTOPT_KEEP_ALIVE );
vtc_query_row(pvci, row_index, &colNames, &rowData);
for(i=0; colNames && colNames[i]; ++i){
lr_output_message("%s: %s", colNames[i], rowData[i]);
}
vtc_free_list(colNames);
vtc_free_list(rowData);
return 0;
}

Exporting ListView content to Excel?

I've been trying to export the content data of my ListView control into an Excel application using C#. I've already done this before using VB.NET and I tried to convert my code to C# but it failed but I made a modification on it and seems everything is fine on code below except for the last part which to save the data as an excel file. I need an assistance to modify this code correctly. I would greatly appreciate your helpful response.
The code below that got an error is:
oBook.SaveAs(SaveFileDialog1.InitialDirectory.ToString() + SaveFileDialog1.FileName);
Error 5 No overload for method 'SaveAs' takes '1' arguments
private void Button4_Click(object sender, System.EventArgs e)
{
int row = 0;
int col = 0;
int row2 = 0;
int col2 = 0;
int ch = 0;
int ctr = 0;
ctr = 0;
row2 = 1;
col2 = 3;
row = 3;
col = 3;
Microsoft.Office.Interop.Excel.Application oExcel = new Microsoft.Office.Interop.Excel.Application();
oExcel.Visible = false;
Microsoft.Office.Interop.Excel.Workbook oBook = oExcel.Workbooks.Add(Microsoft.Office.Interop.Excel.XlSheetType.xlWorksheet);
Microsoft.Office.Interop.Excel.Worksheet oSheet = (Microsoft.Office.Interop.Excel.Worksheet)oExcel.ActiveSheet;
SaveFileDialog SaveFileDialog1 = new SaveFileDialog();
SaveFileDialog1.Filter = "Excel File|*.xlsx|Word File|*.doc|Text File|*.txt";
SaveFileDialog1.Title = "Save As";
if (SaveFileDialog1.ShowDialog() == System.Windows.Forms.DialogResult.OK)
{
switch (SaveFileDialog1.FilterIndex)
{
case 1:
{
for (ch = 1; ch <= ListView1.Columns.Count; ch++)
{
oSheet.Cells[row2, col2] = ListView1.Columns[ctr].Text;
col2 = col2 + 1;
ctr = ctr + 1;
}
foreach (ListViewItem lview in ListView1.Items)
{
foreach (ListViewItem.ListViewSubItem lview2 in lview.SubItems)
{
oSheet.Cells[row, col] = lview2.Text;
col = col + 1;
}
col = 3;
row = row + 1;
}
oBook.SaveAs(SaveFileDialog1.InitialDirectory.ToString() + SaveFileDialog1.FileName);
oExcel.Quit();
SaveFileDialog1.Dispose();
MessageBox.Show("Data has been successfully saved", string.Empty, MessageBoxButtons.OK, MessageBoxIcon.Information);
break;
}
}
The workbook SaveAs method takes 11 parameters. In C# you have to pass something for these optional parameters even if you do not care about them. The easiest thing to do here is pass System.Reflection.Missing.Value for those you do not care about. It is a pain though.

Resources