std::string::erase() segfaults when given an iterator - c++11

I have a string-
std::string out;
and an iterator over that string-
std::string::iterator pos = out.begin();
if I do -
if(!out.empty() || pos > out.begin())
out.erase(pos--);
It segfaults, with this error-
Program received signal SIGSEGV, Segmentation fault.
__memmove_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:2821
2821 ../sysdeps/x86_64/multiarch/memcpy-ssse3.S: No such file or directory.
What am I missing?
For some context, I'm implementing line editing with ncurses and this code defines behaviour when backspace is pressed.
EDIT
It segfaults even after I move the decrement out of the call.
here is an MCVE -
#include <string>
#include <ncurses.h>
int main()
{
initscr();
cbreak();
keypad(stdscr, true); //Seems to be causing the error
std::string out;
std::string::iterator pos = out.begin();
short ch;
bool insert = false;
int y, x;
while(true)
{
ch = getch(); //Get next character
if(ch == '\n')
break; //End of line
else if(ch == KEY_BACKSPACE) //Backspace behaviour defined here
{
if(!out.empty())
{
out.erase(pos); //Character erased from string as per backspace
addch('\b'); //Erasure reflected on screen
std::string temp(pos, out.end()); //
getyx(stdscr, y, x);
mvaddstr(y, x - 1, temp.c_str());
}
}
else if(std::isalnum(ch) || std::ispunct(ch)) //Normal characters
{
if(!insert) //If insert is off
out.insert(pos++, ch); //Character inserted normally
else //Insert is on
{
*pos = ch; //Overwrite characters
pos++;
}
}
}
endwin();
return 0;
}
Compile with
g++ -lncurses -std=c++11
-Wall gives no clues.

This is expected behaviour for std::string. In the C++ standard, validity of string iterators is not garanteed after a modification of the string, contrary to other types with stronger requirements.
In [string.require] (§21.4.1), the standard states:
References, pointers, and iterators referring to the elements of a
basic_­string sequence may be invalidated by the following uses of
that basic_­string object: [...] Calling non-const member functions,
except operator[], at, data, front, back, begin, rbegin, end, and
rend.

Related

compiler segfault when printf is added (gcc 10.2 aarch64_none-elf- from arm)

I know this is not adequate for stack overflow question, but ..
This is a function in scripts/dtc/libfdt/fdt_ro.c of u-boot v2021.10.
const void *fdt_getprop_namelen(const void *fdt, int nodeoffset,
const char *name, int namelen, int *lenp)
{
int poffset;
const struct fdt_property *prop;
printf("uuu0 nodeoffset = 0x%x, name = %s, namelen = %d\n", nodeoffset, name, namelen);
prop = fdt_get_property_namelen_(fdt, nodeoffset, name, namelen, lenp,
&poffset);
//printf("uuu1 prop = 0x%lx, *lenp = 0x%x, poffset = 0x%x\n", prop, *lenp, poffset);
if (!prop)
return NULL;
/* Handle realignment */
if (fdt_chk_version() && fdt_version(fdt) < 0x10 &&
(poffset + sizeof(*prop)) % 8 && fdt32_to_cpu(prop->len) >= 8)
return prop->data + 4;
return prop->data;
}
When I build the program, if I uncomment the second printf, the compiler seg-faults.
I have no idea. Is it purely compiler problem(I think so it should never die at least)? or can it be linked to my fault somewhere in another code? Is there any method to know the cause of the segfault? (probably not.).
If you're getting a segmentation fault when running the compiler itself, the compiler has a bug. There are some errors in your code, but those should result in compile-time diagnostics (warnings or error messages), never a compile-time crash.
The code in your question is incomplete (missing declarations for fdt_get_property_namelen_, printf, NULL, etc.). Reproduce the problem with a complete self-contained source file and submit a bug report: https://gcc.gnu.org/bugzilla/
printf("uuu1 prop = 0x%lx, *lenp = 0x%x, poffset = 0x%x\n", prop, *lenp, poffset);
prop is a pointer, so I'd use %p instead of %lx
lenp is a pointer, so I'd make sure that it points to valid memory

Read binary data from QProcess in Windows

I have some .exe file (say some.exe) that writes to the standard output binary data. I have no sources of this program. I need to run some.exe from my C++/Qt application and read standard output of the process I created. When I'm trying to do this with QProcess::readAll someone replaces byte \n (0x0d) to \r\n (0x0a 0x0d).
Here is a code:
QProcess some;
some.start( "some.exe", QStringList() << "-I" << "inp.txt" );
// some.setTextModeEnabled( false ); // no effect at all
some.waitForFinished();
QByteArray output = some.readAll();
I tried in cmd.exe to redirect output to file like this:
some.exe -I inp.txt > out.bin
and viewed out.bin with hexedit there was 0a 0d in the place where should be 0d.
Edit:
Here is a simple program to emulate some.exe behaviour:
#include <stdio.h>
int main() {
char buf[] = { 0x00, 0x11, 0x0a, 0x33 };
fwrite( buf, sizeof( buf[ 0 ] ), sizeof( buf ), stdout );
}
run:
a.exe > out.bin
//out.bin
00 11 0d 0a 33
Note, that I can't modify some.exe that's why I shouldn't modify my example like _setmode( _fileno( stdout, BINARY ) )
The question is: how can I say to QProcess or to Windows or to console do not change CR with LF CR?
OS: Windows 7
Qt: 5.6.2
how can I say to QProcess or to Windows or to console do not change CR with LF CR?
They don't change anything. some.exe is broken. That's all. It outputs the wrong thing. Whoever made it output brinary data in text mode has messed up badly.
There's a way to recover, though. You have to implement a decoder that will fix the broken output of some.exe. You know that every 0a has to be preceded by 0d. So you have to parse the output, and if you find a 0a, and there's 0d before it, remove the 0d, and continue. Optionally, you can abort if a 0a is not preceded by 0d - some.exe should not produce such output since it's broken.
The appendBinFix function takes the corrupted data and appends the fixed version to a buffer.
// https://github.com/KubaO/stackoverflown/tree/master/questions/process-fix-binary-crlf-51519654
#include <QtCore>
#include <algorithm>
bool appendBinFix(QByteArray &buf, const char *src, int size) {
bool okData = true;
if (!size) return okData;
constexpr char CR = '\x0d';
constexpr char LF = '\x0a';
bool hasCR = buf.endsWith(CR);
buf.resize(buf.size() + size);
char *dst = buf.end() - size;
const char *lastSrc = src;
for (const char *const end = src + size; src != end; src++) {
char const c = *src;
if (c == LF) {
if (hasCR) {
std::copy(lastSrc, src, dst);
dst += (src - lastSrc);
dst[-1] = LF;
lastSrc = src + 1;
} else
okData = false;
}
hasCR = (c == CR);
}
dst = std::copy(lastSrc, src, dst);
buf.resize(dst - buf.constData());
return okData;
}
bool appendBinFix(QByteArray &buf, const QByteArray &src) {
return appendBinFix(buf, src.data(), src.size());
}
The following test harness ensures that it does the right thing, including emulating the output of some.exe (itself):
#include <QtTest>
#include <cstdio>
#ifdef Q_OS_WIN
#include <fcntl.h>
#include <io.h>
#endif
const auto dataFixed = QByteArrayLiteral("\x00\x11\x0d\x0a\x33");
const auto data = QByteArrayLiteral("\x00\x11\x0d\x0d\x0a\x33");
int writeOutput() {
#ifdef Q_OS_WIN
_setmode(_fileno(stdout), _O_BINARY);
#endif
auto size = fwrite(data.data(), 1, data.size(), stdout);
qDebug() << size << data.size();
return (size == data.size()) ? 0 : 1;
}
class AppendTest : public QObject {
Q_OBJECT
struct Result {
QByteArray d;
bool ok;
bool operator==(const Result &o) const { return ok == o.ok && d == o.d; }
};
static Result getFixed(const QByteArray &src, int split) {
Result f;
f.ok = appendBinFix(f.d, src.data(), split);
f.ok = appendBinFix(f.d, src.data() + split, src.size() - split) && f.ok;
return f;
}
Q_SLOT void worksWithLFCR() {
const auto lf_cr = QByteArrayLiteral("\x00\x11\x0a\x0d\x33");
for (int i = 0; i < lf_cr.size(); ++i)
QCOMPARE(getFixed(lf_cr, i), (Result{lf_cr, false}));
}
Q_SLOT void worksWithCRLF() {
const auto cr_lf = QByteArrayLiteral("\x00\x11\x0d\x0a\x33");
const auto cr_lf_fixed = QByteArrayLiteral("\x00\x11\x0a\x33");
for (int i = 0; i < cr_lf.size(); ++i)
QCOMPARE(getFixed(cr_lf, i), (Result{cr_lf_fixed, true}));
}
Q_SLOT void worksWithCRCRLF() {
for (int i = 0; i < data.size(); ++i) QCOMPARE(getFixed(data, i).d, dataFixed);
}
Q_SLOT void worksWithQProcess() {
QProcess proc;
proc.start(QCoreApplication::applicationFilePath(), {"output"},
QIODevice::ReadOnly);
proc.waitForFinished(5000);
QCOMPARE(proc.exitCode(), 0);
QCOMPARE(proc.exitStatus(), QProcess::NormalExit);
QByteArray out = proc.readAllStandardOutput();
QByteArray fixed;
appendBinFix(fixed, out);
QCOMPARE(out, data);
QCOMPARE(fixed, dataFixed);
}
};
int main(int argc, char *argv[]) {
QCoreApplication app(argc, argv);
if (app.arguments().size() > 1) return writeOutput();
AppendTest test;
QTEST_SET_MAIN_SOURCE_PATH
return QTest::qExec(&test, argc, argv);
}
#include "main.moc"
Unfortunately it has nothing to do with QProcess or Windows or console. It's all about CRT. Functions like printf or fwrite are taking into account _O_TEXT flag to add an additional 0x0D (true only for Windows). So the only solution is to modify stdout's fields of your some.exe with WriteProcessMemory or call the _setmode inside an address space of your some.exe with DLL Injection technique or patch the lib. But it's a tricky job.

disable assign integer to string c++

std::string (std::basic_string) have assignment operator for 'char' type.
But, for this reason, std::string may assign any integral types.
See little example.
#include <string>
enum MyEnum{ Va = 0, Vb = 2, Vc = 4 };
int main(){
std::string s;
s = 'a'; // (1) OK - logical.
s = Vc; // (2) Ops. Compiled without any warnings.
s = true; // (3) Ops....
s = 23; // (4) Ops...
}
Q: How disable (or add warning ) (2, 3, 4) situations ??
There is a related Question
Given the constraints of C++03 and GCC 4.8 as in the tags, I could not get -Wconversion to do anything useful (and in GCC 7 it doesn't even generate the warnings for me despite telling it that I'm using --std=c++03).
As such, there's a good practical solution that requires only minimal change at your calling site:
Proxy the assignment via a class object that wraps your string and that allows assignment from char but disallows it from int:
#include <string>
enum MyEnum{ Va = 0, Vb = 2, Vc = 4 };
struct string_wr {
string_wr (std::string& s) : val(s) {}
operator std::string& () const { return val; }
// we explicitly allow assigning chars.
string_wr& operator= (char) { return *this; }
// ww explicitly disable assigning ints by making the operator unreachable.
private:
string_wr& operator= (int);
private:
std::string& val;
};
int main(){
std::string s;
s = 'a'; // (1) OK - logical.
s = Vc; // (2) Ops. Compiled without any warnings.
s = true; // (3) Ops....
s = 23; // (4) Ops...
string_wr m(s); // this is the only real change at the calling site
m = 'a'; // (1) OK - logical.
m = Vc; // (2) Should fail with "assignment is private" kind of error.
m = true; // (3) Should fail...
m = 23; // (4) Should fail...
}
However, if your final goal is to specifically get warnings or errors when using std::string, your best option in C++03 is to patch the <string> header to add the private int-assignment operator shown in the class above. But that means patching a system header and the procedure and results will be dependant on your compiler version (and will have to be repeated in each installation and compiler version).

Accepting and printing a string

Can we accept and print a string like this in c++?
This code is not working properly.
#include<iostream>
#include<string>
using namespace std;
main()
{
string a;char ch;
for(int i=0;i<5;i++)
{cin>>ch;
a[i]=ch;
}
a[5]='\0';
cout<<a;
}
I am able to print individual elements like a[1],a[2],etc but unable to print the entire string.Why?
If you want to take a string, you could do the following.
#include <iostream>
int main() {
std::string str;
std::getline(std::cin, str);
std::cout << str;
}
Also, C++ automatically null terminates any string literal you use.
Well it's not really anywhere near best-practices but to fix your immediate issue you need to actually resize the string.
#include<iostream>
#include<string>
main()
{
std::string a;char ch;
a.resize(5); // <--- reserves memory
for(int i=0;i<5;i++)
{
std::cin>>ch;
a[i]=ch;
}
a[5]='\0'; //<-- unnecessary
st::cout<<a;
}
alternatively you can append the characters
#include<iostream>
#include<string>
main()
{
std::string a;char ch;
for(int i=0;i<5;i++)
{
std::cin>>ch;
a+=ch;
}
std::cout<<a;
}
The real problem here is not that you can't read or can't print the string, is that you are writing to unallocated memory. operator[], which is what you are using when you do something like a[i]=ch, does not do any kind of boundary checking and thus you are causing undefined behavior. In my machine, nothing is printed, for instance.
In short, you need to make sure that you have space to write your characters. If you are certain that you are going to read 5 characters (and adding a \0 at the end, making it 6 in length), you could do something like this:
std::string a(6, '\0')
If you are uncertain of how many characters you are going to read, std::string is ready to allocate space as need, but you need to use std::push_back to give it a chance to do so. Your loop contents would be something like:
cin >> ch;
a.push_back(ch);
If you are uncertain where the std::string object is coming from (as in, this is library code that accepts a std::string as an argument, you could use at(i) (e.g, a.at(i) = ch instead of a[i] = ch), which throws an exception if it is out of range.
You can print the string like this
#include<iostream>
#include<string>
using namespace std;
int main()
{
string a;char ch;
for(int i=0;i<5;i++)
{
cin>>ch;
a.push_back(ch);
}
a.push_back('\0');
cout << a;
return 0;
}

As one MPI process executes MPI_Barrier(), other processes hang

I have an MPI program for having multiple processes read from a file that contains list of file names and based on the file names read - it reads the corresponding file and counts the frequency of words.
If one of the processes completes this and returns - to block executing MPI_Barrier(), the other processes also hang. On debugging, it could be seen that the readFile() function is not entered by the processes currently in process_files() Unable to figure out why this happens. Please find the code below:
#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>
#include <ctype.h>
#include <string.h>
#include "hash.h"
void process_files(char*, int* , int, hashtable_t* );
void initialize_word(char *c,int size)
{
int i;
for(i=0;i<size;i++)
c[i]=0;
return;
}
char* readFilesList(MPI_File fh, char* file,int rank, int nprocs, char* block, const int overlap, int* length)
{
char *text;
int blockstart,blockend;
MPI_Offset size;
MPI_Offset blocksize;
MPI_Offset begin;
MPI_Offset end;
MPI_Status status;
MPI_File_open(MPI_COMM_WORLD,file,MPI_MODE_RDONLY,MPI_INFO_NULL,&fh);
MPI_File_get_size(fh,&size);
/*Block size calculation*/
blocksize = size/nprocs;
begin = rank*blocksize;
end = begin+blocksize-1;
end+=overlap;
if(rank==nprocs-1)
end = size;
blocksize = end-begin+1;
text = (char*)malloc((blocksize+1)*sizeof(char));
MPI_File_read_at_all(fh,begin,text,blocksize,MPI_CHAR, &status);
text[blocksize+1]=0;
blockstart = 0;
blockend = blocksize;
if(rank!=0)
{
while(text[blockstart]!='\n' && blockstart!=blockend) blockstart++;
blockstart++;
}
if(rank!=nprocs-1)
{
blockend-=overlap;
while(text[blockend]!='\n'&& blockend!=blocksize) blockend++;
}
blocksize = blockend-blockstart;
block = (char*)malloc((blocksize+1)*sizeof(char));
block = memcpy(block, text + blockstart, blocksize);
block[blocksize]=0;
*length = strlen(block);
MPI_File_close(&fh);
return block;
}
void calculate_term_frequencies(char* file, char* text, hashtable_t *hashtable,int rank)
{
printf("Start File %s, rank %d \n\n ",file,rank);
fflush(stdout);
if(strlen(text)!=0||strlen(file)!=0)
{
int i,j;
char w[100];
i=0,j=0;
while(text[i]!=0)
{
if((text[i]>=65&&text[i]<=90)||(text[i]>=97&&text[i]<=122))
{
w[j]=text[i];
j++; i++;
}
else
{
w[j] = 0;
if(j!=0)
{
//ht_set( hashtable, strcat(strcat(w,"#"),file),1);
}
j=0;
i++;
initialize_word(w,100);
}
}
}
return;
}
void readFile(char* filename, hashtable_t *hashtable,int rank)
{
MPI_Status stat;
MPI_Offset size;
MPI_File fx;
char* textFromFile=0;
printf("Start File %d, rank %d \n\n ",strlen(filename),rank);
fflush(stdout);
if(strlen(filename)!=0)
{
MPI_File_open(MPI_COMM_WORLD,filename,MPI_MODE_RDONLY,MPI_INFO_NULL,&fx);
MPI_File_get_size(fx,&size);
printf("Start File %s, rank %d \n\n ",filename,rank);
fflush(stdout);
textFromFile = (char*)malloc((size+1)*sizeof(char));
MPI_File_read_at_all(fx,0,textFromFile,size,MPI_CHAR, &stat);
textFromFile[size]=0;
calculate_term_frequencies(filename, textFromFile, hashtable,rank);
MPI_File_close(&fx);
}
printf("Done File %s, rank %d \n\n ",filename,rank);
fflush(stdout);
return;
}
void process_files(char* block, int* length, int rank,hashtable_t *hashtable)
{
char s[2];
s[0] = '\n';
s[1] = 0;
char *file;
if(*length!=0)
{
/* get the first file */
file = strtok(block, s);
/* walk through other tokens */
while( file != NULL )
{
readFile(file,hashtable,rank);
file = strtok(NULL, s);
}
}
return;
}
void execute_process(MPI_File fh, char* file, int rank, int nprocs, char* block, const int overlap, int * length, hashtable_t *hashtable)
{
block = readFilesList(fh,file,rank,nprocs,block,overlap,length);
process_files(block,length,rank,hashtable);
}
int main(int argc, char *argv[]){
/*Initialization*/
MPI_Init(&argc, &argv);
MPI_File fh=0;
int rank,nprocs,namelen;
char *block=0;
const int overlap = 70;
char* file = "filepaths.txt";
int *length = (int*)malloc(sizeof(int));
hashtable_t *hashtable = ht_create( 65536 );
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &nprocs);
char processor_name[MPI_MAX_PROCESSOR_NAME];
MPI_Get_processor_name(processor_name, &namelen);
printf("Rank %d is on processor %s\n",rank,processor_name);
fflush(stdout);
execute_process(fh,file,rank,nprocs,block,overlap,length,hashtable);
printf("Rank %d returned after processing\n",rank);
MPI_Barrier(MPI_COMM_WORLD);
MPI_Finalize();
return 0;
}
The filepaths.txt is a file that contain the absolute file names of normal text files:
eg:
/home/mpiuser/mpi/MPI_Codes/code/test1.txt
/home/mpiuser/mpi/MPI_Codes/code/test2.txt
/home/mpiuser/mpi/MPI_Codes/code/test3.txt
Your readFilesList function is pretty confusing, and I believe it doesn't do what you want it to do, but maybe I just do not understand it correctly. I believe it is supposed to collect a bunch of filenames out of the list file for each process. A different set for each process. It does not do that, but this is not the problem, even if this would do what you want it to, the subsequent MPI IO would not work.
When reading files, you use MPI_File_read_all with MPI_COMM_WORLD as communicator. This requires all processes to participate in reading this file. Now, if each process should read a different file, this obviously is not going to work.
So there are several issues with your implementation, though I can not really explain your described behavior, I would rather first start off and try to fix them, before debugging in detail, what might go wrong.
I am under the impression, you want to have an algorithm along these lines:
Read a list of file names
Distribute that list of files equally to all processes
Have each process work on its own set of files
Do something with the data from this processing
And I would suggest to try this with the following approach:
Read the list on a single process (no MPI IO)
Scatter the list of files to all processes, such that all get around the same amount of work
Have each process work on its list of files independently and in serial (serial file access and processing)
Some data reduction with MPI, as needed
I believe, this would be the best (easiest and fastest) strategy in your scenario. Note, that no MPI IO is involved here at all. I don't think doing some complicated distributed reading of the file list in the first step would result in any advantage here, and in the actual processing it would actually be harmful. The more independent your processes are, the better your scalability usually.

Resources