Using libssh library, I have a trouble with ssh_scp_read function - windows

Question

Now I'm trying to download file from server (Linux) to my host (windows).
It looks working well (other functions) but when I'm trying to read file from server. Data in the file is not proper.
for example)
server text file:
A
B
C
D
E
Reading file and saving to buffer:
A
B
C
I checked small size text file. It was working well. However now I'm in trouble because text file is bigger than before.
Here is my code.
int scp_receive(ssh_session session)
{
ssh_scp scp;
int rc;
int size, mode;
char *filename;
char *buffer;
int fd;
scp = ssh_scp_new(session, SSH_SCP_READ, "/home/abc/27");
if (scp == NULL)
{
fprintf(stderr, "Error allocating scp session: %s\n",
ssh_get_error(session)); return SSH_ERROR;
}
rc = ssh_scp_init(scp);
if (rc != SSH_OK)
{
fprintf(stderr, "Error initializing scp session: %s\n",ssh_get_error(session));
ssh_scp_free(scp);
return rc;
}
rc = ssh_scp_pull_request(scp);
if (rc != SSH_SCP_REQUEST_NEWFILE)
{
fprintf(stderr, "Error receiving information about file: %s\n",ssh_get_error(session));
return SSH_ERROR;
}
size = ssh_scp_request_get_size(scp);
std::cout << "size is:" << size << std::endl;
filename = strdup(ssh_scp_request_get_filename(scp));
mode = ssh_scp_request_get_permissions(scp);
printf("Receiving file %s, size %d, permisssions 0%o\n", filename, size, mode);
free(filename);
buffer = (char *)malloc(sizeof(char)*size);
if (buffer == NULL)
{
fprintf(stderr, "Memory allocation error\n");
return SSH_ERROR;
}
ssh_scp_accept_request(scp);
rc = ssh_scp_read(scp, buffer, sizeof(char)*size);
if(rc == SSH_ERROR)
{
fprintf(stderr, "Error receiving file data: %s\n", ssh_get_error(session));
free(buffer);
return rc;
}
std::cout << buffer << std::endl;
printf("Done\n");
//write(fd, buffer, size);
char path[30];
sprintf(path, "C:/Check/%s", filename);
if (access(path, F_OK) != 0) {
//TODO: file
remove(path);
}
if(0 < (fd=open(path, O_RDWR | O_CREAT | O_TRUNC, mode)))
{
write(fd, buffer, sizeof(char)*size);
close(fd);
}
else
{
std::cerr << "failed to open" << std::endl;
}
free(buffer);
//rc = ssh_scp_pull_request(scp);
//if(rc != SSH_SCP_REQUEST_EOF)
//{
// fprintf(stderr, "Unexpected request: %s\n",ssh_get_error(session));
// return SSH_ERROR;
//}
ssh_scp_close(scp);
ssh_scp_free(scp);
return SSH_OK;
}

Answer 1

The issue here is with the libssh read function. It only reads a max of 65536 bytes at once. It won't read anything large than that block size.
If you take a look at the libssh SCP read function here: http://xtalopt.openmolecules.net/wiki/index.fcgi/browser/src/libssh/scp.c?rev=2b0288492ad2481ee8bdbb8c1f9d5c453a044eee
You will see the size is limited to 65536.
SOLUTION
In your application, you should attempt to read multiple smaller chunks of memory till you finish reading the large text file. And since you have the size of the large text file, this can be done easily via a loop.
I know this is a late answer, but hope this can help someone else having this issue.