I have a quick question: is it possible to send type MPI_Comm (handler of communicator) via MPI_Send/MPI_ISend? Will receiving process be able to use this communicator handler normally?
No, communicator handles are locally valued objects which means that their value makes no sense in other processes. This is also true for most (if not all) MPI handles. For example, in Open MPI communicator handles are either pointers to structures (C bindings) or indices in an array of pointers to such structures (Fortran bindings), so both make no sense outside of the process where they were assigned.
Related
In the Windows system, we can modify the memory of another process across processes. For example, if process A wants to modify the memory of process B, A can call the system function WriteProcessMemory. The approximate form is as follows:
BOOL flag = WriteProcessMemory(handler, p_B_addr, &p_A_buff, write_size); ...
This function return a Boolean value, which represents whether the write operation is successful. It needs to pass four parameters, let's take a look at these four parameters:
handler. This is a process handle, and it can be used to find B process.
p_B_addr. In process B, the address offset to be written into memory.
p_A_buff. In process A, the pointer to the write data buffer.
write_size. The number of bytes to write.
I am confused about the first parameter handler, which is a variable of type HANDLE. For example, when our program is actually running, the ID of process B is 2680, and then I want to write memory to process B. First I need to use this 2680 to get the handle of process B in process A. The specific form is handler=OpenProcess(PROCESS_ALL_ACCESS, FALSE, 2680), and then you can use this handler to fall into the kernel to modify the memory of process B.
Since they are all trapped in kernel functions to modify memory across processes, why is the WriteProcessMemory function not designed to be in the form of WriteProcessMemory(B_procID, p_B_addr, &p_A_buff, write_size)?
Among them, B_procID is the ID of the B process, since each process they all have unique IDs. Can the system kernel not find the physical address that the virtual address of the B process can map through this B_procID? Why must the process handle index of the B process in the A process be passed in?
There are multiple reasons, all touched on in the comments.
Lifetime. The process id is simply a number, knowing the id does not keep the process alive. Having a open handle to a process means the kernel EPROCESS structure and the process address space will stay intact, even if said process finishes by calling ExitProcess. Windows tries to not re-use the id for a new process right away but it will happen some time in the future given enough time.
Security/Access control. In Windows NT, access control is performed when you open a object, not each time you interact with the object. In this case, the kernel needs to know that the caller has PROCESS_VM_WRITE and PROCESS_VM_OPERATION access to the process. This is related to point 3, efficiency.
Speed. Windows could of course implement a WriteProcessMemoryById function that calls OpenProcess+WriteProcessMemory+CloseHandle but this encourages sub optimal design as well as opening you up to race conditions related to point 1. The same applies to "why is there no WriteFileByFilename function" (and all other Read/Write functions).
Sometimes not all ranks are needed for a compute task. I am trying to tailor the communicator to the need but MPI appear to stop after the reduced communicator has been constructed (rank=8). MPI debug says 'fatal error in PMPI_Comm_rank: invalid communicator'. My essential code is:
PROGRAM mpi_comm_create
USE MPI
IMPLICIT NONE
INTEGER comm,i,ierr,group,rank,rank1,root,size,size1,redcomm,redgroup
INTEGER,ALLOCATABLE::ranks(:)
comm=MPI_COMM_WORLD
!---------------------------------------------------------------------------
CALL MPI_init(ierr)
CALL MPI_comm_size(comm,size,ierr)
CALL MPI_comm_rank(comm,rank,ierr)
CALL MPI_comm_group(comm,group,ierr)
!--------------------------------------------------------------------------
size1=size-2
ALLOCATE(ranks(size1))
ranks(1:size1)=[0:size1-1]
!---------------------------------------------------------------------------
!Define new group redgroup and communicator redcomm with size1 processes
CALL MPI_group_incl(group,size1,ranks,redgroup,ierr)
CALL MPI_comm_create(comm,redgroup,redcomm,ierr)
CALL MPI_comm_rank(redcomm,rank1,ierr)
!---------------------------------------------------------------------------
!Use redcomm in a REDUCE operation
!---------------------------------------------------------------------------
CALL MPI_group_free(redgroup,ierr)
CALL MPI_comm_free(redcomm,ierr)
!---------------------------------------------------------------------------
CALL MPI_FINALIZE(ierr)
DEALLOCATE(ranks)
STOP; END
Firstly use MPI_COMM_SPLIT - it is easier as indicated in replies to your essentially identical earlier question.
Secondly the error is because all processes are making the second call to mpi_comm_rank, but not all processes are in the communicator specified in that call.
The root cause of the crash is ranks [size1:size-1] invoke MPI_Comm_rank() on MPI_COMM_NULL, which is not allowed since MPI_COMM_NULL is not a valid communicator. In order to get rid of the crash, you can replace
CALL MPI_comm_rank(redcomm,rank1,ierr)
with
IF (MPI_COMM_NULL.ne.redcomm) CALL MPI_comm_rank(redcomm,rank1,ierr)
From a performance point of view (which is what your question is really about), i do not expect any significant difference between MPI_Comm_create() and MPI_Comm_split().
From a semantic point of view, both MPI_Comm_split() and MPI_Comm_create() are collective operations, and they must be invoked by all MPI tasks from MPI_COMM_WORLD. If MPI tasks [size1:size-1] cannot/should not be involved in the creation of redcomm, then you can use MPI_Comm_create_group(), that should only be called by MPI tasks [0:size1-1].
If all tasks can be involved in the creation of redcomm, then i suggest you stick to MPI_Comm_split() in order to keep your code simpler. If you do not need MPI tasks [size1:size-1] being part of a valid communicator redcomm, then i also suggest you use color=MPI_UNDEFINED on these tasks, so redcomm will be MPI_COMM_NULL instead of a valid communicator. But keep in mind MPI_COMM_NULL is not a valid communicator, so it is up to your code not to invoke MPI_Comm_rank(), MPI_Comm_size(), MPI_Comm_free() and other subroutines on it.
While trying to learn more about linux kernel networking ... I have a kernel module that contains a protocol which runs on top of TCP. Its almost an application layer protocol I'm experimenting with. The calls are passed in via the normal system call interface as executed from userspace.
So network calls from within my (layer above TCP) module generally look like this ...
ret = sock->ops->connect(sock, (struct sockaddr *) &myprot.daddr,
sizeof(myprot.daddr), flags);
I've used sendmsg/recvmsg successfully within my KM to send and receive data from a client to a server (from two separate kernel instances). The calls within the KM generally looks as follows:
ret = sock->ops->sendmsg(iocb, myprot.skt, &msg, sizeof(struct msghdr));
ret = sock->ops->recvmsg(iocb, sock, msg, total_len, flags);
What I'm trying to understand now is how and when to use sk_buff to do the same thing. I.e. when to use system calls such as what I use above, and when to directly access the network stack via sk_buff to send and receive data.
I've found many examples of how to send and receive data from within transport layers using sk_buff, but nothing from a layer above the transport that is also contained in a kernel module and using sk_buff.
Update for clarification.
I've overridden struct proto_ops and replaced the member methods for my own protocols use which do correspond to system calls from user space. I do understand that sk_buff is the buffer system for the kernel and is where packets are enqueued. However. I don't see any reason why I can't use the protocol-specific functions of struct proto_ops which also handles sockets and the data enqueued on them (though at a higher level). So it seems to me there are two ways to access sk_buffs depending upon where one wants to access them.
If I'm working in the transport layer and want to access data anywheres within the network stack (e.g. transport, ip, mac), I could directly access sk_buffs, but if I am working above the transport layer, I would use the abstracted protocol specific member functions that correspond to system calls. After all, they both eventually work on sk_buffs.
I guess my confusion, or what I'm trying to confirm that I'm doing right or wrong by knowing the difference in these two ways to access sk_buffs and from where, is ... if I'm sending data over a transport from TCP within the kernel, than I can just make use of the proto_ops system calls that relate to TCP unless I need more control in which I would then make use of the lower level skb functions to manage the queues.
Not sure to understand because you want to use to different things for the same purpose. The proto_ops in sock->ops are operations invoked during the correspondent system call. The sk_buff is the socket buffer system of the kernel; it is the place where packet are enqueued.
There is not the possibility to do the same thing of proto_ops with sk_buff, if it should be possible one of these structures is useless.
I'm writing a win32 dll for read/write USB HID device. The data for exchange is a 64 byte unsigned char array. The client program is written in C++/CLI.
In order to achieve max speed and minimum overhead, I need an efficient way to sending the array to managed client.
There are two options I can think of right now:
Native: use PostMessage and send the pointer for the array.
Managed: in WndProc, Marshal.Copy the pointer to a new managed Byte array, then delete the pointer.
Native: use function pointer as a callback to process the data.
Managed: use Marshal.GetFunctionPointerForDelegate to pass function pointer to native world.
Thanks.
I would say choosing bizarre ways of marshalling is premature optimization. Use the most simple way to marshal the data, and try to evaluate other methods if that doesn't perform satisfactorily. Barring any architectural need for callbacks or message posting, why not just pass the array to a function?
Is it possible to use IO Completion Ports for Serial I/O? According to Windows via C/C++ it is alluded to that it is possible, and does give an example of using IOCP with physical files showing work with CreateFile, ReadFile, WriteFile, etc. However can this actually work with serial comms - has anyone got it working?
I can't find any examples of this on the web, but I cannot be the first to attempt it?
Yes, using I/O Completion Ports for Serial I/O works fine. There is some setup work needed to create a file handle for a serial port that is appropriate for IOCP. But once the setup is done, you can do asynchronous ReadFile() and WriteFile() operations just like with regular file handles and socket handles.
The setup is basically:
Open serial port with CreateFile() passing in the FILE_FLAG_OVERLAPPED value as the dwFlagsAndAttributes parameter.
Modify the serial port state as desired using GetCommState() and SetCommState(). Do this just like you would do when not using IOCP.
Use GetCommTimeouts() and SetCommTimeouts() to turn off total-timeouts for read operations, since it typically doesn't make sense to have timeouts for asynchronous operations. (You would instead explicitly call CancelIO() to cancel a read operation instead.) Turning off total-timeouts is done by setting the ReadTotalTimeoutMultiplier and ReadTotalTimeoutConstant fields of the COMMTIMEOUTS structure to zero.
Now you can use the handle with IOCP just like you would do with regular file handles and socket handles. I.e. attach the handle to a completion port using CreateIoCompletionPort(), initiate I/O operations with ReadFile() or WriteFile() using an OVERLAPPED structure, dequeue completed, failed or canceled operations from the completion port using the GetQueuedCompletionStatus() function.
Additional serial port specific events can also be retrieved asynchronously using the WaitCommEvent() function.