I'm making a TCP client-server connection and I'm trying to send files, however, I was wondering what is the right way to send the name of the file so that it gets saved on the server side.
The code that the client uses to send the file is the following
func sendFile(connection net.Conn, fileName string) {
//Open the file
file, err := os.Open(strings.TrimSpace(fileName))
if err != nil {
fmt.Println(err)
return
}
// make sure to close the file
defer file.Close()
// Send the file to the connection
n, err := io.Copy(connection, file)
if err != nil {
fmt.Println("We couldn't send the message to the server")
fmt.Println(err)
return
}
fmt.Println(n, "bytes sent")
}
The Server receives the file without any problems, however, I don't know how to send the name of the file. Any thoughts?
Here is the code from the server that receives the file
func (client *Client) read() {
defer client.connection.Close()
for {
// Here I should replace 'ServerFileReceived.jpg' with the name of the
// file that is coming
file, err := os.Create("ServerFileReceived.jpg")
if err != nil {
fmt.Println("2: Woah, there's a mistake here :/")
fmt.Println(err)
}
// make sure to close the file
defer file.Close()
n, err := io.Copy(file, client.connection)
if err != nil {
fmt.Println("3: Woah, there's a mistake here :/")
fmt.Println(err)
}
fmt.Println("########")
fmt.Println(n, "bytes received")
fmt.Println("########")
// Read Client Data
_, error := bufio.NewReader(client.connection).ReadString('\n')
if error != nil {
fmt.Println("4: Woah, there's a mistake here :/")
fmt.Println(error)
fmt.Println("4: Woah, there's a mistake here :/")
}
}
}
Related
I'm trying to make a music app that sends file through tcp protocol using go and microservice architecture. Now I'm creating a player service that should:
Get user token and get claims from it
Check is user exists using claims and user_service microservice
Get song from redis
Check is song exists using music_service
Read file by chunks and send it to client using tcp
Redis data looks like this:
{
"user_id": [{
"song_id": "<song_id>"
}]
}
But I faced with a small problem. My music files stored in a flac format and when I receive it on the client, my player doesn't play it. I don't really know what can be the problem. So here's my code:
SERVER
service_setup.go
//this function is called in main function
func setService() {
ln, err := net.Listen("tcp", config.TCPAddress)
if err != nil {
panic("couldn't start tcp server")
}
defer ln.Close()
for {
conn, err := ln.Accept()
if err != nil {
logger.ErrorLog(fmt.Sprintf("Error: couldn't accept connection. Details: %v", err))
return
}
service.DownloadSong(conn)
}
}
downloader_service.go
func DownloadSong(conn net.Conn) {
token, err := bufio.NewReader(conn).ReadString('\n')
if err != nil {
logger.ErrorLog(fmt.Sprintf("Error: couldn't get token. Details: %v", token))
conn.Close()
return
}
claims, err := jwt_funcs.DecodeJwt(token)
if err != nil {
conn.Close()
return
}
songs, err := redis_repo.Get(claims.Id)
if err != nil {
conn.Close()
return
}
for _, song := range songs {
download(song, conn)
}
}
func download(song models.SongsModel, conn net.Conn) {
filePath, err := filepath.Abs(fmt.Sprintf("./songs/%s.flac", song.SongId))
if err != nil {
logger.ErrorLog(fmt.Sprintf("Errror: couldn't create filepath. Details: %v", err))
conn.Close()
return
}
file, err := os.Open(filePath)
defer file.Close()
if err != nil {
logger.ErrorLog(fmt.Sprintf("Errror: couldn't open file. Details: %v", err))
conn.Close()
return
}
read(file, conn)
}
func read(file *os.File, conn net.Conn) {
reader := bufio.NewReader(file)
buf := make([]byte, 15)
defer conn.Close()
for {
_, err := reader.Read(buf)
if err != nil && err == io.EOF {
logger.InfoLog(fmt.Sprintf("Details: %v", err))
fmt.Println()
return
}
conn.Write(buf)
}
}
CLIENT
main.go
func main() {
conn, _ := net.Dial("tcp", "127.0.0.1:6060")
var glMessage []byte
text := "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjYzYzlhNmE1OWI3ZmQyNTQ2ZjA4ZWEyYSIsInVzZXJuYW1lIjoiMTIiLCJleHAiOjE2NzQyMTE5ODl9.aarSDhrFF1df3i2pIRyjNxTfSHKObqLU3kHJiPreredIhLNCzs7z7jMgRHQIcLaIvCOECN7bX0OaSvKdW7VKsQ\n"
fmt.Fprint(conn, text)
reader := bufio.NewReader(conn)
b := make([]byte, 15)
c := 0
for i, _ := reader.Read(b); int(i) != 0; i, _ = reader.Read(b) {
c += i
glMessage = append(glMessage, b...)
}
os.WriteFile("./test.flac", glMessage, 0644)
}
If you know what can be the problem, please tell me. I'd really appreciate it!
It looks like you're trying to send the music file over the network in 15 byte chunks, which is likely not enough to play the song on the client side.
You can try increasing the chunk size, for example, to 8192 bytes. To do this, replace buf := make([]byte, 15) with buf := make([]byte, 8192).
Also, it's better to write the received data directly to the file rather than storing it in memory. You can do this by creating a file and using os.Create to write the received data to it:
file, err := os.Create("./test.flac")
if err != nil {
fmt.Println("Error: couldn't create file")
return
}
defer file.Close()
for {
i, err := reader.Read(buf)
if err != nil && err == io.EOF {
break
}
file.Write(buf[:i])
}
I believe that this can solve the issue.
Implementing an ssh proxy in Go, errors out with bad packet length, these are the errors with ssh in debug mode:
debug1: SSH2_MSG_KEXINIT sent
Bad packet length 1231976033.
ssh_dispatch_run_fatal: Connection to ::1 port 8080: message authentication code incorrect
Code:
func handleSSH(conn net.Conn, r *bufio.Reader, protocol string) {
target, err := url.Parse("ssh://localhost:3333")
if err != nil {
fmt.Println("Error parsing target", err)
conn.Close()
return
}
targetConn, err := net.Dial("tcp", target.Host)
if err != nil {
fmt.Println("error dialing SSH target:", err)
conn.Close()
return
}
defer targetConn.Close()
var wg sync.WaitGroup
wg.Add(2)
go func() {
_, err := io.Copy(targetConn, conn)
if err != nil {
fmt.Println("error copying data to target:", err)
}
wg.Done()
}()
go func() {
_, err := io.Copy(conn, targetConn)
if err != nil {
fmt.Println("error copying data from target:", err)
}
wg.Done()
}()
wg.Wait()
conn.Close()
}
// EDIT
func connection(conn net.Conn) {
r := bufio.NewReader(conn)
protocol, err := r.ReadString('\n')
if err != nil {
fmt.Println("Error reading first line", err)
conn.Close()
return
}
if protocol[0:3] == "SSH" {
handleSSH(conn, r, protocol)
}
}
func main() {
ln, err := net.Listen("tcp", ":8080")
if err != nil {
panic(err)
}
defer ln.Close()
for {
conn, err := ln.Accept()
if err != nil {
panic(err)
}
go connection(conn)
}
}
EDIT: added code for relevant information on how the connection is initiated and reproduce the error.
My best guess is the ssh negotiation process is being interrupted, and things goes out of sync.
The code is reading the first line from the client and checks the kind of protocol in order to call the appropriate handler:
protocol, err := r.ReadString('\n')
...
if protocol[0:3] == "SSH" {
handleSSH(conn, r, protocol)
}
}
But the code fails to forward the already read bytes to the connected server. These bytes are in protocol and are given to handleSSH. But it fails to send these bytes to the connected server once the connection is established. Instead it only copies new data between client and server.
This means the server does not get the first line from the client. It therefore likely complains about a protocol error with something like Invalid SSH identification string. which gets forwarded to the client and misinterpreted as valid data from an SSH connection.
Can somebody help me understand what am I doing wrong here, all I'm trying to do is write a Ping message over a net.Conn instance (server), and reply back with a Pong which is expected on a net.Conn instance (client).
I have annotated the code with some errors that I receive.
reader.go
func read(conn net.Conn) {
for {
conn.SetReadDeadline(time.Now().Add(2 * time.Second))
_, op, err := wsutil.ReadClientData(conn)
if err != nil {
log.Printf("wsmanager read: %v", err) // read: write pipe: deadline exceeded
return
}
if op != ws.OpPing {
continue
}
c.conn.SetWriteDeadline(time.Now().Add(3 * time.Second))
if err = wsutil.WriteServerMessage(c.conn, ws.OpPong, []byte{}); err != nil {
log.Printf("wsmanager: send pong error: %v", err)
return
}
}
}
// reader_test.go
client, server := net.Pipe()
go read(server) // starts the loop above
err := wsutil.WriteClientMessage(server, ws.OpPing, []byte{})
if err != nil {
t.Fatalf("failed sending pings message %v", err)
}
_, op, err := wsutil.ReadServerData(client)
if err != nil {
t.Errorf("exp no err, got %v", err)
}
if op != ws.OpPong {
t.Errorf("exp ws.OpPong, got %v", op)
}
Thank you for using this library :)
As the doc states, the ReadData functions read data from the connection; that is, application specific data, not the control messages. Control frames are handled implicitly in these functions. If you want to read any kind of message, you could use wsutil.Reader or the plain ws.Read functions.
https://godoc.org/github.com/gobwas/ws/wsutil#ReadClientData
Is there any way how to serve files in Go with GRPC, like in gin-gonic's variant:
router.Static("/static", "/var/www")
You can't do it exactly like that.
But you can use the proto bytes type and put the file bytes in that field.
Also (as pointed out in the comments) with large files you should use streaming instead of a unary call. (most GRPC implementation have a limit of 4MB per message).
Proto example:
syntax = "proto3";
message Response {
bytes fileChunk = 1;
}
message Request {
string fileName = 1;
}
service TestService {
rpc Download(Request) returns (stream Response);
}
Server implementation example:
func (srv *Server) Download(req *pbgo.Request, responseStream pbgo.TestService_DownloadServer) error {
bufferSize := 64 *1024 //64KiB, tweak this as desired
file, err := os.Open(req.GetFileName())
if err != nil {
fmt.Println(err)
return err
}
defer file.Close()
buff := make([]byte, bufferSize)
for {
bytesRead, err := file.Read(buff)
if err != nil {
if err != io.EOF {
fmt.Println(err)
}
break
}
resp := &pbgo.Response{
FileChunk: buff[:bytesRead],
}
err = responseStream.Send(resp)
if err != nil {
log.Println("error while sending chunk:", err)
return err
}
}
return nil
}
Client would call it like this:
conn, err := grpc.Dial("localhost:9090", grpc.WithInsecure())
if err != nil {
log.Fatal("client could connect to grpc service:", err)
}
c := pbgo.NewTestServiceClient(conn)
fileStreamResponse, err := c.Download(context.TODO(), &pbgo.Request{
FileName: "test.txt",
})
if err != nil {
log.Println("error downloading:", err)
return
}
for {
chunkResponse, err := fileStreamResponse.Recv()
if err == io.EOF {
log.Println("received all chunks")
break
}
if err != nil {
log.Println("err receiving chunk:", err)
break
}
log.Printf("got new chunk with data: %s \n", chunkResponse.FileChunk)
}
If you need to be able to serve arbitrary files, you would need to handle which files you allow serving (say someone requests the file /etc/passwd or something).
Not sure what exactly is the use case here.
I'm taking a dive into the networking side of Go, and I'd thought I'd start with a TCP Client and Server.
I am able to get the client to connect to the server and send a simple message ("Hello") successfully. However, I can not get the server to send back a response (or the get the client to read the response).
Here is the code.
Server
Address := "localhost:9999"
Addr, err := net.ResolveTCPAddr("tcp", Address)
if err != nil {
log.Fatal(err)
}
listener, err := net.ListenTCP("tcp", Addr)
if err != nil {
log.Fatal(err)
}
defer listener.Close()
//server loop
for {
conn, err := listener.Accept()
if err != nil {
continue
}
go handle(conn)
}
func handle(c net.Conn) {
totalBytes, message := connRead(c)
fmt.Println(c.RemoteAddr())
fmt.Println(string(message[:totalBytes]))
c.Write([]byte("Hi"))
fmt.Println("Replied")
c.Close()
}
func connRead(c net.Conn) (int, []byte) {
buffer := make([]byte, 4096)
totalBytes := 0
for {
n, err := c.Read(buffer)
totalBytes += n
if err != nil {
if err != io.EOF {
log.Printf("Read error: %s", err)
}
break
}
}
return totalBytes, buffer
}
Client
tcpAddr, err := net.ResolveTCPAddr("tcp", "localhost:9999")
if err != nil {
log.Fatal(err)
}
conn, err := net.DialTCP("tcp", nil, tcpAddr)
if err != nil {
log.Fatal(err)
}
defer conn.Close()
_, err = conn.Write([]byte("Hello"))
if err != nil {
log.Fatal(err)
}
tBytes, resp := connRead(conn)
fmt.Println(tBytes)
fmt.Println(string(resp[:tBytes]))
func connRead(c net.Conn) (int, []byte) {
buffer := make([]byte, 4096)
totalBytes := 0
for {
fmt.Println("Stuck?")
n, err := c.Read(buffer)
fmt.Println("Stuck.")
totalBytes += n
fmt.Println(totalBytes)
if err != nil {
if err != io.EOF {
log.Printf("Read error: %s", err)
}
break
}
}
return totalBytes, buffer
}
From what I can tell it's not a problem with the server. When I run the client, everything stops right after fmt.Println("Stuck?"). This leads me to belive that it's messing up in the n, err := c.Read(buffer) statement somehow. The server doesn't even print out the messeage length (5) and message ("Hello") untill after I Ctrl-C the client. If I comment out the read and printings in the client, then things run smoothly.
I've tried googling for answers, but nothing has come up.
What am I doing wrong? Am I using conn.Read() wrong in the client?
EDIT:
I actually do have access to Linux, so here are the SIGQUIT dumps for the pertinent functions.
Server
http://pastebin.com/itevngCq
Client
http://pastebin.com/XLiKqkvs
for {
n, err := c.Read(buffer)
totalBytes += n
if err != nil {
if err != io.EOF {
log.Printf("Read error: %s", err)
}
break
}
}
It is because you are reading from connection till EOF error occurs
conn.Write([]byte("Hello"))
The above statement won't reach EOF at all until you actually closes the connection
On pressing ctrl+c client side the connection will be closed, So EOF occurs at server side, That is the reason why it is exiting server side for loop and printing these
127.0.0.1:****
Hello
Replied
If you want to make this work you should not read the connection till EOF
There are many other alternatives for this
Chose a delimiter and read at the server until the delimiter occurs and respond back after that. Check out this link
Send number of bytes to read from client side before sending the actual message, First read number of bytes to read from the server side and then read those many bytes from the connection