Using Cowboy Websocket Client for Testing with Elixir - websocket

Question

First of all, there is a genuine lack of documentation for Cowboy altogether and Websockets in particular but in general it is excellent to use once it is decyphered. Then getting that info from Erlang to Elixir is another step. Thanks to this post by 7stud I was able to get a functioning websocket going for testing purposes but I can't get it to listen and optionally send messages at the same time. I think this is because receive is blocking the thread which is needed to send and this is intrinsically linked to to the websocket connection so it can't send while it's waiting to receive. Maybe this understanding is flawed. I would love to be corrected. I have tried spawning to no avail which is why I am thinking the receive is blocking the websocket thread.
def ws do
localhost = 'localhost'
path = '/ws/app/1'
port = 5000
{:ok, _} = :application.ensure_all_started(:gun)
connect_opts = %{
connect_timeout: :timer.minutes(1),
retry: 10,
retry_timeout: 100
}
{:ok, conn_pid} = :gun.open(localhost, port, connect_opts)
IO.inspect(conn_pid, label: "conn_pid")
{:ok, protocol} = :gun.await_up(conn_pid)
IO.inspect(protocol, label: "protocol")
# Set custom header with cookie for device id
stream_ref = :gun.ws_upgrade(conn_pid, path, [{"cookie", "device_id=1235"}])
IO.inspect(stream_ref, label: "stream_ref")
receive do
{:gun_upgrade, ^conn_pid, ^stream_ref, ["websocket"], headers} ->
upgrade_success(conn_pid, headers, stream_ref)
{:gun_response, ^conn_pid, _, _, status, headers} ->
exit({:ws_upgrade_failed, status, headers})
{:gun_error, _conn_pid, _stream_ref, reason} ->
exit({:ws_upgrade_failed, reason})
whatever ->
IO.inspect(whatever, label: "Whatever")
# More clauses here as needed.
after 5000 ->
IO.puts "Took too long!"
:erlang.exit("barf!")
end
:ok
end
def upgrade_success(conn_pid, headers, stream_ref) do
IO.puts("Upgraded #{inspect(conn_pid)}. Success!\nHeaders:\n#{inspect(headers)}\n")
IO.inspect(self(), label: "upgrade self")
# This one runs and message is received
run_test(conn_pid)
# This should spawn and therefore not block
listen(conn_pid, stream_ref)
# This never runs
run_test(conn_pid)
end
def listen(conn_pid, stream_ref) do
spawn receive_messages(conn_pid, stream_ref)
end
def receive_messages(conn_pid, stream_ref) do
IO.inspect conn_pid, label: "conn_pid!"
IO.inspect stream_ref, label: "stream_ref!"
IO.inspect(self(), label: "self pid")
receive do
{:gun_ws, ^conn_pid, ^stream_ref, {:text, msg} } ->
IO.inspect(msg, label: "Message from websocket server:")
other_messages ->
IO.inspect(other_messages, label: "Other messages")
after 5000 ->
IO.puts "Receive timed out"
end
receive_messages(conn_pid, stream_ref)
end
def send_message(message, conn_pid) do
:gun.ws_send(conn_pid, {:text, message})
end
def run_test(conn_pid) do
IO.puts "Running test"
message = "{\"type\":\"init\",\"body\":{\"device_id\":1234}}"
send_message(message, conn_pid)
end
def stop(conn_pid) do
:gun.shutdown(conn_pid)
end

Answer 1

From the gun docs:
Receiving data
Gun sends an Erlang message to the owner process for every Websocket
message it receives.
and:
Connection
...
Gun connections
...
A Gun connection is an Erlang process that manages a socket to a
remote endpoint. This Gun connection is owned by a user process that
is called the owner of the connection, and is managed by the
supervision tree of the gun application.
The owner process communicates with the Gun connection by calling
functions from the module gun. All functions perform their respective
operations asynchronously. The Gun connection will send Erlang
messages to the owner process whenever needed.
Although it's not specifically mentioned in the docs, I'm pretty sure the owner process is the process that calls gun:open(). My attempts also reveal that the owner process has to call gun:ws_send(). In other words, the owner process has to both send messages to the server and receive the messages from the server.
The following code operates gun with a gen_server in such a way that the gen_server both sends messages to the server and receives messages from the server.
When gun receives a message from the cowboy http server, gun sends the message, i.e. Pid ! Msg, to the owner process. In the following code, the gen_server creates the connection in the init/1 callback, which means that gun will bang (!) messages that it receives from cowboy at the gen_server. A gen_server handles messages sent directly to its mailbox with handle_info().
In handle_cast(), the gen_server uses gun to send requests to cowboy. Because handle_cast() is asynchronous, that means you are able to send asynchronous messages to cowboy. And, when gun receives a message from cowboy, gun sends(!) the message to the gen_server, and the gen_server's handle_info() function handles the message. Inside handle_info(), gen_server:reply/2 is called to relay the message to the gen_server client. As a result, the gen_server client can jump into a receive clause whenever it wants to check the server messages sent from gun.
-module(client).
-behavior(gen_server).
-export([start_server/0, send_sync/1, send_async/1, get_message/2, go/0]).
-export([init/1, handle_call/3, handle_cast/2, handle_info/2]).
-export([terminate/2, code_change/3]). %%% client functions
-export([sender/1]).
%%% client functions
%%%
start_server() ->
gen_server:start({local, ?MODULE}, ?MODULE, [], []).
send_sync(Requ) ->
gen_server:call(?MODULE, Requ).
send_async(Requ) ->
gen_server:cast(?MODULE, {websocket_request, Requ}).
get_message(WebSocketPid, ClientRef) ->
receive
{ClientRef, {gun_ws, WebSocketPid, {text, Msg} }} ->
io:format("Inside get_message(): Ref = ~w~n", [ClientRef]),
io:format("Client received gun message: ~s~n", [Msg]);
Other ->
io:format("Client received other message: ~w~n", [Other])
end.
receive_loop(WebSocketPid, ClientRef) ->
receive
{ClientRef, {gun_ws, WebSocketPid, {text, Msg} }} ->
io:format("Client received Gun message: ~s~n", [Msg]);
Other ->
io:format("Client received other message: ~w~n", [Other])
end,
receive_loop(WebSocketPid, ClientRef).
go() ->
{ok, GenServerPid} = start_server(),
io:format("[ME]: Inside go(): GenServerPid=~w~n", [GenServerPid]),
[{conn_pid, ConnPid}, {ref, ClientRef}] = send_sync(get_conn_pid),
io:format("[ME]: Inside go(): ConnPid=~w~n", [ConnPid]),
ok = send_async("ABCD"),
get_message(ConnPid, ClientRef),
spawn(?MODULE, sender, [1]),
ok = send_async("XYZ"),
get_message(ConnPid, ClientRef),
receive_loop(ConnPid, ClientRef).
sender(Count) -> %Send messages to handle_info() every 3 secs
send_async(lists:concat(["Hello", Count])),
timer:sleep(3000),
sender(Count+1).
%%%%%% gen_server callbacks
%%%
init(_Arg) ->
{ok, {no_client, ws()}}.
handle_call(get_conn_pid, From={_ClientPid, ClientRef}, _State={_Client, WebSocketPid}) ->
io:format("[ME]: Inside handle_call(): From = ~w~n", [From]),
{reply, [{conn_pid, WebSocketPid}, {ref, ClientRef}], _NewState={From, WebSocketPid} };
handle_call(stop, _From, State) ->
{stop, normal, shutdown_ok, State}; %Calls terminate()
handle_call(_Other, _From, State) ->
{ok, State}.
handle_cast({websocket_request, Msg}, State={_From, WebSocketPid}) ->
gun:ws_send(WebSocketPid, {text, Msg}), %{text, "It's raining!"}),
{noreply, State}.
handle_info(Msg, State={From, _WebSocketPid}) ->
io:format("[ME]: Inside handle_info(): Msg=~w~n", [Msg]),
gen_server:reply(From, Msg),
{noreply, State}.
terminate(_Reason, _State={_From, WebSocketPid}) ->
gun:shutdown(WebSocketPid).
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%%% private functions
%%%
ws() ->
{ok, _} = application:ensure_all_started(gun),
{ok, ConnPid} = gun:open("localhost", 8080),
{ok, _Protocol} = gun:await_up(ConnPid),
gun:ws_upgrade(ConnPid, "/please_upgrade_to_websocket"),
receive
{gun_ws_upgrade, ConnPid, ok, Headers} ->
io:format("[ME]: Inside gun_ws_upgrade receive clause: ~w~n",
[ConnPid]),
upgrade_success_handler(ConnPid, Headers);
{gun_response, ConnPid, _, _, Status, Headers} ->
exit({ws_upgrade_failed, Status, Headers});
{gun_error, _ConnPid, _StreamRef, Reason} ->
exit({ws_upgrade_failed, Reason})
after 1000 ->
exit(timeout)
end.
upgrade_success_handler(ConnPid, _Headers) ->
io:format("[ME]: Inside upgrade_success_handler(): ~w~n", [ConnPid]),
ConnPid.
=======
Whoops, the answer below shows how to get the server to push data to the client.
Okay, I got it--in erlang. This example is a little bit tortured. You need to do a couple of things:
1) You need to get the pid of the process running the websocket_* functions, which is not the same as the pid of the request:
Post-upgrade initialization
Cowboy has separate processes for handling the connection and
requests. Because Websocket takes over the connection, the Websocket
protocol handling occurs in a different process than the request
handling.
This is reflected in the different callbacks Websocket handlers have.
The init/2 callback is called from the temporary request process and
the websocket_ callbacks from the connection process.
This means that some initialization cannot be done from init/2.
Anything that would require the current pid, or be tied to the current
pid, will not work as intended. The optional websocket_init/1 can be
used [to get the pid of the process running the websocket_ callbacks]:
https://ninenines.eu/docs/en/cowboy/2.6/guide/ws_handlers/
Here's the code I used:
init(Req, State) ->
{cowboy_websocket, Req, State}. %Perform websocket setup
websocket_init(State) ->
io:format("[ME]: Inside websocket_init"),
spawn(?MODULE, push, [self(), "Hi, there"]),
{ok, State}.
push(WebSocketHandleProcess, Greeting) ->
timer:sleep(4000),
WebSocketHandleProcess ! {text, Greeting}.
websocket_handle({text, Msg}, State) ->
timer:sleep(10000), %Don't respond to client request just yet.
{
reply,
{text, io_lib:format("Server received: ~s", [Msg]) },
State
};
websocket_handle(_Other, State) -> %Ignore
{ok, State}.
That will push a message to the client while the client is waiting for a reply to a request that the client previously sent to the server.
2) If you send a message to the process that is running the websocket_* functions:
Pid ! {text, Msg}
then that message will get handled by the websocket_info() function--not the websocket_handle() function:
websocket_info({text, Text}, State) ->
{reply, {text, Text}, State};
websocket_info(_Other, State) ->
{ok, State}.
The return value of the websocket_info() function works just like the return value of the websocket_handle() function.
Because your gun client is now receiving multiple messages, the gun client needs to receive in a loop:
upgrade_success_handler(ConnPid, Headers) ->
io:format("Upgraded ~w. Success!~nHeaders:~n~p~n",
[ConnPid, Headers]),
gun:ws_send(ConnPid, {text, "It's raining!"}),
get_messages(ConnPid). %Move the receive clause into a recursive function
get_messages(ConnPid) ->
receive
{gun_ws, ConnPid, {text, "Greeting: " ++ Greeting} } ->
io:format("~s~n", [Greeting]),
get_messages(ConnPid);
{gun_ws, ConnPid, {text, Msg} } ->
io:format("~s~n", [Msg]),
get_messages(ConnPid)
end.

Answer 2

Thanks to 7stud for the example code and the edits which are reflected below:
Here is my Elixir interpretation to give a basic WebSocket client for gun:
defmodule WebsocketTester.Application do
use Application
def start(_type, _args) do
path = '/ws/app/1'
port = 5000
host = 'localhost'
args = %{path: path, port: port, host: host}
children = [
{ WebSocket.Client, args }
]
Supervisor.start_link(children, strategy: :one_for_one, name: WebsocketTester.Supervisor)
end
end
defmodule WebSocket.Client do
use GenServer
def child_spec(opts) do
%{
id: __MODULE__,
start: {__MODULE__, :start_link, [opts]},
type: :worker,
restart: :permanent,
shutdown: 500
}
end
def start_link(args) do
GenServer.start_link(__MODULE__, args, name: __MODULE__)
end
# GenServer callbacks
def init(args) do
# Set up the websocket connection
# get > upgrade
# Initial state with gun_pid and stream_ref
# %{gun_pid: gun_pid, stream_ref: stream_ref} = ws(args)
{:ok, init_ws(args)}
end
# Give back gun_pid from state
def handle_call(:get_conn, from, %{gun_pid: gun_pid, stream_ref: stream_ref}) do
IO.inspect(gun_pid, label: "handle call gun pid")
{:reply, %{gun_pid: gun_pid, stream_ref: stream_ref}, %{from: from, gun_pid: gun_pid} }
end
# Everything else
def handle_call(other, from, state) do
IO.inspect(other, label: "other call")
IO.inspect(from, label: "from")
{:ok, state}
end
# Client sends message to server.
def handle_cast({:websocket_request, message}, %{gun_pid: gun_pid} = state) do
IO.puts message
IO.inspect(gun_pid, label: "gun_pid")
:gun.ws_send(gun_pid, {:text, message})
{:noreply, state}
end
def handle_info(message, %{from: from} = state) do
IO.inspect(message, label: "Inside handle_info(): ")
GenServer.reply(from, message)
{:noreply, state}
end
def terminate(reason, _state) do
IO.puts "Terminated due to #{reason}."
:ok
end
def code_change(_old_version, state, _extra) do
{:ok, state}
end
## Client functions
# Used for getting gun_pid from state
def send_sync(request) do
GenServer.call(__MODULE__, request)
end
# Send a message async
def send_async(request) do
GenServer.cast(__MODULE__, {:websocket_request, request})
end
# Receive a single message
def get_message(stream_ref, gun_pid) do
receive do
{^stream_ref, {:gun_ws, ^gun_pid, {:text, message} }} ->
IO.puts("Client received gun message: #{message}")
other ->
IO.inspect(other, label: "Client received other message")
end
end
# Receive all messages recursively
def receive_loop(stream_ref, gun_pid) do
IO.puts "Listening"
get_message(stream_ref, gun_pid)
receive_loop(stream_ref, gun_pid)
end
def go() do
# Get the gun_pid from state
%{gun_pid: gun_pid, stream_ref: stream_ref} = send_sync(:get_gun_pid)
IO.inspect(gun_pid, label: "Inside go(): gun_pid=")
# Send messages manually
:ok = send_async(Jason.encode!(%{type: "info", greet: "yo"}))
# Or to send just text
# :ok = send_async("yo")
# Receive messages manually
get_message(stream_ref, gun_pid)
# Start sending loop
spawn sender 1
# Start listening
receive_loop(stream_ref, gun_pid)
end
# Send messages to handle_info() every 3 secs
def sender(count) do
send_async("count is #{count}")
:timer.sleep(3000)
sender(count+1)
end
## End of client functions
# Initialize the websocket connection
def init_ws(args) do
%{ path: path, port: port, host: host} = args
{:ok, _} = :application.ensure_all_started(:gun)
connect_opts = %{
connect_timeout: :timer.minutes(1),
retry: 10,
retry_timeout: 100
}
{:ok, gun_pid} = :gun.open(host, port, connect_opts)
{:ok, _protocol} = :gun.await_up(gun_pid)
# Set custom header with cookie for device id - set_headers can be left out if you don't want custom headers
stream_ref = :gun.ws_upgrade(gun_pid, path, set_headers("I like cookies"))
receive do
{:gun_upgrade, ^gun_pid, ^stream_ref, ["websocket"], headers} ->
upgrade_success(gun_pid, headers, stream_ref)
{:gun_response, ^gun_pid, _, _, status, headers} ->
exit({:ws_upgrade_failed, status, headers})
{:gun_error, _gun_pid, _stream_ref, reason} ->
exit({:ws_upgrade_failed, reason})
whatever ->
IO.inspect(whatever, label: "Whatever")
# More clauses here as needed.
after 5000 ->
IO.puts "Took too long!"
:erlang.exit("barf!")
end
# stop(gun_pid)
end
def set_headers(cookie_value) do
[{"cookie", "my_cookie=#{cookie_value}"}]
end
# This just returns the gun_pid for further reference which gets stored in the GenServer state.
def upgrade_success(gun_pid, headers, stream_ref) do
IO.puts("Upgraded #{inspect(gun_pid)}. Success!\nHeaders:\n#{inspect(headers)}\n")
%{stream_ref: stream_ref, gun_pid: gun_pid}
end
# To stop gun
def stop(gun_pid) do
:gun.shutdown(gun_pid)
end
end
To use this:
iex -S mix
iex> WebSocket.Client.go