when you use netty (4.0.23, java 1.7u67, win8x64) as a client ChannelInboundHandlerAdapter.channelReadComplete(...) should be called 1 time once netty completes reading response, right?
Trying different sites, it's always called twice:
#Test
public void testDoubleReadComplete() throws Exception {
final String host = "www.google.de";
final CountDownLatch count = new CountDownLatch(20);
bootstrap = new Bootstrap()
.group(new NioEventLoopGroup())
.channel(NioSocketChannel.class)
.handler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
String request = String.format(
"GET / HTTP/1.1\n" +
"Host: " + host +"\n" +
"\n\n"
);
System.out.println("sending...");
System.out.println(request);
ByteBuf req = Unpooled.wrappedBuffer(request.getBytes(Charset.defaultCharset()));
ctx.writeAndFlush(req);
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
System.err.println("777 read complete");
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
ByteBuf resp = (ByteBuf) msg;
count.countDown();
System.out.printf("****************************************************>>>>> %s%n", Thread.currentThread().getName());
System.out.println(resp.toString(Charset.defaultCharset()));
System.out.println("<<<<<****************************************************");
resp.release();
}
});
}
});
ChannelFuture future = bootstrap.connect(new InetSocketAddress(host, 80));
future.awaitUninterruptibly();
count.await(5, TimeUnit.SECONDS);
}
outputs 777 read complete 2 times, why?
I'm not sure that the channelReadComplete is intended for how you are trying to use it.
Read the javadoc for the method. The data is not guaranteed to arrive all at once and so netty reads the data as it arrives in a non-blocking manner. This methods notifies you when it has finished the current read operation which is not necessarily the last read operation.
I'm not exactly sure about your use case but here is some non-production code that may be closer to what you are trying to accomplish?
#Test
public void testDoubleReadComplete() throws Exception {
final String host = "www.google.de";
final CountDownLatch count = new CountDownLatch(20);
Bootstrap bootstrap = new Bootstrap()
.group(new NioEventLoopGroup())
.channel(NioSocketChannel.class)
.handler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
String request = String.format(
"GET / HTTP/1.1\r\n" +
"Host: " + host +"\r\n" +
"Connection: close\r\n" +
"\r\n"
);
System.out.println("sending...");
System.out.println(request);
ByteBuf req = Unpooled.wrappedBuffer(request.getBytes(Charset.defaultCharset()));
ctx.writeAndFlush(req);
}
#Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
System.err.println("777 read complete");
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
ByteBuf resp = (ByteBuf) msg;
count.countDown();
System.out.printf("****************************************************>>>>> %s%n", Thread.currentThread().getName());
System.out.println(resp.toString(Charset.defaultCharset()));
System.out.println("<<<<<****************************************************");
resp.release();
}
});
}
});
found the cause and solution: ChannelOption.MAX_MESSAGES_PER_READ
So now I'm testing with .option(ChannelOption.MAX_MESSAGES_PER_READ, Integer.MAX_VALUE) and so far so good - channelReadComplete is called once at the end of response as I need
Related
Following recommendations elsewhere I am attempting to parallelize my final inbound handler in a Netty pipeline as such
public final class EchoServer {
private EventLoopGroup group = new NioEventLoopGroup();
private UnorderedThreadPoolEventExecutor workers = new UnorderedThreadPoolEventExecutor(10);
public void start(int port) throws InterruptedException {
try {
Bootstrap b = new Bootstrap();
b.group(group).channel(NioDatagramChannel.class).option(ChannelOption.SO_BROADCAST, true)
.handler(new ChannelInitializer<NioDatagramChannel>() {
#Override
protected void initChannel(NioDatagramChannel channel) throws Exception {
channel.pipeline().addLast(workers, new SimpleChannelInboundHandler<DatagramPacket>() {
#Override
public void channelRead0(ChannelHandlerContext ctx, DatagramPacket packet) throws Exception {
System.err.println(packet);
// Simulated database delay that I have to wait to occur before repsonding
Thread.sleep(1000);
ctx.write(new DatagramPacket(Unpooled.copiedBuffer("goodbye", StandardCharsets.ISO_8859_1), packet.sender()));
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx) {
ctx.flush();
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
cause.printStackTrace();
}
});
}
});
b.bind(port).sync().channel().closeFuture().await();
} finally {
group.shutdownGracefully();
}
}
public void stop() {
group.shutdownGracefully();
}
}
I have ten clients that connect concurrently, as a test, and I am measuring execution time for handling all the requests. As expected with the 1 second delay and sequential execution it takes just over 10 seconds. I am trying to get execution down to somewhere sub 2 seconds to prove parallel handling.
From what I understand adding the handler to the pipeline with an explicitly assigned executor is supposed to parallelize that handlers work across the thread in the executor.
Instead of seeing a increase in performance, what I am finding is that my client is not receiving the responses when I add the parallel processing. The thread sleep is there to simulate the potential time it will take to write the incoming data to a database. Am I doing something obviously wrong here?
I worked around the apparently lack of Netty support for doing final end UDP processing in parallel using standard java concurrency mechanisms.
public final class EchoServer {
private EventLoopGroup group = new NioEventLoopGroup();
private ExecutorService executors = Executors.newFixedThreadPool(10);
public void start(int port) throws InterruptedException {
try {
Bootstrap b = new Bootstrap();
b.group(group).channel(NioDatagramChannel.class).handler(new ChannelInitializer<NioDatagramChannel>() {
#Override
protected void initChannel(NioDatagramChannel channel) throws Exception {
channel.pipeline().addLast(new SimpleChannelInboundHandler<DatagramPacket>() {
#Override
public void channelRead0(ChannelHandlerContext ctx, DatagramPacket packet) throws Exception {
CompletableFuture.runAsync(() -> {
System.err.println(packet);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
ctx.writeAndFlush(new DatagramPacket(Unpooled.copiedBuffer("goodbye", StandardCharsets.ISO_8859_1),
packet.sender()));
}, executors);
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx) {
ctx.flush();
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
cause.printStackTrace();
}
});
}
});
b.bind(port).sync().channel().closeFuture().await();
} finally {
group.shutdownGracefully();
}
}
public void stop() {
group.shutdownGracefully();
}
}
I am attempting to change the Content-Type header in a request and change it to "application/json" before it reaches my spring rest controller. I have created a servlet request wrapper to change the values, but when the request reaches the controller it is still "text/plain". The logging shows that the header value has been changed before hitting doFilter();
Here is my class extending HttpServletRequestWrapper
class HttpServletRequestWritableWrapper extends HttpServletRequestWrapper {
private final Logger logger = org.slf4j.LoggerFactory.getLogger(HttpServletRequestWritableWrapper.class);
private final ByteArrayInputStream decryptedBody;
HttpServletRequestWritableWrapper(HttpServletRequest request, byte[] decryptedData) {
super(request);
decryptedBody = new ByteArrayInputStream(decryptedData);
}
#Override
public String getHeader(String name) {
String headerValue = super.getHeader(name);
if("Accept".equalsIgnoreCase(name))
{
logger.debug("Accept header changing :");
return headerValue.replaceAll(
MediaType.TEXT_PLAIN_VALUE, MediaType.APPLICATION_JSON_VALUE
);
}
else if ("Content-Type".equalsIgnoreCase(name))
{
logger.debug("Content type change: ");
return headerValue.replaceAll(MediaType.TEXT_PLAIN_VALUE, MediaType.APPLICATION_JSON_VALUE);
}
return headerValue;
}
#Override
public Enumeration<String> getHeaderNames() {
return super.getHeaderNames();
}
#Override
public String getContentType() {
String contentTypeValue = super.getContentType();
if (MediaType.TEXT_PLAIN_VALUE.equalsIgnoreCase(contentTypeValue)) {
logger.debug("Changing on getContentType():");
return MediaType.APPLICATION_JSON_VALUE;
}
return contentTypeValue;
}
#Override
public BufferedReader getReader() throws UnsupportedEncodingException {
return new BufferedReader(new InputStreamReader(decryptedBody, UTF_8));
}
#Override
public ServletInputStream getInputStream() throws IOException {
return new ServletInputStream() {
#Override
public int read() {
return decryptedBody.read();
}
};
}
And here is my filter:
#WebFilter(displayName = "EncryptionFilter", urlPatterns = "/*")
public class EncryptionFilter implements Filter {
private final Logger logger = org.slf4j.LoggerFactory.getLogger(EncryptionFilter.class);
#Override
public void init(FilterConfig filterConfig) throws ServletException {
}
#Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
byte[] data = "{\"currentClientVersion\":{\"majorElement\":\"1\",\"minorElement\":\"2\"}}".getBytes();
logger.debug("data string " + data.toString());
logger.debug("Content-type before: " + servletRequest.getContentType());
HttpServletRequest request = (HttpServletRequest) servletRequest;
HttpServletRequestWritableWrapper requestWrapper = new HttpServletRequestWritableWrapper(request, data);
//logger.debug("Accept Header: " + requestWrapper.getHeader("Accept"));
//logger.debug("Content-Type: " + requestWrapper.getHeader("Content-Type"));
//logger.debug("Contenttype" + requestWrapper.getContentType());
filterChain.doFilter(requestWrapper, servletResponse);
}
#Override
public void destroy() {
}
}
It appears that the getHeaders method was being called somewhere else after my filter and not returning the headers with my updated values.
I added this override in my HttpServletRequestWrapper and it is now working:
#Override
public Enumeration<String> getHeaders(String name) {
List<String> headerVals = Collections.list(super.getHeaders(name));
int index = 0;
for (String value : headerVals) {
if ("Content-Type".equalsIgnoreCase(name)) {
logger.debug("Content type change: ");
headerVals.set(index, MediaType.APPLICATION_JSON_VALUE);
}
index++;
}
return Collections.enumeration(headerVals);
}
I want to write Junit UT case for my websocket serverendpoint code using embedded Jetty.
i tried things explained in below link:
JUnit test with javax.websocket on embedded Jetty throws RejectedExecutionException: NonBlockingThread
I want to test my onMessage callback for websocket.
If i dont use server.join() method then the connection closes as soon as it opens.
If i use server.join() method nothing happens after joining.
Below is My code.
Server startup code::
public class EmbeddedJettyServer {
private final int port;
private Server server;
public EmbeddedJettyServer(int port) {
this.port = port;
}
public void start() throws Exception {
server = new Server();
ServerConnector connector = new ServerConnector(server);
connector.setPort(8080);
server.addConnector(connector);
// Setup the basic application "context" for this application at "/"
// This is also known as the handler tree (in jetty speak)
ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
context.setContextPath("/");
server.setHandler(context);
try {
// Initialize javax.websocket layer
ServerContainer wscontainer = WebSocketServerContainerInitializer.configureContext(context);
// Add WebSocket endpoint to javax.websocket layer
wscontainer.addEndpoint(WebSocketServer.class);
System.out.println("Begin start");
server.start();
server.dump(System.err);
server.join();
} catch (Throwable t) {
t.printStackTrace(System.err);
}
}
public void stop() throws Exception {
server.stop();
LOGGER.info("Jetty server stopped");
}
public URI getWebsocketUri(Class<WebSocketServer> class1) {
return server.getURI();
}
}
Client Code:
#ClientEndpoint()
public class WebSocketClientJetty {
WebSocketContainer container;
public Session connect(URI uri) throws Exception {
WebSocketContainer container = ContainerProvider.getWebSocketContainer();
try {
// Attempt Connect
Session session = container.connectToServer(WebSocketClientJetty.class,uri);
// return container.connectToServer(WebSocketClientJetty.class, uri);
session.getBasicRemote().sendText("Hello");
// Close session
// session.close();
return session;
} finally {
}
}
public void stop() throws Exception{
if (container instanceof LifeCycle) {
((LifeCycle) container).stop();
}
}
#OnOpen
public void onWebSocketConnect(Session sess)
{
System.out.println("Socket Connected: " + sess);
}
#OnMessage
public void onWebSocketText(String message)
{
System.out.println("Received TEXT message: " + message);
}
#OnClose
public void onWebSocketClose(CloseReason reason)
{
System.out.println("Socket Closed: " + reason);
}
#OnError
public void onWebSocketError(Throwable cause)
{
cause.printStackTrace(System.err);
}
}
Serverendpoint code:
#ServerEndpoint(value = "/echo",
encoders={JsonEncoder.class})
public class WebSocketServer {
private static final Logger LOGGER =
#OnOpen
public void onOpen(Session session){
System.out.println("onopen");
some code....
}
#OnMessage
public void onMessage(String message, Session session) throws IOException{
System.out.println("onmessage");
....
}
#OnClose
public void onClose(Session session){
System.out.println("onClose");
...
}
}
Ut case:
public class WebSocketJettyTest {
private static EmbeddedJettyServer server;
#ClassRule
public static final ExternalResource integrationServer = new ExternalResource() {
#Override
protected void before() throws Throwable {
System.out.println("Starting...");
server = new EmbeddedJettyServer(8080);
server.start();
System.out.println("Started");
}
};
#Before
public void setUp() throws Exception {
}
#After
public void shutdown() throws Exception {
server.stop();
}
#Test
public void testSocket() throws Exception {
/*URI uri = server.getWebsocketUri(WebSocketServer.class);*/
URI uri = URI.create("ws://localhost:8080/echo");
WebSocketClientJetty client = new WebSocketClientJetty();
Session session = client.connect(uri);
session.getBasicRemote().sendText("hello");
Thread.sleep(6000);
client.stop();
}
}
Drop the call to
server.join();
That just makes the current thread wait until the server thread stops.
Which is making it difficult for you.
I using httpcomponenets nio server to handle post request file upload.
Below is the sample code. I have the complete data in data byte array including params, uploaded file etc. separated by boundary. Is there a parser utility to parse data and get the parameters? Something like request.getParameter("param1"), request.getFile() etc.
public static void main(String[] args) throws Exception {
int port = 8280;
// Create HTTP protocol processing chain
HttpProcessor httpproc = HttpProcessorBuilder.create()
.add(new ResponseDate())
.add(new ResponseServer("Test/1.1"))
.add(new ResponseContent())
.add(new ResponseConnControl()).build();
// Create request handler registry
UriHttpAsyncRequestHandlerMapper reqistry = new UriHttpAsyncRequestHandlerMapper();
// Register the default handler for all URIs
reqistry.register("/test*", new RequestHandler());
// Create server-side HTTP protocol handler
HttpAsyncService protocolHandler = new HttpAsyncService(httpproc, reqistry) {
#Override
public void connected(final NHttpServerConnection conn) {
System.out.println(conn + ": connection open");
super.connected(conn);
}
#Override
public void closed(final NHttpServerConnection conn) {
System.out.println(conn + ": connection closed");
super.closed(conn);
}
};
// Create HTTP connection factory
NHttpConnectionFactory<DefaultNHttpServerConnection> connFactory;
connFactory = new DefaultNHttpServerConnectionFactory(
ConnectionConfig.DEFAULT);
// Create server-side I/O event dispatch
IOEventDispatch ioEventDispatch = new DefaultHttpServerIODispatch(protocolHandler, connFactory);
// Set I/O reactor defaults
IOReactorConfig config = IOReactorConfig.custom()
.setIoThreadCount(1)
.setSoTimeout(3000)
.setConnectTimeout(3000)
.build();
// Create server-side I/O reactor
ListeningIOReactor ioReactor = new DefaultListeningIOReactor(config);
try {
// Listen of the given port
ioReactor.listen(new InetSocketAddress(port));
// Ready to go!
ioReactor.execute(ioEventDispatch);
} catch (InterruptedIOException ex) {
System.err.println("Interrupted");
} catch (IOException e) {
System.err.println("I/O error: " + e.getMessage());
}
System.out.println("Shutdown");
}
public static class RequestHandler implements HttpAsyncRequestHandler<HttpRequest> {
public void handleInternal(HttpRequest httpRequest, HttpResponse httpResponse, HttpContext httpContext) throws HttpException, IOException {
HttpEntity entity = null;
if (httpRequest instanceof HttpEntityEnclosingRequest)
entity = ((HttpEntityEnclosingRequest)httpRequest).getEntity();
byte[] data;
if (entity == null) {
data = new byte [0];
} else {
data = EntityUtils.toByteArray(entity);
}
System.out.println(new String(data));
httpResponse.setEntity(new StringEntity("success response"));
}
#Override public HttpAsyncRequestConsumer<HttpRequest> processRequest(HttpRequest request, HttpContext context) throws HttpException, IOException {
return new BasicAsyncRequestConsumer();
}
#Override
public void handle(HttpRequest request, HttpAsyncExchange httpExchange, HttpContext context) throws HttpException, IOException {
HttpResponse response = httpExchange.getResponse();
handleInternal(request, response, context);
httpExchange.submitResponse(new BasicAsyncResponseProducer(response));
}
}
MIME content parsing (as well handling of content of any type) is out of scope for Apache HttpComponents. Please consider using Apache Mime4J.
I have a Sample WebSocket Program whown below which works fine
When ever the user closes the browser or if there is any excetion Or any disconnect , the onClose Method is
being called
My question is that , Is it possible to know from the program what is the reason for onClose being called ??
Please share your views , Thanks for reading .
public class Html5Servlet extends WebSocketServlet {
private AtomicInteger index = new AtomicInteger();
private static final List<String> tickers = new ArrayList<String>();
static{
tickers.add("ajeesh");
tickers.add("peeyu");
tickers.add("kidillan");
tickers.add("entammo");
}
/**
*
*/
private static final long serialVersionUID = 1L;
public WebSocket doWebSocketConnect(HttpServletRequest req, String resp) {
//System.out.println("doWebSocketConnect");
return new StockTickerSocket();
}
protected String getMyJsonTicker() throws Exception{
return "";
}
public class StockTickerSocket implements WebSocket.OnTextMessage{
private Connection connection;
private Timer timer;
#Override
public void onClose(int arg0, String arg1) {
System.out.println("onClose called!"+arg0);
}
#Override
public void onOpen(Connection connection) {
//System.out.println("onOpen");
this.connection=connection;
this.timer=new Timer();
}
#Override
public void onMessage(String data) {
//System.out.println("onMessage");
if(data.indexOf("disconnect")>=0){
connection.close();
timer.cancel();
}else{
sendMessage();
}
}
public void disconnect() {
System.out.println("disconnect called");
}
public void onDisconnect()
{
System.out.println("onDisconnect called");
}
private void sendMessage() {
if(connection==null||!connection.isOpen()){
//System.out.println("Connection is closed!!");
return;
}
timer.schedule(new TimerTask() {
#Override
public void run() {
try{
//System.out.println("Running task");
connection.sendMessage(getMyJsonTicker());
}
catch (IOException e) {
e.printStackTrace();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}, new Date(),5000);
}
}
}
The signature for onClose is the following ...
#Override
public void onClose(int closeCode, String closeReason) {
System.out.println("onClose called - statusCode = " + closeCode);
System.out.println(" reason = " + closeReason);
}
Where int closeCode is any of the registered Close Status Codes.
And String closeReason is an optional (per protocol spec) close reason message.