Ron Cecchini
2018-09-12 22:50:08 UTC
Hi, there. I'm experiencing something weird here and I'm desperate for an answer.
I realize this is a long message, but it's only because I'm verbose.
It's a really straight forward setup, so I'm hoping it's just an obvious configuration issue...
I have a simple Server and Client setup for serving and fetching a list of automobiles.
The Client is an "adapter" that translates the returned JSON list to a Protobuf,
and then writes it to a RabbitMQ bus (port 5672).
----------
I implemented 2 Client-Server approaches -- a simple Pull & Push:
1. Pull: The Client uses a Camel timer to do a periodic GET (using Jetty),
translates the JSON to a Protobuf, writes it to the bus.
2. Push: The Server uses a Camel timer to do a periodic GET (from itself, using Rest),
writes the data to a UDP port (using Netty4 or Mina2).
The write is one-way; i.e. no response is expected, so I set 'sync=false'.
The Client reads from that UDP port (using Netty4 or Mina2),
translates the JSON to a Protobuf, writes it to the bus.
Here are the 2 problems I am experiencing:
----------
1. The first problem I'm experiencing is that I simply can not get Netty4 to work.
Whenever I use Netty4 in the Client / Server routes below, no data is written at all.
I don't get an error. I simply get nothing. I turned up logging to DEBUG and don't see anything.
I have a post-write log statement in the route ("... AFTER write to UDP ..."), which never prints.
The Client, of course, doesn't budge as it's not seeing anything.
However, when I use Mina2, I *can* write and read. Data flows, and it all looks good.
So what's up with that?
-----
(let me sneak in another question here)
I determined that the max size of the data I can send with Mina2 is 2048 bytes.
I'm on a Linux/CentOS machine. I looked around, and I think my system max is >> 2048.
So, is Mina setting something under the hood? And if so, how would I change it?
For Netty, I see that there is a 'sendBufferSize' and 'receiveBufferSize' which default to 65,536 bytes.
But I don't see any similar send/receive sizes for Mina.
(Of course, if I can figure out how to use Netty4, I won't care about Mina...)
----------
2. I can't write to RabbitMQ if I use UDP.
Ok, this one is *really* weird...
You can see my routes below.
When I set up the "pull" model so that the Client does a periodic GET,
I can connect and write to the bus no problem, and the data is fine.
However, when I set up a "push" model so that the Server does a UDP write,
when the Client then gets the data (and remember, this is only when using Mina2),
when I make the same exact call to write to the RabbitMQ bus -- it hangs.
The connection times out after 20 seconds, goes back and gets the next UDP data -- and hangs again.
Over and over...
So what's up with that?
What in the world can the UDP write/read be messing up that interferes with connecting to RabbitMQ??
Thank you so much for any help. I'm really in a pickle here.
Thank you, thank you.
--------------------------------------------------------------------------------
SERVER
restConfiguration()
.host("localhost").port(8080)
.bindingMode(RestBindingMode.json);
// NOTE: if uncommented, we're doing a Server push, and the 'timer' in CLIENT is commented out
from("timer:suss?period={{timer.period}}")
.streamCaching()
.to("rest:get:auto/list")
.to("direct:udp");
from("direct:udp")
.log("*** from direct:udp: BEFORE write to UDP: ${body}")
// .to("mina2:udp://localhost:40000?sync=false")
.to("netty4:udp://localhost:40000?sync=false")
// .to("netty4:udp://localhost:40000?sync=false&udpConnectionlessSending=true")
.log("*** from direct:udp: AFTER write to UDP: ${body}");
// --------------------------------------------------------------------------------
CLIENT
// NOTE: if uncommented, we're doing a Client pull, and the 'timer' in SERVER is commented out
/*
from("timer:adapter?period={{timer.period}}")
.streamCaching()
.to("jetty:http://localhost:8080/auto/list")
.log("*** got something from REST: ${body}")
.to("direct:autolist");
*/
// from("mina2:udp://localhost:40000?sync=false")
from("netty4:udp://localhost:40000?sync=false")
.streamCaching()
.log("*** got something from UDP: ${body}")
.to("direct:autolist");
// convert the automobile JSON to a protobuf Java object
from("direct:autolist")
.unmarshal(autoListJson)
.process(new AdapterProcessor())
.marshal().protobuf()
.to("direct:protobuf");
from("direct:protobuf")
.log("*** about to write to RabbitMQ ...")
.to("rabbitmq:{{rabbitmq.exch}}?connectionFactory=#rabbitmq
&routingKey={{rabbitmq.rkey}}&exchangeType=topic&durable=false&autoDelete=true");
I realize this is a long message, but it's only because I'm verbose.
It's a really straight forward setup, so I'm hoping it's just an obvious configuration issue...
I have a simple Server and Client setup for serving and fetching a list of automobiles.
The Client is an "adapter" that translates the returned JSON list to a Protobuf,
and then writes it to a RabbitMQ bus (port 5672).
----------
I implemented 2 Client-Server approaches -- a simple Pull & Push:
1. Pull: The Client uses a Camel timer to do a periodic GET (using Jetty),
translates the JSON to a Protobuf, writes it to the bus.
2. Push: The Server uses a Camel timer to do a periodic GET (from itself, using Rest),
writes the data to a UDP port (using Netty4 or Mina2).
The write is one-way; i.e. no response is expected, so I set 'sync=false'.
The Client reads from that UDP port (using Netty4 or Mina2),
translates the JSON to a Protobuf, writes it to the bus.
Here are the 2 problems I am experiencing:
----------
1. The first problem I'm experiencing is that I simply can not get Netty4 to work.
Whenever I use Netty4 in the Client / Server routes below, no data is written at all.
I don't get an error. I simply get nothing. I turned up logging to DEBUG and don't see anything.
I have a post-write log statement in the route ("... AFTER write to UDP ..."), which never prints.
The Client, of course, doesn't budge as it's not seeing anything.
However, when I use Mina2, I *can* write and read. Data flows, and it all looks good.
So what's up with that?
-----
(let me sneak in another question here)
I determined that the max size of the data I can send with Mina2 is 2048 bytes.
I'm on a Linux/CentOS machine. I looked around, and I think my system max is >> 2048.
So, is Mina setting something under the hood? And if so, how would I change it?
For Netty, I see that there is a 'sendBufferSize' and 'receiveBufferSize' which default to 65,536 bytes.
But I don't see any similar send/receive sizes for Mina.
(Of course, if I can figure out how to use Netty4, I won't care about Mina...)
----------
2. I can't write to RabbitMQ if I use UDP.
Ok, this one is *really* weird...
You can see my routes below.
When I set up the "pull" model so that the Client does a periodic GET,
I can connect and write to the bus no problem, and the data is fine.
However, when I set up a "push" model so that the Server does a UDP write,
when the Client then gets the data (and remember, this is only when using Mina2),
when I make the same exact call to write to the RabbitMQ bus -- it hangs.
The connection times out after 20 seconds, goes back and gets the next UDP data -- and hangs again.
Over and over...
So what's up with that?
What in the world can the UDP write/read be messing up that interferes with connecting to RabbitMQ??
Thank you so much for any help. I'm really in a pickle here.
Thank you, thank you.
--------------------------------------------------------------------------------
SERVER
restConfiguration()
.host("localhost").port(8080)
.bindingMode(RestBindingMode.json);
// NOTE: if uncommented, we're doing a Server push, and the 'timer' in CLIENT is commented out
from("timer:suss?period={{timer.period}}")
.streamCaching()
.to("rest:get:auto/list")
.to("direct:udp");
from("direct:udp")
.log("*** from direct:udp: BEFORE write to UDP: ${body}")
// .to("mina2:udp://localhost:40000?sync=false")
.to("netty4:udp://localhost:40000?sync=false")
// .to("netty4:udp://localhost:40000?sync=false&udpConnectionlessSending=true")
.log("*** from direct:udp: AFTER write to UDP: ${body}");
// --------------------------------------------------------------------------------
CLIENT
// NOTE: if uncommented, we're doing a Client pull, and the 'timer' in SERVER is commented out
/*
from("timer:adapter?period={{timer.period}}")
.streamCaching()
.to("jetty:http://localhost:8080/auto/list")
.log("*** got something from REST: ${body}")
.to("direct:autolist");
*/
// from("mina2:udp://localhost:40000?sync=false")
from("netty4:udp://localhost:40000?sync=false")
.streamCaching()
.log("*** got something from UDP: ${body}")
.to("direct:autolist");
// convert the automobile JSON to a protobuf Java object
from("direct:autolist")
.unmarshal(autoListJson)
.process(new AdapterProcessor())
.marshal().protobuf()
.to("direct:protobuf");
from("direct:protobuf")
.log("*** about to write to RabbitMQ ...")
.to("rabbitmq:{{rabbitmq.exch}}?connectionFactory=#rabbitmq
&routingKey={{rabbitmq.rkey}}&exchangeType=topic&durable=false&autoDelete=true");