Concurrent consumers from SEDA to CXF

[Mirek Borsky]

Greetings,

I have encountered a strange behavior with a simple camel route and I would
like to ask for help of more experienced users. First, let me quickly show
the route:

The relevant part:
            <from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
            <to uri="MonitoringService" />

The whole blueprint:
<?xml version="1.0" encoding="UTF-8"?>
<osgi:blueprint xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v
1.0.0 https://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd" xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance" xmlns:cxf="http://camel.apache.
org/schema/blueprint/cxf" xmlns:osgi="http://www.osgi.org/xmlns/blueprint/v
1.0.0">
    <camelContext id="TestSAMLoggerSeda-ctx" xmlns="http://camel.apache.org/
schema/blueprint" streamCache="true">
        <route>
            <from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
            <to uri="MonitoringService" />
        </route>
    </camelContext>
    <cxf:cxfEndpoint xmlns:tns="http://www.talend.org/esb/sam/
MonitoringService/v1" id="MonitoringService" address="http://localhost:8042/
MonitoringMock" serviceName="tns:MonitoringWebServiceService" endpointName=
"tns:MonitoringWebServicePort" wsdlURL="deploy/MonitoringService.wsdl">
        <cxf:properties>
            <osgi:entry key="dataFormat" value="PAYLOAD"/>
        </cxf:properties>
    </cxf:cxfEndpoint>
</osgi:blueprint>

Now the expected behavior:
What I want to achieve is a queue from which 10 parallel threads are
processing messages, calling the cxf endpoint. The cxf endpoint takes
several seconds to answer, so I am expecting the speed of processing
somewhere about 1-2 messages per second.

Observed behavior:
Using JMX I can see X+N inflight messages in the route, where X is the
maximum number of concurrent http connections to the cxf endpoint and N is
the number of concurrent users. For some reason, the workers that are
waiting for the cxf endpoint to answer are not counted against the
concurrentConsumers limit. Only the workers that are waiting for the
connection to be established are counted against the limit of
concurrentConsumers.

Questions:
What is the correct behavior for the route?
If my configuration is incorrect, how can I achieve my target behavior?

The camel version used is 2.20.

If someone would be able to help me, and explain what I did wrong, I would
very much appreciate it.

thank you for help
kind regards
Miroslav BorskÜ

[Claus Ibsen]

Hi

Can you try to explain the numbers a bit more, maybe with some example numbers?

For http connections etc, then CXF has its own set of limitations,
specially if you call the same remote server via the same client, eg
same cxf-endpoint in Camel.
That may have limitations on number of concurrent requests, that can
be lower. So try to look at the CXF documentation about this.

Post by Mirek Borsky
Greetings,
I have encountered a strange behavior with a simple camel route and I would
like to ask for help of more experienced users. First, let me quickly show
<from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
<to uri="MonitoringService" />
<?xml version="1.0" encoding="UTF-8"?>
<osgi:blueprint xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v
1.0.0 https://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd" xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance" xmlns:cxf="http://camel.apache.
org/schema/blueprint/cxf" xmlns:osgi="http://www.osgi.org/xmlns/blueprint/v
1.0.0">
<camelContext id="TestSAMLoggerSeda-ctx" xmlns="http://camel.apache.org/
schema/blueprint" streamCache="true">
<route>
<from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
<to uri="MonitoringService" />
</route>
</camelContext>
<cxf:cxfEndpoint xmlns:tns="http://www.talend.org/esb/sam/
MonitoringService/v1" id="MonitoringService" address="http://localhost:8042/
MonitoringMock" serviceName="tns:MonitoringWebServiceService" endpointName=
"tns:MonitoringWebServicePort" wsdlURL="deploy/MonitoringService.wsdl">
<cxf:properties>
<osgi:entry key="dataFormat" value="PAYLOAD"/>
</cxf:properties>
</cxf:cxfEndpoint>
</osgi:blueprint>
What I want to achieve is a queue from which 10 parallel threads are
processing messages, calling the cxf endpoint. The cxf endpoint takes
several seconds to answer, so I am expecting the speed of processing
somewhere about 1-2 messages per second.
Using JMX I can see X+N inflight messages in the route, where X is the
maximum number of concurrent http connections to the cxf endpoint and N is
the number of concurrent users. For some reason, the workers that are
waiting for the cxf endpoint to answer are not counted against the
concurrentConsumers limit. Only the workers that are waiting for the
connection to be established are counted against the limit of
concurrentConsumers.
What is the correct behavior for the route?
If my configuration is incorrect, how can I achieve my target behavior?
The camel version used is 2.20.
If someone would be able to help me, and explain what I did wrong, I would
very much appreciate it.
thank you for help
kind regards
Miroslav Borský

[Mirek Borsky]

Hi Claus,

I would try to explain. Please ask for the clarification as needed.

In the route in my example, if there will a delayer as the component that
take 2 seconds to proceed, the route will look like this:
<route>
  <from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
  <delay><constant>2000</constant></delay>
</route>

there will be 10 inflight messages in this route when the seda queue is
filled. The rate of processing of the messages from the seda queue will be
around 5 messages per second. This is the expected behaviour when I set
concurrentConsumers=10 on the seda queue..

However, when instead of <delay/> there will be the cxf call to a service
that take 2 seconds to answer, things will change. In my runtime
nevironment, I can see 110 inflight messages in the route. Also the rate of
processing of the messages from the seda queue is much higher, around 50 per
second.

The problem I perceive is not the number of concurrent requests for the cxf
endpoint. Problem is the fact, that the combination of seda queue and cxf
endpoint seems to exhaust all the available connections regardless of the
concurrentConsumers setting.

I hope I have provided the example that help you better understand the issue
I am facing. If I have failed in doing so, please bear with me and try to
ask differently.

kind regards
Miroslav BorskÜ


---------- Původní e-mail ----------
Od: Claus Ibsen <***@gmail.com>
Komu: ***@camel.apache.org
Datum: 30. 7. 2018 17:04:45
Předmět: Re: Concurrent consumers from SEDA to CXF
"Hi

Can you try to explain the numbers a bit more, maybe with some example
numbers?

For http connections etc, then CXF has its own set of limitations,
specially if you call the same remote server via the same client, eg
same cxf-endpoint in Camel.
That may have limitations on number of concurrent requests, that can
be lower. So try to look at the CXF documentation about this.

Post by Mirek Borsky
Greetings,
I have encountered a strange behavior with a simple camel route and I would
like to ask for help of more experienced users. First, let me quickly show
<from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
<to uri="MonitoringService" />
<?xml version="1.0" encoding="UTF-8"?>
<osgi:blueprint xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v
1.0.0 https://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd" xmlns:xsi

=

Post by Mirek Borsky
"http://www.w3.org/2001/XMLSchema-instance" xmlns:cxf="http://camel.

apache.

Post by Mirek Borsky
org/schema/blueprint/cxf" xmlns:osgi="http://www.osgi.org/xmlns/blueprint/

v

Post by Mirek Borsky
1.0.0">
<camelContext id="TestSAMLoggerSeda-ctx" xmlns="http://camel.apache.org/
schema/blueprint" streamCache="true">
<route>
<from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
<to uri="MonitoringService" />
</route>
</camelContext>
<cxf:cxfEndpoint xmlns:tns="http://www.talend.org/esb/sam/

8042/

Post by Mirek Borsky
MonitoringMock" serviceName="tns:MonitoringWebServiceService" endpointName

=

Post by Mirek Borsky
"tns:MonitoringWebServicePort" wsdlURL="deploy/MonitoringService.wsdl">
<cxf:properties>
<osgi:entry key="dataFormat" value="PAYLOAD"/>
</cxf:properties>
</cxf:cxfEndpoint>
</osgi:blueprint>
What I want to achieve is a queue from which 10 parallel threads are
processing messages, calling the cxf endpoint. The cxf endpoint takes
several seconds to answer, so I am expecting the speed of processing
somewhere about 1-2 messages per second.
Using JMX I can see X+N inflight messages in the route, where X is the
maximum number of concurrent http connections to the cxf endpoint and N is
the number of concurrent users. For some reason, the workers that are
waiting for the cxf endpoint to answer are not counted against the
concurrentConsumers limit. Only the workers that are waiting for the
connection to be established are counted against the limit of
concurrentConsumers.
What is the correct behavior for the route?
If my configuration is incorrect, how can I achieve my target behavior?
The camel version used is 2.20.
If someone would be able to help me, and explain what I did wrong, I would
very much appreciate it.
thank you for help
kind regards
Miroslav BorskÜ

[Claus Ibsen]

Hi

Ah okay its likely because CXF can process asynchronously and you may
not have a limit on the seda/vm queue so it can buffer pending
inflight exchanges.
So try setting cxf to be synchronous, via synchronous=true on its endpoint.

Post by Mirek Borsky
Hi Claus,
I would try to explain. Please ask for the clarification as needed.
In the route in my example, if there will a delayer as the component that
<route>
<from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
<delay><constant>2000</constant></delay>
</route>
there will be 10 inflight messages in this route when the seda queue is
filled. The rate of processing of the messages from the seda queue will be
around 5 messages per second. This is the expected behaviour when I set
concurrentConsumers=10 on the seda queue..
However, when instead of <delay/> there will be the cxf call to a service
that take 2 seconds to answer, things will change. In my runtime
nevironment, I can see 110 inflight messages in the route. Also the rate of
processing of the messages from the seda queue is much higher, around 50 per
second.
The problem I perceive is not the number of concurrent requests for the cxf
endpoint. Problem is the fact, that the combination of seda queue and cxf
endpoint seems to exhaust all the available connections regardless of the
concurrentConsumers setting.
I hope I have provided the example that help you better understand the issue
I am facing. If I have failed in doing so, please bear with me and try to
ask differently.
kind regards
Miroslav Borský
---------- Původní e-mail ----------
Datum: 30. 7. 2018 17:04:45
Předmět: Re: Concurrent consumers from SEDA to CXF
"Hi
Can you try to explain the numbers a bit more, maybe with some example numbers?
For http connections etc, then CXF has its own set of limitations,
specially if you call the same remote server via the same client, eg
same cxf-endpoint in Camel.
That may have limitations on number of concurrent requests, that can
be lower. So try to look at the CXF documentation about this.

Post by Mirek Borsky
Greetings,
I have encountered a strange behavior with a simple camel route and I

would

Post by Mirek Borsky
like to ask for help of more experienced users. First, let me quickly show
<from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
<to uri="MonitoringService" />
<?xml version="1.0" encoding="UTF-8"?>
<osgi:blueprint xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v
1.0.0 https://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd" xmlns:xsi

=

Post by Mirek Borsky
"http://www.w3.org/2001/XMLSchema-instance" xmlns:cxf="http://camel.

apache.

Post by Mirek Borsky
org/schema/blueprint/cxf" xmlns:osgi="http://www.osgi.org/xmlns/blueprint/

v

Post by Mirek Borsky
1.0.0">
<camelContext id="TestSAMLoggerSeda-ctx" xmlns="http://camel.apache.org/
schema/blueprint" streamCache="true">
<route>
<from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
<to uri="MonitoringService" />
</route>
</camelContext>
<cxf:cxfEndpoint xmlns:tns="http://www.talend.org/esb/sam/

8042/

Post by Mirek Borsky
MonitoringMock" serviceName="tns:MonitoringWebServiceService" endpointName

=

Post by Mirek Borsky
"tns:MonitoringWebServicePort" wsdlURL="deploy/MonitoringService.wsdl">
<cxf:properties>
<osgi:entry key="dataFormat" value="PAYLOAD"/>
</cxf:properties>
</cxf:cxfEndpoint>
</osgi:blueprint>
What I want to achieve is a queue from which 10 parallel threads are
processing messages, calling the cxf endpoint. The cxf endpoint takes
several seconds to answer, so I am expecting the speed of processing
somewhere about 1-2 messages per second.
Using JMX I can see X+N inflight messages in the route, where X is the
maximum number of concurrent http connections to the cxf endpoint and N is
the number of concurrent users. For some reason, the workers that are
waiting for the cxf endpoint to answer are not counted against the
concurrentConsumers limit. Only the workers that are waiting for the
connection to be established are counted against the limit of
concurrentConsumers.
What is the correct behavior for the route?
If my configuration is incorrect, how can I achieve my target behavior?
The camel version used is 2.20.
If someone would be able to help me, and explain what I did wrong, I would
very much appreciate it.
thank you for help
kind regards
Miroslav Borský

--
Claus Ibsen
-----------------
Camel in Action 2: https://www.manning.com/ibsen2
"