Discussion:
File polling issue for filename with multiple dots
DEIMAT Guillaume
2018-11-15 14:17:39 UTC
Permalink
Hi guys,

I'm working on un upgrade from Camel 2.6.0 to 2.22.x (yeah... this is huge step !!) and need some help for an issue with file polling.
I'm not very familiar with this framework, so I'll try to give enough info to expose our issue. If I'm not, sorry for that !

Our application read file from a given directory which follow the format : <mySignificativeName>_yyyyMMdd_HHmmss.SSS.ebt
It was working fine in 2.6.0, but after the upgrade our file are not consumed and stay in entry directory.

After a few hours of debugging, I finally found why.
It's because our file names contains two dots, and the GenericFileEndpoint class uses FileUtil.stripExt() function to generate the done file name.
This function has been updated for CAMEL-9064 issue to work in "single mode" and the GenericFileEndpoint does not appear to have been updated too (don't know this stuff very well to say if he should have been or not).

Code from GenericFileEndpoint.createDoneFileName
--------------------------------------------------------------------------
[...]
pattern = pattern.replaceFirst("\\$\\{file:name.noext\\}", FileUtil.stripExt(onlyName));
pattern = pattern.replaceFirst("\\$simple\\{file:name.noext\\}", FileUtil.stripExt(onlyName));
[...]

Code from GenericFileEndpoint.createDoneFileName
--------------------------------------------------------------------------
public static String stripExt(String name) {
return stripExt(name, false);
}

public static String stripExt(String name, boolean singleMode) {
[...]
}

Our road is defined this way :
--------------------------------------------------------------------------
from("file:<entryDirectory>?sortBy=reverse:file:modified&include=B_[A-Z]{3,30}_01.*\.ebt$&charset=UTF-8&delete=true&doneFileName=${file:name.noext}.done", "direct:flux04")
.process(new OurProcessor())
.to("cxf://ourEndpoint...")


We updated our source application to generate file names with a single dot for extension, but for compatibility reason we need to read both file names (with a single dot & with multiple dots).
I don't know how to proceed :
1) It's really an issue in GenericFileEndpoint => so we just have to wait for a patch or even submit it if we are good enough to :)
2) Maybe we can set a parameter to accept files with multiple dots in their file names => does not seem to be possible in all documentations I read
3) Or we should extends GenericFileEndpoint to get the behavior that we need => I don't know how to proceed for that

Thanks a lot for your help,

Guillaume
Alex Dettinger
2018-11-15 19:56:22 UTC
Permalink
Hi Guillaume,

Welcome on camel 2.22.x :) I've reported issue
https://issues.apache.org/jira/browse/CAMEL-12940 and provided a fix.
You should be able to check the fix against 2.23.0-SNAPSHOT tomorrow, or
wait for the release of CAMEL-2.22.3.

Enjoy,
Alex

On Thu, Nov 15, 2018 at 3:17 PM DEIMAT Guillaume <
Post by DEIMAT Guillaume
Hi guys,
I'm working on un upgrade from Camel 2.6.0 to 2.22.x (yeah... this is huge
step !!) and need some help for an issue with file polling.
I'm not very familiar with this framework, so I'll try to give enough info
to expose our issue. If I'm not, sorry for that !
<mySignificativeName>_yyyyMMdd_HHmmss.SSS.ebt
It was working fine in 2.6.0, but after the upgrade our file are not
consumed and stay in entry directory.
After a few hours of debugging, I finally found why.
It's because our file names contains two dots, and the GenericFileEndpoint
class uses FileUtil.stripExt() function to generate the done file name.
This function has been updated for CAMEL-9064 issue to work in "single
mode" and the GenericFileEndpoint does not appear to have been updated too
(don't know this stuff very well to say if he should have been or not).
Code from GenericFileEndpoint.createDoneFileName
--------------------------------------------------------------------------
[...]
pattern = pattern.replaceFirst("\\$\\{file:name.noext\\}",
FileUtil.stripExt(onlyName));
pattern = pattern.replaceFirst("\\$simple\\{file:name.noext\\}",
FileUtil.stripExt(onlyName));
[...]
Code from GenericFileEndpoint.createDoneFileName
--------------------------------------------------------------------------
public static String stripExt(String name) {
return stripExt(name, false);
}
public static String stripExt(String name, boolean singleMode) {
[...]
}
--------------------------------------------------------------------------
from("file:<entryDirectory>?sortBy=reverse:file:modified&include=B_[A-Z]{3,30}_01.*\.ebt$&charset=UTF-8&delete=true&doneFileName=${file:name.noext}.done",
"direct:flux04")
.process(new OurProcessor())
.to("cxf://ourEndpoint...")
We updated our source application to generate file names with a single dot
for extension, but for compatibility reason we need to read both file names
(with a single dot & with multiple dots).
1) It's really an issue in GenericFileEndpoint => so we just have to
wait for a patch or even submit it if we are good enough to :)
2) Maybe we can set a parameter to accept files with multiple dots in
their file names => does not seem to be possible in all documentations I
read
3) Or we should extends GenericFileEndpoint to get the behavior that
we need => I don't know how to proceed for that
Thanks a lot for your help,
Guillaume
DEIMAT Guillaume
2018-11-16 09:06:05 UTC
Permalink
Hi Alex,

Thanks a lot for the quick fix !
We'll wait for the next release so :)

Guillaume

-----Message d'origine-----
De : Alex Dettinger <***@gmail.com>
Envoyé : jeudi 15 novembre 2018 20:56
À : ***@camel.apache.org
Objet : Re: File polling issue for filename with multiple dots

Hi Guillaume,

Welcome on camel 2.22.x :) I've reported issue
https://issues.apache.org/jira/browse/CAMEL-12940 and provided a fix.
You should be able to check the fix against 2.23.0-SNAPSHOT tomorrow, or wait for the release of CAMEL-2.22.3.

Enjoy,
Alex
Post by DEIMAT Guillaume
Hi guys,
I'm working on un upgrade from Camel 2.6.0 to 2.22.x (yeah... this is
huge step !!) and need some help for an issue with file polling.
I'm not very familiar with this framework, so I'll try to give enough
info to expose our issue. If I'm not, sorry for that !
<mySignificativeName>_yyyyMMdd_HHmmss.SSS.ebt
It was working fine in 2.6.0, but after the upgrade our file are not
consumed and stay in entry directory.
After a few hours of debugging, I finally found why.
It's because our file names contains two dots, and the
GenericFileEndpoint class uses FileUtil.stripExt() function to generate the done file name.
This function has been updated for CAMEL-9064 issue to work in "single
mode" and the GenericFileEndpoint does not appear to have been updated
too (don't know this stuff very well to say if he should have been or not).
Code from GenericFileEndpoint.createDoneFileName
----------------------------------------------------------------------
----
[...]
pattern = pattern.replaceFirst("\\$\\{file:name.noext\\}",
FileUtil.stripExt(onlyName));
pattern =
pattern.replaceFirst("\\$simple\\{file:name.noext\\}",
FileUtil.stripExt(onlyName));
[...]
Code from GenericFileEndpoint.createDoneFileName
--------------------------------------------------------------------------
public static String stripExt(String name) {
return stripExt(name, false);
}
public static String stripExt(String name, boolean singleMode) {
[...]
}
----------------------------------------------------------------------
----
from("file:<entryDirectory>?sortBy=reverse:file:modified&include=B_[A-
Z]{3,30}_01.*\.ebt$&charset=UTF-8&delete=true&doneFileName=${file:name
.noext}.done",
"direct:flux04")
.process(new OurProcessor())
.to("cxf://ourEndpoint...")
We updated our source application to generate file names with a single
dot for extension, but for compatibility reason we need to read both
file names (with a single dot & with multiple dots).
1) It's really an issue in GenericFileEndpoint => so we just have
to wait for a patch or even submit it if we are good enough to :)
2) Maybe we can set a parameter to accept files with multiple dots
in their file names => does not seem to be possible in all
documentations I read
3) Or we should extends GenericFileEndpoint to get the behavior
that we need => I don't know how to proceed for that
Thanks a lot for your help,
Guillaume
Loading...