NiFi version 1.5.
i use FetchFTP configured as shown below:
Hostname: x.x.x.x
port: 21
username: yyy
password: zzz
Remote File: ${path}/${file_name}
Completion Strategy: Delete File
Run Schedule: 5 sec
50 files are processed by FetchFTP and out of which only 46 is successfully deleted the file from the FTP server.
Instantly, the processor showed the below error message and also in the log file:
2019-11-20 23:33:25,542 WARN [Timer-Driven Process Thread-6] o.a.nifi.processors.standard.FetchFTP FetchFTP[id=53ee29a5] Successfully fetched the content for StandardFlowFileRecord[uuid=8ec8219e,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=15742604, container=default, section=175], offset=49, length=598026],offset=0,name=20191118145221190.pdf,size=598026] from x.x.x.x:21/<folder>/20191118145221190.pdf but failed to remove the remote file due to java.io.IOException: Failed to remove file /<folder>/20191118145221190.pdf due to 550 The process cannot access the file because it is being used by another process.
Appreciate for any help regarding this.
Related
In my NiFi workflow I need to download .zip file from SOAP web-server, save it on machine (optional) and unpack content to sub-folder. Everything works on my local Win 10 machine, but issues occur when I try to move to remote Linux server. Here is the part of my flow when error happens:
So we have a FlowFile entered UpdateAttribute where filename attributed is set with required name and .zip extension. The file is correct as can be seen in queue after starting the processor.
Problems start happen when I pass the FlowFile to PutFile processors. I tried different scenarios based on selected directory:
relative to NiFi main folder ./out:
12:30:01 MSK ERROR
PutFile[id=05788ae5-64e5-32af-bb40-88d50d4c886c] Penalizing StandardFlowFileRecord[uuid=3e0c5e38-76f8-4ce3-b911-90f6901c35a4,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1586337594333-49, container=default, section=49], offset=2335, length=375628606],offset=0,name=,size=375628606] and transferring to failure due to /opt/nifi/nifi-1.11.4/./out: java.nio.file.DirectoryNotEmptyException: /opt/nifi/nifi-1.11.4/./out
12:30:01 MSK ERROR
PutFile[id=05788ae5-64e5-32af-bb40-88d50d4c886c] Unable to remove temporary file /opt/nifi/nifi-1.11.4/./out/. due to /opt/nifi/nifi-1.11.4/./out/.: Invalid argument: java.nio.file.FileSystemException: /opt/nifi/nifi-1.11.4/./out/.: Invalid argument
Full path /opt/nifi/nifi-1.11.4/out/file/ :
12:32:45 MSK ERROR
PutFile[id=0171102b-c82d-149d-c9ae-ea4da99b1750] Penalizing StandardFlowFileRecord[uuid=0573803f-8407-46e4-93f0-e52a5fc35a07,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1586337594333-49, container=default, section=49], offset=2335, length=375628606],offset=0,name=,size=375628606] and transferring to failure due to Failed to export StandardFlowFileRecord[uuid=0573803f-8407-46e4-93f0-e52a5fc35a07,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1586337594333-49, container=default, section=49], offset=2335, length=375628606],offset=0,name=,size=375628606] to /opt/nifi/nifi-1.11.4/out/file/. due to java.io.FileNotFoundException: /opt/nifi/nifi-1.11.4/out/file/. (No such file or directory): org.apache.nifi.processor.exception.FlowFileAccessException: Failed to export StandardFlowFileRecord[uuid=0573803f-8407-46e4-93f0-e52a5fc35a07,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1586337594333-49, container=default, section=49], offset=2335, length=375628606],offset=0,name=,size=375628606] to /opt/nifi/nifi-1.11.4/out/file/. due to java.io.FileNotFoundException: /opt/nifi/nifi-1.11.4/out/file/. (No such file or directory)
So it adds dot ('.') to path which causes exception. All folders are created and permissions granted. I tried to run simple test flow with file of 42B and the same path (GenerateFlowFile -> PutFile) and everything is OK.
What am I doing wrong?
The problem was in Linux system rights: there was 'nifi' user for flow execution and one more user for accessing Linux filesystem.
Assigning rwxrwxrwx for used folders solved the issue.
my oobjective is simple, just a backup and retsore it on other machine , which have no raltion with running cluter .
My steps .
1. Remotly pg_basebackup on new machine .
2. rm -fr ../../main/
3. mv bacnkup/main/ ../../main/
4. start postgres service
** During backup no error occur. **
But getting error:
2018-12-13 10:05:12.437 IST [834] LOG: database system was shut down in recovery at 2018-12-12 23:01:58 IST
2018-12-13 10:05:12.437 IST [834] LOG: invalid primary checkpoint record
2018-12-13 10:05:12.437 IST [834] LOG: invalid secondary checkpoint record
2018-12-13 10:05:12.437 IST [834] PANIC: could not locate a valid checkpoint record
2018-12-13 10:05:12.556 IST [833] LOG: startup process (PID 834) was terminated by signal 6: Aborted
2018-12-13 10:05:12.556 IST [833] LOG: aborting startup due to startup process failure
2018-12-13 10:05:12.557 IST [833] LOG: database system is shut down
Based on the answer to a very similar question (How to mount a pg_basebackup on a stand alone server to retrieve accidently deleted data and on the fact that that answer helped me get this working glitch-free, the steps are:
do the basebackup, or copy/untar previously made one, to the right location /var/lib/postgresql/9.5/main
remove the file backup_label
run /usr/lib/postgresql/9.5/bin/pg_resetxlog -f /var/lib/postgresql/9.5/main
start postgres service
(replying to this old question because it is the first one I found when looking to find the solution to the same problem).
When I entered Oracle Application Server Control panel, an error message was appeared "Too few bytes (-1) received from OPMN response". It doesn't allow to do any deployment. How can I fix this error?
Trace Information:
oracle.ias.opmn.optic.OpticBadConnectException: Too few bytes (-1) received from OPMN response at oracle.ias.opmn.optic.OpmnPhone.rcvResponse(OpmnPhone.java:529) at oracle.ias.opmn.optic.OpmnPhone.makePhoneCall(OpmnPhone.java:193) at oracle.ias.opmn.optic.OpmnPhone.request(OpmnPhone.java:130) at oracle.ias.opmn.optic.OpmnQuery.getBuf(OpmnQuery.java:347) at oracle.ias.opmn.optic.OpmnQuery.getDom(OpmnQuery.java:467) at oracle.ias.opmn.optic.OpmnQuery.getIasCluster(OpmnQuery.java:941) at oracle.sysman.ias.studio.cluster.OpticTopologyAdminBean.initializeAppServers(OpticTopologyAdminBean.java:1117) at oracle.sysman.ias.studio.cluster.TopologyHelper.prepareData(TopologyHelper.java:1278) at oracle.sysman.ias.studio.sdk.AbstractController.prepareData(AbstractController.java:875) at oracle.sysman.emSDK.svlt.PageHandler.handleRequest(PageHandler.java:391) at oracle.sysman.emSDK.svlt.EMServlet.myDoGet(EMServlet.java:765) at oracle.sysman.emSDK.svlt.EMServlet.doGet(EMServlet.java:283) at oracle.sysman.ias.studio.app.StudioConsole.doGet(StudioConsole.java:297) at javax.servlet.http.HttpServlet.service(HttpServlet.java:743) at javax.servlet.http.HttpServlet.service(HttpServlet.java:856) at com.evermind.server.http.ResourceFilterChain.doFilter(ResourceFilterChain.java:64) at oracle.sysman.ias.studio.app.BrowserVersionFilter.doFilter(BrowserVersionFilter.java:75) at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:15) at oracle.sysman.ias.studio.app.MultipleJVMFilter.doFilter(MultipleJVMFilter.java:85) at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:17) at oracle.sysman.ias.studio.app.PostLogonFilter.doFilter(PostLogonFilter.java:80) at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:17) at oracle.sysman.ias.studio.app.ShortHostnameRedirectFilter.doFilter(ShortHostnameRedirectFilter.java:68) at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:619) at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:368) at com.evermind.server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:866) at com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:448) at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:302) at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:190) at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260) at com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:303) at java.lang.Thread.run(Thread.java:595)
Error logs in opmn.log file :
.
.
.
16/02/03 10:14:29 [ons-connect] Local connection 127.0.0.1,6100 invalid form factor
16/02/03 10:16:29 [ons-connect] Local connection 127.0.0.1,6100 invalid form factor
16/02/03 10:18:42 [ons-connect] Local connection 127.0.0.1,6100 invalid form factor
16/02/03 10:19:29 [ons-connect] Local connection 127.0.0.1,6100 invalid form factor
.
.
After a long search process, I find that my error is related with owner permissions on oracle server. oracle user manages the instances so files under oracle/ folder must have oracle owner permissions. If you enter with root or other user and change some file permissions by running or rewriting them especially config files under j2ee/home/persistence, opmn/conf, opmn/bin, oracle user can not execute that files. I set files owner permissions with oracle user and restart the server, error was gone.
On starting TeamCity from URL, I get following error :
It was working fine till last week. And when opened today, it is showing error.
I logged in as administrator to view logs. Below is the stacktrace :
<pre>jetbrains.buildServer.processes.ProcessTreeTerminatorException: Process has not exited within 30 seconds
at jetbrains.buildServer.processes.ProcessTreeTerminatorImplBase.readCommandOutput(ProcessTreeTerminatorImplBase.java:159)
at jetbrains.buildServer.processes.ProcessTreeTerminatorImplBase.buildProcessTree(ProcessTreeTerminatorImplBase.java:315)
at jetbrains.buildServer.processes.ProcessTreeTerminatorImplBase.getProcessVisitor(ProcessTreeTerminatorImplBase.java:82)
at jetbrains.buildServer.processes.ProcessTreeTerminatorImplBase.getCurrentPid(ProcessTreeTerminatorImplBase.java:28)
at jetbrains.buildServer.processes.ProcessTreeTerminator$1.getCurrentPid(ProcessTreeTerminator.java:144)
at jetbrains.buildServer.processes.ProcessTreeTerminator.getCurrentPid(ProcessTreeTerminator.java:101)
at jetbrains.buildServer.maintenance.StartupProcessor.createPidFile(StartupProcessor.java:476)
at jetbrains.buildServer.maintenance.StartupProcessor.doInitialStage(StartupProcessor.java:326)
at jetbrains.buildServer.maintenance.StartupProcessor.processConcreteStage(StartupProcessor.java:83)
at jetbrains.buildServer.maintenance.StartupProcessor.processConcreteStageSafe(StartupProcessor.java:503)
at jetbrains.buildServer.maintenance.StartupProcessor.processTeamCityLifecycle(StartupProcessor.java:558)
at jetbrains.buildServer.maintenance.StartupProcessor.access$000(StartupProcessor.java:92)
at jetbrains.buildServer.maintenance.StartupProcessor$1.run(StartupProcessor.java:2)
at java.lang.Thread.run(Thread.java:745)
Some more information :
Current Startup State
Startup status
Current step: TeamCity server startup error
Next step: not defined yet
Data Directory
Data Directory path is not specified/detected yet
Database
Not connected to the database yet.
Versions
Software version: 722
Data directory version: unknown
Database version: unknown
Logs
Logs path: C:\TeamCity\logs
What should be done to resolve error?
As this is an EAP build, you should report this issue to JetBrains with complete information logs here
I am experiencing some problems with RabbitMQ started as a service on Windows.
Operative System: Windows 8 (Microsoft Windows NT version 6.2 Server)
(build 9200)
Erlang: R16B03 (erts-5.10.4)
RabbitMQ: 3.2.2
Goal: create a RabbitMQ cluster with three servers: Srv1, Srv2, Srv3.
Note: I have carefully followed the official documentation
All the following operations are executed as user "Administrator".
FIRST SCENARIO: start RabbitMQ from command line as a background process
I used the command "rabbitmq-server -detached" on Srv1.
Result: a file ".erlang.cookie" is created under C:\Users\Administrator
The execution of the command "rabbimqctl status" is successful and gives me the current state of the node.
I can then copy the file .erlang.cookie in the same folder on Srv2 and Srv3 and successfully create a cluster.
SECOND SCENARIO: start RabbitMQ as a service (this is requirement I have)
Result: the file ".erlang.cookie" is created under C:\Windows.
When I type the command "rabbitmqctl status" another file .erlang.cookie is created under C:\Users\Administrator and I receive the following result:
C:\Program Files\Aspect\DashBoard\RabbitMQ\sbin>rabbitmqctl.bat status
Status of node 'rabbit#RABBITMQ-NODE4' ...
Error: unable to connect to node 'rabbit#RABBITMQ-NODE4': nodedown
DIAGNOSTICS
===========
nodes in question: ['rabbit#RABBITMQ-NODE4']
hosts, their running nodes and ports:
- RABBITMQ-NODE4: [{rabbit,49428},{rabbitmqctl3045334,49434}]
current node details:
- node name: 'rabbitmqctl3045334#rabbitmq-node4'
- home dir: C:\Users\Administrator
- cookie hash: 0DLAKf8pOVrGC016+6BDBw==
We know that this is ok because the two cookies are different.
So I copy the .erlang.cookie file from C:\Windows into C:\Users\Administrator and I try again the same command. This time I get:
C:\Program Files\Aspect\DashBoard\RabbitMQ\sbin>rabbitmqctl.bat status
Status of node 'rabbit#RABBITMQ-NODE4' ...
Error: unable to connect to node 'rabbit#RABBITMQ-NODE4': nodedown
DIAGNOSTICS
===========
nodes in question: ['rabbit#RABBITMQ-NODE4']
hosts, their running nodes and ports:
- RABBITMQ-NODE4: [{rabbitmqctl1178095,49471}]
current node details:
- node name: 'rabbitmqctl1178095#rabbitmq-node4'
- home dir: C:\Users\Administrator
- cookie hash: TIuqp21HOQSoUJT8JfgRQw==
C:\Program Files\Aspect\DashBoard\RabbitMQ\sbin>rabbitmqctl.bat status
Status of node 'rabbit#RABBITMQ-NODE4' ...
Error: {badarith,[{rabbit_vm,bytes,1,[]},
{rabbit_vm,'-mnesia_memory/0-lc$^0/1-0-',1,[]},
{rabbit_vm,mnesia_memory,0,[]},
{rabbit_vm,memory,0,[]},
{rabbit,status,0,[]},
{rpc,'-handle_call_call/6-fun-0-',5,
[{file,"rpc.erl"},{line,205}]}]}
Please notice the Error at the end: "badarith" in rpc.erl, line 205.
I think that the file is Erlang\lib\kernel-2.16.4\src\rpc.erl
The function is this one:
handle_call_call(Mod, Fun, Args, Gleader, To, S) ->
RpcServer = self(),
%% Spawn not to block the rpc server.
{Caller,_} =
erlang:spawn_monitor(
fun () ->
set_group_leader(Gleader),
Reply =
%% in case some sucker rex'es
%% something that throws
case catch apply(Mod, Fun, Args) of
{'EXIT', _} = Exit ->
{badrpc, Exit};
Result ->
Result
end,
RpcServer ! {self(), {reply, Reply}}
end),
{noreply, gb_trees:insert(Caller, To, S)}.
and line 205 is 'case catch apply(Mod, Fun, Args) of'
THIRD SCENARIO: start RabbitMQ as a named user to avoid it to create the file .erlang.cookie under C:\Windows
I set the RabbitMQ service to log on as the user "Administrator", this way it does not create the file under C:\Windows but only under C:\User\Administrator.
Result: when the service starts, the file ".erlang.cookie" is created only under C:\User\Administrator.
When I type the command "rabbitmqctl status" I get the same error as in the provious case (badarith...).
Now the question: I have not found any information about this error (badarith).
Could anyone give me a suggestion about how to troubleshoot/avoid this?