I have web server with autoindex module that lists all files and folders. Something like that:
How can i recursively fetch all files using Apache NiFi? I tried to use ListFTP/FetchFTP processors, but without any luck.
Can someone tell me what processors should i use for this task?
As remote path you should specify the root folder from which you want to List/Fetch all files and sub folders.
In your case, you put /FILE/store/0002550267/ as remote path, so only files and folders in this directory will be picked up.
Change remote path to /File (or / if you want all files on the server) and you should see all files being listed.
Related
I'm using the below ansible-playbook code to archive multiple folders under IBM folder.
Below is my absolute path Directory structure:
/app
|-- /IBM
|--/test
|--/log
|--/common
|--/api
|--/<unknown folders>
The number of folders and files under IBM folder is unknown hence it seems impossible to specify all the files / folders under IBM under exclude_path attribute.
I wish to build an archive (gz) that has only IBM folder containing only two folders i.e common and api folders while ignoring the rest of the folders and files.
Thus, I wrote the below playbook:
- name: Creating the archive
archive:
path:
- /was/IBM/common
- /was/IBM/api
dest: /var/backup/mysetup.tar.gz
exclude_path:
- /was/IBM/common/log
- /was/IBM/api/tmp
format: gz
This gives me the archive file mysetup.tar.gz.
I want the mysetup.tar.gz file to have a folder called IBM which should have the two folders common and api. Thus, I'm expecting the below in the mysetup.tar.gz no matter what other files are under the IBM folder.
IBM
|--/common
|--/api
But, the mysetup.tar.gz does not have IBM folder but has common and api folders.
I was not specific hence my question did not get answered here: How to archive multiple folders under one folder using Ansible
Can you please guide me as to how I can get the archive to have both the folders inside the IBM folder inside the mysetup.tar.gz?
Requirement you have can't be achieved straight using archive module. Two specific reasons are listed below:
If you specify the path say /app/IBM then the content of the zip file will have the path after IBM something like common/, api/. To get IBM as a root directory inside the tar.gz file, you need to specify path as /app. This brings to the next point about the usage of exclude_path.
exclude_path doesn't work as you would expect. Say, if the path is define as /app and you want to exclude /app/IBM/test, this wouldn't work. exclude_path works only with direct sub folder/file of a defined path (with wildcard) which means that exclusion of /app/IBM/test would work if path defined as /app/IBM/* but that is not what is expected and brings back the previous point. There is already an issue reported on this topic.
So you probably better off using regular tar command using command/shell module.
I have nifi-1.3.0-RC1 and i want to get log files i used this :
nifi-1.3.0-RC1/logs/nifi-app.log
but tailFail processor throws and exception :Java.nio.file.NosuchFieleExcption maybe this version of nifi collects its log file somewhere else?
The log files are kept in a directory under $NIFI_HOME, which is the directory containing the bin, conf, lib, logs, etc. directories that Apache NiFi runs from. You can determine this directory by running
If you built NiFi from source, it will be a directory path like:
/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT
If you downloaded the binary, it will be something like (after unzipping/untarring):
/Users/alopresto/Downloads/nifi-1.3.0/
It's a good idea to use the absolute path for the filesystem rather than the relative path you have above.
My .csv recreates on every release, and, as I understand, to keep its data unchanged between deploys I need to put it in /shared directory and simlink to it from my deploy.rb.
Is this the right route? (I have this question because I don't seem to find much info on how to do this with respect to, eg, databases, for some reason. /shared directory is mostly used for .conf files and paperclip-like directories).
When using capistrano, your application code will be "uploaded" to some directory on the server. Capistrano uses this structure:
/path_to_folder:
current - symlink to the directory with the current release
releases - contains all kept releases
shared - files that should persist between releases
So to your question - copy the .csv file somewhere into "shared" directory and then in the config/deploy.rb add this:
namespace :deploy do
task :create_symlinks do
run "ln -s #{shared_path}/something.csv #{latest_release}/db/something.csv"
end
end
after 'deploy:update_code', 'deploy:create_symlinks'
Replace "something" with the file name that you copied. You can also put the csv file into some directory under "shared" if you want to, I'd use "db" in this case. If you do so, don't forget to update path in the symlink.
I am working on setting up a load balancing cluster on windows server 2012 and have a shared drive where I want the configuration files for Apache to exist at. This way each member of the LB can load the exact same config files. How do I change where the config file is located independently of where the ServerRoot is?
Start the Apache process with the -d parameter and give your alternative ServerRoot as an argument, though I'd imagine it would be a much better idea for you to use some mechanism to sync the files locally to each server.
Also read http://httpd.apache.org/docs/2.4/mod/core.html#mutex, as it's advised if you're running from a networked file system.
If you just want to specify the main config file, start the process with the -f parameter and the path to the config file as an argument.
I am having directory issues when using sublime sftp to upload files. Currently am using ftp type, and the issue is this:
My local and remote folders match, but when I take a file from the local side that is say 3 directories deep and modify/upload, it's putting the file to the root folder and not the directory location.
Any ideas are appreciated.
NOt sure if this helps, but
Uploading "/Volumes/--/Dropbox/ItsJustFood/web/wp-content/themes/justfood/library/css/style.css" to "/library/css/style.css" ..... success
looks like it's only going two directories up when it should be going to the root?
Turns out you have to make sure the .json file is also in the root folder of your remote copy.