When setting up an FTP warehouse, what time zone is used for the scheduled run times?
Based on my testing, the scheduled run time will be dependent on the timezone configured within Basic Info of the warehouse settings.
Related
Lets say we have kerberos secured Hadoop Cluster. ( For example Kdc is configured to provide kerberos tickets with validity of 2 hour and renewal period of 3 hours after this so total 5 hours. )
Cluster also includes tools like Hive and Pig.
With respect to given scenario can any one help me understand the behaviors in each cases mentioned below(Note: Job: Map-Reduce Job, Job Submitted Through Hive Or Pig)
Lets say job takes 4 hours to run, I am triggering the job, for next two hours user who triggered the job has valid ticket, job runs fine after two hours Job is out of valid kerberos ticket but it has renewal period left, Will job renew it? Will it stop execution? Is there any property to enable or disable such automatically renewal behavior.
Lets say job takes 10 hours to run, I am trigerring the job, for next two hours user who triggered the job has valid ticket, lets say this time after two hours we don't have renewal period left, Will Job continue? Will Job fail with authentication error. Will Job request for new ticket?
Let say Job takes 10 hours to run And Kdc is configured to provide 2 hours ticket. we setup a cron job which runs at every 6 hours and gets a new ticket(Technically User will not run out of valid kerberos ticket any time), Will new generated ticket cause any problem of the running job through old ticket?
I want to understand working behavior of this Jobs as well as tools like kerberos, hive and pig.
I am new to Apache NIFI tool. I am trying to import data from mongo db and put that data into the HDFS. I have created 2 processors one for MongoDB and second for HDFS and I configured them correctly. The job is running successfully and storing the data into HDFS but the job should terminate automatically on success. But it is not, and creating too many files in HDFS. I want to know how to make On Demand Job in NIFI and how to determine that a job is successfull.
GetMongo will continue to pull data from MongoDB based on the provided properties such as Query, Projection, Limit. It has no way of tracking the execution process, at least for now. What you can do, however, is changing the Run Schedule and/or Scheduling Strategy. You can find them by right clicking on the processor and clicking Configure. By default, Run Schedule will be 0 sec which means running continuously. Changing it to, say, 60 min will make the processor run every one hour. This will still read the same documents from MongoDB again every one hour but since you have mentioned that you just want to run it only once, I'm suggesting this approach.
I am using a jenkins configuration where the same job is being executed in different locations: one in farm1 and another in an overseas farm2.
The Jenkins master server is located in farm1.
I encounter a situation where the job on farm2 takes much more time to finish, sometimes twice the elapsed time.
Do you have an idea what could be the reason for that?
is there a continuous master-slave discussion during the build that can cause such delay?
The job is a maven junit test + ui seleniun using vnc server on the slave
Thanks in advance,
Roy
I assume your server farms have identical hardware specs?
Network differences while checking out code, downloading dependencies, etc. Workspace of Master and Slave are on different servers
If you are archiving artifacts, they are usually archived back on Master, even when the job is run on Slave.
Install Timestamper plugin, enable it, and then review the logs of both the Master and the Slave runs, and see where there is a big time difference (you can configure Timestamper to show time as increments from the start of job, this would be helpful here)
I have a cron job that is set to run at midnight. I would like to know if this is midnight from the server timezone or is it as soon as i create the cron job i have to wait 24h until it's being executed?
You don't have to wait 24 hours. The cron in launched in the scheduled hour based on server's time.
Is there a similar event scheduler from MySQL available in PostgreSQL?
While a lot of people just use cron, the closest thing to a built-in scheduler is PgAgent. It's a component to the pgAdmin GUI management tool. A good intro to it can be found at Setting up PgAgent and doing scheduled backups.
pg_cron is a simple, cron-based job scheduler for PostgreSQL that runs
inside the database as an extension. A background worker initiates
commands according to their schedule by connecting to the local
database as the user that scheduled the job.
pg_cron can run multiple jobs in parallel, but it runs at most one
instance of a job at a time. If a second run is supposed to start
before the first one finishes, then the second run is queued and
started as soon as the first run completes. This ensures that jobs run
exactly as many times as scheduled and don’t run concurrently with
themselves.
If you set up pg_cron on a hot standby, then it will start running the
cron jobs, which are stored in a table and thus replicated to the hot
standby, as soon as the server is promoted. This means your periodic
jobs automatically fail over with your PostgreSQL server.
Source: citusdata.com