i have query which give me information of ETL Error like this:
i create a Job: tOracleinput --->tsendMail which sends this Error logs to my Email.
What i want now is that: when i execute the Job again and again, it sends me only new Error logs. The Error logs which has been sent, will be not send again.
Can you please help me?
There are several ways to accomplish this.
This is one. Pls treat this answer as an algo.
Create a variable, called ID_error_log = 1 in some properties file.
Read this ID from properties file at the start of ETL job.
In your ETL, when you're trying to fetch errorData, use this ID in query to fetch data like
select * from bla_bla_bla where id > context.ID
Send emails for these errors.
At the end of the job, update the ID_error_log in properties file to the latest ID for which you sent an email.
So next time, as the ID is updated - records will not be duplicated. Thanks.
What I understand from your question is that you are trying to persist the data that you what to send through mail.
One possible solution is to write the error logs into a file/database, and then read all of the data from this file/database, and send it into tSendMail.
Related
Situation: We get audit emails every morning. The body of the email contains a table with various columns. The column labelled 'ID' is the unique key for each row. I have to copy the data from the ID column, format it in note++, and then paste it in a pre-filled query in SQL were I run it.
Question: Is it possible to automate this process? if so, where could I start? I would be nice if I could have something that either runs automatically or manually, reads the email, extracts the data from the column, formats it, and throws it in a query and executes.
Additional Details: The emails are always from the same distro, fire at the same time every day, and the table columns are static.
My skill Level: Beginner but resourceful and eager to learn, so please don't crucify me if I am not clear.
Yes, it is possible. You can develop a VBA macro or a COM add-in if you need to distribute your solution on multiple machines. Basically you are interested in handling the NewMailEx event of the Application class which is fired when a new message arrives in the Inbox and before client rule processing occurs. You can use the Entry ID returned in the EntryIDCollection array to call the NameSpace.GetItemFromID method and process the item. This event fires once for every received item that is processed by Microsoft Outlook. The item can be one of several different item types, for example, MailItem, MeetingItem, or SharingItem.
After retrieving the incoming item you can process it by getting property values and adding them to the SQL Db.
You may find the Walkthrough: Create your first VSTO Add-in for Outlook article helpful.
I created a Flow where if a record is created in Common Data Service it will delay for 10 seconds an email containing the information from the record just entered. I had everything working up until I decided to make the email format a little better looking, deleted some dummy data I had entered and now it pops up this error in Flow:
The request failed. Response content: '{"status":412,"message":"A record with matching key values already exists.","source":"org50841ab4.crm.dynamics.com","errors":[],"debugInfo":"clientRequestId: dbdfdece-dad5-431e-abd7-ede83be8bdf2"}'.
From what I can tell the record is entered into Common Data Service just fine but I receive the same error every time. Can anyone help me with this?
I need to create to send a message to all new users registered on my website. I created a table called messages that admins can store (insert) the messages from the admin panel to this table and these messages are simply shown to all users with a foreach.
I don't know if this the best way to do something like that!
Anyway, the problem is that when any new user register and open his dashboard he just found the old messages
This is the table :
image
And this is the simple code for foreach
$msgForAll = Message::latest()->get();
I'm not sure how to display new messages to the users.
Again the way I made this idea is wrong, I know that ):
You can use eloquent's where spefically the whereDate queries in your case to get certain rows past or before date.
As an example relating to what you want to do, it can be along the lines of this:
// Given that you are using Auth to get the user's data...
// Get messages that were created after the user's creation date
$messages = Message::whereDate('created_at','>',Auth::User()->created_at)->get();
I am using the command :
# bin/connect-standalone.sh config/connect-standalone.properties config/debezium-config.properties
My debezium-config.properties is :
name=publications-connector
database.hostname=localhost
database.port=5432
database.user=andy
database.password=postgres
database.dbname=postgres
database.server.name=dbserver1
table.whitelist=public.publications
In Postgres, I have a table called publications
When I insert a new record into the publications table, I can see that my consumer shows the new message in json format.
However when I update an existing record into the publications, no new message is published to the topic and hence nothing to consume.
How can I fix this ?
Also, I would like to add another table 'comments' to my database. What changes do I need to make to the debezium-config.properties file or anywhere else to have those messages be published to its own topic ?
I see the following logs from the console :
WARN: no values found for table 'public.publications' from update message at 'source_info[server=dbserver1'db='postgres', .... schema=public, table=publications]'; skipping record (io.debezium.connector.postgres.RecordsStreamProducer:333)
Thanks,
After some detailed reading into many debezium topics, i was able to solve this by setting the REPLICA IDENTITY to FULL. Once I did this, on updates I was able to see the update messages and consume them.
I think the reason is this:
If a table does not have a primary key, the connector does not emit UPDATE or DELETE events for that table. For a table without a primary key, the connector emits only create events. Typically, a table without a primary key is used for appending messages to the end of the table, which means that UPDATE and DELETE events are not useful.
reference resources
How do I export the user emails from the database in a fast and easy way. I want the emails in a list so I can send out news letters. Thanks for help!
There's a button in the data browser to export data. I believe this goes to a .CSV file, so you can delete the other columns and then just use a batch email sender to send your newsletter.
OR
Create a program that queries the _User table and takes the email column and creates a .csv file.