I am using Talend Open StudioV5.4 for Data Integration. I am providing a brief description of the jobs I created in Talend. I have created a job name Refresh_bFO_Table which connects to Salesforce.com and Fetch Data from Account and Opportunity Table and with those data its populate a local database named OFOTaccdb. In this Local database there are two tables bFO_Account and bFO_Opportunity which get populated with these data retrieved from SalesForce.
Now, for connecting to salesforce we have used OAuth2.0 for authentication. We have provided the consumerSecret, consumerkey, redirect URI in Talend tSalesforceComponent configuaration and jobs are running perfectly from Talend.
We have created a UI Interface using VB2010.so,after successful completion of Talend job we are exporting this job into Visual Studio's bin folder as executable code, now, when we are running the project from Visual Studio it gets an authentication error regarding OAuth2.0
Exception in component tSalesforceConnection_1
java.io.IOException: Server returned HTTP response code: 400 for URL: https://test.salesforce.com/services/oauth2/token
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
at org.talend.salesforce.oauth.OAuthClient.refreshToken(OAuthClient.java:104)
at ofotenhancement.refresh_bfo_table_0_1.Refresh_bFO_Table.tSalesforceConnection_1Process(Refresh_bFO_Table.java:533)
at ofotenhancement.refresh_bfo_table_0_1.Refresh_bFO_Table.runJobInTOS(Refresh_bFO_Table.java:2318)
at ofotenhancement.refresh_bfo_table_0_1.Refresh_bFO_Table.main(Refresh_bFO_Table.java:2169)
I have inspected the source code of tSalesforceComponent from Talend with line no which VS2010 is mentioning in this exception.Source code is as follows:
token_tSalesforceConnection_1 = oauthClient_tSalesforceConnection.refreshToken(storedRefreshToken_tSalesforceConnection_1);
Well my refresh token is in a property file named token.properties:
#Wed Jun 18 10:34:11 IST 2014
refreshtoken=5Aep861vKMLoVl4zz9FUGsd2kHAPryhoIRCvNajnyB2TR4OVopNSy
N1zDjeFZfa6Ehq0A1IPVRX_JcWtgvV6pgk
Jobs are running properly from Talend; the problem is occurring from VS2010.
Related
I've imported an Azure SQL DB into VS so i can work in Azure Devops with it.
I'm getting this error when i try to build the DB:
SQL71589: Master Key must be created before a database scoped credential
The DB has a master key, which was created before the user, credential, and external data source which uses it.
I've tried importing into VS with just the master key and no DB Scoped cred, and get no error, then when i create the cred and load it the error comes back. So it seems to be with the creation of the cred.
Also when i click into the error it says there are syntax errors on my cred script, but I've checked against MS docs and it seems fine.
I've tried dropping and recreating in this order the master key, user, credential and external data source.
Seems to be a couple of threads here about this but they haven't helped so far.
I'm using VS 2019 v16.5.4
Any help would be much appreciated.
Open the file that caused the error. Add the creation of Master Key to the top of the file before the Scoped Credential and save it. After the change the content of the file will look like below:
CREATE MASTER KEY ENCRYPTION
BY PASSWORD = '{Your Password}'
GO
CREATE DATABASE SCOPED CREDENTIAL [Your Credential]
WITH IDENTITY = N'SHARED ACCESS SIGNATURE';
Build the project, the build should now be successful.
After performing an upgrade of a TFS server to 2018 Update 2 the search and indexing seems to be broken on one of our environments.
Any search gives "We encountered an unexpected error when processing your request" and I have work through all the troubleshooting docs to clean and reindex all collections. Also completely reinstalled the search package to the separate server we run for search and indexing to make sure we got the correct version running.
In the event logs on the TFS App Server a large number of these exceptions are logged:
Events (81277) completed with status FailedAndRetry. Event 81277
completed with message 'BeginBulkIndex-PushEventNotification: The
operation did not complete successfully because of exception
Microsoft.VisualStudio.Services.Search.Common.FeederException: Lots of
files rejected by Elasticsearch, failing this job. Failure Reason :
Microsoft.VisualStudio.Services.Search.Common.SearchPlatformException:
ES Exception: [HTTP Status Code: [200] BULK_API_ERROR: [ index
returned 404 _index: codesearchshared_1_0 _type:
SourceNoDedupeFileContractV3 _version: 0 error: Type:
type_missing_exception Reason: "type[SourceNoDedupeFileContractV3]
missing"
And another exception type also logged a lot of times indicating failure to index work items:
Microsoft.VisualStudio.Services.Search.Common.SearchPlatformException:
ES Exception: [HTTP Status Code: [200] BULK_API_ERROR: [ update
returned 404 _index: workitemsearchshared_0_2 _type: workItemContract
_version: 0 error: Type: type_missing_exception Reason: "type[workItemContract] missing" update returned 404 _index:
workitemsearchshared_0_2 _type: workItemContract _version: 0 error:
Type: type_missing_exception Reason: "type[workItemContract] missing"
The exceptions seems to indicate that some type registrations are missing like the workItemContract and SourceNoDedupeFileContractV3 but I can not find any errors on the search server installation logs.
Anyone got some suggestions on how to solve this and get the Elastic Search back into a working state?
We resolved to situation by completely uninstalling and then reinstalling everything related to search.
Uninstalled all Code/Work/Wiki extensions from all collections from the extension management in web admin
Removed the TFS Search Service feature from the TFS Admin Console.
Uninstalled the elasticsearch service from the separate search server, using the PowerShell script .\Configure-TFSSearch.ps1 -operation remove
Restart TFS Job Agent service
Deleted old Search related databas content from ALL collection databases using
DELETE FROM [Search].[tbl_IndexingUnit]
DELETE FROM [Search].[tbl_IndexingUnitChangeEvent]
DELETE FROM [Search].[tbl_IndexingUnitChangeEventArchive]
DELETE FROM [Search].[tbl_JobYield]
DELETE FROM [Search].[tbl_TreeStore]
DELETE FROM [Search].[tbl_DisabledFiles]
DELETE FROM [Search].[tbl_ItemLevelFailures]
DELETE FROM [Search].[tbl_ResourceLockTable]
Restart TFS Job Agent service
Reboot Search server
Run Configure Search feature wizard from TFS Admin Console, using existing search server
Install search package according to instructions by PowerShell script
.\Configure-TFSSearch.ps1 -Operation install -TFSSearchInstallPath
D:\ES -TFSSearchIndexPath D:\ESDATA -Port 9200 -Verbose
Completed Search configuration wizard from TFS Admin Console enabling code search for all existing collections
Checked that services were running and tested searching from the web application, it works!
According to your error info and TFS version, this issue similar to Unable to start search after upgrade to TFS 2018 Update 2
Try the solution in the question below:
It seemed that I had an invalid/problematic setting in following reg
key that update/install did not fix.
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Apache Software
Foundation\Procrun
2.0\elasticsearch-service-x64\Parameters\Java\Options
Value contained:
-Xms0m
-Xmx0m
Changing both from '0m' to '1g' fixed the issue. As far as I
understand '0m' defaults to 4GB, which might collide with my server
only having 2,8 GB of RAM. I will upgrade the server to follow the
minimum requirements.
Maybe the configuration tool could warn about this problem or set the
value to something that is possible.
Also take a look at this article, which maybe helpful: Elasticsearch 6.2.2 fails to run as a windows service
[This post refers to azure devops server 2019 to 2020 upgrade]
I also got
"We encountered an unexpected error when processing your request"
for any search - this was after migrating azure devops server 2019 to 2020 on a new box (was unable to perform the upgrade on the same box)
http://localhost:9200/_cat/health?v showed the only cluster TFS_Search_AZURE-DEVOPS in red status
In my particular case everything on the same box - windows 2019 + sql server 2019 + search service
Followed Per Salmi instructions and unfortunately that did not resolve the issue.
The solution in my case was rebuilding elastic search indexes (details and scripts):
Reindex repo
To reindex a Git or TFVC repository, execute the appropriate version of the script Re-IndexingCodeRepository.ps1 for your Azure DevOps Server or TFS version with administrative privileges. You're prompted to enter: [my values in brackets]
The SQL server instance name where the Azure DevOps Server or TFS configuration database is [azure-devops]
The name of the Azure DevOps Server or TFS collection database [Tfs_DefaultCollection]
The name of the Azure DevOps Server or TFS configuration database [Tfs_Configuration]
The type of reindexing to execute. Can be one of the following values: Git_Repository TFVC_Repository [Git_Repository]
The name of the collection [DefaultCollection]
The name of the repository to reindex. [your repo name here]
Reindex collection
To reindex a collection, execute the script TriggerCollectionIndexing.ps1 with administrative privileges. You're prompted to enter: [my values in brackets]
The SQL server instance name where the Azure DevOps Server or TFS configuration database is [azure-devops]
The name of the Azure DevOps Server or TFS collection database [Tfs_DefaultCollection]
The name of the Azure DevOps Server or TFS configuration database [Tfs_Configuration]
The name of the collection [DefaultCollection]
The entities to reindex. Can be one of the following values: All Code WorkItem Wiki [All]
The scripts above place reindexing jobs that typically take a few minutes to complete.
Right after execution I was getting an encouraging "We are not able to show results because one or more projects in your organization are still being indexed"
And after a few minutes results started to come in, http://localhost:9200/_cat/health?v showing the only cluster TFS_Search_AZURE-DEVOPS in green status
> [OLE DB Source [113]] Error: SSIS Error Code
DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The
AcquireConnection method call to the connection manager "msAccess"
failed with error code 0xC0202009. There may be error messages posted
before this with more information on why the AcquireConnection method
call failed. .
I am new in SSIS Packages designing for learning purpose I am Designing dtsx packages. i have a .mdb file that I am trying to import in my sql server 2016 . when I design packages it running successfully but when I deploy that packages and try to run that packages through package utility it showing me above error.
i searched a lot i changed my packages properties like delay validation= true and validate external metadata = false.
But after deploying packages data is not loading into my respective table.
The most likely answer is that you don't have the .mdb file in the same local folder on the SQL Server:
C:\Users\Administrator\Desktop\msAccess.mdb
When you deploy the package and run it through the package utility, the package is running ON the SQL Server, and not on your local box anymore. So any local path in the package will be interpreted as a local path on the SQL Server.
sorry for late reply. I gave SQL server agent permission access to my packages and they are running through my stored procedure and also my mdb file i gave access to SQL server agent to access that file –
I solved this issue by adjusting which elements of my connection managers are parameterized.
I learned that you should parameterize either the entire connection string or the individual elements of the connection string. Not both.
As soon as I removed the ConnectionString parameter it worked fine. I just have the Password and DataSource parameterized. I used to have those plus the ConnectionString parameterized. But it is not necessary and just trips up SSIS. Only parameterize what you absolutely need to.
I am in the process of migrating our existing TFS 2012 to Visual Studio Online, and I have already migrated 3 of our 5 projects from our own server.
The largest one of our projects, the fourth in the migration queue fails.
Here is the error received:
Error in execution of Job : teamfoundation\DefaultCollection to [Name
Removed].visualstudio.com _1| teamfoundation\DefaultCollection to
[Name Removed].visualstudio.com _1| TFS Source 1412855236297 Source
TFS 1412855236297 'WebSite' Commit Information
teamfoundation\DefaultCollection to [Name Removed].visualstudio.com
_1| TFS Source 1412855236297 Source TFS 1412855236297 TFS Commit Information
Fatal Error: One of the end-point not available. Server Error: OpsHub-012013: Process execution failed on node: callMethod, in
process definition: 'SCM Default Integration Workflow'; nested
exception is : script evaluation exception
I am running OpsHub Integration Manager Framework-OVSMU-V1.0.1.006
Here is an image of the error received
I have tried to restart the process twice, and it times out after about 20 minutes - And gives the same error.
The error message indicates loss of connection to one of the end point. Can you please ping your end points for some while and see if the response is consistent?
Background:
I am using the deployment tools in Visual Studio 2010.
I right clicked my project and selected Package/Publish settings. Put all my settings in there ...
I am then using "web deploy" to tranfer the files to my remote server running a remote agent service and this is working fine. The transforms i have on my Web.Release.config do their thing and the server can access the database I created manually.
Problem:
My next step was to get the Database Deployment working too.
I went into the Package / Publish SQL tab and entered my Connection string for the destination database.
(Data Source=MyDBServer;Initial Catalog=Database2;User ID=User;Password=pass)
This database is empty ready to accept the import.
I also enter in the connection string for the source database. This lives on the same server.
(Data Source=MyDBServer;Initial Catalog=Database;User ID=User;Password=pass)
Database Scripting options are set to Schema and Data (changing this makes no difference) and the database scripts are set to [Auto Generated Schema and Data]
When i deploy this now, i get the error:
Error 4 Web deployment task
failed.((09/06/2010 16:41:51) An error
occurred when the request was
processed on the remote computer.)
(09/06/2010 16:41:51) An error
occurred when the request was
processed on the remote computer. The
entry type 'Unknown' was not expected
at this time. The serialization stream
may be corrupted.
Additional Info:
I can successfully create a package with no problems. I looked at the contents in the zip and can see the SQL is generated fine (so no problems connecting to the database). I can then copy this SQL and run it as a new query on the new database and the tables and data are created fine.
I can not seem to work out where this is going wrong, i googled the error and there are no entries on the whole internet. Anyone have any ideas?
Addendum:
To get some further idea of what might be going on, i sent the package across to the server and imported it using IIS. It told me i needed SQL Server Management Objects. So I installed that.
Next attempt it told me my user did not have permission to create the database, I thought excellent this must be the problem. :Granted access - Re-run. Passed!
So i deleted all the tables and went back to VS2010 clicked publish and i get the same error. :(
Sorted it!
Thank goodness, i was totally out of ideas when i went back to a video by hanselman. He mentioned that the Web Deployment Agent can have permissions. I went in had a look and there was a tab in it's properties called log on.
I entered the detials of an account with a decent level of access and clicked okay.
I then restarted the service as requested to enable the changes.
I then went back to VS 2010 and clicked Publish Web.
Music to my eyes, i see the words "Publish succeeded", I check the database and the tables are there. Excellent!
I think i scared the office by getting a little over excited, if you get this problem and this solution fixes it for you, try to hold in the temptation to shout out "YES!, yes, get in!" while laughing maniacally or people will think you're weird like me.