Edit
I've discovered that the service is not actually performing any database calls when running my process but it does for the normal application. I am going to troubleshoot a problem with my SSIS package / code under the assumption that something is silently failing or looping there it shouldn't.
Original Question
I have an SSIS package that has a table of user accounts. I need to loop through them (386,000 rows) and call a WCF service to create the users in our system. The WCF call is sub-second but even at 1/2 second this process would take over 50 hours. To improve through-put I modified the package to divide the result set into 5 batches and process them asynchronously.
The process works and I get the expected 1/5 run time in Visual Studio debugging. Problem is that after 40,000ish users my VS will crash.
So, I attempted an execution using the SQL Server Execute Package Utility but the iteration run time seems to be growing and is destroying any improvement that I made.
Execution start
Execution after 2 hours of processing
Any solution to my problem would be much appreciated, thank you.
Related
I tried to sync database on Visual Studio 2015 after creating a project, EDT, Enum and a Table in order to create a new screen on Dynamics 365.
When I tried to synchronize it, it was stopped in the middle during schema checking process. Though it seems that the DB synchronization doesn't have problem for the first few minutes, it always stops during this process as I describe below.
Log Details:
"Schema has not changed between new table 'DPT_TableDT' and old table
'DPT_TableDT' with table id '3997'. Returning from
ManagedSyncTableWorker.ExecuteModifyTable() Syncing Table Finished:
DPT_TableDT. Time elapsed: 0:00:00:00.0010010"
Could you tell me how to solve this issue?
Thanks in advance.
Full database synchronization log
DB Sync Log
From what you've described and also shown in your screenshot, this does not look like an error but is simply describing X++ and Dynamics AX/365FO behaviour.
When you say that it "doesn't have a problem for the first few minutes" I'm guessing you're just not being patient enough. Full database syncs should generally take between 10-30 minutes, but can take shorter or longer depending on a variety of factors such as how much horsepower your development environment has, how many changes are being sync'd etc. I would wait at least one hour before considering the possibility that the sync engine has errors (or even run it overnight and see what information it has for you in the morning).
The message you've posted from the log ("Schema has not changed") isn't an error message; it is just an informational log from the sync engine. It is simply letting you know that the table did not have any changes to propagate to SQL Server.
Solution: Run the sync overnight and post a screenshot of the results or the error list window in Visual Studio.
I've recently been stymied by a long running application where Access v2003 replicas refused to synchronize. The message returned was "not enough memory". This was on machines running Windows 10. The only way I was able to force synchronizing was to move the replicas onto an old machine still running Windows 98 with Office XP, which allowed synchronizing and conflict resolution. When I moved the synchronized files back to the Windows 10 machine they still would not synchronize.
I finally had to create a blank database and link to a replica, then use make-table queries to select only data fields to create new tables. I was then able to create new replicas that would synchronize.
From this I've come to suspect the following:
Something in Windows 10 has changed and caused the problem with synchronizing/conflict resolution.
Something in the hidden/protected fields added to the replica sets is seen as a problem under Windows 10 that is not a problem under Windows 98.
One thing I noticed is that over the years the number of replicas in the synchronizing list had grown to over 900 sets, but the only way to clear the table was to create a new clean database.
I am using Visual Studio 2015 Community. Problem is when I start VS2015 it start quickly but when I open website project then it takes too much time apprx (30-60 mins) to open particular code file (Only for first time). Once it loads then it works normally. How do I resolve this problem?
It's very difficult to give an correct answer without more details as there might be a lot of things that can happen but I suggest you check the logs from Process Monitor.
It will log all the events that happening in your OS.
After gathering the logs you need to go through them (try to filter them - there might be a LOT of them) and check for some indications of WAIT operation.
Maybe some network operation is time-outing (but 30-60 min is quite huge timeout) or some IO operation.
No matter what I do, which DB I connect with, EF seems to take around 15-30 minutes to generate a model. While it's doing this, I get a "Visual Studio is busy" message in the system tray.
The first DB I connected to was complex and had a lot of data, lots of views so I thought, may be that's why. Now I have a local DB file with 1 table that has 2 columns and 3 rows. It still takes the same amount of time.
Eventually VS crashes and restarts. Has anyone had this problem before? Any idea?
I've looked at resource monitor, devenv.exe does not seem to be consuming any resources that would indicate it's doing a lot of work.
What credentials do you use to access your DB? Let's try ruling out domain latency issues first. If this is at work, can you verify the same flow on 2 separate machines using the same domain credentials? This wouldn't apply if you were using local creds.
Turns out I had a Visual Studio DVD in my DVD drive. Each time I did something with EF, VS started to read from the disc. I have no idea what or why, but the little LED would blink. Once I ejected the disk, everything ran fine. Go figure!
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Service runs then dies.
Hello,
I'm running Win 7 Pro 64-bit. I wrote a service in C# using the .NET 4 framework. It installs properly and starts to run. I know that it runs because it writes some output to a log file. However, after a few seconds it dies. When I use Visual Studio 2010 Pro to run this same code not as a service it never dies. So, my obvious question is regarding the appropriate approach for debugging this since I can't figure out why it should dies as a service but not die as a non-service. I've put writes to the log file in several places in the code but it seems to die in a different place every time. The application has 3 threads. Any suggestions are welcomed.
Thanks,
Ray
I suspect some sort of thread unsafe code. This is a typical symptom of code that is not thread safe. At some point two or more threads crash into each other (such as when one iterates a list, and another modifies that list).
When you are doing debugging in VS the code runs significantly slower, so it is less likely/takes longer for the threads to crash.
I'm executing a test-execution project using visual studio 2005 (using the pnunit framework and C# but its not relevant). The total execution time is more than 40 hours. In between if there are any environment problems on the network (external factors like remote server is down, db is down etc) the code stops executing. This creates a problem because when I notice the error I'll have to manually set external factors right and again resume the run. So it becomes important to keep checking the execution progress frequently to make sure we are not losing time due to halted execution, which is troublesome.
I looking for a way to somehow either triggering an email/batch script/exe/anything when the code stops running. Is there any way I could achieve this? Any ideas?
Thanks,
Mugen
If you only need to trigger an email at the end of the application, then I would suggest putting the code here at the end of your execution.