I'm having a very strange problem. I'm using dfs-datastores Pail abstraction to write data to HDFS in Java. I don't think the Pail piece is important to the problem though.
When it calls org.apache.hadoop.fs.FileSystem getFS(java.lang.String path) with a path on my local filesystem it pauses for about 2 minutes seemingly doing nothing then returns. This is on my laptop.
The weird thing is that it worked really fast when I was on the network at my office today, but now that I'm home it's doing it again. I'm running Ubuntu 10.10 64-bit with Java 1.7.
Anyone have any ideas what it's doing? What could be different between being at work and being at home?
UPDATE:
I've been stepping through code with the debugger and it seems to be having trouble in Configuration.loadResource(). It's calling that multiple times and it will take 5-10 seconds to return from that function.
UPDATE2:
I've narrowed this down a little further. The biggest hang up seems to be when it calls KerberosName.setConfiguration(). Which would explain why it runs fast at work since the Active Directory acts as a Kerberos server. I don't have one here at home, so it can't find one. Now they question is why in the world it's trying to load the Java Kerberos stuff.
I found a solution (or at least a work around). I installed the krb5-kdc package and now my little program runs fast without any unexplained pauses. After this I removed krb5-kdc, tested and it was still running fast. I removed /etc/krb5.conf and it started doing the pause again. It looks like using the Hadoop library on Ubuntu (at least) requires a /etc/krb5.conf file.
Maybe this will help someone else.
Related
Longshot, but anyone had this issue?
I have code running (or not) which is set to run on the GPU. It fails to run unless I bump the GPU to make it run a little bit, for example by watching YouTube or playing a game.
I believe there may be some computer resource/application prioritisation/limitation configurations causing vscode to stop running, and wondered if anyone else had run into this issue.
I am running the code in a .ipynb notebook file in vscode (not sure if that might contribute to the issue). Sometimes the code will just freeze permanently, and I usually go about restarting the code to get things going properly.
The code should normally takes about 7 seconds for the training epoch, and 0.7 seconds for the validation epoch. But I was away for the first epoch and found it hadn't started, and so I opened up Youtube and it began.
Code timings
I can't think of what settings to change for this, but have tried a few
Power options
Anyone had a similar issue before? My second theory is that I think perhaps I am using too much GPU ram in my python code which is slowing it down and effectively made it freeze. And then when I load another application to use the GPU it forces the GPU ram to reconfigure and somehow this RAM reconfiguring might be unblocking the GPU allowing it to run again.
I am working on Laravel with docker.
If I run php unit test in mac os, it takes few seconds.
However on windows 10, it takes few mins.
Is there anyway to fix this problem?
Thanks.
If you're running on a non-Linux OS, Docker has to virtualise your file system, and this requires a certain amount of time per file. For programs that are compiled into one executable, this is less of a problem at runtime (but clearly with its own compilation-time implications), but for scripting languages like PHP this can mean that every request runs super slowly, since every file that is used has to be 'translated' every time it is read. This is also a problem on Docker for Mac (so you're actually experiencing problems there, too, but less so, since at least it's a Linux system under the hood.) Linux is, I believe, completely virtualised on Windows which is going to add even more time.
This Reddit discusses the problem to an extent:
https://www.reddit.com/r/docker/comments/7xvlye/docker_for_macwindows_performances_vs_linux/
With this being particularly interesting (I have not tried it myself):
https://nickjanetakis.com/blog/setting-up-docker-for-windows-and-wsl-to-work-flawlessly
There is also a good community-created solution which we have used to solve our Docker for Mac problem. I don't see why their Windows options wouldn't work similarly well in your case. You can find it here:
https://github.com/EugenMayer/docker-sync/wiki/docker-sync-on-Windows
It basically sets up an intermediate service that copies all the files over into an intermediate volume (that uses the 'correct' filesystem) only when the file is updated, therefore speeding up run speed immensely.
I know it looks like quite an intimidating process, but this problem is fundamental, so you're going to have to do a certain amount of work to fix things!
FWIW I had that working on Docker 4 Mac, but it added a layer of complexity to our dev process that I found annoying, so in the end I've got myself a Linux box for work. To be honest, installing Linux as dual boot on my Windows machine (which has been my at-home solution) was probably easier than tweaking Docker 4 Mac to my satisfaction, so you might want to consider that. I have used this page twice:
https://itsfoss.com/install-ubuntu-1404-dual-boot-mode-windows-8-81-uefi/
And it's worked fine each time. One caveat - it suggests a low amount of disk for your root (/) volume, but Docker gets mounted on root so give it around 100G (not the 10-20G that page recommends.)
Since some years ago I started using Qt in both Windows 7 as well as in Linux Ubuntu and it would always compile fast with MinGW being used for Windows. But in the last couple of years or so, maybe thanks to updates in the version of both Qt and MinGW, I started detecting a slow down in the compiling speed inside Windows. I did some research trying to find why MinGW had started to become so slow compared to Linux (it wasn't before!) and everything people told me was that MinGW was slower in Windows and that it would be better, if possible, to just use Linux.
Since I wanted to continue my project, I followed the suggestion and since I've being using Linux with relatively no problems. The situation now is that I must go back to Windows (now updated to Windows 10) to make visual corrections for this OS and I need to once again work with MinGW having to face the same problem as before.
But for some reason it seems that the slowness of MinGW became even worse! While before I at least was able to compile the app in around 4 minutes, now the last time I tried it took 38 minutes before I gave up and went to sleep - and this is for a project that takes only 1:03 minute to be compiled in Linux [under the same compile configuration]!
Well I'm still aware about the slowness of MinGW, but as a quick research around this problem on the web reveals, that is just too slow: all backtesting one can find in other threads here on SO reveals at best 2x-3x more time to compile a project, not 38x+!!
So I would like to know what kind of possible problems I might have in my Windows for this exaggerated slowness to happen. I know I ended up installing at least 4 different versions of MinGW; could this have brought the problem?
It's interesting also to notice that when compiling using the -j option and watching the Compile Output log in Qt Creator alongside Process Explorer, there are moments when the compiling simple pauses for 10 seconds or more and the CPU usage drops from its ~100% to close to 5% with nothing happening till it suddenly continues the compilation process. I'm sure this constant pauses are part of the above average time, but I have no idea why MinGW is showing this behaviour.
You might want to check where the time is spent.
There a lot of tools that allow you to capture what a certain process is doing, I name just two of them:
ProcMon
XPerf or its successor
But to analyze the reports generated by these tools you need a rather deep understanding. If this doesn't help temporarily disable other running services and program step-by-step (if you want to know which program causes the problem) or disable all of them at once.
Looking at the spikes of cpu usage that TaskManager or Procexp by sysinternals show might help too to identify those components that block your cpu.
If your antivirus is the cause of the collision that makes the compile so slow you can define exceptions, then the antivirus will not scan certain programs or paths.
So perhaps it is easier to first try the compilation process with a disabled antivirus software or even from a clean live boot Windows CD.
I installed Eclipse Galileo and after trouble with the JDK, its starting well. But I have big problems with performance. Every third second, Eclipse is hanging for a while. It runs not smoothly. I need a efficient IDE as Eclipse for work. So, it would be very nice when you have a fast answer :)
Both Eclipse as the JDK are 64-bit versions.
Have you any ideas?
Update:
I can´t really explain the problem from scratch. But in my case, it was a trouble between Eclipse´s and the auto-complete-function of my OSK. If I disabled auto-complete, there was no hangs anymore. I don't know why the using of the OSK blocks the thread (?) of the whole editor.
Maybe anyone of you, has an idea why?
From your description it sounds like the garbage collector is being triggered. How much RAM have you got in the system? Depending on the plugins you're loading Eclipse can need quite a lot of it. I think the bare minimum is 256 Mb, and realistically you need at least 1 Gb, more if you're doing web development
Have you got an up-to-date JVM? Eclipse generally runs much quicker with a 1.6 JVM.
One other thing to check, do you have an aggressive virus scanner? Eclipse plugins are collections of small files in jars, some virus scanners can really slow down the performance. If you are able, remove the Eclipse install directory from the scanned files.
See this EclipseZone article or this question for some general performance tips.
Run Process Monitor and see what kind of system calls and/or file system calls the JVM is doing. Use filters aggressively to pinpoint a specific process. I had a similar issue where a graphics card utility triggered a flood of registry lookups for every UI update which just made Eclipse incredibly slow. (Somehow SWT was hit exceptionally hard by this bug, I'm not sure why.)
EDIT: I meant "Process Monitor", not "Process Explorer". But the link was correct.
You could try to run it from within a virtual machine set up on your computer to see if the problem is still there. If it's not, it might be faster for you to just work from within the virtual machine environment. Doesn't address the issue, but it may help avoid it altogether.
I had same problem so I just switched to the 32 bit version of Eclipse and it runs fine with no performance issues.
I can´t really explain the problem from scratch. But in my case, it was a trouble between Eclipse´s and the auto-complete-function of my OSK. If I disabled auto-complete, there was no hangs anymore. I don´t know why the using of the OSK blocks the thread (?) of the whole editor.
Maybe anyone of you, has an idea why?
Thanks for any help!
Same problem for me
I have Windows 7 professional 64 bit and 8gb of RAM
Eclipse is extremely slow, probably 5 times slower than the Windows Vista 32 bit machine I have recently upgraded from (Europa version) - and that machine was a complete dog!
Adding -Xmx1024m -XX:+UseParallelGC -vm C:\Program Files\Java\jdk1.6.0_20\jre\bin\server\jvm.dll has made a pretty big difference
I have same problem as not respoinding.
I searched in internet for a solution. I found one by adding the below to
eclipse helios config file.
-vm
C:\Program Files\Java\jre7\bin\javaw.exe
Initially it looks Ok to start and click on the different buttons and
running on several files in eclips project. But when I click on debug
and step by step process. Then it is again showing not respoding.
I have a new laptop win7 installed.
I have the same problems with the 32 bit version, running with a 32 bit JVM.
It's more that my RCP Application which I developed with Eclipse is slow. I've tried both -Xmx1024m and -XX:+UseParallelGC, with no noticable effect. Has this issue been registed with eclipse.org?
How can I find what's hanging all new installations on a Windows box?
While testing an installation script on Windows (XP Pro, if it matters) I've run into a situation wherein any and all attempts to install anything on the system hang waiting on who knows what. When the system is restarted, all queued up attempts at installation then go through their exit paths with pop-ups that report the installation is being aborted due to system shutdown having been requested. Of course, reboots do not cure the problem. The system otherwise runs fine.
So... How can I determine what part of the OS I've wedged? (Something in the registry, I suppose, but I'm a real greenhorn when it comes to Windows.) Most likely, something from a preceding install attempt went awry and is now blocking even though I saw no errors reported. Once I figure this out, I want to put in a check for this sort of thing, possibly at both ends of my install scripts, if that seems reasonable.
Thanks for your input.
UPDATE:
Unfortunately for me, rebuilding from scratch to get to the point the system's in now is about 9 hours. I'd like to unwedge it from where it is now rather than reload (again). Procmon seems great but I haven't got SP2 installed, only SP1! -frown- So, other ideas are welcome.
I assume you've tried logging the install to see where things go wrong?
Try rolling back to before things went wrong using "System Restore", if that doesn't solve it and the MSI log files show nothing useful then I'd take the plunge and reload before wasting any more time on it.
That said, if you're developing installers then taking an image of this PC in it's crappy state could be a worthwhile exercise. Some point in the future when you have more time to debug you can try and figure out what the problem is.
P.S. I'm assuming you're asking this question from the point of view of someone developing an installer and not as a tech-support question... otherwise this question should probably be closed as not-programming-related ;)
Try using Procmon to figure out where the installer is having problems, if you set a filter it will report all file and registry activity for that process.