I'm working on a uboot test application that will work with a special DMA engine. The DMA engine will transfer data between memories without "notify" cache. Therefore, I expect that if I keep transferring different data to the same destination, I should get the stale data.
However, I found that I always get the correct data the DMA engine sent. This makes me think that maybe the dcache is not enabled. So I tried the uboot build-in cmd - dcache. It shows my data cache is enabled. And I checked the TLB table and all pages are marked as "write back write allocate". So it means the cache is enabled?
And more interesting thing I found is that, I wrote a simple program that just keeps reading the same address. And I found that by disabling the dcache using the dcache cmd, the time to run the test just tripled. I tried a similar simple test in Linux on the same hardware and the cache can enable more than 15 times performance boost. So this must not be a hardware issue.
In summary, I found that my cache is working to some extent but not fully working. And it might be a configuration issue. Is there any theory can explain what I found? How can I continue to debug... Thanks
Let me answer it myself...
Code in Uboot is a little misleading... it run
set_section_dcache(i, DCACHE_WRITEBACK_WRITETHROUGH)
but after checking the MMU, it turns out that the memory type is set to be device.
Related
I am working with openmote-cc2538 for the project: https://github.com/cetic/6lbr/wiki
Here when I was trying to flash openmote with this command
sudo make TARGET=cc2538dk BOARD=openmote-CC2538 bootload0/dev/ttyUSB0 slip-radio.upload
This error message is displayed
ERROR:Cant CONNECT TO TARGET:Ensure bootloader is started(no answer on sync sequence)
cc2538 bsl.py script is available for uploading binary to openmote. But I don't have much experience about this. Can anybody give me some suggestions regarding how to proceed??
Grab a scope and probe different digital I/O pins on the CC2538. Typically a bootloader will initialize all the ports on the chip and you will see them change state, which is an indication that the bootloader is indeed running. I would guess one of the LEDs would change state as well (which doesn't require a scope).
Apparently the Windows file cache flushes data to disk asynchronously, even when using the synchronous WriteFile() API. Quoting "File Caching" on MSDN:
By default, [...] write operations write file data to the system
file cache rather than to the disk, and this type of cache is
referred to as a write-back cache.
Assuming that write-through and no-buffering flags are not used, what happens if the actual write to disk fails? Can clients be notified of such failures? What is the expected client error handling model for such failures? "Fire and forget" and "Write and pray" come to mind but maybe there is something else.
Secondary question: are there certain classes of errors that are guaranteed to be detected early? E.g. will WriteFile() always return an error if the disk is full? -- even though the actual write to disk would be deferred?
I would like to know how to write reliable file i/o that responds to these kinds of errors without disabling the Windows File Cache.
Bonus points: is this handled differently on other operating systems? Can you recommend a good resource on the topic?
In Windows 7, the user is notified via a pop-up dialog from the notification area.
Normal errors (such as the disk being full, lack of permissions, etc.) are reported back to the application immediately, these do not cause late failures.
Late failures can only happen in a handful of situations, such as a hardware failure or operating system crash. They can also happen when writing to a network share if the connection drops unexpectedly for any reason.
In most cases, it doesn't make sense for an application to worry about this. Data loss is to be expected under these circumstances; let the user deal with it.
If the data you are writing is unusually important, then you may need to worry, in which case you will have to use the write-through and/or no-buffering flags.
There is no third option.
Does anyone know how well tested LevelDB is and what is its status for use in production? It's a relatively new library and when I checked the source code it didn't appear to be handling errors too well. Does anyone use LevelDB in production and can comment on my question?
We use LevelDB in our website, but wrapped in SSDB(https://github.com/ideawu/ssdb), the LevelDB network server, with hash/zset data types support. Our SSDB instance serves 100 million queries per day.
LevelDB has a lot of high-visibility problems https://github.com/bitcoin/bitcoin/issues/2770 and the code is so poorly written that a bounty was needed to find a fix https://bitcointalk.org/index.php?topic=337294.0;all And the leveldb discussion group is predominantly bug reports about very fundamental database functionality that fails to work as advertised. https://groups.google.com/forum/#!forum/leveldb (e.g., "snapshots" aren't actually snapshots, and can be tainted by subsequent writes https://groups.google.com/forum/#!topic/leveldb/IAKJaL2zqZM etc...)
On the date that this question was asked, LevelDB was certainly NOT production ready and anyone who thought so was delusional. The code quality is abysmal, as confirmed by independent developers https://twitter.com/rescrv/status/406106256890286080
One place it is used in a production environment is the Bitcoin project. Within bitcoin, it's usage is critical for the security of the platform. See the release notes for Bitcoin QT 0.8.0
How do you qualify "relatively new" as it was out in 2011?
Can you please give more detail on "not handling errors too well"?
LevelDB is used as a backend in Riak and Hyperdex, which have both customised it to improve throughput under huge loads. There was a great video from Ricon East 2013 explaining the Riak changes made by Basho. (taken down at some point prior to 2019-03).
Note that RocksDB is another major fork, by Facebook, which is recommended for serverside. History of it forking from LevelDB is on WikiPedia. You can read about how RocksDB handles errors on this page:
Currently in RocksDB, any error during a write operation (write to
WAL, Memtable Flush, background compaction etc) causes the database
instance to go into read-only mode by default and further user writes
are not accepted....
Call DB::Resume() to manually resume the DB and put it in read-write
mode. This function will clear the error, purge any obsolete files,
and restart background flush and compaction operations. At present, it
only supports resuming from background errors that happen during
compaction. In the future, we will add more cases.
I implemented a web application to start the Tomcat service works very quickly, but spending hours and when more users are entering is getting slow (up to 15 users approx.).
Checking RAM usage statistics (20%), CPU (25%)
Server Features:
RAM 8GB
Processor i7
Windows Server 2008 64bit
Tomcat 7
MySql 5.0
Struts2
-Xms1024m
-Xmx1024m
PermGen = 1024
MaxPernGen = 1024
I do not use Web server, we publish directly on Tomcat.
Entering midnight slowness is still maintained (only 1 user online)
The solution I have is to restart the Tomcat service and response time is again excellent.
Is there anyone who has experienced this issue? Any clue would be appreciated.
Not enough details provided. Need more information :(
Use htop or top to find memory and CPU usage per process & per thread.
CPU
A constant 25% CPU usage in a 4 cores system can indicate that a single-core application/thread is running 100% CPU on the only core it is able to use.
Which application is eating the CPU ?
Memory
20% memory is ~1.6GB. It is a bit more than I expect for an idle server running only tomcat + mysql. The -Xms1024 tells tomcat to preallocate 1GB memory so that explains it.
Change tomcat settings to -Xms512 and -Xmx2048. Watch tomcat memory usage while you throw some users at it. If it keeps growing until it reaches 2GB... then freezes, that can indicate a memory leak.
Disk
Use df -h to check disk usage. A full partition can make the issues you are experiencing.
Filesystem Size Used Avail Usage% Mounted on
/cygdrive/c 149G 149G 414M 100% /
(If you just discovered in this example that my laptop is running out of space. You're doing it right :D)
Logs
Logs are awesome. Yet they have a bad habit to fill up the disk. Check logs disk usage. Are logs being written/erased/rotated properly when new users connect ? Does erasing logs fix the issue ? (copy them somewhere for future analysis before you erase them)
If not. Logs are STILL awesome. They have the good habit to help you track bugs. Check tomcat logs. You may want to set logging level to debug. What happens last when the website die ? Any useful error message ? Do user connections are still received and accepted by tomcat ?
Application
I suppose that the 25% CPU goes to tomcat (and not mysql). Tomcat doesn't fail by itself. The application running on it must be failing. Try removing the application from tomcat (you can eventually put an hello world instead). Can tomcat keep working overnight without your application ? It probably can, in which case the fault is on the application.
Enable full debug logging in your application and try to track the issue. Run it straight from eclipse in debug mode and throw users at it. Does it fail consistently in the same way ?
If yes, hit "pause" in the eclipse debugger and check what the application is doing. Look at the piece of code each thread is currently running + its call stack. Repeat that a few times. If there is a deadlock, an infinite loop, or similar, you can find it this way.
You will have found the issue by now if you are lucky. If not, you're unfortunate and it's a tricky bug that might be deep inside the application. That can get tricky to trace. Determination will lead to success. Good luck =)
For performance related issue, we need to follow the given rules:
You can equalize and emphasize the size of xms and xmx for effectiveness.
-Xms2048m
-Xmx2048m
You can also enable the PermGen to be garbage collected.
-XX:+UseConcMarkSweepGC -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled
If the page changes too frequently to make this option logical, try temporarily caching the dynamic content, so that it doesn't need to be regenerated over and over again. Any techniques you can use to cache work that's already been done instead of doing it again should be used - this is the key to achieving the best Tomcat performance.
If there any database related issue, then can follow sql query perfomance tuning
rotating the Catalina.out log file, without restarting Tomcat.
In details,There are two ways.
The first, which is more direct, is that you can rotate Catalina.out by adding a simple pipe to the log rotation tool of your choice in Catalina's startup shell script. This will look something like:
"$CATALINA_BASE"/logs/catalina.out WeaponOfChoice 2>&1 &
Simply replace "WeaponOfChoice" with your favorite log rotation tool.
The second way is less direct, but ultimately better. The best way to handle the rotation of Catalina.out is to make sure it never needs to rotate. Simply set the "swallowOutput" property to true for all Contexts in "server.xml".
This will route System.err and System.out to whatever Logging implementation you have configured, or JULI, if you haven't configured.
See more at: Tomcat Catalina Out
I experienced a very slow stock Tomcat dashboard on a clean Centos7 install and found the following cause and solution:
Slow start up times for Tomcat are often related to Java's
SecureRandom implementation. By default, it uses /dev/random as an
entropy source. This can be slow as it uses system events to gather
entropy (e.g. disk reads, key presses, etc). As the urandom manpage
states:
When the entropy pool is empty, reads from /dev/random will block until additional environmental noise is gathered.
Source: https://www.digitalocean.com/community/questions/tomcat-8-5-9-restart-is-really-slow-on-my-centos-7-2-droplet
Fix it by adding the following configuration option to your tomcat.conf or (preferred) a custom file into /tomcat/conf/conf.d/:
JAVA_OPTS="-Djava.security.egd=file:/dev/./urandom"
We encountered a similar problem, the cause was "catalina.out". It is the standard destination log file for "System.out" and "System.err". It's size kept on increasing thus slowing things down and ultimately tomcat crashed. This problem was solved by rotating "catalina.out". We were using redhat so we made a shell script to rotate "catalina.out".
Here are some links:-
Mulesoft article on catalina (also contains two methods of rotating):
Tomcat Catalina Introduction
If "catalina.out" is not the problem then try this instead:-
Mulesoft article on optimizing tomcat:
Tuning Tomcat Performance For Optimum Speed
We had a problem, which looks similar to yours. Tomcat was slow to respond, but access log showed just milliseconds for answer. The problem was streaming responses. One of our services returned real-time data that user could subscribe to. EPOLL were becoming bloated. Network requests couldn't get to the Tomcat. And whats more interesting, CPU was mostly idle (since no one could ask server to do anything) and acceptor/poller threads were sitting in WAIT, not RUNNING or IN_NATIVE.
At the time we just limited amount of such requests and everything became normal.
If you are using XCF you can write all your data into an Active Memory Server. Other processes can now access all this data. Taking the first steps with XCF, my question now is what steps have to be done to get some example data in the active memory?
To get a short answer to this question to give some pointers for the active memory stuff:
First you have to create a working xcf Environment. This is mainly configuring your dispatcher.
Then you have to run an active memory server. You can use the command line tool xcfinfo to look if your active memory server is active in your xcf environment.
If your memory server is running you can do some initial testing with the java memory interface tool to see how and if your server is working. For inserting and listenging to the memory have a look at this tutorial.