enter image description hereOur group members used beeswax to do the hive queries.
When few people do the query,everything is okay.
But when lots people do the query at the same time or in a short time,something strange happened. The database list keep loading and the whole browser will be no response.At this time, the cores of hive servers and metastore servers were always more than one core.
So I had to wait for a long time or restart the whole hive server and everything going okay again.
I had checked the log of hue and hive server,but found nothing useful.(so sad of it)
I had worked for this question for a every log time but could not get the solution of this problem.
Because of the limit of network,I can not paste the screenshots and any logs here.
Are there anyone can help me.
I search this picture from internet.I made a mark on it.When the problem happened,there is a loading icon on the it and it was impossible to change database.
Related
I have a Laravel App running on Azure Web App Linux service, all running nice and smoothly until I reach a feature that exports a query to an XLS for download. Then I receive the ERROR 502.
On my local environment works normally, I can export the query to XLS with no issues, it is not a large query, just a few rows.
In the same app, I have a function that exports to XLS just 1 row at a time and works fine, so it is just when I go for a larger(ish) query.
Any ideas? I have tried scaling up, restarting the app, apache, changed .ini (via .htaccess to increase execution time).
There is no trace in the logs either, there is something about the container crashing but cannot trace it to this particular error.
Ok, managed to solved it... was not straight forward at all. It has to do with the size of the query, even tough is not big by any means (a couple thousands max) raising memory limit to 1024M or further ended up in 502 Error. Decided to try different and moved from Laravel Excel to Fast-Excel which is less featured but man... it works. Now everything downloads perfectly. In case you are having this issue give fast-excel a try.
I'm new to use oracle BIEE.My development enviromnent now is installed,and the project is a little big.Multi user development is using for developing now.The problems happens when one developer publish the rpd to network and want to test the data,the server reloading the rpd file takes too much time and I can hardly wait!When multi users want to test rpd file,e,can't stand it... is there any other way to solve the problem?or how to make the biee sever reload the rpd file quickly?
It's hard to say specifically without knowing a bit more about your setup, but here are a few general advice pointers:
When stopping the service OBI will wait for any running queries to complete before stopping the service, so making sure there's nothing running before you try to do this.
Make sure you're only restarting the BI Server component, you don't need to wait for the other services to restart if you're just changing the RPD (if you're on 11g then deploying through EM should mean this happens anyway so you don't need to worry).
If you're using 11g, you could try incremental updates by creating patches.
Check whether the hardware you're running on is adequate, most importantly that you've enough RAM so it's not having to page out to disk when it loads the RPD.
Remove anything unused from the RPD to make it smaller.
Folks,
I'm trying to set up a regular backup of a rather large production database (half a gig) that has both InnoDB and MyISAM tables. I've been using mysqldump so far, but I find that it's taking increasingly longer periods of time, and the server is completely unresponsive while mysqldump is running.
I wanted to ask for your advice: how do I either
Make mysqldump backup non-blocking - assign low priority to the process or something like that, OR
Find another backup mechanism that will be better/faster/non-blocking.
I know of the existence of MySQL Enterprise Backup product (http://www.mysql.com/products/enterprise/backup.html) - it's expensive and this is not an option for this project.
I've read about setting up a second server as a "replication slave", but that's not an option for me either (this requires hardware, which costs $$).
Thank you!
UPDATE: more info on my environment: Ubuntu, latest LAMPP, Amazon EC2.
If replication to a slave isn't an option, you could leverage the filesystem, depending on the OS you're using,
Consistent backup with Linux Logical Volume Manager (LVM) snapshots.
MySQL backups using ZFS snapshots.
The joys of backing up MySQL with ZFS...
I've used ZFS snapshots on a quite large MySQL database (30GB+) as a backup method and it completes very quickly (never more than a few minutes) and doesn't block. You can then mount the snapshot somewhere else and back it up to tape, etc.
Edit: (previous answer was suggestion a slave db to back up from, then I noticed Alex ruled that out in his question.)
There's no reason your replication slave can't run on the same hardware, assuming the hardware can keep up. Grab a source tarball, ./configure --prefix=/dbslave; make; make install; and you'll have a second mysql server living completely under /dbslave.
EDIT2: Replication has a bunch of other benefits, as well. For instance, with replication running, you'll may be able to recover the binlog and replay it on top your last backup to recover the extra data after certain kinds of catastrophes.
EDIT3: You mention you're running on EC2. Another, somewhat contrived idea to keep costs down is to try setting up another instance with an EBS volume. Then use the AWS api to spin this instance up long enough for it to catch up with writes from the binary log, dump/compress/send the snapshot, and then spin it down. Not free, and labor-intensive to set up, but considerably cheaper than running the instance 24x7.
Try mk-parallel-dump utility from maatkit (http://www.maatkit.org/)
regards,
Something you might consider is using binary logs here though a method called 'log shipping'. Just before every backup, issue out a command to flush the binary logs and then you can copy all except the current binary log out via your regular file system operations.
The advantage with this method is your not locking up the database at all, since when it opens up the next binary log in sequence, it releases all the file locks on the prior logs so processing shouldn't be affected then. Tar'em, zip'em in place, do as you please, then copy it out as one file to your backup system.
An another advantage with using binary logs is you can restore up to X point in time if the logs are available. I.e. You have last year's full backup, and every log from then to now. But you want to see what the database was on Jan 1st, 2011. You can issue a restore 'until 2011-01-01' and when it stops, your at Jan 1st, 2011 as far as the database is concerned.
I've had to use this once to reverse the damage a hacker caused.
It is definately worth checking out.
Please note... binary logs are USUALLY used for replication. Nothing says you HAVE to.
Adding to what Rich Adams and timdev have already suggested, write a cron job which gets triggered on low usage period to perform the slaving task as suggested to avoid high CPU utilization.
Check mysql-parallel-dump also.
In my attempt to extract data (dumps and selective reading of columns) from a diverse collection of edb databases I got faced with a fundamental problem. I have an edb database coming with a couple of log files. I know what information there is within the database, but I just get half of it extracted. I fear that the remaining half sleeps somewhere in the log files. I assumed the EDB engine knows where the log files are and automagically loads them when attaching the database (JET_paramSystemPath, JET_paramLogFilePath and JET_paramBaseName are properly set). Is that a wrong assumption? If so, what should I do to have the logs loaded as well?
Alternatively, would it be possible to simply commit the transactions to the EDB file and get rid of the logs?
If there are uncommitted transactions then the database will be marked as 'inconsistent'. You can check this using ESENTUTL /MH against the database. Calling JetAttachDatabase against an inconsistent database will always fail.
So, if your program is able to attach and open the database then it is consistent. There are two ways a database can be made consistent:
A clean shutdown of ESENT.
Running recovery using the logfiles at JetInit time.
The first thing that JetInit does is to look for the logfiles specified by JET_paramLogFilePath and JET_paramBaseName. Logfiles contain the full paths of the database(s) they reference and the transactions in the logfiles are then committed to the database(s). So, if you set the system parameters properly then ESENT will load the logs when attaching the database.
On the other hand, if you don't set the parameters properly then your program will actually work on databases that don't require recovery. JetInit won't find any logfiles so it won't do anything and the attach will succeed because the database is consistent.
One further twist is that the logfiles contain the full path to the database. This means that if you have copied/moved the database then recovery will not work. Starting with Windows Server 2003 you can deal with this by setting JET_paramAlternateDatabaseRecoveryPath to the directory containing the database.
Important: to be safe you should always attach and open the database using the read-only flags. This will avoid any problems caused by bad logfile settings. A common problem is that read-only applications end up creating a set of logfiles in a different directory which prevent the database from being recovered properly.
How can I stop the indexing process? I had imported 15,000 products and started the indexing, but now it's stuck on processing for "Category search index". It has been stuck on processing for two days now, how can I stop the indexing process?
Any help would be appreciated, thank you in advance.
When Magento starts indexing of each item in the 'Index Management' list it sets 'status' field of correlative entire of 'index_process' table to 'working'. If error happens during the indexing process script halts and status remains as 'working'. As a result 'processing' image hangs at the 'Index Management' page. Of course there's no way to make sure that indexing process is still not running but checking your server's processes list (or brutal restarting or your apache).
The only intelligent way I know to solve this issue is to find out what causing this error and fix it. You can do it by running indexing from your shell and looking for the output. Here is the command:
php shell/indexer.php reindexall
If you don't have an SSH access you can also try locating this error from your server's error logs.
If you just want to remove 'processing' image you can just run the following MYSQL query:
UPDATE `index_process` SET `status` = 'pending' WHERE `status` = 'working'
However this will not solve the problem, but only hide the output.
Probably the easiest way is to restart Apache. It will gracefully close any open visitor connections, and should terminate the PHP process.
--------- EDIT ----------
There are a set of tables in the database that look to be related to the indexing, index_process and index_process_event. You could try reverting the state of the relevant record there to pending and try again...
Keep in mind, if you are running magento under a different user than the one you are currently using (www-data vs mySshLoginUser), you might open yourself up to permissions problems on the lock files. If you have manually run the shell indexer, those lock files will belong to your ssh user.
Worst case scenario, delete those lock files in var/locks/* and try it.