I am trying to devise a crosstab report for COGNOS. I am joining a main query to a dimension query so that all crosstabs will nicely line up with one another. When I ask to view the tabular data for either the main query or the dimension query, it displays fine. The main query as about 10,000 records and the dimension query 70 records.
However, when I run the report as normal, or view the tabular data for the crosstab query, I get an RQP-DEF-0177 error.
An error occurred while performing operation 'sqlOpenResult' status='-237'.
and the error details begin:
UDA-SOR-0005 Unable to write the file.RSV-SRV-0042 Trace back:RSReportService.cpp(724)
followed by a bunch of QFException clauses.
How could I fix this problem?
This is due to insufficient disk space for Temporary files. Check IBM Cognos configuration -> Environment -> Temporary files location.
When a report is running a UDA file is created inside the temp folder and disk may not have enough space to for UDA file to expand. By increasing the disk space where the temp folder is located will resolve this issue.
This is generic error, most of the times cognos throws this error, if you want go in details go to /logs/cogserver.log , you will find more details there.
Related
I am working on Apache Drill and HDFS in my project.
I am dealing with v.big file (e.g 150GB) and that file is stored in HDFS system. I am writing my Drill query such a way that i will get some amount of data and i will process that (e.g 100 rows) and then again fire a query on that file, so my performance will increase.
(e.g SELECT * FROM dfs.file path LIMIT 100 )
But every time when i perform a query on that File which is in HDFS system, i am not getting consistent data. It changes every time as Hadoop will fetch that data from any cluster.
Because of that, it may be the case that during the entire process of getting all the record, i may get the same records which i have already.
You might be lucky with using pagination with LIMIT and OFFSET, altough I am not sure about it's behaviour with HDFS.
There is a question with a similar approach How to use apache drill do page search and the documentation says:
The OFFSET clause provides a way to skip a specified number of first rows in a result set before starting to return any rows.
(Source)
I am using clickhouse to store data, and I'm getting the following error while querying the column cid from the click table.
Checksum doesn't match: corrupted data.
I don't have any replicate for now, any suggestions for recovery?
The error comes down to the fact the checksum of the CityHash128 and the compressed data doesn't match and throws this exception in the readCompressedData function.
You can try to disable this check using the disable_checksum via the disableChecksumming method.
It could work, but a corrupted most probably means that something is wrong with your raw data and there is small chances for recovery unless you did backups.
Usually, you will get data part name and column name in exception message.
You could locate specific data part, remove files related to that single column, and restart the server. You will lose (already corrupted) data for one column in one data part (it will be filled with default values on read), but all other data will remain.
I am trying to reformat over 600gb of csv files into parquet using apache drill in a single node setup.
I run my sql statement:
CREATE TABLE AS Data_Transform.'/' AS
....
FROM Data_source.'/data_dump/*'
and it is creating parquet files but I get the error:
Query Failed: An Error Occurred
org.apache.drill.common.exceptions.UserRemoteException: RESOURCE ERROR:
One or more nodes ran out of memory while executing the query.
is there a way around this?
Or is there an alternative way to do the conversion?
I don't know if querying all those GB on a local node is feasible. If you've configured the memory per the docs, using a cluster of Drillbits to share the load is the obvious solution, but I guess you already know that.
If you're willing to experiment, and you're converting csv files using a select * to query the csv, rather than selecting individual columns, change the query to something like select columns[0] as user_id, columns1 as user_name. Cast any columns to types like int, float, datetime if possible. This avoids the read overhead storing data in the varchars and prepares data for your future queries that need to be cast for any analysis.
I've also seen the following recommendation from a Drill developer: split files into smaller files manually to overcome the local file system capability limitations. Drill doesn't split files on block splits.
I am getting some error in loading my files onto big sheets both directly from the HDFS( files that are output of pig scripts) and also raw data that is lying on the local hard disk.
I have observed that whenever I am loading the files and issuing a row count to see if all data is loaded into bigsheets, then I see lesses number of rows being loaded.
I have checked that the files are consistent and proper delimeters(/t or comma separated fields).
Size of my file is around 2GB and I have used either of the format *.csv/ *.tsv.
Also in some cases when i have tired to load a file from windows os directly then the files sometimes load successfully with row count matching with actual number of lines in the data, and then sometimes with lesser number of rowcount.
Even sometimes when a fresh file being used 1st time it gives the correct result but if I do the same operation next time some rows are missing.
Kindly share your experience your bigsheets, solution to any such problems where the entire data is not being loaded etc. Thanks in advance
The data that you originally load into BigSheets is only a subset. You have to run the sheet to get it on the full dataset.
http://www-01.ibm.com/support/knowledgecenter/SSPT3X_3.0.0/com.ibm.swg.im.infosphere.biginsights.analyze.doc/doc/t0057547.html?lang=en
I was wondering, for folks familiar with DataStage, if Oracle SQLLDR can be used on DataStage. I have some sets of control files that I would like to incorporate into DataStage. A step by step way of accomplishing this will greatly be appreciated. Thanks
My guess is that you can run it with external stage in data stage.
You simply put the SQLLDR command in the external stage and it will be executed.
Try it and tell me what happens.
We can use ORACLE SQL Loader in DataStage .
If you check Oracle Docs there are two types of fast loading under SQL Loader
1) Direct Path Load - less validation in database side
2) Conventional Path Load
There is less validation in Direct Load if we compare to Conventional Load.
In SQL Loader process we have to specify points like
Direct or not
Parallel or not
Constraint and Index options
Control and Discard or Log files
In DataStage , we have Oracle Enterprise and Oracle Connector Stages
Oracle Enterprise -
we have load option in this stage to load data in fast mode and we can set Environment variable OPTIONS
for Oracle , example is below
OPTIONS(DIRECT=FALSE,PARALLEL=TRUE)
Oracle Connector -
We have bulk load option for it and other properties related to SQL Loader are available in properties tab .
Example - control and discard file values all set by DataStage but you can set these properties and others manually.
As you know SQLLDR basically loads data from files to database so datastage allows you to use any input data file, that would take input in any data file like sequential file, pass them format, pass the schema of the table, and it’ll create an in memory template table, then you can use a database connecter like odbc or db2 etc. and that would load your data in your table, simple as that.
NOTE: if your table does not exist already at the backend then for first execution make it create then set it to append or truncate.
Steps:
Read the data from the file(Sequential File Stage)
Load it using the Oracle Connector(You could use Bulk load so that you could used direct load method using the SQL loader and the data file and control file settings can be configured manually). Bulk Load Operation: It receives records from the input link and passes them to Oracle database which formats them into blocks and appends the blocks to the target table as opposed to storing them in the available free space in the existing blocks.
You could refer the IBM documentation for more details.
Remember, there might be some restriction in loading when it comes to handling rejects, triggers or constraints when you use bulk load. It all depends on your requirement.