dbt + clickhouse incremental materialization - error 62 - clickhouse

i am struggling a bit with the incremental materialization via dbt-clickhouse (+ dbt 1.3). My model looks like this (i simplified as much as i could):
{{
config(
materialized='incremental',
engine='MergeTree()',
order_by=['ts'],
unique_key=['impid']
)
}}
select ts, impid from gam.data
{% if is_incremental() %}
where ts >= now() - toIntervalHour(1)
{% endif %}
the error I am receiving is this when running dbt run:
10:16:18 :HTTPDriver for http://localhost:8123 returned response code 400)
10:16:18 Code: 62. DB::Exception: Syntax error: failed at position 312 ('empty') (line 15, col 9): empty
10:16:18 as (
10:16:18
10:16:18 select ts, impid from gam.data
10:16:18
10:16:18 )
10:16:18 . Expected one of: token, Arrow, DoubleColon, MOD, DIV, NOT, BETWEEN, LIKE, ILIK
10:16:18
10:16:18 Done. PASS=6 WARN=0 ERROR=1 SKIP=0 TOTAL=7
the funny part is that the exact same model works with the basic view materialization but not with the incremental one. I tried every single combination I could come up with but so far no luck. Did you encounter something similar?

Ok, problem solved. I reached out on dbt's slack and it seems like the dbt-clickhouse adapter requires a more recent version of Clickhouse. It is a bug as the adapter should work with as versions as low as 22.7.1 but it does not. That being said, after upgrading the server, everything works fine.

Related

Indent error in the if statement | Discord.py

await ctx.send('Enter the channel name you want to be set as default welcome channel: ')
else:
await ctx.send('Only server moderators and owner can use this command')
the code is giving me error:
IndentationError: expected an indented block
The error isn't a DiscordError subclass.
It's only an IndentationError.
It means that your code isn't indented well.
Example
>>> if (True):
print('True')
IndentationError raised
You have to indent the statement block:
>>> if (True):
print('True')
'True'
And nothing goes wrong.
In conclusion
If your error was simply because of a wrong indentation, please fix it and accept my answer.

Nunjucks error reporting on imported file

With Nunjucks I use the throwOnUndefined option to see where I use an undefined or null value in my output.
In my code structure I have a main file, and import files containing macro's for the actual rendering of output.
I want to report errors on, but the errors are always reported on the main file. Both in my own code, and when I use for example nunjucks-cli.
An example with nunjucks-cli:
file options.json:
{
"throwOnUndefined": true,
}
file test.njk:
{% import "macro.njk" as doit %}
{{ doit.giveError("x") }}
file macro.njk, containing an error because writing value of y:
{% macro giveError(x) %}
{{ y }}
{% endmacro %}
Nunjucks commando:
nunjucks test.njk --path . --options options.json
Error output:
Template render error: (C:\Q\nunjucks-errorreporting\test.njk) [Line 1, Column 17]
Template render error: (C:\Q\nunjucks-errorreporting\test.njk) [Line 2, Column 4]
attempted to output null or undefined value
Both errors are reported on test.njk. The first error on Line 1, Column 17, but on both files there is no relevance on column 17. The second error is Line 2, Column 4, which matches with writing out {{ y }} in the macro.njk file, but is reported on test.njk.
I get the same behaviour in my custom code.
Is there a way to get correct error reporting on the correct files, because I have file importing files, importing other files...
Update:
Based on input Lesha Ogonkov I added "dev": true to the options.json.
This results in the following output:
Error: attempted to output null or undefined value
at new TemplateError (C:\Q\nunjucks-errorreporting\node_modules\nunjucks\src\lib.js:89:17)
at Object.ensureDefined (C:\Q\nunjucks-errorreporting\node_modules\nunjucks\src\runtime.js:242:11)
at Object.eval (eval at _compile (C:\Q\nunjucks-errorreporting\node_modules\nunjucks\src\environment.js:631:18), <anonymous>:18:38)
at Object.giveError (C:\Q\nunjucks-errorreporting\node_modules\nunjucks\src\runtime.js:131:17)
at Context.<anonymous> (C:\Q\nunjucks-errorreporting\node_modules\nunjucks\src\runtime.js:259:23)
at Object.callWrap (C:\Q\nunjucks-errorreporting\node_modules\nunjucks\src\runtime.js:273:14)
at eval (eval at _compile (C:\Q\nunjucks-errorreporting\node_modules\nunjucks\src\environment.js:631:18), <anonymous>:13:88)
at C:\Q\nunjucks-errorreporting\node_modules\nunjucks\src\environment.js:613:9
at Template.root [as rootRenderFunc] (eval at _compile (C:\Q\nunjucks-errorreporting\node_modules\nunjucks\src\environment.js:631:18), <anonymous>:29:1)
at Template.getExported (C:\Q\nunjucks-errorreporting\node_modules\nunjucks\src\environment.js:609:10) {
lineno: 2,
colno: 4,
firstUpdate: false,
Update: [Function: Update]
}
I still not see a relation to the file where the error is occurring.
Try to set dev flag next to throwOnUndefined. It is undocumented nunjucks feature, that prevent stacktrace collapse.

Loading of json files from S3 to sparkR dataframe

I have jason files saved in S3 bucket. I am trying to load them as dataframe in spark R and I am getting error logs. Following is my code. Where am I going wrong?
devtools::install_github('apache/spark#v2.2.0',subdir='R/pkg',force=TRUE)
library(SparkR)
sc=sparkR.session(master='local')
Sys.setenv("AWS_ACCESS_KEY_ID"="xxxx",
"AWS_SECRET_ACCESS_KEY"= "yyyy",
"AWS_DEFAULT_REGION"="us-west-2")
movie_reviews <-SparkR::read.df(path="s3a://bucketname/reviews_Movies_and_TV_5.json",sep = "",source="json")
I have tried all combinations of s3a , s3n, s3 and none seems to work.
I get following error log in my sparkR console
17/12/09 06:56:06 WARN FileStreamSink: Error while looking for metadata directory.
17/12/09 06:56:06 ERROR RBackendHandler: loadDF on org.apache.spark.sql.api.r.SQLUtils failed
java.lang.reflect.InvocationTargetException
For me it works
read.df("s3://bucket/file.json", "json", header = "true", inferSchema = "true", na.strings = "NA")
What #Ankit said should work, but if you are trying to get something that looks more like a dataframe, you need to use a select statement. i.e.
rdd<- read.df("s3://bucket/file.json", "json", header = "true", inferSchema = "true", na.strings = "NA")
Then do a printSchema(rdd) to see the structure of the data.
If you see something that has root followed by no indentations to your data, you can probably go ahead and select using the names of the "columns" you want. If you see branching down your schema tree, you may have to put a headers.blah or a payload.blah in you select statement. Like this:
sdf<- SparkR::select(rdd, "headers.something", "headers.somethingElse", "payload.somethingInPayload", "payload.somethingElse")

Calendar agent - error: sql cached statement NSSQLiteStatement

So after spending quite a lot of time trying to fix my iCalendar app
from the Spinning beach ball lag every time I make a new event or edit an event,
I found this error in the console application:
"error: sql cached statement NSSQLiteStatement <0x7f8eef4c0fd0> on entity 'Group' with sql text 'SELECT t0.Z_ENT, t0.Z_PK, t0.Z_OPT, t0.ZCOLORSTRING, t0.ZISENABLED, ....... ( t0.Z_PK IN (SELECT * FROM _Z_intarray0) AND t0.Z_ENT >= ? AND t0.Z_ENT <= ?) ' failed due to missing variable binding for (null) with expecting bindings (
"<NSSQLBindVariable: 0x7f8eef47bdd0>",
"<NSSQLBindVariable: 0x7f8eef47be70>"
) but actual substitution variables {
objects = "{<NSManagedObject: 0x7f8eef540140> (entity: ExchangePrincipal; id: 0x240092b <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/ExchangePrincipal/p9> ; data: <fault>)}";
}
error: sql cached statement NSSQLiteStatement <0x7f8eef48daa0> on entity 'Attendee' with sql text 'SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZADDRESSSTRING, t0.ZCOMMONNAME, t0.ZEMAIL, t0.ZINCLUDEDINALLRESPONDED, t0.ZINVITERNAME, t0.ZISSELFINVITED, t0.ZLIKENESSDATASTRING, t0.ZOMITSYNCRECORD, t0.ZPROPOSALENDDATE, t0.ZPROPOSALSTARTDATE, t0.ZPROPOSALSTATUS, t0.ZROLE, t0.ZRSVP, t0.ZSCHEDULEAGENT, t0.ZSCHEDULEFORCESEND, t0.ZSCHEDULESTATUS, t0.ZSTATUS, t0.ZSTATUSMODIFIEDDATE, t0.ZTYPE, t0.ZEVENT, t0.ZMYATTENDEEFOREVENT FROM ZATTENDEE t0 WHERE t0.ZMYATTENDEEFOREVENT IN (SELECT * FROM _Z_intarray0) ' failed due to missing variable binding for (null) with expecting bindings (
) but actual substitution variables {
destinations = "{0x122c009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1163>, 0x1230009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1164>, 0x1234009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1165>, 0x123c009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1167>, 0x1258009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1174>, 0x1264009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1177>, 0x126c009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1179>, 0x127c009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1183>, 0x1280009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1184>, 0x1284009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1185>}";
}"
There are around 8-10 of such errors each time I make a new event.
Can you please help me with this issue?
I already reinstalled mac os sierra few times,
but it made no difference.
What fixed it for me was a new clean install of the Mac OS.
Now I don't get the same issue (and Calendar is a pleasure to use),
but if the issue re-appears I will make an update.

CUDAPP 1.1 cudppSort configuration error (Invalid configuration argument)

I am trying to call cudppSort to sort a set of keys/values. I'm using the following code to set up the sort algorithm:
CUDPPConfiguration config;
config.op = CUDPP_ADD;
config.datatype = CUDPP_UINT;
config.algorithm = CUDPP_SORT_RADIX;
config.options = CUDPP_OPTION_KEY_VALUE_PAIRS | CUDPP_OPTION_FORWARD | CUDPP_OPTION_EXCLUSIVE;
CUDPPHandle planHandle;
CUDPPResult result = cudppPlan(&planHandle, config, number_points, 1, 0);
if (CUDPP_SUCCESS != result) {
printf("ERROR creating CUDPPPlan\n");
exit(-1);
}
The program exits, however on the line:
CUDPPResult result = cudppPlan(&planHandle, config, number_points, 1, 0);
and prints to stdout:
Cuda error: allocScanStorage in file 'c:/the/path/to/release1.1/cudpp/src/app/scan_app.cu' in line 279 : invalid configuration argument.
I looked at the line in scan_app.cu. It is,
CUT_CHECK_ERROR("allocScanStorage");
So apparently my configuration has an error that is causing the allocScanStorage to bomb out. There are only two calls to CUDA_SAFE_CALL in the function and I don't see a reason why either has anything to do with the configuration.
What is wrong with my configuration?
So that this doesn't sit around as an unanswered question (I'm not sure if this is the right SO etiquette but it seems like an answered question shouldn't sit around unanswered...), I'm copying the comment I made above here as an answer since it was the solution:
I figured this out (I'm still learning CUDA at the moment.) Because the error checking is asynchronous errors can show up in strange places if you don't check for them from time to time. My code had caused an error before I called cudppPlan but because I didn't check for errors the cudppPlan reported the error as if it was in cudppPlan.

Resources