I'm trying (new to OrientDb) to load my old fashioned self joined mysql table to OrientDb.
I'm kinda stuck, I want to create vertices and edges with ETL but edges are created to empty vertices. I spent many hours in the documentation but can't find what I am missing.
Here is my oetl json file :
{
"config": {
"log": "debug"
},
"extractor" : {
"jdbc": { "driver": "com.mysql.jdbc.Driver",
"url": "***",
"userName": "***",
"userPassword": "***",
"query": "select nid, pnid, label from prod_arbo limit 500" }
},
"transformers" : [
{ "vertex": { "class": "Noeud", "skipDuplicates": true} },
{ "field": { "fieldName": "titre", "expression": "label"}},
{ "field": { "fieldName": "titre", "operation": "remove"} },
{ "field": { "fieldName": "pnid", "operation": "remove"} },
{ "edge": {
"class": "Parent",
"joinFieldName": "pnid",
"lookup": "Noeud.nid",
"unresolvedLinkAction": "CREATE"
} }
],
"loader" : {
"orientdb": {
"dbURL": "***",
"dbUser": "***",
"dbPassword": "***",
"standardElementConstraints": false,
"tx": false,
"wal": false,
"dbType": "graph"
}
}
}
Mysql db is like :
nid -> autoincrement unique id
label -> need to put this in field "titre"
pnid -> parent self join
I get logs like this :
[43:edge] DEBUG Transformer input: v(Noeud)[#13:57377]
[43:edge] DEBUG joinCurrentValue=null, lookupResult=null
[43:edge] DEBUG created new vertex=Noeud#13:57378{nid:null} v36
[43:edge] DEBUG created new edge=e[#17:56380][#13:57377-Parent->#13:57378]
[43:edge] DEBUG Transformer output: v(Noeud)[#13:57377]
And I don't understand why nid is null on new created vertex while created edge, how can I do to make it use pnid field here ?
Many thanks for your help
Laurent
Related
I'm using the go-elasticsearch API in my application to create indices in an Elastic.co cloud cluster. The application dynamically creates an index with a template and then starts indexing documents. The template includes an alias name and look like this:
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
},
"mappings": {
"properties": {
"title": {
"type": "text"
},
"created_at": {
"type": "date"
},
"updated_at": {
"type": "date"
},
"status": {
"type": "keyword"
}
}
},
"aliases": {
"rollout-nodes-f0776f0": {}
}
}
The name of the alias can change, so we pass it to the template when we create a new index. This is done with the Create indices API in Go:
indexTemplate := getIndexTemplate()
res, err := n.client.Indices.Create(
indexName,
n.client.Indices.Create.WithBody(indexTemplate),
n.client.Indices.Create.WithContext(ctx),
n.client.Indices.Create.WithTimeout(time.Second),
)
Doing some testing, this code works on localhost (without security enabled) but is not working with the cluster in Elastic.co, the index is created but not the alias.
I think it should be a problem related with either the API Key permissions or some configuration in the server, but I was unable to find yet which permission I'm missing.
For more context, this is the API Key I'm using:
{
"id": "fakeID",
"name": "index-service-key",
"creation": 1675350573126,
"invalidated": false,
"username": "fakeUser",
"realm": "cloud-saml-kibana",
"metadata": {},
"role_descriptors": {
"logstash_writer": {
"cluster": [
"monitor",
"transport_client",
"read_ccr",
"read_ilm",
"manage_index_templates"
],
"indices": [
{
"names": [
"*"
],
"privileges": [
"all"
],
"allow_restricted_indices": false
}
],
"applications": [],
"run_as": [],
"metadata": {},
"transient_metadata": {
"enabled": true
}
}
}
}
Any ideas? I know I can use the POST _aliases API, but the index creation option should be working too.
I try to implement NLog into my .NET Core Api web service.
I want to log to an Oracle database. All works well through an nlog.config XML file.
But the goal is to implement NLog config into appsettings.json and here problem occurs.
I get the error set in title:
LayoutRenderer cannot be found: 'aspnet-user-identity
My config file is like this :
"NLog": {
"autoReload": true,
"throwConfigExceptions": true,
"internalLogLevel": "info",
"internalLogFile": "c:/app/log/dev/internal-appsetting-nlog.txt",
"extensions": {
"NLog.Extensions.Logging": {
"assembly": [
"NLog.Extensions.Logging",
"NLog.Web.AspNetCore"
]
}
},
"variables": {
"var_logdir": "c:/app/log/dev"
},
"default-wrapper": {
"type": "AsyncWrapper",
"overflowAction": "Block"
},
"targets": {
"all-file": {
"type": "File",
"fileName": "${var_logdir}/nlog-all-${shortdate}.log",
"layout": {
"type": "JsonLayout",
"Attributes": [
{
"name": "timestamp",
"layout": "${date:format=o}"
},
{
"name": "level",
"layout": "${level}"
},
{
"name": "logger",
"layout": "${logger}"
},
{
"name": "message",
"layout": "${message:raw=true}"
},
{
"name": "properties",
"encode": false,
"layout": {
"type": "JsonLayout",
"includeallproperties": "true"
}
}
]
}
},
"db": {
"type": "Database",
"commandText": "INSERT INTO logtable (LOGLEVEL,LOGGER,MESSAGE,MACHINENAME,USERNAME,CALLSITE, THREADID,EXCEPTIONMESSAGE,STACKTRACE,SESSIONID) VALUES (:pLEVEL,:pLOGGER,:pMESSAGE,:pMACHINENAME, :pCALLSITE,:pTHREADID,:pEXCEPTIONMESSAGE,:pSTACKTRACE)",
"parameters": [
{
"name": "#pLEVEL",
"layout": "${level}"
},
{
"name": "#pLOGGER",
"layout": "${logger}"
},
{
"name": "#pMESSAGE",
"layout": "${message}"
},
{
"name": "#pMACHINENAME",
"layout": "${machinename}"
},
{
"name": "#pUSERNAME",
"layout": "${aspnet-user-identity}"
},
{
"name": "#pCALLSITE",
"layout": "${callsite:filename=true}"
},
{
"name": "#pTHREADID",
"layout": "${threadid}"
},
{
"name": "#pEXCEPTIONMESSAGE",
"layout": "${exception}"
},
{
"name": "#pSTACKTRACE",
"layout": "${stacktrace}"
},
{
"name": "#pSESSIONID",
"layout": "${aspnet-sessionid}"
}
],
"dbProvider": "Oracle.ManagedDataAccess.Client.OracleConnection, Oracle.ManagedDataAccess",
"connectionString": "xxxxxxxxxxxx"
}
},
"rules": [
{
"logger": "*",
"minLevel": "Trace",
"writeTo": "all-file"
},
{
"logger": "*",
"minLevel": "Trace",
"writeTo": "db"
},
{
"logger": "Microsoft.*",
"maxLevel": "Info",
"final": true
}
]
},
The internal debugger reports:
2019-10-09 16:48:48.6665 Info Adding target AsyncTargetWrapper(Name=all-file)
2019-10-09 16:48:48.7859 Warn Error when setting property 'Layout' on 'NLog.Targets.DatabaseParameterInfo' Exception: System.ArgumentException: LayoutRenderer cannot be found: 'aspnet-user-identity'. Is NLog.Web not included?
at NLog.Config.Factory`2.CreateInstance(String itemName)
at NLog.Layouts.LayoutParser.GetLayoutRenderer(ConfigurationItemFactory configurationItemFactory, String name)
at NLog.Layouts.LayoutParser.ParseLayoutRenderer(ConfigurationItemFactory configurationItemFactory, SimpleStringReader stringReader)
at NLog.Layouts.LayoutParser.CompileLayout(ConfigurationItemFactory configurationItemFactory, SimpleStringReader sr, Boolean isNested, String& text)
at NLog.Layouts.SimpleLayout.set_Text(String value)
at NLog.Internal.PropertyHelper.TryNLogSpecificConversion(Type propertyType, String value, Object& newValue, ConfigurationItemFactory configurationItemFactory)
at NLog.Internal.PropertyHelper.SetPropertyFromString(Object obj, String propertyName, String value, ConfigurationItemFactory configurationItemFactory)
Error occurs on ${aspnet-sessionid}. If I comment out both layout, everything works well.
I found different things on GitHub issue report but all I tried was a fail.
Could someone help?
The unknown aspnet-user-identity is probably an issue with your extensions:
"extensions": [
{ "assembly": "NLog.Extensions.Logging" },
{ "assembly": "NLog.Web.AspNetCore" }
],
Could you try the above suggestion?
P.S. Updated the wiki to include example of multiple "extensions"
I have the following query:
Criteria crit = Criteria.where("nestedObj.date").lt(LocalDate.now())
.and("nestedObj.active").is(true)
.and("someId").is(null)
.and("somethingElse").exists(false);
How can I make sure that nestedObj.active and nestedObj.date are checked on the same nestedObj?
I only want this to match if a document has a nestedObj that is active AND has a date older than today.
Example:
If the nestedObj array on a document loos like this, the query should match:
[
{
"nestedObj": {
"active": "true",
"date": "2010-29-10"
},
{
"nestedObj": {
"active": "false",
"date": "2010-29-10"
},
{
"nestedObj": {
"active": "true",
"date": "2022-29-10"
}
]
But if it looks like this, it shouldn't:
[
{
"nestedObj": {
"active": "false",
"date": "2010-29-10"
},
{
"nestedObj": {
"active": "true",
"date": "2022-29-10"
}
]
Check the element match in https://docs.mongodb.com/manual/reference/operator/query/elemMatch/
for instance
where("nestedObj.date").elemMatch( where("attribute1").is("value1").and("attribute2").regex("(?i).*$something.*")
I'm working on a call to readQuery. I'm getting an error message:
modules.js?hash=2d0033b4773d9cb6f118946043f7a3d4385825fe:25847
Error: Can't find field resolutions({"id":"Resolution:DHSzPa8bvPCDjuAac"})
on object (ROOT_QUERY) {
"resolutions": [
{
"type": "id",
"id": "Resolution:AepgCCio9KWGkwyMC",
"generated": false
},
{
"type": "id",
"id": "Resolution:DHSzPa8bvPCDjuAac", // <==ID I'M SEEKING
"generated": false
}
],
"user": {
"type": "id",
"id": "User:WWv57KsvqWeAoBNHY",
"generated": false
}
}.
The object with that id appears to be plainly visible as the second entry in the list of resolutions.
Here's my query:
const GET_CURRENT_RESOLUTION_AND_GOALS = gql`
query Resolutions($id: String!) {
resolutions(id: $id) {
_id
name
completed
goals {
_id
name
completed
}
}
}
`;
...and here's how I'm calling it:
<Mutation
mutation={CREATE_GOAL}
update={(cache, {data: {createGoal}}) => {
let id = 'Resolution:' + resolutionId;
const {resolutions} = cache.readQuery({
query: GET_CURRENT_RESOLUTION_AND_GOALS,
variables: {
id
},
});
}}
>
What am I missing?
Update
Per the GraphQL Dev Tools extension for Chrome, here's the whole GraphQL data store:
{
"data": {
"resolutions": [
{
"_id": "AepgCCio9KWGkwyMC",
"name": "testing 123",
"completed": false,
"goals": [
{
"_id": "TXq4nvukpLcqQhMRL",
"name": "test goal abc",
"completed": false,
"__typename": "Goal"
},
],
"__typename": "Resolution"
},
{
"_id": "DHSzPa8bvPCDjuAac",
"name": "testing 345",
"completed": false,
"goals": [
{
"_id": "PEkg5oEEi2tJ6i8LH",
"name": "goal abc",
"completed": false,
"__typename": "Goal"
},
{
"_id": "X4H4dFzGm5gkq5bPE",
"name": "goal bcd",
"completed": false,
"__typename": "Goal"
},
{
"_id": "hYunrXsMq7Gme7Xck",
"name": "goal cde",
"completed": false,
"__typename": "Goal"
}
"__typename": "Resolution"
}
],
"user": {
"_id": "WWv57KsvqWeAoBNHY",
"__typename": "User"
}
}
}
Posted as answer for fellow apollo users with similar problems:
Remove the prefix of Resolution:, the query should only take the id.
Then the question arises how is your datastore filled?
To read a query from cache, the query needs to have been called with exactly the same arguments on the remote API before. This way apollo knows what the result for a field is with specific arguments. If you never called the remote endpoint with the arguments you want to use but know what the result would be, you can circumvent that and resolve the query locally by implementing a cache resolver. Have a look at the example in the documentation. Here the store contains a list of books (in your case resultions) and the query for a single book by id can be resolved with a simple cache lookup.
I would like to find the datetime for 1 day ago so that I can create link to kibana in an email sent from the watcher. Using Elasticsearch 5.0.2
I've tried the watch below but it returns an error of
ScriptException[runtime error]; nested: IllegalArgumentException[Unable to find dynamic method [minusDays] with [1] arguments for class [org.joda.time.DateTime].];
minusDays does exist in the joda DateTime spec
but it doesn't exist in the elastic codebase
here's the watch
PUT /_xpack/watcher/watch/errors-prod
{
"trigger": {
"schedule": {
"daily": {
"at": [
"08:36"
]
}
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"<das-logstash-{now}>",
"<das-logstash-{now-1d}>"
],
"types": [
"redis-input"
],
"body": {
"size": 0,
"query": {
"match_all": {}
}
}
}
}
},
"actions": {
"send_email": {
"transform": {
"script" : "return [ 'from' : ctx.trigger.scheduled_time.minusDays(1) ]"
},
"email": {
"profile": "standard",
"from": "noreply#email.com",
"to": [
"me#email.com"
],
"subject": "errors",
"body": {
"html": "<html><body><p>from {{ctx.payload.from}}</p><p>to {{ctx.trigger.scheduled_time}}</p></body></html>"
}
}
}
}
}
I needed something similar and was able to hack this together by modifying a comment that almost worked from an elastic forum.
"transform": {
"script" : {
"source" : "def payload = ctx.payload; DateFormat df = new SimpleDateFormat(\"yyyy-MM-dd'T'HH:mm:ss.SSS'Z'\"); ctx.payload.from = df.format(Date.from(Instant.ofEpochMilli(ctx.execution_time.getMillis() - (24 * 60 * 60 * 1000) ))); return payload"
}
},
Hope that helps!