I'm solving, find the largest prime factor of the number, Project Euler problem3.
Following Elixir code throw warnings, and do not evaluate in if block(assigning) I think:
num = 13195
range = num
|> :math.sqrt
|> Float.floor
|> round
for dv <- 2..range do
if rem(num, dv) == 0 and div(num, dv) != 1 do
num = div(num, dv)
end
end
num
|> IO.puts
Warnings are:
$ elixir 3.exs
warning: variable "num" is unused
3.exs:10
warning: the result of the expression is ignored (suppress the warning by assigning the expression to the _ variable)
3.exs:10
13195
$ elixir -v
Erlang/OTP 20 [erts-9.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Elixir 1.5.3
How can I update(assign) the num?
(following Python and Javascript codes are work for same the problem):
# 3.py
from math import ceil, sqrt
num = 600851475143
for div in range(2, ceil(sqrt(num)) + 1):
if num%div == 0 and num/div != 1:
num /= div
assert int(num) == 6857
// 3.js
var num = 600851475143;
var range = Array.from({length: Math.trunc(Math.sqrt(num))}, (x, i) => i + 2)
for (const div of range) {
if (num%div === 0 && num/div != 1) {
num /= div;
}
}
var assert = require('assert');
assert(num === 6857)
You are actually creating a new variable and shadowing the one from outer scope
You can rewrite it like this
num = 13195
range =
num
|> :math.sqrt()
|> Float.floor()
|> round
num =
2..range
|> Enum.reduce(num, fn elem, acc ->
if rem(acc, elem) == 0 and div(acc, elem) != 1 do
div(acc, elem)
else
acc
end
end)
IO.puts num
More on shadowing:
+------------------------------------------------------------+
| Top level |
| |
| +------------------------+ +------------------------+ |
| | Module | | Module | |
| | | | | |
| | +--------------------+ | | +--------------------+ | |
| | | Function clause | | | | Function clause | | |
| | | | | | | | | |
| | | +----------------+ | | | | +----------------+ | | |
| | | | Comprehension | | | | | | Comprehension | | | |
| | | +----------------+ | | | | +----------------+ | | |
| | | +----------------+ | | ... | | +----------------+ | | |
| | | | Anon. function | | | | | | Anon. function | | | |
| | | +----------------+ | | | | +----------------+ | | |
| | | +----------------+ | | | | +----------------+ | | |
| | | | Try block | | | | | | Try block | | | |
| | | +----------------+ | | | | +----------------+ | | |
| | +--------------------+ | | +--------------------+ | |
| +------------------------+ +------------------------+ |
| |
+------------------------------------------------------------+
Any variable in a nested scope whose name coincides with a variable from the surrounding scope will shadow that outer variable. In other words, the variable inside the nested scope temporarily hides the variable from the surrounding scope, but does not affect it in any way.
source
I'm trying to determine how a Turing Machine (consisting of only 0's and 1's, no blanks) could recognize a sequence of 8 1's. Every algorithm I've found has a TM searching for an indeterminate number of 1's or 0's, not a specific number.
Essentially, if you have this tape:
1 1 1 1 1 1 1 1 0 0 0 1 0 0 0 1
How can you recognize that the 8 1's represent addition, and you want to add 0 0 0 1 and 0 0 0 1?
I take it that 11111111 is like an opcode and 0001, 0001 are the operands for that opcode. At least, that's the only interpretation I am seeing.
A TM can look for a fixed, finite number of symbols by using a similar fixed, finite number of states, the sole purpose of each one being to recognize that another of the expected symbols has been seen. For instance, here's a four-tape TM that recognizes addition and does the binary addition:
|----|----|----|----|----||----|-----|-----|-----|-----|----|----|----|----|
| Q | T1 | T2 | T3 | T4 || Q' | T1' | T2' | T3' | T4' | D1 | D2 | D3 | D4 |
|----|----|----|----|----||----|-----|-----|-----|-----|----|----|----|----|
// read the opcode /////////////////////////////////////////////////////////
| qA | 1 | x | y | z || qB | 1 | x | y | z | R | S | S | S |
| qB | 1 | x | y | z || qC | 1 | x | y | z | R | S | S | S |
| qC | 1 | x | y | z || qD | 1 | x | y | z | R | S | S | S |
| qD | 1 | x | y | z || qE | 1 | x | y | z | R | S | S | S |
| qE | 1 | x | y | z || qF | 1 | x | y | z | R | S | S | S |
| qF | 1 | x | y | z || qG | 1 | x | y | z | R | S | S | S |
| qG | 1 | x | y | z || qH | 1 | x | y | z | R | S | S | S |
| qH | 1 | x | y | z || qI | 1 | x | y | z | R | S | S | S |
// read the first addend ///////////////////////////////////////////////////
| qI | w | x | y | z || qJ | w | w | y | z | R | R | S | S |
| qJ | w | x | y | z || qK | w | w | y | z | R | R | S | S |
| qK | w | x | y | z || qL | w | w | y | z | R | R | S | S |
| qL | w | x | y | z || qM | w | w | y | z | R | R | S | S |
// read the second addend //////////////////////////////////////////////////
| qM | w | x | y | z || qN | w | x | w | z | R | S | R | S |
| qN | w | x | y | z || qO | w | x | w | z | R | S | R | S |
| qO | w | x | y | z || qP | w | x | w | z | R | S | R | S |
| qP | w | x | y | z || qQ | w | x | w | z | R | S | R | S |
// prepare the output tape /////////////////////////////////////////////////
| qQ | w | x | y | z || qR | w | x | y | z | S | S | S | R |
| qR | w | x | y | z || qS | w | x | y | z | S | S | S | R |
| qS | w | x | y | z || qT | w | x | y | z | S | S | S | R |
| qT | w | x | y | z || qU | w | x | y | z | S | S | S | R |
// handle addition when no carry ///////////////////////////////////////////
| qU | w | 0 | 0 | z || qU | w | 0 | 0 | 0 | S | L | L | L |
| qU | w | 0 | 1 | z || qU | w | 0 | 1 | 1 | S | L | L | L |
| qU | w | 1 | 0 | z || qU | w | 1 | 0 | 1 | S | L | L | L |
| qU | w | 1 | 1 | z || qV | w | 1 | 1 | 0 | S | L | L | L |
| qU | w | B | B | B || hA | w | B | B | B | S | R | R | R |
// handle addition when carry //////////////////////////////////////////////
| qV | w | 0 | 0 | z || qU | w | 0 | 0 | 1 | S | L | L | L |
| qV | w | 0 | 1 | z || qV | w | 0 | 1 | 0 | S | L | L | L |
| qV | w | 1 | 0 | z || qV | w | 1 | 0 | 0 | S | L | L | L |
| qV | w | 1 | 1 | z || qV | w | 1 | 1 | 1 | S | L | L | L |
| qV | w | B | B | B || hA | w | B | B | B | S | R | R | R |
|----|----|----|----|----||----|-----|-----|-----|-----|----|----|----|----|
Legend:
Q: current state
T1: current tape symbol, input tape
T2: current tape symbol, scratch tape #1
T3: current tape symbol, scratch tape #2
T4: current tape symbol, output tape (not used)
Q': state to transition into
T1': symbol to write to input tape (not used)
T2': symbol to write to scratch tape #1
T3': symbol to write to scratch tape #2
T4': symbol to write to output tape
D1: direction to move input tape head
D2: direction to move scratch tape #1 head
D3: direction to move scratch tape #2 head
D4: direction to move output tape head
Conventions:
w, x, y and z are variables and represent either 0 or 1. A transition using all four of these can be thought of as a shorthand notation for writing sixteen (2^4) concrete transitions.
directions are L=left, S=same, R=right.
B is a blank symbol; it can be dispensed with if you add more states to assist U and V in the addition.
I have a graph (User-[Likes]->Item) with millions nodes and billions nodes (roughly 50G in disk) built on a powerful machine with 256G RAM and 40 cores. Currently, I'm computing the allshortestpath() between two items.
To improve the cypher query performance, I set dbms.pagecache.memory=100g and wrapper.java.additional=-Xmx32g, with the hope that the whole neo4j can be loaded into meomory. However, when I execute the shortestpath query, the CPU usage is 1625% while MEMORY usage is only 5.7%, and I didn't see performance improvements on the cypher query. Am I missing something in the setting? Or can I setup something to run the query faster? I have read the Performance Tuning guide in the developer manual but didn't find solution.
EDIT1:
The cypher query is to count the number of unique users that like both two items. The full pattern would be (Brand)-[:Has]->(Item)<-[:LIKES]-(User)-[:LIKES]->(Item)<-[:HAS]-(Brand)
profile
MATCH p = allShortestPaths((p1:Brand {FID:'001'})-[*..4]-(p2:Brand {FID:'002'}))
with [r in RELS(p)|type(r)] as relationshipPath,
[n in nodes(p)|id(n)][2] as user, p1, p2
return p1.FID, p2.FID, count(distinct user);
EDIT2:
Below is a sampler query plan. It now seems that I'm not using shortestsPath efficiently (380,556,69 db hits). I use shortestsPath to get the common user node between start/end nodes, and then use count(distinct) to get the unique user. Is it possible to tell cypher to eliminate paths which contain the node that have been visited before?
Can you try to run this instead:
MATCH (p1:Brand {FID:'001'}),(p2:Brand {FID:'002'})
MATCH (u:User)
WHERE (p1)-[:Has]->()<-[:LIKES]-(u) AND
(p2)-[:Has]->()<-[:LIKES]-(u)
RETURN p1,p2,count(u);
This starts at the user and checks against both brands, the explain plan looks much better
+----------------------+----------------+------------------------------------------+---------------------------+
| Operator | Estimated Rows | Variables | Other |
+----------------------+----------------+------------------------------------------+---------------------------+
| +ProduceResults | 0 | count(u), p1, p2 | p1, p2, count(u) |
| | +----------------+------------------------------------------+---------------------------+
| +EagerAggregation | 0 | count(u) -- p1, p2 | p1, p2 |
| | +----------------+------------------------------------------+---------------------------+
| +SemiApply | 0 | p2 -- p1, u | |
| |\ +----------------+------------------------------------------+---------------------------+
| | +Expand(Into) | 0 | anon[78] -- anon[87], anon[89], p1, u | (p1)-[:Has]->() |
| | | +----------------+------------------------------------------+---------------------------+
| | +Expand(All) | 0 | anon[87], anon[89] -- p1, u | (u)-[:LIKES]->() |
| | | +----------------+------------------------------------------+---------------------------+
| | +Argument | 1 | p1, u | |
| | +----------------+------------------------------------------+---------------------------+
| +SemiApply | 0 | p1 -- p2, u | |
| |\ +----------------+------------------------------------------+---------------------------+
| | +Expand(Into) | 0 | anon[119] -- anon[128], anon[130], p2, u | (p2)-[:Has]->() |
| | | +----------------+------------------------------------------+---------------------------+
| | +Expand(All) | 0 | anon[128], anon[130] -- p2, u | (u)-[:LIKES]->() |
| | | +----------------+------------------------------------------+---------------------------+
| | +Argument | 1 | p2, u | |
| | +----------------+------------------------------------------+---------------------------+
| +CartesianProduct | 0 | u -- p1, p2 | |
| |\ +----------------+------------------------------------------+---------------------------+
| | +CartesianProduct | 0 | p2 -- p1 | |
| | |\ +----------------+------------------------------------------+---------------------------+
| | | +Filter | 0 | p1 | p1.FID == { AUTOSTRING0} |
| | | | +----------------+------------------------------------------+---------------------------+
| | | +NodeByLabelScan | 0 | p1 | :Brand |
| | | +----------------+------------------------------------------+---------------------------+
| | +Filter | 0 | p2 | p2.FID == { AUTOSTRING1} |
| | | +----------------+------------------------------------------+---------------------------+
| | +NodeByLabelScan | 0 | p2 | :Brand |
| | +----------------+------------------------------------------+---------------------------+
| +NodeByLabelScan | 0 | u | :User |
+----------------------+----------------+------------------------------------------+---------------------------+
I have next graph model:
(:PaveView {Number:int, Page:string}), (:Page {Name:string})
(:PageView)-[:At]->(:Page)
(:PageView)-[:Next]->(:PageView)
Schema:
Indexes
ON :Page(Name) ONLINE (for uniqueness constraint)
ON :PageView(Page) ONLINE
ON :PageView(Revision) ONLINE (for uniqueness constraint)
Constraints
ON (pageview:PageView) ASSERT pageview.Number IS UNIQUE
ON (page:Page) ASSERT page.Name IS UNIQUE
I want to do something similar to this post
I have tried to find popular paths without loops of this structure:
(:PageView)-[:Next*2]->(:PageView)
That my tries:
1. Nicole White's method from post
MATCH p = (:PageView)-[:Next*2]->(:PageView)
WITH p, EXTRACT(v IN NODES(p) | v.Page) AS pages
UNWIND pages AS views
WITH p, COUNT(DISTINCT views) AS distinct_views
WHERE distinct_views = LENGTH(NODES(p))
RETURN EXTRACT(v in NODES(p) | v.Page), count(p)
ORDER BY count(p) DESC
LIMIT 10;
profile output:
10 rows
177270 ms
Compiler CYPHER 2.2-rule
ColumnFilter(0)
|
+Extract(0)
|
+ColumnFilter(1)
|
+Top
|
+EagerAggregation(0)
|
+Extract(1)
|
+ColumnFilter(2)
|
+Filter(0)
|
+Extract(2)
|
+ColumnFilter(3)
|
+EagerAggregation(1)
|
+UNWIND
|
+ColumnFilter(4)
|
+Extract(3)
|
+ExtractPath
|
+Filter(1)
|
+TraversalMatcher
+---------------------+---------+----------+------------------------------------------------------------------------+------------------------------------------------------------------------------------------------+
| Operator | Rows | DbHits | Identifiers | Other |
+---------------------+---------+----------+------------------------------------------------------------------------+------------------------------------------------------------------------------------------------+
| ColumnFilter(0) | 10 | 0 | EXTRACT(v in NODES(p) | v.Page), count(p) | keep columns EXTRACT(v in NODES(p) | v.Page), count(p) |
| Extract(0) | 10 | 0 | FRESHID225, FRESHID258, EXTRACT(v in NODES(p) | v.Page), count(p) | EXTRACT(v in NODES(p) | v.Page), count(p) |
| ColumnFilter(1) | 10 | 0 | FRESHID225, FRESHID258 | keep columns , |
| Top | 10 | 0 | FRESHID225, INTERNAL_AGGREGATEf7fa022b-cdb5-4ef2-bec5-a2f4f10706b6 | { AUTOINT0}; Cached( INTERNAL_AGGREGATEf7fa022b-cdb5-4ef2-bec5-a2f4f10706b6 of type Integer) |
| EagerAggregation(0) | 212828 | 0 | FRESHID225, INTERNAL_AGGREGATEf7fa022b-cdb5-4ef2-bec5-a2f4f10706b6 | |
| Extract(1) | 1749120 | 10494720 | FRESHID225, distinct_views, p | |
| ColumnFilter(2) | 1749120 | 0 | distinct_views, p | keep columns distinct_views, p |
| Filter(0) | 1749120 | 0 | FRESHID196, distinct_views, p | CoercedPredicate(anon[196]) |
| Extract(2) | 2115766 | 0 | FRESHID196, distinct_views, p | |
| ColumnFilter(3) | 2115766 | 0 | distinct_views, p | keep columns p, distinct_views |
| EagerAggregation(1) | 2115766 | 0 | INTERNAL_AGGREGATEb0939c81-a40c-4012-afd6-4852b17cf2e4, p | p |
| UNWIND | 6347298 | 0 | p, pages, views | |
| ColumnFilter(4) | 2115766 | 0 | p, pages | keep columns p, pages |
| Extract(3) | 2115766 | 12694596 | p, pages | pages |
| ExtractPath | 2115766 | 0 | p | |
| Filter(1) | 2115766 | 2115766 | | hasLabel(anon[34]:PageView(0)) |
| TraversalMatcher | 2115766 | 16926150 | | , , , |
+---------------------+---------+----------+------------------------------------------------------------------------+------------------------------------------------------------------------------------------------+
Total database accesses: 42231232
2.
match (p1:PageView)-[:Next]->(p2:PageView)-[:Next]->(p3:PageView)
where p1.Page<>p2.Page and p1.Page<>p3.Page and p2.Page<>p3.Page
RETURN [p1.Page,p2.Page,p3.Page], count(*) as count
ORDER BY count DESC
LIMIT 10;
profile output:
10 rows
28660 ms
Compiler CYPHER 2.2-cost
Projection(0)
|
+Top
|
+EagerAggregation
|
+Projection(1)
|
+Filter(0)
|
+Expand(0)
|
+Filter(1)
|
+Expand(1)
|
+NodeByLabelScan
+------------------+---------------+---------+----------+------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Operator | EstimatedRows | Rows | DbHits | Identifiers | Other |
+------------------+---------------+---------+----------+------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Projection(0) | 1241 | 10 | 0 | FRESHID146, [p1.Page,p2.Page,p3.Page], count | [p1.Page,p2.Page,p3.Page], count |
| Top | 1241 | 10 | 0 | FRESHID146, count | { AUTOINT0}; count |
| EagerAggregation | 1241 | 212828 | 0 | FRESHID146, count | |
| Projection(1) | 1542393 | 1749120 | 10494720 | FRESHID146, p1, p2, p3 | |
| Filter(0) | 1542393 | 1749120 | 17872173 | p1, p2, p3 | (((hasLabel(p3:PageView(0)) AND NOT(Property(p1,Page(3)) == Property(p3,Page(3)))) AND NOT(anon[20] == anon[43])) AND NOT(Property(p2,Page(3)) == Property(p3,Page(3)))) |
| Expand(0) | 1904189 | 1985797 | 3971596 | p1, p2, p3 | (p2)-[:Next]->(p3) |
| Filter(1) | 1904191 | 1985799 | 10578840 | p1, p2 | (NOT(Property(p1,Page(3)) == Property(p2,Page(3))) AND hasLabel(p2:PageView(0))) |
| Expand(1) | 2115767 | 2115768 | 4231538 | p1, p2 | (p1)-[:Next]->(p2) |
| NodeByLabelScan | 2115770 | 2115770 | 2115771 | p1 | :PageView |
+------------------+---------------+---------+----------+------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
3. (With loops!? And I don't know why! I suggested that if identifiers are different then nodes are different)
match (pv1:PageView)-[:Next]->(pv2:PageView)-[:Next]->(pv3:PageView),
(pv1)-[:At]->(p1),(pv2)-[:At]->(p2),(pv3)-[:At]->(p3)
RETURN [p1.Name,p2.Name,p3.Name], count(*) as count
ORDER BY count DESC
LIMIT 10;
profile output:
10 rows
27678 ms
Compiler CYPHER 2.2-cost
Projection(0)
|
+Top
|
+EagerAggregation
|
+Projection(1)
|
+Filter(0)
|
+Expand(0)
|
+Filter(1)
|
+Expand(1)
|
+Filter(2)
|
+Expand(2)
|
+Filter(3)
|
+Expand(3)
|
+Expand(4)
|
+NodeByLabelScan
+------------------+---------------+---------+----------+------------------------------------------------+------------------------------------------------------------+
| Operator | EstimatedRows | Rows | DbHits | Identifiers | Other |
+------------------+---------------+---------+----------+------------------------------------------------+------------------------------------------------------------+
| Projection(0) | 1454 | 10 | 0 | FRESHID139, [p1.Name,p2.Name,p3.Name], count | [p1.Name,p2.Name,p3.Name], count |
| Top | 1454 | 10 | 0 | FRESHID139, count | { AUTOINT0}; count |
| EagerAggregation | 1454 | 223557 | 0 | FRESHID139, count | |
| Projection(1) | 2115760 | 2115764 | 12694584 | FRESHID139, p1, p2, p3, pv1, pv2, pv3 | |
| Filter(0) | 2115760 | 2115764 | 0 | p1, p2, p3, pv1, pv2, pv3 | (NOT(anon[116] == anon[80]) AND NOT(anon[80] == anon[98])) |
| Expand(0) | 2115760 | 2115764 | 4231530 | p1, p2, p3, pv1, pv2, pv3 | (pv1)-[:At]->(p1) |
| Filter(1) | 2115762 | 2115766 | 2115766 | p2, p3, pv1, pv2, pv3 | (hasLabel(pv1:PageView(0)) AND NOT(anon[21] == anon[45])) |
| Expand(1) | 2115762 | 2115766 | 4231532 | p2, p3, pv1, pv2, pv3 | (pv2)<-[:Next]-(pv1) |
| Filter(2) | 2115764 | 2115766 | 0 | p2, p3, pv2, pv3 | NOT(anon[116] == anon[98]) |
| Expand(2) | 2115764 | 2115766 | 4231534 | p2, p3, pv2, pv3 | (pv2)-[:At]->(p2) |
| Filter(3) | 2115766 | 2115768 | 2115768 | p3, pv2, pv3 | hasLabel(pv2:PageView(0)) |
| Expand(3) | 2115765 | 2115768 | 4231536 | p3, pv2, pv3 | (pv3)<-[:Next]-(pv2) |
| Expand(4) | 2115767 | 2115768 | 4231538 | p3, pv3 | (pv3)-[:At]->(p3) |
| NodeByLabelScan | 2115770 | 2115770 | 2115771 | pv3 | :PageView |
+------------------+---------------+---------+----------+------------------------------------------------+------------------------------------------------------------+
System info:
windows 8.1
250G ssd
neo4j enterprise 2.2.0-M02
cache: hpc
ram: 8G
jvm heap size: 4G
memory mapping: 50%
149 (:Page) nodes
2115770 (:PageView) nodes
Why even the fastest of this three methods is so slow? (I guess that all my data is in RAM)
What is the best way to filter paths with loops?
By specifying labels for all identifiers, you force Cypher to open the node headers and filter all labels in it.
This is where the names of your relationships are important. Relationships are made to drive you into the graph, for performance there would be no need to specify the labels, so if your sure all nodes along the Path have the Pageview label, just omit it except for the start of your query :
match (p1:PageView)-[:Next]->(p2)-[:Next]->(p3)
where p1.Page<>p2.Page and p1.Page<>p3.Page and p2.Page<>p3.Page
RETURN [p1.Page,p2.Page,p3.Page], count(*) as count
ORDER BY count DESC
LIMIT 10;
I posted some query plan results in this answer related to your question : Neo4j: label vs. indexed property?