I am trying to batch-write to a sqlite3 db using a pooled connection as described in clojure-cookbook.
It works up to two rows. When I insert three rows I got a java.lang.ClassCastException: clojure.lang.MapEntry cannot be cast to clojure.lang.Named exception.
Here's my code:
(def db-spec {:classname "org.sqlite.JDBC"
:subprotocol "sqlite"
:subname "sqlite.db"
:init-pool-size 1
:max-pool-size 1
:partitions 1})
(jdbc/db-do-commands
*pooled-db*
(jdbc/create-table-ddl
:play
[[:primary_id :integer "PRIMARY KEY AUTOINCREMENT"]
[:data :text]]))
(jdbc/insert! *pooled-db* :play {:data "hello"}{:data "hello"})
(jdbc/insert! *pooled-db* :play {:data "hello"}{:data "hello"}{:data "hello"})
What am I missing here?
Thanks
See the docs for this example: https://github.com/clojure/java.jdbc
(j/insert-multi! mysql-db :fruit
[{:name "Apple" :appearance "rosy" :cost 24}
{:name "Orange" :appearance "round" :cost 49}])
The API docs say this:
insert-multi!
function
Usage: (insert-multi! db table rows)
(insert-multi! db table cols-or-rows values-or-opts)
(insert-multi! db table cols values opts)
Given a database connection, a table name and either a sequence of maps (for
rows) or a sequence of column names, followed by a sequence of vectors (for
the values in each row), and possibly a map of options, insert that data into
the database.
When inserting rows as a sequence of maps, the result is a sequence of the
generated keys, if available (note: PostgreSQL returns the whole rows).
When inserting rows as a sequence of lists of column values, the result is
a sequence of the counts of rows affected (a sequence of 1's), if available.
Yes, that is singularly unhelpful. Thank you getUpdateCount and executeBatch!
The :transaction? option specifies whether to run in a transaction or not.
The default is true (use a transaction). The :entities option specifies how
to convert the table name and column names to SQL entities.
Related
I have two facts tables, First and Second, and two dimension tables, dimTime and dimColour.
Fact table First looks like this:
and facet table Second looks like this:
Both dim-tables have 1:* relationships to both fact tables and the filtering is one-directional (from dim to fact), like this:
dimColour[Color] 1 -> * First[Colour]
dimColour[Color] 1 -> * Second[Colour]
dimTime[Time] 1 -> * First[Time]
dimTime[Time] 1 -> * Second[Time_]
Adding the following measure, I would expect the FILTER-functuion not to have any affect on the calculation, since Second does not filter First, right?
Test_Alone =
CALCULATE (
SUM ( First[Amount] );
First[Alone] = "Y";
FILTER(
'Second';
'Second'[Colour]="Red"
)
)
So this should evaluate to 7, since only two rows in First have [Alone] = "Y" with values 1 and 6 and that there is no direct relationship between First and Second. However, this evaluates to 6. If I remove the FILTER-function argument in the calculate, it evaluates to 7.
There are thre additional measures in the pbix-file attached which show the same type of behaviour.
How is filtering one fact table which has no direct relationship to a second fact table affecting the calculation done on the second table?
Ziped Power BI-file: PowerBIFileDownload
Evaluating the table reference 'Second' produces a table that includes the columns in both the Second table, as well as those in all the (transitive) parents of the Second table.
In this case, this is a table with all of the columns in dimColour, dimTime, Second.
You can't see this if you just run:
evaluate 'Second'
as when 'evaluate' returns the results to the user, these "Parent Table" (or "Related") columns are not included.
Even so, these columns are certainly present.
When a table is converted to a row context, these related columns become available via RELATED.
See the following queries:
evaluate FILTER('Second', ISBLANK(RELATED(dimColour[Color])))
evaluate 'Second' order by RELATED(dimTime[Hour])
Similarly, when arguments to CALCULATE are used to update the filter context, these hidden "Related" columns are not ignored; hence, they can end up filtering First, in your example. You can see this, by using a function that strips the related columns, such as INTERSECT:
Test_ActuallyAlone = CALCULATE (
SUM ( First[Amount] ),
First[Alone] = "Y",
//This filter now does nothing, as none of the columns in Second
//have an impact on 'SUM ( First[Amount] )'; and the related columns
//are removed by the INTERSECT.
FILTER(
INTERSECT('Second', 'Second')
'Second'[Colour]="Red"
)
)
(See these resources that describe the "Expanded Table"
(this is an alternative but equivalent explanation of this behaviour)
https://www.sqlbi.com/articles/expanded-tables-in-dax/
https://www.sqlbi.com/articles/context-transition-and-expanded-tables/
)
When using the seedfile in a Phoenix app, is it possible to restart the identity for a table if the seed is run more than once?
For example, I'm currently working with the following seed:
PhoenixApp.Repo.delete_all PhoenixApp.Role
PhoenixApp.Repo.insert!(%PhoenixApp.Role{role: "admin"})
PhoenixApp.Repo.insert!(%PhoenixApp.Role{role: "non-admin"})
The first line clears the table (so that the records don't pile up if the seed is run multiple times), and the lines following create the seed records. Running this code once would create two records with the autoincremented primary keys '1' and '2', as you would normally expect. However, if I want to add another entry to the table later on, such as
PhoenixApp.Repo.insert!(%PhoenixApp.Role{role: "superuser"})
the ids of the rows will now be '3', '4', and '5', because the identity was not restarted.
Does Ecto have a command that restarts the table identity as well? I realize that I could add additional records to my table via IEx, but I'd prefer to restart the identity, if possible.
In PostgreSQL, you can execute ALTER SEQUENCE <sequence name> RESTART to reset the value of the sequence to its original value. The Sequence Name for the ID primary key will be #{table_name}_id_seq. You can run the query using Repo.query, for example:
Repo.query("ALTER SEQUENCE comments_id_seq RESTART")
iex(1)> Repo.insert!(%Comment{}).id
11
iex(2)> Repo.delete_all(Comment)
{11, nil}
iex(3)> Repo.insert!(%Comment{}).id
12
iex(4)> Repo.query("ALTER SEQUENCE comments_id_seq RESTART")
{:ok,
%Postgrex.Result{columns: nil, command: :alter_sequence, connection_id: 3360,
num_rows: 0, rows: nil}}
iex(5)> Repo.insert!(%Comment{}).id
1
I am looking for Clojure data structure, that works like relation table (as in relational databases).
Map (even biderectional) id -> (val1, val2, val3, ...) does not do the job. If I, for example, want to find all rows with val2 = "something" it will took O(n).
But I want to perform search in a column in O(log n)!
Searching for rows in database with a column predicate without an index is O(n) as every row has to be checked if it matches the predicate. If there is an index for a column that your predicate uses then the index can be used to find all the rows for a specific value by looking up that value as the key in the index to fetch all the matching rows. Then it is usually log(n) operation (it depends on the internal implementation of the index, e.g. for B-tree it is log(n)).
I am not aware of out-of-the-box implementation of a Clojure data structure having such characteristics as they have usually single-purpose (e.g. map is an associative datastructure for lookup by a single key, not multiple keys as in DB with multiple indexes). You would rather need a library providing some kind of an in-memory database, for example (as mentioned by Thumbnail in his comment) DataScript, or even in-memory SQL DB with JDBC interface (e.g. H2SQL, HSQLDB or DerbyDB using their in-memory stores).
I am not sure of your specific requirements, but you could also implement some of the features yourself using basic API from Clojure. For example, you could use a Clojure set as your "table" and enhance it with some functions from clojure.set:
Your table:
(def my-table #{{:id 1 :name "John" :age 30 :gender :male}
{:id 2 :name "Jane" :age 25 :gender :female}
{:id 3 :name "Joe" :age 40 :gender :male}})
And specify your indices:
(def by-id (clojure.set/index my-table [:id]))
(def by-age (clojure.set/index my-table [:age]))
(def by-gender (clojure.set/index my-table [:gender]))
And then use your indices when querying/filtering your table:
(clojure.set/intersection
(by-age {:age 30})
(by-gender {:gender :male}))
;; => #{{:id 1, :name "John", :age 30, :gender :male}}
This is too tough for me guys. It's for Jeremy!
I have two tables (although I can also envision needing to join a third table) and I want to sum one field and count rows, in the same, table while joining with another table and return the result in json format.
First of all, the data type field that needs to be summed, is numeric(10,2) and the data is inserted as params['amount'].to_f.
The tables are expense_projects which has the name of the project and the company id and expense_items which has the company_id, item and amount (to mention just the critical columns) - the "company_id" columns are disambiguated.
So, the following code:
expense_items = DB[:expense_projects].left_join(:expense_items, :expense_project_id => :project_id).where(:project_company_id => company_id).to_a.to_json
works fine but when I add
expense_total = expense_items.sum(:amount).to_f.to_json
I get an error message which says
TypeError - no implicit conversion of Symbol into Integer:
so, the first question is why and how can this be fixed?
Then I want to join the two tables and get all the project names form the left (first table) and sum amount and count items in the second table. I have tried
DB[:expense_projects].left_join(:expense_items, :expense_items_company_id => expense_projects_company_id).count(:item).sum(:amount).to_json
and variations of this, all of which fails.
I would like a result which gets all the project names (even if there are no expense entries and returns something like:
project item_count item_amount
pr 1 7 34.87
pr 2 0 0
and so on. How can this be achieved with one query returning the result in json format?
Many thanks, guys.
Figured it out, I hope this helps somebody else:
DB[:expense_projects___p].where(:project_company_id=>user_company_id).
left_join(:expense_items___i, :expense_project_id=>:project_id).
select_group(:p__project_name).
select_more{count(:i__item_id)}.
select_more{sum(:i__amount)}.to_a.to_json
I have a set of rows which I've fetched from a table. Let's say the object Rating. After fetching this object, I have say 100 entries from the database.
The Rating object might look like this:
table_ratings
t.integer :num
So what I now want to do is perform some calculations on these 100 rows without performing any other queries. I can do this, running an additional query:
r = Rating.all
good = r.where('num = 2') # this runs an additional query
"#{good}/#{r.length}"
This is a very rough idea of what I want to do, and I have other more complex output to build. Let's imagine I have over 10 different calculations I need to perform on these rows, and so aliasing these in a sql query might not be the best idea.
What would be an efficient way to replace the r.where query above? Is there a Ruby method or a gem which would give me a similar query api into ActiveRecord Association collection objects?
Rating.all returns an array of all Rating objects. From there, shift your focus to selecting and mapping from the array. eg.:
#perfect_ratings = r.select{|x| x.a == 100}
See:
http://www.ruby-doc.org/core-1.9.3/Array.html
ADDITIONAL COMMENTS:
Going over the list of methods available for array, I find myself using the following frequently:
To check a variable against multiple values:
%w[dog cat llama].include? #pet_type # returns true if #pet_type == 'cat'
To create another array(map and collect are aliases):
%w[dog cat llama].map(|pet| pet.capitalize) # ["Dog", "Cat", "Llama"]
To sort and drop duplicates:
%w[dog cat llama dog].sort.uniq # ["cat", "dog", "llama"]
<< to add an element, + to add arrays, flatten to flatten embedded arrays into a single level array, count or length or size for number of elements, and join are the others I tend to use a lot.
Finally, here is an example of join:
%w[dog cat llama].join(' or ') # "dog or cat or llama"