Error while using mosql mongodb to postgres translator - ruby

I am using MOSQL to create a live database replica of a mongodb database in postgres. I install all the requirements.
My mosql query i ran in terminal:
mosql -c coll.yaml --sql postgres://postgres:postgres#localhost:5432/test --mongo mongodb://localhost:27017/
My mongodb collection:
{
"_id" : ObjectId("562dd2ef5bd6681ad0f63868"),
"name" : "manish",
"score" : 23,
"class" : "Engg"
}
...
My coll.yaml:
Flipkart:
temp:
:columns:
- name: TEXT
- score: INTEGER
- class: TEXT
:meta:
:table: temp
:extra_props: true
Error
/usr/lib/ruby/1.9.1/psych.rb:203:in parse': (coll.yaml): found character that cannot start any token while scanning for the next token at line 2 column 1 (Psych::SyntaxError)
from /usr/lib/ruby/1.9.1/psych.rb:203:inparse_stream'
from /usr/lib/ruby/1.9.1/psych.rb:151:in parse'
from /usr/lib/ruby/1.9.1/psych.rb:127:inload'
from /usr/lib/ruby/1.9.1/psych.rb:297:in block in load_file'
from /usr/lib/ruby/1.9.1/psych.rb:297:inopen'
from /usr/lib/ruby/1.9.1/psych.rb:297:in load_file'
from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/cli.rb:140:inload_collections'
from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/cli.rb:152:in run'
from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/cli.rb:16:inrun'
from /var/lib/gems/1.9.1/gems/mosql-0.4.3/bin/mosql:5:in <top (required)>'
from /usr/local/bin/mosql:23:inload'
from /usr/local/bin/mosql:23:in `'
How to remove the error?

Related

Ruby Sequel Gem unable to look up tables with different owner

I'm connecting to my company's database with the Sequel Gem using the odbc adaptor.
DB = Sequel.odbc('myserver', :user => "USER1", :password => "1234")
I connected fine but I was not able to get the dataset off of tables that belongs to other users.
When I connect to the database with Interactive SQL, I see the following tables.
USER1.TABLE1
USER1.TABLE2
USER1.TABLE3
USER2.TABLE4
USER2.TABLE5
When in interactive SQL. I have complete right to do anything with all the tables since user1 is a superuser. However, when I try to pull table4 or table5 with Sequel gem.
dataset = DB[:TABLE4].all.count
I get the following error.
Traceback (most recent call last):
16: from C:/Ruby26-x64/bin/irb:23:in 'load'
15: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/irb-1.2.0/exe/irb:11:in '<top (required)>'
14: from (irb):17
13: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/dataset/actions.rb:51:in 'all'
12: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/dataset/actions.rb:1006:in `_all'
11: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/dataset/actions.rb:51:in 'block in all'
10: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/dataset/actions.rb:152:in 'each'
9: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/adapters/odbc.rb:90:in 'fetch_rows'
8: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/dataset/actions.rb:1089:in 'execute'
7: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/adapters/odbc.rb:40:in 'execute'
6: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/database/connecting.rb:270:in 'synchronize'
5: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/connection_pool/threaded.rb:92:in 'hold'
4: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/adapters/odbc.rb:42:in 'block in execute'
3: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/database/logging.rb:38:in 'log_connection_yield'
2: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/adapters/odbc.rb:42:in 'block (2 levels) in execute'
1: from C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/sequel-5.29.0/lib/sequel/adapters/odbc.rb:42:in 'run'
Sequel::DatabaseError (ODBC::Error: S0002 (-141) [Sybase][ODBC Driver][SQL Anywhere]Table 'TABLE4' not found)
However when using the same code to pull TABLE1, 2 or 3 is fine tho.
dataset = DB[:TABLE1].all.count
=>999
Maybe because I'm not looking up table 4 or 5 correctly? Or I have to somehow specify the owner of those table? In Interactive SQL I have to do USER2.TABLE4 to identify the table.
Many thanks!
You need to use a qualified identifier in Sequel, just as you need to in SQL:
dataset = DB[Sequel[:USER2][:TABLE4]].all.count
For more information, see:
http://sequel.jeremyevans.net/rdoc/files/README_rdoc.html#label-Qualifying+identifiers+-28column-2Ftable+names-29
http://sequel.jeremyevans.net/rdoc/files/doc/sql_rdoc.html#label-Identifiers

On Heroku I got FFMPEG::Error (Failed encoding.Errors: encoded file is invalid. Full output: )

The code is:
tempfile = Tempfile.open('content')
tempfile.write(content)
movie = FFMPEG::Movie.new(tempfile.path)
screenshot_temp_file = Tempfile.open('screenshot')
movie.screenshot(screenshot_temp_file.path)
everything works fine locally, but when I try this on Heroku I got error at last row:
irb(main):013:0> movie.screenshot(screenshot_temp_file.path)
I, [2020-10-07T15:48:27.132114 #7] INFO -- : Running transcoding...
["/app/vendor/ffmpeg/ffmpeg", "-y", "-i", "/tmp/content20201007-7-1g1955b", "-vframes", "1", "-f", "image2", "/tmp/screenshot20201007-7-1mmts7s"]
E, [2020-10-07T15:48:27.190572 #7] ERROR -- : Failed encoding...
["/app/vendor/ffmpeg/ffmpeg", "-y", "-i", "/tmp/content20201007-7-1g1955b", "-vframes", "1", "-f", "image2", "/tmp/screenshot20201007-7-1mmts7s"]
Errors: encoded file is invalid.
Traceback (most recent call last):
7: from /app/bin/irb:30:in `<main>'
6: from /app/bin/irb:30:in `load'
5: from /app/vendor/ruby-2.6.6/lib/ruby/gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
4: from (irb):13
3: from /app/vendor/bundle/ruby/2.6.0/gems/streamio-ffmpeg-3.0.2/lib/ffmpeg/movie.rb:204:in `screenshot'
2: from /app/vendor/bundle/ruby/2.6.0/gems/streamio-ffmpeg-3.0.2/lib/ffmpeg/transcoder.rb:45:in `run'
1: from /app/vendor/bundle/ruby/2.6.0/gems/streamio-ffmpeg-3.0.2/lib/ffmpeg/transcoder.rb:112:in `validate_output_file'
FFMPEG::Error (Failed encoding.Errors: encoded file is invalid. Full output: )
I've try to pass some options, like :seek_time and validate: false but nothing changed. What am I doing wrong? Thank you so much!
EDIT:
After Full output: ) there is nothing.
I've install FFMPEG buildpack on Heroku, and can see it in ui:

Mongo Ruby Driver Cursor not found error

I find documents from mongodb using ruby driver, collect them to array and iterate on them to update the same document as in the code below:
crawlarray = ##mongoclient[:crawlarray].find({searchresults:[]},:timeout => false).limit(500)
crawlarray.each do |elm|
finalsearchstring = elm['searchstring']
if elm["searchresults"].blank?
ap "SEARCHING: #{finalsearchstring}"
results = searchG(finalsearchstring)
elm["searchresults"] = results
##mongoclient[:crawlarray].update_one({"_id" => elm['_id']}, elm)
else
ap "ALREADY SEARCHED: #{finalsearchstring}"
end
end
There are 90K records but as you see I just get 500 to not to get the error.Everytime, after about 150 iterations I get this error;
D, [2016-08-02T22:32:08.853065 #10098] DEBUG -- : MONGODB | 127.0.0.1:27017 | posluga-dev.getMore | FAILED | Cursor not found, cursor id: 463388278686 (43) | 0.008009s
/Users/apple/.rvm/gems/ruby-2.2.1/gems/mongo-2.2.7/lib/mongo/operation/result.rb:256:in `validate!': Cursor not found, cursor id: 463388278686 (43) (Mongo::Error::OperationFailure)
from /Users/apple/.rvm/gems/ruby-2.2.1/gems/mongo-2.2.7/lib/mongo/operation/executable.rb:36:in `block in execute'
from /Users/apple/.rvm/gems/ruby-2.2.1/gems/mongo-2.2.7/lib/mongo/server/connection_pool.rb:107:in `with_connection'
from /Users/apple/.rvm/gems/ruby-2.2.1/gems/mongo-2.2.7/lib/mongo/server/context.rb:63:in `with_connection'
from /Users/apple/.rvm/gems/ruby-2.2.1/gems/mongo-2.2.7/lib/mongo/operation/executable.rb:34:in `execute'
from /Users/apple/.rvm/gems/ruby-2.2.1/gems/mongo-2.2.7/lib/mongo/cursor.rb:163:in `block in get_more'
from /Users/apple/.rvm/gems/ruby-2.2.1/gems/mongo-2.2.7/lib/mongo/retryable.rb:51:in `call'
from /Users/apple/.rvm/gems/ruby-2.2.1/gems/mongo-2.2.7/lib/mongo/retryable.rb:51:in `read_with_retry'
from /Users/apple/.rvm/gems/ruby-2.2.1/gems/mongo-2.2.7/lib/mongo/cursor.rb:162:in `get_more'
from /Users/apple/.rvm/gems/ruby-2.2.1/gems/mongo-2.2.7/lib/mongo/cursor.rb:88:in `each'
from /Users/apple/.rvm/gems/ruby-2.2.1/gems/mongo-2.2.7/lib/mongo/collection/view/iterable.rb:44:in `each'
Can anybody help me to sort it out?
Use 'no_cursor_timeout' option along with the find query while using Mongo Ruby Driver.
This will disable all cursor timeouts. By default MongoDB tries to kill all cursors which have been inactive for more than 10 mins.
For more information, check this post.

Chef-solo and cookbook_path: multiple paths are not woking

Contrary to the docs...in my solo.rb file the below does not work:
root = File.absolute_path(File.dirname(__FILE__))
cookbook_path [ root + "/cookbooks",root + "/site-cookbooks", root + "/berks-cookbooks" ]
This works but defeats the purpose because I need to add the other two dirs in the cookbook paths:
root = File.absolute_path(File.dirname(__FILE__))
cookbook_path root + "/cookbooks"
Below is a sample role
{
"name": "coordinator_do_server",
"environment":{"git_repos":[]},
"java":{ "install_flavor":"oracle","jdk_version":"7"},
"run_list": ["recipe[build-essential]"
]
}
Error I get is the below:
Starting Chef Client, version 11.14.2
================================================================================
Error expanding the run_list:
================================================================================
Unexpected Error:
-----------------
TypeError: can't convert Array into String
[2014-12-07T18:22:07+08:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
Chef Client failed. 0 resources updated in 2.259210544 seconds
[2014-12-07T18:22:07+08:00] ERROR: can't convert Array into String
[2014-12-07T18:22:07+08:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
How do I resolve?
Generated at 2014-12-08 00:12:42 +0800
TypeError: can't convert Array into String
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/environment.rb:248:in `directory?'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/environment.rb:248:in `load_from_file'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/environment.rb:241:in `load'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/node.rb:380:in `apply_expansion_attributes'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/node.rb:369:in `expand!'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/policy_builder/expand_node_object.rb:135:in `expand_run_list'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/policy_builder/expand_node_object.rb:122:in `build_node'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/client.rb:259:in `build_node'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/client.rb:420:in `do_run'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/client.rb:213:in `block in run'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/client.rb:207:in `fork'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/client.rb:207:in `run'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/application.rb:237:in `run_chef_client'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/application/solo.rb:226:in `block in run_application'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/application/solo.rb:218:in `loop'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/application/solo.rb:218:in `run_application'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/application.rb:55:in `run'
/usr/local/lib/ruby/gems/1.9.1/gems/chef-11.14.2/bin/chef-solo:25:in `<top (required)>'
/usr/local/bin/chef-solo:23:in `load'
/usr/local/bin/chef-solo:23:in `<main>'
The problem is that the chef_repo_path derives its default value from that of cookbook_path, and then the default environment_path is based on chef_repo_path. To fix this add chef_repo_path root to your config.

peatio: what is default password for database.yml?

I just tried to install peatio and I see that password is empty in peatio/config/database.yml
what should be written there?
here is screenshot: http://s14.postimg.org/4ijmxjwtt/untitled.png
here is my error from terminal when I tried to start server:
bundle exec rails server
/usr/local/lib/ruby/2.1.0/psych.rb:370:in `parse': (): did
not find expected key while parsing a block mapping at line 6 column 3
(Psych::SyntaxError)
edited, here is full error:
from /usr/local/lib/ruby/2.1.0/psych.rb:370:in parse_stream'
from /usr/local/lib/ruby/2.1.0/psych.rb:318:inparse'
from /usr/local/lib/ruby/2.1.0/psych.rb:245:in load'
from /usr/local/lib/ruby/gems/2.1.0/gems/figaro-0.7.0/lib/figaro.rb:21:i
from /usr/local/lib/ruby/gems/2.1.0/gems/figaro-0.7.0/lib/figaro.rb:17:i
from /usr/local/lib/ruby/gems/2.1.0/gems/figaro-0.7.0/lib/figaro/railtie
from /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.0.2/lib/active_
from /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.0.2/lib/active_
from /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.0.2/lib/active_
from /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.0.2/lib/active_
from /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.0.2/lib/active_
from /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.0.2/lib/rails/applic
from /peatio/config/application.rb:16:in'
from /peatio/config/application.rb:15:in <top (required)>'
from /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.0.2/lib/rails/comman
from /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.0.2/lib/rails/comman
from /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.0.2/lib/rails/comman
from /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.0.2/lib/rails/comman
from bin/rails:4:inrequire'
from bin/rails:4:in `'
Peatio default password for user admin#barong.io is Qwerty123 or Chah5YohWm

Resources