I'm trying to use Sequel to manage migrations for my database, and sometimes, due to feature-branching, there are moments when the database has migrations applied that don't exist in the filesystem.
By default, when I apply migrations with sequel -m on those situations I get this error:
Error: Sequel::Migrator::Error: Applied migration files not in file system
"Migrations" says there's an option for that:
Ignoring missing migrations
In some cases, you may want to allow a migration in the database that does not exist in the filesystem (deploying to an older version of code without running a down migration when deploy auto-migrates, for example). If required, you can pass :allow_missing_migration_files => true as an option. This will stop errors from being raised if there are migrations in the database that do not exist in the filesystem.
Nice!
How do I pass that allow_missing_migration_files option to sequel -m?
I think you'll have to use the Sequel::Migrator API for that. Something like
Sequel::Migrator.run(DB, '/path/to/migrations/dir', allow_missing_migration_files: true)
You can also wrap this behavior in a Rake task, so you don't have to start a console to run migrations.
namespace :db do
desc "Run migrations ignoring the missing ones"
task :migrate_without_missing, [:version] do |t, args|
require "sequel"
Sequel.extension :migration
db = Sequel.connect(ENV.fetch("DATABASE_URL"))
if args[:version]
puts "Migrating to version #{args[:version]}"
Sequel::Migrator.run(db, "db/migrations", target: args[:version].to_i, allow_missing_migration_files: true)
else
puts "Migrating to latest"
Sequel::Migrator.run(db, "db/migrations", allow_missing_migration_files: true)
end
end
end
Then run rake db:migrate_without_missing.
Related
In Rails 5, I am no longer able to use a rake task that migrates files in a different directory. I get the error ActiveRecord::UnknownMigrationVersionError: No migration with version number 20180209214145.
Any idea how I could do this in Rails 5? I think that the line ActiveRecord::Migrator.migrations_paths = ['db/manual_migrate'] no longer works as expected, but I am not quite sure how to fix it.
desc 'Run db:migrate:up on a task we wish to migrate manually (e.g. not on all servers).' \
'VERSION is required.'
task manual_migrate: [:environment, :load_config] do
raise 'VERSION of migration is required for this task!' unless ENV['VERSION']
default_paths = ActiveRecord::Migrator.migrations_paths
ActiveRecord::Migrator.migrations_paths = ['db/manual_migrate']
begin
Rake::Task['db:migrate:up'].invoke
ensure
ActiveRecord::Migrator.migrations_paths = default_paths
end
end
I generated model with hanami generate model stimulus. Then I fixed "stimuluses" to "stimuli" in the migration file name and inside, the table name.
Everytime I load a page I get this error in the server console window:
[ROM::Relation[Stimuluses]] failed to infer schema. Make sure tables exist before ROM container is set up. This may also happen when your migration tasks load ROM container, which is not needed for migrations as only the connection is required (schema parsing returned no columns, table "stimuluses" probably doesn't exist)
I looked into the libraries and found that this functionality has Inflecto library. Then I tried both adding to hanami project this:
# /config/initializers/inflecto.rb
require 'inflecto'
Inflecto.inflections do |inflect|
inflect.irregular('stimulus', 'stimuli')
end
And editing the defualt library file:
# gems/inflecto-0.0.2/lib/inflecto/defaults.rb
Inflecto.inflections do |inflect|
...
inflect.irregular('stimulus', 'stimuli')
...
end
But the message is still there after restarting the server.
Is this something I should solve and if yes, how to do this?
EDIT:
Also tried:
# /config/initializers/inflector.rb
require 'hanami/utils/inflector'
Hanami::Utils::Inflector.inflections do
exception 'stimulus', 'stimuli'
end
I'm assuming we are talking about Hanami v1.0.0, right?
You nearly succeeded. What hit you is that initializers seem to be not loaded when executing hanami commands and maybe a bug in code reloading. So instead of an initializer put it into a file that gets loaded when hanami commands are executed or require the initializer file in such a place. E.g.,
# config/initializers/inflections.rb
require 'hanami/utils/inflector'
Hanami::Utils::Inflector.inflections do
exception 'stimulus', 'stimuli'
end
and then in your environment file
# config/environment.rb
# ...
require_relative 'initializers/inflections.rb'
# ...
I'm not sure if that is a good place to put custom inflection rules, but at least it works.
I'm trying to perform database related operations on my newly upgraded app(Rails 5) and I'm unable to perform destructive database commands locally.
rails db:reset or rails db:drop .
The trace results with the following data,
rails db:drop --trace
** Invoke db:drop (first_time)
** Invoke db:load_config (first_time)
** Execute db:load_config
** Invoke db:check_protected_environments (first_time)
** Invoke environment (first_time)
** Execute environment
** Invoke db:load_config
** Execute db:check_protected_environments
rails aborted!
ActiveRecord::NoEnvironmentInSchemaError:
Environment data not found in the schema. To resolve this issue, run:
bin/rails db:environment:set RAILS_ENV=development
What I've tried so far are,
Setting bin/rails db:environment:set RAILS_ENV=development, doesn't change anything still the error occurs.
Setting Environment variable manually to development.
None of these helped. I'm Looking for a fix or workaround.
Two Fixes to ActiveRecord::NoEnvironmentInSchemaError
The other answers here describe the problem very well, but lack proper solutions. Hoping this answer helps someone experiencing this issue.
Why this error is happening
This incorrect error message is a result of this pull request designed to prevent destructive actions on production databases. As u/pixelearth correctly points out, Rails 4.2 defines the 'key' field in the ar_internal_metadata table to be an integer, and Rails 5+ (including Rails 6) expects this to be a string with the value, environment. When Rails 5 and Rails 6 run this safety check, they incorrectly raise an ActiveRecord::NoEnvironmentInSchemaError as the code is incompatible with the Rails 4 schema format.
Fix #1: Override the safety check in Rails 5+
**Please remember, the safety check was implemented as a result of so many users dropping their production databases by accident. As the question describes, the operations are destructive.
From the terminal:
rails db:drop RAILS_ENV=development DISABLE_DATABASE_ENVIRONMENT_CHECK=1
# and/or
rails db:drop RAILS_ENV=test DISABLE_DATABASE_ENVIRONMENT_CHECK=1
As noted here, the DISABLE_DATABASE_ENVIRONMENT_CHECK=1 flag disables the environment check. After, you can run a rake db:create RAILS_ENV=development, for example, to recreate your database with the correct schema in the ar_internals_metadata table.
Fix #2: Revert to Rails 4, drop database, go back to Rails 5+ and recreate
From the terminal:
git log
# grab the commit hash from before the upgrade to Rails 5+
git checkout **hash_from_rails_4**
rake db:drop RAILS_ENV=development
rake db:drop RAILS_ENV=test
git checkout master
# now things should work
rails db:migrate
Again, please ensure you are not pointing at a production database when overriding this functionality. Alternatively, it would be possible to directly modify the schema of this table. If you're experiencing this error in production, you may need to take this approach.
This happened because, for some reason, your table ar_internal_metadata got deleted or changed.
If you cannot drop the database via the command line, you need to do it via your DBMS or database client.
Then, just run rails db:create db:migrate to create and run the migrations.
For posterity, my issue was that this schema was generated by a rails 4 app and the current app using it is rails 5. With rails 5 the structure of the ar_internal_metadata table has changed slightly. The key field needs to be a string and contain the word 'environment', not an integer. This error only goes away when this is changed.
It should look like this in Rails 5
ie, change the type of ar_internatl_metadata #key to string...
My situation is a bit uncommon involving a rails 4 app and a rails 5 app sharing the same db. When I need to "update", I have a task:
puts "Modifying Rails 4 schema to fit Rails 5 schema"
file_name = "./db/schema.rb"
rails_4_ar_internal_metadata = 'create_table "ar_internal_metadata", primary_key: "key", force: :cascade do |t|'
rails_5_ar_internal_metadata = 'create_table "ar_internal_metadata", primary_key: "key", id: :string, force: :cascade do |t|'
new_schema = File.read(file_name).gsub(rails_4_ar_internal_metadata, rails_5_ar_internal_metadata)
File.write(file_name, new_schema)
Console rails
bin/rails db:environment:set RAILS_ENV=development
I had
bundle exec rake db:drop
rake aborted!
ActiveRecord::NoEnvironmentInSchemaError:
Environment data not found in the schema. To resolve this issue, run:
rails db:environment:set RAILS_ENV=development
And, indeed, simply running rails db:environment:set RAILS_ENV=development made the problem go away.
Why did this happen?
It happened because I tried to drop the database and create it / migrate it, however, I had a syntax error in the migration (datatype and column name in the wrong places). Check your migration file for any silly errors
Specifically I had
t.submitted :boolean, default: false
instead of
t.boolean :submitted, default: false
I see that the Sequel gem supports migrations here, but I don't see any type of generator documented. Does one exist; or should I be manually creating all of my migrations (or alternately creating my own task to generate migrations)?
From the documentation:
Sequel doesn't come with generators that create migrations for you. However, creating a migration is as simple as creating a file with the appropriate filename in your migrations directory that contains a Sequel.migration call.
The contents of the migration file doesn't have to specify a timestamp or index and it's an extremely simple format.
I generally just copy a previous migration (maybe one similar to the migration I'm creating) and alter the filename. See existing migrations with:
$ ls -1 db/migrate/
20170320075430_check_postgres_extensions.rb
...
For running migrations, I use the rake task that's available here.
Sequel has no migration generator. But you can easily write your own, for example using a rake task. Here is one:
# Rakefile
require 'fileutils'
namespace :db do
MIGRATIONS_DIR = 'db/migrations'
desc "generates a migration file with a timestamp and name"
task :generate_migration, :name do |_, args|
args.with_defaults(name: 'migration')
migration_template = <<~MIGRATION
Sequel.migration do
up do
end
down do
end
end
MIGRATION
file_name = "#{Time.now.strftime('%Y%m%d%H%M%S')}_#{args.name}.rb"
FileUtils.mkdir_p(MIGRATIONS_DIR)
File.open(File.join(MIGRATIONS_DIR, file_name), 'w') do |file|
file.write(migration_template)
end
end
end
Then run the rake task like this rake db:generate_migration[my_migration_name]. This command will create a Sequel migration file 20220720233750_my_migration_name.rb in the directory db/migrations.
I dont want Rails 3 to generate my schema every time I do migration. How to properly disable it?
Thanks
For anyone who is still looking for a way to disable the db dump after migration a configuration is now available in rails 4 which can be set to false like this:
config.active_record.dump_schema_after_migration = false
will prevent it. The configuration has been added in this change - https://github.com/rails/rails/pull/13948
Create an application specific task (as Alex Kaushovik suggested) like so...
Create a file lib\tasks\db_schema_override (actual name doesn't matter, you need a .rake file in lib\tasks) with contents as below (credit to Matthew Bass for remove_task)
Rake::TaskManager.class_eval do
def remove_task(task_name)
#tasks.delete(task_name.to_s)
end
end
Rake.application.remove_task('db:schema:dump')
namespace :db do
namespace :schema do
task :dump do
# Overridden to do nothing
end
end
end
A modification based on David Waller's answer that retains the ability to run db:schema:dump manually. Covers all tasks where schema dumping is automatically invoked. Based on rails 3.2.14 versions of the relevant tasks - when you change rails versions, you'll want to ensure that the task definitions are still valid.
Seriously, there should be a config option to disable automatic schema dumps.
Rake::TaskManager.class_eval do
def remove_task(task_name)
#tasks.delete(task_name.to_s)
end
Rake.application.remove_task("db:migrate")
Rake.application.remove_task("db:rollback")
Rake.application.remove_task("db:forward")
Rake.application.remove_task("db:migrate:up")
Rake.application.remove_task("db:migrate:down")
end
namespace :db do
desc "Migrate the database (options: VERSION=x, VERBOSE=false)."
task :migrate => [:environment, :load_config] do
ActiveRecord::Migration.verbose = ENV["VERBOSE"] ? ENV["VERBOSE"] == "true" : true
ActiveRecord::Migrator.migrate(ActiveRecord::Migrator.migrations_paths, ENV["VERSION"] ? ENV["VERSION"].to_i : nil) do |migration|
ENV["SCOPE"].blank? || (ENV["SCOPE"] == migration.scope)
end
# db_namespace['_dump'].invoke
end
desc 'Rolls the schema back to the previous version (specify steps w/ STEP=n).'
task :rollback => [:environment, :load_config] do
step = ENV['STEP'] ? ENV['STEP'].to_i : 1
ActiveRecord::Migrator.rollback(ActiveRecord::Migrator.migrations_paths, step)
# db_namespace['_dump'].invoke
end
# desc 'Pushes the schema to the next version (specify steps w/ STEP=n).'
task :forward => [:environment, :load_config] do
step = ENV['STEP'] ? ENV['STEP'].to_i : 1
ActiveRecord::Migrator.forward(ActiveRecord::Migrator.migrations_paths, step)
# db_namespace['_dump'].invoke
end
namespace :migrate do
# desc 'Runs the "up" for a given migration VERSION.'
task :up => [:environment, :load_config] do
version = ENV['VERSION'] ? ENV['VERSION'].to_i : nil
raise 'VERSION is required' unless version
ActiveRecord::Migrator.run(:up, ActiveRecord::Migrator.migrations_paths, version)
# db_namespace['_dump'].invoke
end
# desc 'Runs the "down" for a given migration VERSION.'
task :down => [:environment, :load_config] do
version = ENV['VERSION'] ? ENV['VERSION'].to_i : nil
raise 'VERSION is required' unless version
ActiveRecord::Migrator.run(:down, ActiveRecord::Migrator.migrations_paths, version)
# db_namespace['_dump'].invoke
end
end
end
I've tested the following - it works:
You could modify the file "databases.rake" inside of the Rails gem's lib/tasks folder on your server(s). Comment out lines with the following code:
Rake::Task["db:schema:dump"].invoke if ActiveRecord::Base.schema_format == :ruby
By default on Ubuntu (and Ubuntu server) it is here: /var/lib/gems/1.8/gems/rails-x.x.x/lib/tasks/databases.rake.
Tested with Rails 2.3.11, but I'm pretty sure it'll work with Rails 3.x.x too.
Another possible solution (didn't test):
You wouldn't have to modify Rails files, just your application.
A rails plugin called "override_rake_task" could be used to override Rake task "db:schema:dump" which is defined inside if Rails gem.
Apparently if you are not using this plugin and if you define a task in your rails application with the same name, rake would execute both tasks: the default and yours.
The default behavior of Active Record is to dump the schema, but it only does it if the schema dump format is set to ruby (the default).
You can disable the schema dump after migrations by setting the schema format to sql. You can safely reverse it to ruby after the migration finishes without consequences.
You can do it in this way:
1) If you have a line like this one setting explicitly the schema to ruby:
config.active_record.schema_format = :ruby
(it could be inside config/application.rb, on config/environment or in the specific environment you are running the migration in).
Then comment it out:
#config.active_record.schema_format = :ruby
2) Then add a line to set the schema format to SQL, either in the environment you are using, or at config/application.rb, as follows:
config.active_record.schema_format = :sql
Rather than overriding the db:migrate behavior (which is obviously necessary when you're building the migration in development, but unnecessary when deploying) I would suggest creating a separate task like:
# http://stackoverflow.com/questions/13646840/how-to-make-rake-dbmigrate-generate-schema-rb-when-using-sql-schema-format
namespace :db do
desc "Migrate the database, without dumping the schema.rb"
task :deploy => :environment do
ActiveRecord::Migration.verbose = ENV["VERBOSE"] ? ENV["VERBOSE"] == "true" : true
ActiveRecord::Migrator.migrate("db/migrate/", nil)
end
end
Then during your deployments you can simply do
rake db:deploy