I have a Python Scrapy spider that I want to run at regular intervals on Heroku or similar.
It produces a JSON file that I would like to commit to a Github repo.
Given Heroku or other similar platform, how can I set it up to automatically commit, post-run?
You could write an item pipeline that keeps a static list of items.
Give this pipeline a function called spider_closed and use the dispatcher to attach that function as a signal handler for spider_closed signal.
In spider_closed use json.loads to serialize your data. Save it to a file. And commit it to github.
This repo has good code examples:
https://github.com/dm03514/CraigslistGigs/blob/master/craigslist_gigs/pipelines.py
This seems to be a good library to use for the git part:
https://pypi.python.org/pypi/GitPython/
HTH - if you need anymore help I'd be happy to update this response.
Can anyone suggest a way to have any application errors in a Padrino app send these errors via email?
I already have the Padrino Mailer configured properly and can send test emails, I just have no idea how to configure the app to send me a report via mail (as well as logging it, of course) whenever an error occurs.
Thanks.
I ended up using the padrino-contrib gem. With it, you can install the plugin you need (of course you can do it manually also):
padrino-gen plugin exception_notifier
Which will add it to your gem file and also edit your app/app.rb and boot.rb files to load this gem.
Then in app/app.rb you put something like:
register Padrino::Contrib::ExceptionNotifier
set :exceptions_from, "noreply#domain.com"
set :exceptions_to, "your_address.domain.com"
And that's it.
The nice thing about letting the plugin install it for you is that if you're more familiar with Rails than Padrino (as is my case) this will not only set things up for you, but also show you were the directives need to go.
Hope this helps someone else.
A good approach should be using exception handlers. Adding a begin..rescue block to your code, and if there is an exception, you send the email and continue to the desired behavior.
def some_action
begin
# some code that could go wrong
rescue SomeExceptionClass => some_variable
# here you send the email with the errors
# render stuff, redirect stuff, etc
end
end
I have a model, Car.
When I update an instance of Car through a form in my application, the PaperTrail plugin is called and a new version is created.
But if I test this model using rspec, the PaperTrail plugin is never called. Why not?
How are you creating the objects? I found that when I created them through a factory, a papertail version was not created. But when I created them through ActiveRecord (e.g. Car.create(params)) a paptertrail version was created
First of all check this:
# in config/environments/test.rb
config.after_initialize do
PaperTrail.enabled = false
end
read this manual or try this gist
I want to delete all the files it created and roll back any changes made, but not necessarily to the database, but more to the config files.
I'd like to automatically delete all the resource mappings for the model/controller deleted in the routes.rb file and everywhere else that changes might have been made?
rails destroy controller lalala
rails destroy model yadayada
rails destroy scaffold hohoho
Rails 3.2 adds a new d shortcut to the command, so now you can write:
rails d controller lalala
rails d model yadayada
rails d scaffold hohoho
It's worth mentioning the -p flag here ("p" for pretend).
If you add this to the command it will simply do a "test" run and show you what files will be deleted without actually deleting them.
$ rails d controller welcome -p
remove app/controllers/welcome_controller.rb
invoke erb
remove app/views/welcome
invoke test_unit
remove test/controllers/welcome_controller_test.rb
invoke helper
remove app/helpers/welcome_helper.rb
invoke test_unit
remove test/helpers/welcome_helper_test.rb
invoke assets
invoke coffee
remove app/assets/javascripts/welcome.js.coffee
invoke scss
remove app/assets/stylesheets/welcome.css.scss
If you're happy with it, run the command again without the -p flag.
rails destroy controller Controller_name was returning a bunch of errors. To be able to destroy controller I had to remove related routes in routes.rb. P.S. I'm using rails 3.1
This is a prototype to generate or destroy a controller or model in Rails:
rails generate/destroy controller/model [controller/model Name]
For example, if you need to generate a User Controller:
rails generate controller User
or
rails g controller User
If you want to destroy the User controller or revert to above action
then use:
rails destroy controller User
or:
rails d controller User
You could use rails d model/controller/migration ... to destroy or remove the changes generated by using the rails generate command.
For example:
rails g model Home name:string
creates a model named home with attribute name. To remove the files and code generated from that command we can use
rails d model Home
If you use Rails, use rails d controller Users.
And, if you use Zeus, use zeus d controller Users.
On the other hand, if you are using git or SVN, revert your changes with the commit number. This is much faster.
Before reverting the rails generate, please make sure you rollback the migration first.
Case 1: if you want to revert scaffold then run this command:
rails destroy scaffold MODEL_NAME
Case 2: if you want to revert model then run this command:
rails destroy model MODEL_NAME
Case 3: if you want to revert controller then run this command:
rails destroy controller CONTROLLER_NAME
Note: you can also use shortcut d instead of destroy.
You can destroy all things that was created same way except little thing change.
For controller,
rails d controller_name (d stands for destroy)
For Model
rails d model_name
you just put d(destroy) instead of g(generate) in your migration.
If you prefer to delete the controller manually:
For controller welcome
rm app/controllers/welcome_controller.rb
rm app/views/welcome
rm test/controllers/welcome_controller_test.rb
rm app/helpers/welcome_helper.rb
rm test/helpers/welcome_helper_test.rb
rm app/assets/javascripts/welcome.js.coffee
rm app/assets/stylesheets/welcome.css.scss
You can revert your
rails g/generate controller/model/migration xxx
output by using:
rails d/destroy controller/model/migration xxx
Suppose I have created a controller named "sample" like:
rails generate controller sample
If I have to destroy this controller, all I have to do is swap generate with destroy, as in
rails destroy controller sample.
If you want to reverse the generation, all you have to do is swap generate with destroy.
You can undo a rails generate in the following ways:
For the model: rails destroy MODEL
For the controller: rails destroy controller_name
To reverse rails generate, use rails destroy:
rails destroy Model
See "rails destroy" for more information.
All versions of rails have a "destroy", so-, if you create a (for example) scaffold named "tasks" using a generator, to destroy all the changes of that generate step you will have to type:
rails destroy scaffold Tasks
Hope it helps you.
To reverse that, we just destroy it. Open the Terminal application and go to the project directory, then, type this:
rails destroy model CamelCase
rails destroy controller CamelCase
Where CamelCase is a name of any model or controller.
It will remove the model, migration and some of the related test files. (You can see the result in the Terminal window after you have run the command.)
We use generate as
rails generate app.
So regenerating any generate statement can be reversed using destroy statement.
Just replace generate with destroy
i.e. rails generate app can be written as rails destroy app'
rails generate ____asrails destroy ____`
Removed scaffolding for selected model:
bin/rails d scaffold <AccessControl> //model name
I'm still working on my first Grails application. This time, my problem is to limit access to some actions for particular users.
Assume users add some object, e.g. books. I would like to give access to edit a book only to admin and the user that added the book. I'm currently using Acegi plugin. I know there is newer version of that plugin, but I'm not sure if it changes anything in my problem.
The second thing is some kind similar. I have a sidebar and there is "Hello ${currentUser.username}. currentUser is a method that returns an instance of currently logged user. But the problem is that I don't have any idea where can I put this message to be able to use it everywhere. Should I put it in some service and include it everywhere? I tried to create an ApplicationController that is extended by all other controllers, but that doesn't seem to work. Have you got any ideas?
Thanks!
Grzegorz
You should use the newer Spring Security Core plugin since it has an ACL add-on plugin that does exactly what you're looking for. See http://grails.org/plugin/spring-security-acl for details.
For the second question, there's a taglib for that. In the Acegi plugin use this:
Hello <g:loggedInUserInfo field="username"/>
(see http://www.grails.org/AcegiSecurity+Plugin+-+Artifacts) and in the Spring Security Core plugin use this:
Hello <sec:username/>
(see the "Security Tags" section of http://burtbeckwith.github.com/grails-spring-security-core/docs/manual/)
For ROLE access you'll just need to specify that a particular ROLE for a particular URL has access to that action. That is if you are using the plugin's RequestMap approach. If you're using the annotation approach, just annotate the action in the controller with:
#Secured(['WHATEVER_ROLE'])
As far as only allowing the user who created the book to edit it, you can pull the user domain out of the authentication with authenticateService.userDomain(), then you can compare that user with the user who created the book (assuming you have some sort of createdBy property on your Book domain.
def loggedInUser = authenticateService.userDomain()
if (book.createdBy.equals(loggedInUser)) {
// allow editing
}
Something like that, anyway.