I have a Flask app hosted on Heroku but want the static files to be served from Amazon S3.
In my templates I use url_for() for all references to static files. In the initialization of the Flask app I then want to put
app = Flask(__name__, static_url_path="http://my-bucket.s3.amazonaws.com")
to make sure instead of mysite.com/static/, http://my-bucket.s3.amazonaws.com/static/ is used. However I get this error:
ValueError: urls must start with a leading slash
If I change it to a value with a leading slash it works, but I want the static URL to point to S3, an external domain so it needs to start with http://.
What am I doing wrong? How can I use S3 for static files with Flask and Heroku?
If you are using any of the static_* options to the Flask object, it is assuming that it'll be responsible for serving those files. A static route is configured that serves both as the view to serve the static files and as the url_for() target to generate the URLs.
So, with Flask alone, you'd have to replace all url_for('static', ...) calls with hardcoded URLs to your CDN instead.
Instead, you should switch to using Flask-CDN, a handy Flask add-on to handle seamless switching between static files hosted by Flask and by a CDN:
from flask_cdn import CDN
app = Flask(__name__)
cdn = CDN(app)
and set the CDN_DOMAIN configuration option to http://my-bucket.s3.amazonaws.com when deploying to production.
In debug mode, url_for() will then generate the old /static/.. urls for Flask to handle, in production mode, url_for() prefixes those urls with the CDN_DOMAIN value. If the latter is left to the default None setting, no such URL alterations take place, making it possible to run Flask locally with debug switched off as needed.
Note that the behaviour of url_for() is only altered for Jinja templates; if you need to generate static URLs in your views, you'll have to swap flask.url_for() out for flask_cdn.url_for().
Related
this is a little bit strange because most of the questions here wanted to force https.
While learning AWS elastic beanstalk. I am hosting a laravel site there. Everything is fine, except that none of my javascripts and css files are being loaded.
If have referenced them in the blade view as :
<script src="{{asset('assets/backend/plugins/jquery/jquery.min.js')}}"></script>
First thing I tried was looking into the file/folder permissions in the root of my project by SSHing into EC2 instance. Didn't work even when I set the permission to public folder to 777.
Later I found out that, the site's main page url was http while all the assets url were 'https'.
I dont want to get into the SSL certificates things just yet, if it is possible.
Is there anyway I can have my assets url be forced to Http only?
Please forgive my naiveity. Any help would be appreciated.
This usually happens if your site is for example behind an reverse proxy, As the URL helper facade, trusts on your local instance that is beyond the proxy, and might not use SSL. Which can be misleading/wrong.
Which is probaly the case on a EC2 instance... as the SSL termination is beyond load balancers/HA Proxies.
i usually add the following to my AppServiceProvider.php
public function boot()
{
if (Str::startsWith(config('app.url'), 'https')) {
\URL::forceScheme('https');
} else {
\URL::forceScheme('http');
}
}
Of course this needs to ensure you've set app.url / APP_URL, if you are not using that, you can just get rid of the if statement. But is a little less elegant, and disallows you to develop on non https
I am developing a Flask app running on Heroku that allows users to upload images. The app has a page displaying the user's images in a table.
For developing purposes, I am saving the uploaded files to Heroku's ephemeral file system, and everything works fine: the images are correctly loaded and displayed (I am using the last method shown here implying the use of send_from_directory()). Now I have moved the storage to S3 and I am trying to adapt the code. I use boto3 to upload the files to the bucket: it works fine. My doubts are related to the download to populate the users' pages with their images.
As explained here, I could set the file as "public-read" and use the URL (I think this is what Flask-S3 does), but I'd rather prefer not to leave free access to the files. So, my solution attempt is to download the file to Heroku's filesystem and serve the image using again the send_from_directory() as follows:
app.py
#app.route('/download/<resource>')
def download_image(resource):
""" resource: name of the file to download"""
s3 = boto3.client('s3',
aws_access_key_id=current_app.config['S3_ACCESS_KEY'],
aws_secret_access_key=current_app.config['S3_SECRET_KEY'])
s3.download_file(current_app.config['S3_BUCKET_NAME'],
resource,
os.path.join('tmp',
resource))
return send_from_directory('tmp', # Heroku's filesystem
resource,
as_attachment=False)
Then, in the template I generate the URL for the image as follows:
...
<img src="{{ url_for('app.download_image',
resource=resource) }}" height="120" width="120">
...
It works, but I don't think this is the proper way for some reasons: among them, I should manage the Heroku's filesystem to avoid using up all the space between dynos restart (I should delete the images from the filesystem).
Which is the best/preferred way, also considering the performance?
Thanks a lot
The preferred way is to simply create a pre-signed URL for the image, and return a redirect to that URL. This keeps the files private in S3, but generates a temporary, time limited, URL that can be used to download the file directly from S3. That will greatly reduce the amount of work happening on your server, as well as the amount of data transfer being consumed by your server. Something like this:
#app.route('/download/<resource>')
def download_image(resource):
""" resource: name of the file to download"""
s3 = boto3.client('s3',
aws_access_key_id=current_app.config['S3_ACCESS_KEY'],
aws_secret_access_key=current_app.config['S3_SECRET_KEY'])
url = s3.generate_presigned_url('get_object', Params = {'Bucket': 'S3_BUCKET_NAME', 'Key': resource}, ExpiresIn = 100)
return redirect(url, code=302)
If you don't like that solution, you should at least look into streaming the file contents from S3 instead of writing it to the file system.
I have my Rails 3.1.0 application running with passenger in production environment and I have a section where the application allows the user to change his profile picture so I upload the image using an ajax uploader and in my controller I upload the file and generate different sizes for the image with rmagick then I render the new image with an image_tag but the application won't show the image till I restart the server.
What I get is No route matches [GET] "assets/path/to/image.png"
If I restart the server It will show the image, but obviously I can't be restarting the server every once a user uploads a new image.
How can I solve the keeping the assets working the right way?
The Rails asset pipeline is really meant for structural / design images, such as backgrounds, icons, banners, etc..). Dynamic assets should go in the public directory [source below]
It's probably a good idea to serve static assets through Nginx or Apache or whatever your web-server is, or place them in the public directory of your Rails app.
That should solve your problem right there.. e.g. make a separate path for static assets into which you upload those images with rmagick / carrierwave, or whatever gem you prefer.
The asset pipeline only knows about images which are present during start-up. So separating static / uploaded assets into a separate directory , and serving it directly through the web-server, will help -- it should also be much faster.
you'll need something like this in your configuration:
# Disable Rails's static asset server (Apache or nginx will already do this)
config.serve_static_assets = false
# Compress JavaScripts and CSS
config.assets.compress = true
# Don't fallback to assets pipeline if a precompiled asset is missed
config.assets.compile = false
# Generate digests for assets URLs
config.assets.digest = true
# UNCOMMENT the header that your server uses for sending files
# config.action_dispatch.x_sendfile_header = "X-Sendfile" # for apache
# config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' # for nginx
More General:
http://railscasts.com/episodes/279-understanding-the-asset-pipeline
http://guides.rubyonrails.org/asset_pipeline.html
Rails 3.1: Should File Uploads be added to asset pipeline?
Regarding serving images outside asset pipeline:
http://mrjaba.posterous.com/rails-31-asset-pipeline-with-nginx-and-passen
http://trackingrails.com/posts/rails-31-and-asset-pipeline-problems-with-apache
http://pastebin.com/kC4Ba40U
https://github.com/defunkt/resque/issues/418
What is the correct way to route your request through Sinatra so that it serves up the file with no processing? I'm looking for the most common way people do this in the Sinatra framework? I normally place all of my static content in a "content" path.
examples:
/content/css
/content/img
/content/js
How can I use a wildcard to serve up everything under content?
I was surprised there were no real examples of this here:
http://sinatra-book.gittr.com/
Sinatra and Rails use the path public for static content - e.g., ./public/javascripts/. All files in these paths would then be served by the web server (e.g. Thin, Passenger), but without the need for /public in the URL (e.g. the file at #{my_app_root}/public/javascripts/application.js would be available via the Web at the URL http://#{my_domain}/javascripts/application.js).
get '/notes/images/:file' do
send_file('/root/dev/notes/images/'+params[:file], :disposition => 'inline')
end
I am developing a http module that hooks into the FormsAuthentication Module through the Authenticate event.
While debugging i noticed that the module (and all other modules registered) gets hit every single time the client requests a resource (also when it requests images, stylesheets, javascript files (etc.)).
This happens both when running on a IIS 7 server in integrated pipeline mode, and debugging through the webdev server (in non- integrated pipeline mode)
As i am developing a website with a lot images which usually wont be cached by the client browser it will hit the modules a lot of unnessecary times.
I am using MVC and its routing mechanishm (System.Web.Routing.UrlRoutingModule).
When creating a new website the runAllManagedModulesForAllRequests attribute for the IIS 7 (system.webServer) section is per default set to true in the web.config, which as the name indicates make it call all modules for every single request.
If i set the runAllManagedModulesForAllRequests attribute to false, no modules will get called.
It seems that the reason for this is because of the routing module or mvc (dont know excactly why), which causes that the asp.net (aspx) handler never gets called and therefore the events and the modules never gets called (one time only like supposed).
I tested this by trying to call "mydomain.com/Default.aspx" instead of just "mydomain.com/" and correctly it calls the modules only once like it is supposed.
How do i fix this so it only calls the modules once when the page is requested and not also when all other resources are requested ?
Is there some way i can register that all requests should fire the asp.net (aspx) handler, except requests for specific filetype extensions ?
Of course that wont fix the problem if i choose to go with urls like /content/images/myimage123 for the images (without the extension). But i cant think of any other way to fix it.
Is there a better way to solve this problem ?
I have tried to set up an ignoreRoute like this routes.IgnoreRoute("content/{*pathInfo}"); where the content folder contains all the images, javascripts and stylesheets in seperat subfolders, but it doesnt seem to change anything.
I can see there a many different possibilites when setting up a handler but I cant seem to figure out how it should be possible to setup one that will make it possible to use the routing module and have urls like /blog/post123 and not call the modules when requesting images, javascripts and stylesheets (etc.).
Hope anyone out there can help me ?
Martin
The problem seems to be the routing module.
The solution is to move images, css, js to a subdomain, or you can probably register which filetypes/extensions the routing module should ignore.
The following code is what I use in every MVC Application in order to avoid the overhead caused by the routing system on serving static files, javascript, css, etc:
public static void RegisterRoutes(RouteCollection routes)
{
routes.RouteExistingFiles = false;
routes.LowercaseUrls = true;
routes.AppendTrailingSlash = true;
routes.IgnoreRoute("Content/{*pathInfo}");
routes.IgnoreRoute("Scripts/{*pathInfo}");
routes.IgnoreRoute("{resource}.axd/{*pathInfo}");
routes.IgnoreRoute("{*favicon}", new { favicon = #"(.*/)?favicon.ico(/.*)?" });
/* ... */
}